Device Array -> Agent JSON -> website

Agent Code
1

I’ve been working on this for about a week now - learning by reading, trial and error - more reading. It’s been about a week and I’ve gotten pretty far considering I started with nothing. The basics are this:

I have a energy monitoring board (openenergymonitor.org) TX3 which I connected an April Imp to via serial UART57. I went from getting a stream of integers to being able to send an array to the agent, which then posts to my own webserver. I have all the bits I think, but I’m stuck on how to try and format my JSON stream.

The array from the Device is:

[agent] [ "9", "9", "1215", "245", "0", "11838" ]

It represents:

["node", "sensor1", "sensor2", "sensor3", "sensor4", "ACVolts"]

The array reaches the agent (after lots of trial and error), but I’m stuck trying to figure out how to pull the integers from the array with the correct inputs for the JSON stream.

The format of the JSON should be:

http://staging.mywebsite.com/emoncms/input/post.json?json={node:"9",sensor1:”9”,sensor2:”1215”,sensor3:”245”,sensor4:”0”,ACvolts:"118.38"}&apikey=xxxxxxxxxxxxxxxxxxxx

I know the JSON format is right from making a manual post and getting values, but I can’t for the life of me figure out how to take the variables from the array and preface them with the correct values. I’ve been scratching my head for a day or so trying to sort out the best way to do this. The website auto updates, so as long as the JSON stream is there, it will update.

I won’t bore with my Device code, but here is my current Agent code:

device.on("senddata", function(data)

{
local data = (data)
sensor =

// Set URL to your web service
local url = mywebsite.com is available for purchase - Sedo.com";

// Set Content-Type header to json
local headers = { “Content-Type”: “application/json” };

// encode data and log
local body = http.jsonencode(data);
server.log(body);

// send data to your web service
http.post(url, headers, body).sendsync();
});

I’d really appreciate some guidance. I feel like the solution is just around the corner, but I can’t quite see it. I’m sure if someone can give me a bit of a shove it will all fall into place. I’m more than an newbee when it comes to coding - so everything is a learning experience. The forums and the documentation are great - and I keep referencing them for help. I managed to get A->B->C, but need some help getting C->D (the website in JSON).

I’m sure I’m missing something simple - just not sure what.

2
` // Set URL to your web service local url = http://staging.mywebsite.com/emoncms/input/post.json?json={node:"9",sensor1:”9”,sensor2:”1215”,sensor3:”245”,sensor4:”0”,ACvolts:"118.38"}&apikey=xxxxxxxxxxxxxxxxxxxx"; `

I didn’t quite understand the syntax of some of the code you posted. But, by the look of things, the code above doesn’t contain valid JSON. It’s better and safer to use the existing JSON encoder to do the work for you…

` local pathdata={ "node":9, "sensor1":9, "sensor2":1215, "sensor3":245, "sensor4":0, "ACvolts":118.38 }

// Set URL to your web service
local url = “mywebsite.com is available for purchase - Sedo.com”;
`

Alternately, you can represent pathdata as…

` local pathdata={ node=9, sensor1=9, sensor2=1215, sensor3=245, sensor4=0, ACvolts=118.38 } `

Are you sure you want to put the sensor data as a query parameter with the path? I’m not familiar with the emon input. Could you just pass it as the body of the http POST?

3

If the JSON is right there in the URL, doesn’t it need to be url-encoded?

Peter

4

It’s not usual to pass JSON in the URL like that. As Peter states you would at least need to url encode it.

But it’s best to send JSON data in the body of the request. Assume your data object is a squirrel table of data with the keys: node, sensor1, sensor2 etc and the values: 9, 9, 1215 etc. you would do:

``
device.on(“senddata”, function(data)

{

// Set URL to your web service
local url = http://staging.mywebsite.com/emoncms/input/post.json?apikey=xxxxxxxxxxxxxxxxxxxx";

// Set Content-Type header to json
local headers = {“Content-Type”=“application/json”};

// encode data and log
local body = http.jsonencode(data);
server.log(body);

// send data to your web service
http.post(url, headers, body).sendsync();
});
``

Then on your server you JSON decode the request body to get your data back. What’s your server code written in and I can provide an example?

5

Thanks for all the feedback. I think I was a little vague in my post because my terminology is probably a little off. Still learning :slight_smile: What I have is the Agent receiving the Device ‘senddata’

It’s presented simply as an array of integers (I believe that’s what it’s called)

[ "9", "9", "1215", "245", "0", "11838" ]

I don’t have key values - which I guess would be what identifies each integer in the ‘senddata’ There would be six key values:

node, sensor1, sensor2, sensor3, sensor4, ACVolts

Do I create a second array and then merge them into a table? (I don’t know the exact syntax here, and I’m writing from my office - not trying to be lazy)

node 9 sensor1 9 sensor2 1215 sensor3 245 sensor4 0 ACvolts 11838 (I'm sure I can figure out how to add the decimal to make it 118.38!)

When I look at what the OpenEnergyMonitor.org CMS is wanting for inputs, I see this:

{"id":"599","nodeid":"","name":"","description":"","processList":"","time":"1440510681","value":""}

The “id” and “time” is generated by the software. That said, the API Input Helper section seems to let me define the JSON format by sending a formatted url to the server:

http://staging.mywebsite.com/emoncms/input/post.json?={"id":"621","nodeid":"","name":"","description":"","processList":"","time":"1440513691","value":""}&apikey=xxxxxxxxxxxxx

So it seems to me I need to slot the array values into that format. What I’m struggling with is how to insert the array values in a string that is enclosed in " ". Doesn’t that mean everything inside the double quotes is static?

6

@cubenrg - I’ll try to answer your very specific question of how you can translate the array of integers into the json table format you’re looking for. I’m going to assume you’re relatively new to programming, so forgive me if some of the below explanation is really simple.

You can index an array (which means “read the value at a specific location”) by using square brackets to indicate which value you want (note: arrays start at 0, not 1… so if we want to read the third element, we would ask for 2).

So given the following array:

local values = [9,9,1215,245,0,11838];

We can access the third element with the following code:

local thirdElement = values[2];

Since you create the array, and know the order of the array, it’s not too difficult to translate these values to a table.

local values = [9,9,1215,245,0,11838]; local valuesTable = { "node": values[0], "sensor1": values[1], "sensor2": values[2], "sensor3": values[3], "sensor4": values[4], "ACvolts": values[5], };

Once you have the table, you should be able to jsonencode it to a string using http.jsonencode.

Hopefully that helps - if you’re still struggling, it may help to link to the documentation you’re talking about… might give some additional clues :slight_smile:

7

First off - thank you for your patience with me. Yes - I am very new to coding. I work in a massive datacenter by day with lots of tools and scripts available to me - some of which I modify to meet unique needs, but I’m always wanting to do a little bit more - make my own scripts and code - explore and understand how some of the bits fit together.

Here is basically what I’m trying to achieve. Perhaps I’m going about it all wrong, but I thought that leveraging as much of the Imp device was better than using the agent.

A simple UART signal comes to the Imp on pins 5/7 via what is basically an arduino with 4 sensors plugged into it.

http://openenergymonitor.org/emon/modules/emonTxV3

Perhaps I’m overthinking the whole process? The ‘retail’ version of this setup uses 433mhz to contact a base station that can post to http://emoncms.org/ or your own hosted emoncms - which in this case I’m doing.

The Device code is:

`inputString <- "";
emonTX3 <- hardware.uart57;
powerreading <- []

function readback() {
    local byte = emonTX3.read();
    if (byte == -1) return;
    if (byte == 13) {       
      local reading = split(inputString, " ")
     agent.send("senddata", reading);
       inputString = "";
    }
    else {  
        inputString = inputString + chr(byte);
    }
}
function chr(asciiValue) { 
    if (asciiValue < 32) return "";
    return format("%c", asciiValue);
}
function chr(asciiValue) {   
    if (asciiValue < 32) return "";
    return format("%c", asciiValue);
}
// PROGRAM STARTS HERE
emonTX3.configure(9600, 8, PARITY_NONE, 1, NO_CTSRTS, readback);`

The Agent code is:

`device.on("senddata", function(data) 
{
  local data = (data)

  // Set URL to your web service
local url = "http://staging.mywebsite.com/emoncms/input/post.json?node=9&json={nodeid:[0],name:[1],description:[2],processlist:[3],value:[4]}&apikey=xxxxxxxxxxxxxxx";

  local headers = { "Content-Type": "application/json" };

  local body = http.jsonencode(data);
  server.log(body);
  
   http.post(url, headers, body).sendsync();
});`

On my webserver, this is on the access.log:

"POST /emoncms/input/post.json?node=9&json={nodeid:[0],name:[1],description:[2],processlist:[3],value:[4]}&apikey=xxxxxxxxxxxxxxxxx HTTP/1.1" 200 177 "-" "Electric Imp Agent (454c339 - jenkins-ei-release-5126 - Thu Aug 13 06:10:18 2015)"

What has become evident it that I’m forming the JSON wrong! It looks like the only values I need to provide are “nodeid”:9,“sensor”:[reading] x the 4 sensors and the ACpower:[reading]

Am I just going about this the wrong way? The relationship between the simple integers that the IMP device captures and how I post them just seems to escape me! I’ve tried dozens of different ways from reading the forums, good 'ole Google and just about every other reference to objective-c or squirrel I can find.

I’m quite certain this is the kind of documentation you are asking for: https://github.com/emoncms/emoncms/blob/v8.5/Modules/input/input_model.php

https://github.com/openenergymonitor/documentation/blob/master/BuildingBlocks/emoncms/developinputproc.md

The Imp is such a perfect device for this application because it can take multiple serial inputs, and I’m looking forward to experimenting with the Ic2 and one-wire applications too.

I won’t be able to try anything out tonight it looks like - gonna be a late one at the office.

8

I think your agent code is muddled. It’s doing two things. It creates a url with url-encoded values and sends the data as an HTTP POST. I’d say you only need one of these. Judging by the documentation you mention (though I’m not a PHP coder), you need to send the data in a URL-encoded form.

As such, your URL is malformed: you’re listing a stack of array indexes (the [0], [1] etc) without naming the array (eg. data[0]) and not providing a way to insert actual values into the string.

I think all that you need to do is correctly form the URL string and call it using a simple HTTP GET (because the values you’re passing are in the url, not the HTTP request body), ie. something like:

`device.on("senddata", function(data) {
  // Set URL to your web service
  // Note: data is an array of strings sent via UART,
  // so data[x] does not require converting to a string
  local url = "http://staging.mywebsite.com/emoncms/input/post.json?node=9&json={nodeid:" + data[0] + ",name:" + data[1] + ",description:" + data[2] + ",processlist:" + data[3] + ",value:" + data[4] + "}&apikey=xxxxxxxxxxxxxxx";
  local response = http.get(url).sendsync();
});`

This should give the “JSON-like string” that emoncms expects, populated with the data values (from the array data[] passed into the function).

9

PS. I’ve added a variable, response, to capture the result of the sendsync() operation. This is useful as it allows you to get information on whether the sending of the data succeeded or failed and possibly why. For response’s data structure, see the sendsync() documentation.