Realtime Interaction with Imp

A way to incorporate websockets or other frameworks like SignalR, Socket.IO, to eliminate the polling.

I did some experimentation with signalR and works great but needs another server(I use Azure) to establish the conecction between clients and IMP.

This is something we’re looking at / working on, but it’s a fairly big feature for a number of reasons.

Right now the best way to poll imps is using the notion of long polling… this will help you identify when certain events happen, but help if you are trying to stream real time data (like sensor data).

Here’s a rundown of how it works (note - this requires agents):

  1. Make a request from your client (the request should have a very long timeout)
  2. Each time you get a request, push the response object into a queue
  3. When an event is triggered in the device (say a button press), send a message to the agent. In the agent handler for that event, send responses to all of the queued events.

I’ll work up an example for this later today to make it a bit more clear :slight_smile:

Here’s the code… this example can handle both traditional polling (where you send a response immediately), and long polling (where you wait until the state changes to send the response):

Agent Code
`server.log(“Use " + http.agenturl() + " to get last known value of pin1”);
server.log("Use " + http.agenturl() + “/longpoll to long poll the device and be notified as soon as the state of pin1 changes”);

responses <- [];
currentState <- null;

// when we get a request
http.onrequest(function(request, resp) {
server.log(request.path);
if (request.path == “/longpoll”) {
// push the response onto the queue
responses.push(resp);
return;
} else {
resp.send(200, http.jsonencode({ state = currentState }));
}

});

// when we get a “stateChanged” message from the device
device.on(“stateChanged”, function(data) {
currentState = data;

// if there are responses in the queue
if (responses.len() > 0 ) {
    // send a response for every queued message
    foreach(r in responses) {
        r.send(200, http.jsonencode({ state = data }));
    } 
    // clear responses
    responses = [];
}

});`

Device Code
`imp.configure(“Long Polling Example”, [], []);

// whenever pin1 changes, send it’s state to agent
function pin1Changed() {
local data = hardware.pin1.read();
agent.send(“stateChanged”, data);
}
hardware.pin1.configure(DIGITAL_IN, pin1Changed);`

Thanks beardedinventor .

Additional info
On the Agent add
server.log(request.path); resp.header("Access-Control-Allow-Origin", "*"); if (request.path == "/longpoll") {

The Client that I use , works great even multiple clients connected.
I wish WebSockets support. but Long Polling works great also

`

Long Polling

Imp Long Polling

`

@jfossi - I modified your post to wrap your code in a < code></ code> tag so all of the HTML + formatting works and is displayed properly :slight_smile:

EDIT: Looks like @jfossi beat me to the magic jQuery code to do longpolling… Mine includes HTTP Basic Auth and a resync function to be called in case events have happened in between longpolls (like that happen on Hannah when a push button is pressed and released). @jfossi have you had any issues where your javascript goes crazy and starts hammering your Agent? I get 429 errors on a regular basis that I haven’t gotten to the bottom of…

I’ve been debating releasing some code “as is” before I can finish it and I think it is ready enough to do now…

My Hannah Project on Github is meant to be a full blown example/tutorial on implementing all the hardware onboard Hannah using the imp and connecting it with an Agent to all the best Webservices available. I’ve also begun a simple webpage UI that implements longpolling using jQuery ajax requests.

The goal is to create the “ultimate” template for doing standard tasks and creating an example project with lots of best practices for developing with the imp - the Agent includes things like Basic Authentication over https for security for example. The code is NOWHERE near where I wanted to complete it but I haven’t had time to work on it for quite a while so maybe the community can pitch in and give me a hand. Let me know what you guys think and feel free to test/contribute on Github!

@beardedinventor thanks, first time posting code

@deldrid1 yes I have seen the 429 error and is usually problem in the my code not failing properly when there is an error, or making the call to the non Long polling address. I am still playing with the code I am learning as i go, specially $.ajax events order.

@deldrid1 i will test your code soon thanks , seems to me adding realtime is the cherry on top for the Imp.

Imp ecosystem is getting better …

A 429 error means too many requests. It can be caused by a few things:

  1. You’ve sent too many requests too quickly.
  2. You have too many open requests at once.

Right now, the only way to ‘close’ or complete a request is to send a send a response. That means if youre http.onerror handler encounters a runtime error before it does it’s response.send(statusCode, message) you essentially ‘lose’ one request (after 10 minutes the request times out and closes).

To avoid this, you should always (always, always, always) wrap you http.onrequest handler in a try / catch, and have the catch code send back an error message.

http.onrequest(function(req, resp) { try { // your code } catch (ex) { resp.send(501, "Internal Server Error: " + ex); } });

I think the issue is normally on the Javascript side of things and is the first thing you mentioned… jQuery gets tripped up somewhere and begins to flood the imp cloud with messages (until very quickly the agent gets blocked and 429 errors begin to flow). I’m still trying to figure out how to detect/handle these corner cases.

A few followup questions:

  1. If a request is closed by the client does that close an open request or does resp.send() still need to happen for it to not be counted as a “strike” against open requests?
  2. How many open requests are allowed at one time?
  3. Is the 10 minute request timeout currently implemented? I haven’t checked in a while but it was a known issue that it was limited only to 1 minute not too long ago.

@beardedinventor I dont think the http.onrequest(function(req, resp) fires on 429 I have a server.log(request.path); after it and I see the 429 on the client but not log on the server

Please advice?
@deldrid1 I saw your Github post great start but I am trying to get away from setting a server like node.js or asp.net-signalr to deal with the communication just straight from the browser to the agent(of course I will need to host the page but it is a plain HTML and JS).

@jsoffi - That is correct. If you’re getting a 429 response code it means you’re http requests are getting rate limited (which occurs before we pass the request to your agent).

  1. I believe you need to do a resp.send() - you this would be a fairly easy thing to test (make an agent that never responds, open some requests, close them, and then try to make more requests)
  2. Right now you are allowed 4 concurrent requests. I added some code to my long polling example to handle this. Basically, the code makes sure we’ll never have more open requests than we’re allowed:
    `http.onrequest(function(request, resp) {
    if (request.path == “/longpoll”) {
    server.log(“in longpoll”);
    // we only ever want to have 3 open requests at a time.
    // 4 is max, and if we get a 5th, it is queued, which we don’t want
    if (responses.len() >= 3) {
    // respond to oldest request, and push everything else down the queue
    responses[0].send(0, http.jsonencode({ state = currentState }));
    responses[0] = responses[1];
    responses[1] = responses[2];
    responses[2] = resp;
    }
    // push the response onto the queue
    responses.push(resp);
    return;
    } else {
    resp.send(200, http.jsonencode({ state = currentState }));
    }

});`

  1. Hmm, I think it should be 10 minutes… I’ll make a quick test (well… an up to 10 minute test) and let you know what happens.

@deldrid1: You’re right! Looks like timeout is actually 1 minute right now - we’ll look into it :slight_smile:

@deldrid1 - We want light sensor code! We want light sensor code! We want light sensor code! (crowd roars) It’s the easiest of all, but nobody appears to have done it yet? I guess it’s up to me…

I did some a long long time ago, but it wasn’t very complete… sorry!

@jfossi I added the nodeServer just as a simple file server - all communication is done directly from the browser to the Agent.

@sbright33 I released what I had so that someone like you could implement the light sensor! I’ve included commented out arduino code for it so it should be pretty easy :wink:

@sbright33 I have some basic code for the TSL2561 in Nora here. I am going to be updating it to Nora v2 very soon.

Where can I find yours Hugo? Simple is good.

Is tsl2561 code compatible with
adjd-s311-cr999?

Here is what is in V3 Hannah?