Preserving persistent storage through server maintenance

My agent code makes use of persistent storage. I have a power meter that accumulates kWhr and keeps track of overall, each day, each week. I have found it seems to get reset when there is server maintenance. My plot of kWhr has climbed up but then reset to zero on September 3, 2014 and again on 9/18.

I guess I don’t know for sure, but is my persistent storage being wiped out when the server is updated? If so, what kind of workaround could be done. One thought is that if the Agent could be notified in advance it could push the table to the device to be retrieved later. or, maybe I just have a bug in my code?

Hi mjkuwp94,

The persistent agent storage, server.save() and server.load(), work fine through reboots and upgrades. Why don’t you post some code and we can review it for bugs.

Aron.

sure, the code will be shared in due time. It is an open project so it all will be published. Right now it would be a really tough read.

Looks like I will get another chance at troubleshooting. A server maintenance has been announced for Friday. I’ll build up a php script to log the table and see what happens.

How does it act across Build and Runs / agent restarts?

If you’ve done things correctly, it should persist. Here’s an example of persistent storage I use for external credentials required by the agent (this isn’t the most efficient code, but it’s really good for demonstrating how the flow generally looks):

`data <- server.load();
user <- null;
pw <- null;

// load password if applicable
if (“pw” in data) {
pw = data.pw;
server.log(“Loaded Password”)
}
if (“user” in data) {
user = data.user;
server.log(“Loaded User”);
}

http.onrequest(function(req, resp) {
if (“pw” in req.query) {
pw = req.query[“pw”];
server.log(“Set Password”);

    // save password
    local data = server.load();
    data["pw"] <- pw;
    server.save(data);
}
if ("user" in req.query) {
    user = req.query["user"];
    server.log("User Password");

    // save user
    local data = server.load();
    data["user"] <- user;
    server.save(data);
}

});`

thanks! I think I will learn something tomorrow.

So - the agents are stopped to do server maintenance and then they are restarted at some point - i assume.

That is a unique case that I cannot test for because normally when I press ‘Build and Run’ both the agent and device restart. I have some ping-pong between the two sides during boot. For one thing, the device requests its calibration constants from the agent and the agents sends them back.

And of course - the agent loads data from persistent storage.

So - the agents are stopped to do server maintenance and then they are restarted at some point - i assume.

When we deploy a new version of the server-side code, we bring up a new server running the new version and migrate any connected devices and their agents from the old server to the new server. Then we migrate any remaining agents which don’t have connected devices. Then we turn off the old server.

This will cause your agent to be stopped on the old server and started on the new server.

The migration takes a few minutes for each server affected. From the point of view of an individual device or agent, it’s effectively instant.

This occurs during our scheduled maintenance windows, which are announced in the IDE, on Twitter (@impstatus), and on http://status.electricimp.com/. The interval between maintenance windows varies, but we will try to announce them several days in advance.

As an aside, today’s maintenance is infrastructure, and should not affect devices and agents whatsoever.

Your agent will also move from one server to another if your device decides to connect to an alternate back-end server (this will only happen if it can’t reach its preferred server). This also results in your agent code restarting.

To persist information across a restart, you can use server.save – which stores it in the Electric Imp back-end database, or you can use a third-party service, such as Firebase (see reference/webservices/firebase at master · electricimp/reference · GitHub).

thanks for the details. This may help a lot.

I do use server.save and that has worked for the most part. Two times this month my total reverted to zero and I think it corresponded to server maintenance by electric imp. I don’t know for sure what the cause is. It could be that my code flow results in the data being zeroed in that circumstance.

One of my troubles is that I cannot find a way to simulate this. I can restart a device by power cycle. I don’t have a way to reboot the agent for test that does not also restart the device.

I have seen Firebase - went through their tutorial a while back. It looks really nice. Given that I have use of the agent I have not found a use for Firebase yet.

Update: my issue was one of logic flow in my code. I tested two different models on different hardware during the last server maintenance. One worked, one did not so I know how to fix the problem. Just had to add a line to simulate an action that the Device would have done were it booting/restarting the regular way.

A problem I have is that I do not have a way to reboot the agent without booting the device so I cannot troubleshoot or test code. Of course, with a well-thought out flowchart and good coding the computer will do what you ask but…in this real world : )

Have you tried crashing the agent? You can force it to restart by allocating all of its memory

cool idea! no, I’ve not done that. I’d be a little worried about getting stuck in some unrecoverable state or worse - affecting other peoples agents? I don’t know how this system works to be perfectly honest.

would I just write a loop that does something like recursively adds a table or array to itself? that would cause a geometric progression and hog memory quite quickly I’d imagine.

Yep, basically everything in the Electric Imp environment must recover from a crash. This includes their Agent VM, DeviceOS, your Agent code, Device code and finally any cloud services that depend on the imps. I believe it’s good to crash early and often in the development cycle to see how everything copes. This link shows you one way of doing it.