How to deal with out of memory while offline?

if out of memory occur while the unit offline it will enter snooze state, and without internet it will always enter this state, network outage error will also snooze the unit. is there any way to prevent or exit this state while offline? and if i forgot the wifi ssid then is there no other way than blinkup?

Restarting the device is the only useful thing we can do after running out of memory; there’s no other way that squirrel code can handle it. When the device restarts after an error, it will try to connect to the server to look for new squirrel that might fix the problem. If there is no path available to the server, then the existing squirrel code will be restarted after a few seconds, which might then attempt a network operation and cause a snooze. There are some techniques and tips here for running your squirrel code offline without snoozing:

You can set the wifi ssid and password from squirrel code instead of blinkup, but you would need some way to pass the wifi credentials to squirrel in the first place (e.g. uart, ble, agent.on) and also some way of getting that squirrel code onto the device in the first place.

i deliberately put the unit in out of memory state when offline (after blessing) 10 second after unit turned on
but when the out of memory error occur it will enter snooze immediately.

server.setsendtimeoutpolicy(RETURN_ON_ERROR, WAIT_TIL_SENT, 10); didnt do anything after out of memory error occur.

my method:
bless unit → unit become offline → code running → out of memory occur → unit enter snooze
then i
disconnect the power → connect the power ->code running → out of memory occur → unit enter snooze

i do not intend to use the unit as development device but need to leave it at client side with no internet. so changing wifi is meaningless
and if error occur in this state(offline) it will freeze/enter snooze. and need to reconnect the power to restore the unit.

That sounds a bit wrong, because as Phil notes, snooze only happens if it can’t contact the server.

Do you have a small repeatable test case?

HI hugo, i test it with imp001 and imp004, the behaviour was different.
when out of memory error occur:
imp001 will reset → try to connect → re run code if fail.
imp001 will reset → try to connect → freeze until connected to internet

here is the code i use for imp001

#require “WS2812.class.nut:3.0.0”
server.setsendtimeoutpolicy(RETURN_ON_ERROR, WAIT_TIL_SENT, 60.0);

// Set up global variables
spi ← null;
led ← null;
state ← false;

// Define the loop flash function
function flash() {
state = !state;
local color = state ? [0,0,255] : [0,0,0];
led.set(0, color).draw();
imp.wakeup(0.5, flash);
}

// Set up the SPI bus the RGB LED connects to
spi = hardware.spi257;
spi.configure(MSB_FIRST, 7500);
hardware.pin1.configure(DIGITAL_OUT, 1);

// Set up the RGB LED
led = WS2812(spi, 1);

// Start the flash loop
flash();

array ← ;
array2 ← ;
array3 ← ;
array4 ← ;
array5 ← ;
array6 ← ;
array7 ← ;
array8 ← ;
array9 ← ;

filler ← “1234567890”

function fillmem(){

for(local x = 0;x<10;x++){
    array.append(filler)
    array2.append(filler)
    array3.append(filler)
    array4.append(filler)
    array5.append(filler)
    array6.append(filler)
    array7.append(filler)
    array8.append(filler)
    array9.append(filler)

}
imp.wakeup(0.1, fillmem);

}

fillmem();

function freemem() {
imp.wakeup(5, freemem);
server.log("free mem : " + imp.getmemoryfree());
}

freemem();

function arrcount(){
imp.wakeup(5, arrcount);
server.log("count : " + array.len());

}

arrcount();

and here the code for imp004

server.setsendtimeoutpolicy(RETURN_ON_ERROR, WAIT_TIL_SENT, 60.0);
hardware.pinP.configure(DIGITAL_OUT, 0);

state ← false;

// Define the loop flash function
function flash() {
state = !state;
if(state){
hardware.pinP.configure(DIGITAL_OUT, 1);
}
else {
hardware.pinP.configure(DIGITAL_OUT, 0);
}
imp.wakeup(0.5, flash);
}

flash();

array ← ;
array2 ← ;
array3 ← ;
array4 ← ;
array5 ← ;
array6 ← ;
array7 ← ;
array8 ← ;
array9 ← ;

filler ← “1234567890”

function fillmem(){

for(local x = 0;x<50;x++){
    array.append(filler)
    array2.append(filler)
    array3.append(filler)
    array4.append(filler)
    array5.append(filler)
    array6.append(filler)
    array7.append(filler)
    array8.append(filler)
    array9.append(filler)

}
imp.wakeup(0.1, fillmem);

}

fillmem();

function freemem() {
imp.wakeup(5, freemem);
server.log("free mem : " + imp.getmemoryfree());
}

freemem();

function arrcount(){
imp.wakeup(5, arrcount);
server.log("count : " + array.len());

}

arrcount();

i use imp001 dev board
and imp004 custom board

imp001 004 test (1).zip (3.1 MB)
here is the video of the code running on imp001(top) and imp004(bot)

0-10s the device is online
10s i cut off the internet
10-44s only imp001 running after restart, imp004 stay freeze

How do you “cut off the internet”? Behavior may differ between 001 and 004 due to different wifi chips, but also there’s a difference between losing wifi association (ie, AP turned off) - and losing uplink (WiFi still up, but no packets get to the internet). Here I could venture a guess that the 001 runs out of RAM just before you disable the network, but the 004 does not (as it has more free RAM), so the behavior seen is different.

The “60.0” in the setsendtimeoutpolicy will cause the system to block for up to 60 seconds in the case of not being able to send, before it returns an error (as you are in RETURN_ON_ERROR mode). As you are calling server.log every 5 seconds, expect a 60s block … or change the 60 to (eg) 0.1s if you want to stop blocking happening.

I would generally recommend reading How To Run An imp-enabled Device Offline | Dev Center and setting up offline logging with a serial cable Debugging Disconnected Applications | Dev Center in order to get more visibility into what’s happening on each system.

Going back to the top of this thread…

What are you actually looking to do? Use the imp as an MCU with no network connection? If so, then you should clear the wifi credentials totally once it’s running the code you want it to. This way, if it hits out of memory then the forced server connection on OOM (to check for new code, as it’s assumed that OOM was not intentional and hence due to a bug that might have been fixed) will be skipped.

I recently rediscovered the SpiflashLogger class and have moved all my server.log and server.error calls to that. It is heartwarming seeing the device go offline, continue logging without running out of memory, and spew out all the logs when it eventually comes online again.

hi hugo, i test it with both scenario “no wifi (AP off)” and “wifi ON but no internet”, both have same response as the video.

i tried every delay scenario and still have the same problem.

i also tried to debug with wire. but as the error occur when no wifi/internet the debug will stop responding

i want to avoid my controller from freezing when error while offline

i recently try to use imp004 breakout board with the same code. and it behave the same as imp001 explorer board. so i thought it was the pcb design problem with my custom imp004. are there any pins i shouldnt’ mess with so this problem not ocurring?

image

imp 004 breakout n custom.zip (3.3 MB)
i cut off the wifi/internet on 10s

As I said, if you “cut off the internet” (presuming this is unplugging your wifi router’s upstream network connection), you’ll just be hitting the blocking timeout you set in your code.

THIS BLOCKING TIMEOUT WILL AFFECT YOUR SERIAL LOGGING. This is because it’s blocking, as documented. If you don’t want it to block for so long, reduce the 60.0 to something much shorter?

I’ve always had a wrapper around server.log() that checks for a connection before execution. If I’ve set up a local debug port, I will also send the output to that stream, so I can see what’s happening while offline

yeah already tried this one but no difference,

the imp004 breakout board work fine even if i set 60 or 0.1, but my custom pcb will definitely freeze.

even if i put “if(server.isconnected()){” before server.log or using “message manager” the freeze still happen. the offline debug port also stop responding

I would guess that if you’re now only seeing this on your own board, then there’s a problem with your hardware design. Did it get reviewed? Can you post the schematic here?

Unexplained issues - once software has been ruled out - are almost always power related.

yes we got the hardware reviewed, and my head developer submit the ticket but no solution until now.

we have 2 imp product
and 1 of them behave the same as dev board, the other was the one i showed above.

so i’m sure it’s hardware design flaw, still not sure which one cause it.