Using server.log for code tracing is so far the best/only way to get a feel for the program flow in squirrel (haven’t tried some of the simulation/emulation attempts yet as I fear it will take too much time to get the behaviour of the ImpAPi integrated).
To enrich those logs, it would be very helpfull if we could print the current call stack position as part of the message.
As the generic exception handler is able to print it, is it also possible to use it in non-exception cases (ie expose it to the Api ?)
2nd question : some sort of memory dump (which object is taking up what part of the RAM memory) would be very helpfull too in the debugging of the ‘imp restart : out of memory memory’ situations. In first instance as part of the crash/restart dump but even better as a new Imp.getmemoryallocation() API function.
I have a custom-built tool which helps a lot with monitoring RAM. I’ve been mulling releasing it as it’s not perfect, and a bit more input from electric imp and other developers could make it even tighter.
To anyone that’s interested, here is a link to some source code I use to review RAM use in both the agent and device. I’ve removed all reference to my http handling which would need to be customised to your particular case. Implementing it is quite simple.
If you create an endpoint such as https://{agent}/getSize?limit=200 and then make sure that the endpoint calls getSize, it should work. The returned tree structure can be very large indeed. Use the limit value to prune the tree to only keep object>=200 bytes in size. You can try it with limit=0, but it may crash the agent/device. If you are using a browser, use a JSON
formatter like JSONview.
If your entire memory is too large to traverse, you could try reviewing only a part of it by calling getSize(200,“myData.subtable.element”) if “myData” is a valid object at the root level.
I’d be interested in any feedback, fixes. The calculations it uses for non-scalar values are not perfect. Arrays, tables and classes all pre-allocate slots (presumably for efficiency), which affects precision. String-keys are preallocated at compile time and are only stored once. Thus multiple instances of “application/json” all correspond to the same string reference.
The RAM usage of identical agent and device code will be somewhat different. Devices have 32-bit ARM CPUs, but agents run on our 64-bit Intel cloud servers: this makes a different to pointer sizes etc. in the Squirrel data structures.
For devices only, if the string “application/json” appears in the source code, then the actual character data will be referenced from flash (“ROM”) and not appear in RAM at all. Only strings which are constructed from other strings, or which are read from a peripheral or an agent, will have the character data taking up RAM. Unfortunately you can’t really tell ROM strings from RAM strings at the Squirrel level. And even ROM strings still have a control structure in RAM.
Generators and functions (closures) can have inner data inside them that isn’t reachable by that style of tree-walking.
Thanks Peter, I’ll review the stats for the device and adjust accordingly. All of my testing was on the agent. I know it will never be precise.
For instance, an array like this: t <- []
uses more ram than this: t <- [1,1,1]
My primary use was to analyse container structures and to review the merits of instances vs tables. Transient use of RAM by generators and functions wasn’t so much my concern.
As more people use squirrel, I think it’s helpful to have tools that provide guidance on how best to use RAM.
When I started writing code on the agents, I thought 1MB was plenty. It’s amazing how arrays and tables gobble it up.
Have you ever considered a feature that auto-compresses containers that are serialisable (ie they don’t contain functions, generators or self references)? The OS could unpack them whenever internally referenced (the Squirrel equivalent of DriveSpace). There’d be a tradeoff in performance versus RAM.