Code size - what influences it

Hi,

I’m really strapped for code space in the device. Currently running at 99%-100%…
I know some changes are coming in a future release that may help, but I’ve just hit the 100% on code that needs to go in production now (and actually not everything I need is in there)
=> I’d like to understand what infuences the space code takes so I can trim it where it makes sense.

  • pure text size of the code => can I save by using shorter var names ?
  • purging all server.log (which will make me run blind) is an obvious one, but does it make a big difference ?
  • use of enums vs. strings for keys ?

Strangely enough, I’ve noticed that it happens that I remove a piece of code (just delete it) but that by doing that the % used code goes up … strange…
Thx,

Here’s a good initial checklist: http://electricimp.com/docs/resources/efficientsquirrel/

thx.
Free memory size isn’t the big problem. I’ve got > 30K free. It’s the program storage which is the problem. I understood that most of the hints in the checklist have to do with memory usage, right ?

Code space has been an ongoing challenge for me too. Strings appear to be quite costly. In the device, I’ve taken to replacing strings with integer codes that are used to index a big look-up table of strings in the agent. That has definitely saved me space. I’ve implemented a lot of code-reuse in my device code (using classes and delegates). It also been helpful to have a hard think about what stuff must absolutely be executed at the device level, and what can be deferred until sent up to the agent.

Strings and variable names do take space, so shorter ones (ie minimizing the code) will help. You can try running your code through a js minimizer, but it’ll likely require fixup afterwards.

We’ve got a minor compiler change which should go out in the next maintenance period (which is tentatively scheduled for 10/22). This could get you another 5%, but the huge win (roughly halving code size) is in release 31.

The halving of code size sounds very promising. I desperately need extra room in the device. Is it some sort of Minification? I’m slightly concerned that I might have trouble referencing some objects. My agent and device share (sort of) the same namespace. I often pass strings from the agent to the device that are the names of the methods I want to invoke.

No, the halving of size is without minification; there are various changes (shared string pools, new bytecode ops, etc). Minification would get you even more savings :slight_smile:

@coverdrive. I was thinking about replacing strings with lookup codes as well, as most of them are for logging so can be done from agent as well. If you have good experience with that, I’ll give it a go.
Invoking device methods from the agent by passing strings is an interesting concept. How do you do that ? By using the ‘compilestring’ squirrel function? (I was thinking about trying that out but never got to it and wasn’t sure that’s actually supported on the imp). I’m using a shared event & messaging system that at least makes notifications of what happens transparent to device and agent…

‘compilestring()’ is sadly unavailable, but it is possible to invoke device methods from the agent (or vice versa) by passing the function details as data.

For example, from the agent I dispatch a request to read i2c as
dispatch("i2c.writestring","i2c",{ addr=0x001C, data="00112233" }, callback)
where the arguments are (funcPath, contextPath, parameters, callbackfunc);

The dispatcher in the Agent packages all this up and sends it to the Device, except for the callback, which it holds until a response is returned. The callback will then be called with the response from the Device.

In the Device, the request is received and the funcPath and contextPath are parsed and decoded into objects, ie “i2c.writestring” becomes i2c.writestring

If the function expects a single argument which is a table of parameters, I then invoke the function by using:
result = func.bindenv(context)(parameters)
or if the function had a normal list of arguments, e.g i2c.writestring(addr,data), it would do the following:
`
In Agent
dispatch(“i2c.writestring”,“i2c”,[0x001C,“00112233”],callback)

In Device
if (typeof(parameters)==“array”) {
parameters.insert(0,context);
result = func.acall(parameters);
}
`

It both cases “result” would be passed back to the callback function in the Agent.

Thx !
I assume the parsing and decoding from the string to the actual function is done through a lookup table (ie the functions you’re calling need to be ‘prepped’ for it (not a big deal).
I’ve been doing this with the Bullwinkle class but that requires specific sending and reception code that is more elaborate then what you’ve described. E.g. to do EEPROM backups from agent.

One other question. I’ve been trying a few times (in vain) to make this type of calls (or agent device requests) that are by nature asynchronous, behave in a more synchronous way as seen by the caller . i.e. the calling function makes a call like result = Dispatch(…); and doesn’t need to know the complex set of actions and callbacks needed to get that result. Off course without locking up the execution thread completely while it waits for the result:-) A bit like the sendsync commands on http.
I’ve been experimenting with generators to achieve this, but didn’t get it working so far. Has this been tried before ?

Decoding the string is pretty straightforward. My explanation above probably skipped over some vital steps. Suffice it to say, it is effective as I’m using it regularly.

I am mulling over releasing the code to do all of this as an alternative to Rocky and Bullwinkle. R&B and my approach have advantages in different areas. The lack of the ability to import reference libraries in the IDE means that improvements to what I’ve written (doubtless there are plenty to be found) would get lost if the code weren’t kept separate.

Anyway, here is the code for decoding…

// given an array of strings, combines them together in an object function resolve(pathArray,context=this){ foreach (element in pathArray) if (element in context) context = context[element]; else return null; return context; }

If you have a string myStr (eg “hardware.pin1.configure”), just call resolve(split(myStr,"."),myContext) and test for null afterwards.

Yes, it is possible to make asynchronous calls synchronous. I’m doing this with generators in the agent but not in the device. I don’t think the way I’ve done it is applicable for general use, though. If you can, I think it’s much easier to accept the asynchronous nature of the environment and use callbacks wherever required.

Libraries are coming… though the form will vary.

@coverdriven., Thx. The solution for resolve is beautifull in its simplicity.

Regarding building with libraries, i’ve also been mulling over releasing my solution for that. I’m doing all code writing in VS (with squirrel syntax highlighting) with files nicely organised in a project under git control. Project is defined in a separate .pnut file that is using #include statement to define a combination of files that make up the agent and device code. Building code files is done with a purpose build C# command line tool that is called as an external tool in VS and which sends its feedback to the VS output window. The only thing I do in the ide is copying the code to it, check the syntax and build/run.
Current functionality includes :

  • building the squirrel agent and device code files from separate library & application portion files
  • using arbitrary library/class definition file location folders apart from the application folders
  • auto generation of a header file & startup log message
    -creation of a build nr tracking file and inclusion of an autoincrementing const in the output code with every build
  • detection and replacement of all server.log statements by a statement of choice in the output

If there’s interest i’m willing to post the building tool C# code on github for anyone to fork + some VS configuration guide to make it all work. Just don’t ask me for custom changes to the tool or for elaborate documentation as i won’t have the time for it :slight_smile: it’s pretty simple C# so anyone with a basic C# experience can make changes. I’m not sure it the express versions of VS support external tools but this can as well be run from simple batch files instead.

@vedecoid That sounds very promising. I’m not a VS nor C# user myself. But, it sounds like you’ve addressed many of the development environment gaps that Electric Imp are aware of (and are working on, I believe).
I’m actually ok with the IDE in the browser, I just don’t need to see/edit my standard libraries, so they should simply be represented by a #include “dispatcher.nut” etc.

That’s what my Mac OS X-based compiler tool does. You add

#import <library_filepath>

to your device and/or agent code and it drops the appropriate file in for you. A button copies the ‘compiled’ code to the pasteboard so it can be dropped into the IDE. When we launch the IDE API, you’ll be able to upload the code straight to the server (this works now, but uses an non-public API).

It saves .squirrelproj files which package up the locations of all the .nut files that are part of a model/project.

You can download a recent version here.

As for the original question: I’ve updated the “Writing Efficient Squirrel” page with notes on which techniques help with code size (as opposed to performance or memory usage) and added a new optimisation section.

Peter

Strings and variable names do take space, so shorter ones (ie minimizing the code) will help. You can try running your code through a js minimizer, but it'll likely require fixup afterwards.

We’ve got a minor compiler change which should go out in the next maintenance period (which is tentatively scheduled for 10/22). This could get you another 5%, but the huge win (roughly halving code size) is in release 31.

how about the schedule now? i find a js and css minifier for a long time.