What's best way to determine size of table when using server.save(table) method

Squirrel Code
1

According to “server.save” documentation, the table to be saved must not exceed 64KB in size, otherwise an exception will be thrown.

What is the best way to check my table size as cannot see any suitable table object methods in documentation (http://electricimp.com/docs/squirrel/table/)

2

Is this on the device or in the agent? In the agent you can use http.jsonencode(table).length().

Peter

3

Whoops, that should be http.jsonencode(table).len().

Peter

4

Thanks Peter! Yes, it’s for the agent.

5

Can I validate something on this?

The following code:
local table = {“12345”:{“key1”:“val1”,“key2”:“val2”,“key3”:“val3”,“key4”:“val4”,“key5”:“val5”,“key6”:“val6”}};
server.log("size: " + http.jsonencode(table).len() + " table: " + http.jsonencode(table));

produces:
size: 111 table: { "12345": { "key1": "val1", "key3": "val3", "key2": "val2", "key5": "val5", "key4": "val4", "key6": "val6" } }

Note the spaces. Without spaces, it has a length of 95 so 16 used up in spaces.

Question: Is the data really being stored with spaces or is that just the way http.jsonencode() formats the string?

Reason: I’m using an imp005 (need ethernet). Need the agent to store data if it can’t upload to a db server (maybe network outage or other reason). The example table with spaces would allow 9.8hrs of downtime @ 1 record per minute. Without spaces, it would allow 11.5hrs.

6

The spaces are introduced by http.jsonencode and do not exist in the original data. You may well find, however, that even with the spaces, the jsonencoded form takes up less memory than the original, which includes all the Squirrel VM’s table overhead.

(But if, on the other hand, you have a lot of these tables and many of the keys and values are the same among all of them, the Squirrel representation pools the strings and stores each only once, where the JSON version must store the strings repeatedly.)

It’s also worth noting, if you have many very similar tables, that a table of arrays uses much less memory than an array of tables. So, as it were:

myreadings = {
   timestamps = [101, 102, 103], temperatures = [5.6, 6.7, 7.8]
}

is more space-efficient than:

myreadings = [
  { timestamp = 101, temperature = 5.6 },
  { timestamp = 102, temperature = 6.7 },
  { timestamp = 103, temperature = 7.8 }
];

especially as the number of readings gets pretty large.

(That Squirrel syntax possibly not exactly correct, but you get the idea.)

Peter

7

But if you’re planning on using server.save(), where the JSON is an implementation detail, you should (probably) prefer saving the squirrel object, rather than JSON-encoding it yourself. Otherwise you end up with a JSON string encoding a JSON string, which is going to be larger.

Also note that – at some point – we’ll be changing the server.save() format from JSON to something else that’ll have better fidelity with Squirrel types, and will remove some of the arbitrary-looking restrictions on what you can store.

8

Catch the exception?