"cannot resize stack while in a metamethod"


#1

I have a class that has an integer-based “get_” metamethod (see below). In my device Squirrel code I have an object of this class which invokes the metamethod by calling obj[10]. I get the error “stack overflow, cannot resize stack while in a metamethod”. If I “inline” and access the object directly without invoking the metamethod, then everything works OK, so I know I’m not really running out of stack space.

I saw a few forum messages about this error from 4 years ago. Is it still a bug in Squirrel? How to avoid “cannot resize stack while in a metamethod”?

function get(i)
{
return blob
[offset_ + i];
}


#2

This may be related to using blobs; unfortunately, Peter who is the real expert on this stuff, is on holiday for another week…

It’s not so much a bug as a limitation in the way metamethods work as I remember.


#3

I think that impOS cannot resize the squirrel stack while executing a metamethod (even though memory is available), because the squirrel stack would be moved and it’s non-trivial to update all of the references to it’s current memory address.

The code path goes something like this:

  1. impOS initialises the squirrel VM
  2. squirrel VM starts interpreting your squirrel code
  3. squirrel code references obj[10]
  4. squirrel VM finds metamethod and makes recursive call
  5. squirrel VM starts interpreting metamethod
  6. squirrel metamethod exhausts squirrel stack

At step 6 we cannot easily resize (i.e. move) the squirrel stack because earlier steps are still holding references to it.

You may be able to avoid the limitation by either reducing stack usage or by provoking a stack resize prior to referencing obj[10] (since the squirrel stack is only resized upwards).


#4

My project has a lot of code. I can’t go through all the code to manually manage the stack. Are there plans to fix Squirrel so that it can manage its own stack? If not, then I have no choice but to stop using metamethods.


#5

It may be possible to resize the stack at the very start; @philmy do you have any suggestions as how to make the stack much bigger then release the space?


#6

Maybe with a bit of recursion?

`function growStack(depth=0) {
  if (depth < 500) {
    growStack(depth+1);
  }
}

server.log(imp.getmemoryfree());
growStack();
server.log(imp.getmemoryfree());
`

2016-10-03 21:20:44 UTC+1 [Device] 82308
2016-10-03 21:20:44 UTC+1 [Device] 47764


#7

I just ran into this issue as well, notably while on an imp005. While the growStack method might be a quick fix during development, I don’t see how this would help during production, where paths through the code, and thus how the stack builds up, cannot always be foreseen.

From my perspective, this needs to be fixed in impOS. Based on philmy’s outline (1-6) above, I would expect impOS to check the stack size, before it enters the metamethod (4), and if the free part is below a threshold (which can even be static), the stack size is increased accordingly.

@philmy does this sound viable? Any other way possible?

@hugo when can we expect this? My impression is that this issue could happen any time in the field, without us even being able to track it. This would make it quite urgent to address.


#8

I don’t think that would help in the case when one metamethod calls another, or when metamethods themselves are greedy about stack usage.

You can catch the “cannot resize” exception in squirrel if you want to track this happening. We might also be able to provide some visibility of the stack watermark for tuning growStack() to your application.

Bear in mind though that the squirrel stack can only be increased, not decreased, and that this memory resource is shared with other squirrel allocations such as blobs. It’s more difficult to anticipate “out of memory” restarts, and therefore good to know the deepest code paths of your application.