Out of memory problem when reading from UART

Squirrel Code
1

I am reading in data, to a blob, and I get ‘imp restarted, out of memory’.
I am reading in about 35 blocks of 256 bytes and get to about block 24 when it crashes.
The two functions are shown below.
while (lenya < len1)
{
writeAck();
pos2++;
if(pos2 == 0){pos1++};// address to read from
lenya++;
readblock(); // read in block of data

}

function readblock()
{

block.flush();
block.seek(0,'b');
local aa2 = imp.getmemoryfree();
server.log("Device aa2: free memory: "+aa2);

readincount = 0;
while (readincount < 1000)
{
    readincount++;
    local byte = hardware.uart57.read();    // read the UART buffer 
    if (byte != -1)  // otherwise, we keep reading until there is no data to be read.
    {
        readincount = 0;
        block.writen(byte,'b')

        if (block.tell() == 256) 
        {
            readincount = 1001;
            server.log("Block done");
        }           
    }
}

}

As I flush the blocks every time I go through the routine surely memory usage should be the same.
The imp.getmemoryfree() shows that the memory is being used up.
Can any one help.

2

Does it help if you do block.resize(256) before starting to read? (If you try that, note that resize() on a blob works more like a “reserve” than a “resize” operation: it doesn’t change len() unless the new size is smaller than len().)

Peter

3

Tried it but it makes no difference.

4

This is likely due to fragmentation of the heap; there are things coming that will make this better-behaved. Until then, I’d suggest you allocate a single large blob (of 35*256 bytes) and read into offsets within it.

5

Thanks I’ll try that.
When I ran the program I noticed that on each loop of writing the block the imp memory was being reduced by about 700 bytes until it crashed. Should that happen?

6

Even when you preallocated with resize, as peter suggested? If you don’t preallocate, then every time it wants to grow it’ll grow another power of 2. It’s possible it decided to grow to 512 bytes, though even then the overhead seems large to get to 700.

As we can’t see all the code, we can’t tell what you’re doing with the blobs after you’ve read them in - I presume you’re keeping a reference to them somewhere, which explains why they’re not being freed by squirrel?

7

Thanks Hugo
Tried what you suggested but still runs out of memory.

Below is the new routine. I’ve set the blob size to 8192 but still runs out of memory.
The routine is reading in blocks from a camera, the camera protocol sends 256 byte blocks and then waits for an ACK. At the moment I am just trying to read in the data and throwing it away, but when successful I will send the blocks via agent to an android app.
I’ve attached the log which shows memory usage.
Should what I am trying to do be possible?

function readblock()
{

if (blockcount == 0)
{
    block.flush();
    block.seek(0,'b');
    block.resize(8192);
}
block.seek(blockcount * 256);
blockcount++;
if (blockcount == 32)
{
    server.log("Block count zeroed")
    blockcount = 0;
}
local aa2 = imp.getmemoryfree();

server.log("Device aa2: free memory: "+aa2);

readincount = 0;
while (readincount < 1000)
{
    readincount++;
    local byte = hardware.uart57.read();    // read the UART buffer 
    if (byte != -1)  // otherwise, we keep reading until there is no data to be read.
    {
        readincount = 0;
        block.writen(byte,'b')

        if (block.tell() == 256) 
        {
            readincount = 1001;
            server.log("Block done");
        }           
    }
}

}

8

You need to change the block.tell() line, otherwise it’ll just keep reading into the block without bounds. The code below (replacing uart57 with a random number generator) runs on release-25 with agreeably static memory usage.
`blockcount <- 0;
block <- blob();
readincount <- 0;

function readblock()
{
if (blockcount == 0)
{
block.flush();
block.seek(0,‘b’);
block.resize(8192);
}
block.seek(blockcount * 256);
blockcount++;
local blockend = blockcount * 256;
if (blockcount == 32)
{
server.log(“Block count zeroed”)
blockcount = 0;
}
local aa2 = imp.getmemoryfree();

server.log("Device aa2: free memory: "+aa2);

readincount = 0;
while (readincount < 1000)
{
    readincount++;
    local byte = math.rand() % 256; //hardware.uart57.read(); // read the UART buffer
    if (byte != -1) // otherwise, we keep reading until there is no data to be read.
    {
        readincount = 0;
        block.writen(byte,'b')

        if (block.tell() == blockend)
        {
            readincount = 1001;
            server.log("Block done");
        }
    }
}

imp.wakeup(1.0, readblock);
}

readblock();`

Peter