Twitter Library Out of memory errors


I’m getting ‘Out of memory’ errors using the Twitter stream but only with some (popular ?) search terms.See code below.

Searching for the word ‘Old’ is fine but if I change the search string to"new" i get ‘too many timers’ and ‘out of memory’ errors.
My error routine is never called .
I’m thinking the Twitter library can’t cope with the number of matching tweets returned for popular words ?

Any help appreciated


#require "Twitter.agent.lib.nut:2.0.0"

local twitter = Twitter(consumerKey, consumerSecret, accessToken,accessSecret ,true);

function onTweet(tweetData)

   if (count>old_count+20) 
        server.log (count);


function onError(errors)

server.log("unhandled error");
// Close the stream then re-open it
twitter.closeStream();, onTweet,onError);
}, onTweet,onError);



I’m replying to my own post , I know (I’m not sure how to post a follow up )

I’m pretty confident its the Twitter library being swamped trying to store all the matching tweets. ‘2018’ as a search term crashes straight away, ‘1998’ is fine.
I’m not sure how feasible it would be to limit processing of matching tweets in the library when they exceed a certain rate? At the very least it would be nice if the code failed nicely rather than with an ‘out of memory’ error and subsequent restart.


I think the problem may just be that twitter is very capable of sending A LOT of data, more than the 1MB allocation for an agent, in a single chunk.

Is there anything in the twitter API where you can specify the max number of items to return?


For the moment I’ll just have to be more selective in my search terms.

I’m out of my depth with the intricacies of the Streaming API :frowning: I’ve not seen any reference to limiting the number of tweets returned, There are references to 'chunks and delimiters so maybe there’s a way of reducing the size of the data packet .

It’s only a trivial project ( a piece of artwork) so I can live with a few restrictions on choice of filter terms :smile:
I’ll delve into Twitter in the mean time