Twitter Library Out of memory errors

Libraries
1

I’m getting ‘Out of memory’ errors using the Twitter stream but only with some (popular ?) search terms.See code below.

Searching for the word ‘Old’ is fine but if I change the search string to"new" i get ‘too many timers’ and ‘out of memory’ errors.
My error routine is never called .
I’m thinking the Twitter library can’t cope with the number of matching tweets returned for popular words ?

Any help appreciated

Blockquote

#require “Twitter.agent.lib.nut:2.0.0”
search_string<-“old”;

local twitter = Twitter(consumerKey, consumerSecret, accessToken,accessSecret ,true);
count<-0;
old_count<-0;

function onTweet(tweetData)
{

   count++;
   if (count>old_count+20) 
    {
        server.log (count);
        old_count=count;
    }

}

function onError(errors)
{

server.log("unhandled error");
// Close the stream then re-open it
twitter.closeStream();

twitter.stream(search_string, onTweet,onError);
}

twitter.stream(search_string, onTweet,onError);

Blockquote

2

I’m replying to my own post , I know (I’m not sure how to post a follow up )

I’m pretty confident its the Twitter library being swamped trying to store all the matching tweets. ‘2018’ as a search term crashes straight away, ‘1998’ is fine.
I’m not sure how feasible it would be to limit processing of matching tweets in the library when they exceed a certain rate? At the very least it would be nice if the code failed nicely rather than with an ‘out of memory’ error and subsequent restart.

3

I think the problem may just be that twitter is very capable of sending A LOT of data, more than the 1MB allocation for an agent, in a single chunk.

Is there anything in the twitter API where you can specify the max number of items to return?

4

For the moment I’ll just have to be more selective in my search terms.

I’m out of my depth with the intricacies of the Streaming API :frowning: I’ve not seen any reference to limiting the number of tweets returned, There are references to 'chunks and delimiters so maybe there’s a way of reducing the size of the data packet .

It’s only a trivial project ( a piece of artwork) so I can live with a few restrictions on choice of filter terms :smile:
I’ll delve into Twitter in the mean time