HTTP limits and streaming

Agent Code
1

Currently the HTTP support in agents is limited to 64Kbyte fetches… this shouldn’t often be a problem in practice, because that’s more data than can sensibly be dealt with on the imp side anyway, but there is a workaround for the rare situations in which fetching a larger file is needed: the file can be streamed, by fetching it in sections using the HTTP “Range” header. Here’s some example code which streams the Electric Imp homepage in 1K lumps (the page is in fact well under 64K, so could have been fetched in one go, but it demonstrates the technique):
`
function fetch() {

offset <- 0;

const LUMP = 1024;

do {
response <- http.get(“http://electricimp.com”,
{ Range=format(“bytes=%u-%u”, offset, offset+LUMP) }
).sendsync();

server.log(response.statuscode)
foreach(k,v in response.headers) {
server.log(k+" = "+v);
}
got <- response.body.len()-1;
offset += got;
} while (response.statuscode == 206 && got == LUMP);

server.log(“Got “+offset+” bytes total”);
}
`

Peter

2

Some new life for this old forum thread:

Peter, is there anyway to get the length of the thing you’re trying to fetch without fetching the whole thing?

3

Pivotal tells me you figured this one out yourself, but for the benefit of the rest of the forum: use http.request() and make an HTTP/1.1 HEAD request.

Peter