My new issue is that when I’m sending a binary string containing raw bytes. It is wrapped in JSON from a HTML web page, and when the imp005 agent receives and decodes the JSON, it appears a 0xC2 character gets inserted. My javascript contains code like this (inspired by examples in the imp webpage):
function sendRawData(settings){
$.ajax({
async: true,
type: "POST",
url : agentURL + '/raw',
data: JSON.stringify(settings),
dataType: "json",
success : function(response) {
//if ('locale' in response) {
//$('.locale-status span').text(response.locale);
//}
alert(response.d);
}
});
}
My agent code looks like this on the other hand:
api.post(“/raw”, function(context) {
try {
local data = http.jsondecode(context.req.rawbody);
local len, idx;
if ("bytes" in data) {
//server.log("data received is: " + data);
foreach(key, value in data) {
if (key == "bytes") continue;
server.log(key + ": " + data[key]);
device.send(key, data[key]);
}
device.send("raw", data.bytes);
context.send(200, "Okay");
} else {
context.send(500, "Request JSON error receiving raw bytes");
}
} catch (err) {
context.send(500, "Bad data posted: " + err);
}
});
Now, I found out it is related to UTF-8 encoding (in JSON?). I’ve observed 0xC2 is inserted whenever I have bytes whose value > 0x7f. Not sure where it gets inserted, whether from web page side or at the agent side. Not sure how can I remove or prevent this from happening. I checked some similar problems here in the forum and I’m reading this:
https://discourse.electricimp.com/t/receiving-a-json-data/6138
As someone coming from firmware background, I’m quite new to this high-level web software abstractions, as well on electric imp. I’m not sure how to do the Base64 encoding on both sides (web and agent?). That is if I want to completely eliminate this issue of 0xC2 insertions. Another thing I can think of is to traverse the string for its individual bytes, check for 0xC2 and check if its succeeding byte is 0x80 or greater, then remove the 0xC2. But I think this latter solution is not that clean?
So how to keep this 0xC2 problem from happening? I certainly can’t limit the bytes I’m sending to 0x7f and below since it came from application layer itself and have no control to what raw bytes they send to me (HTML page).