Efficient hexadecimal string to binary blob function


#1

Currently using the following to take a string (like MAC address) and convert it to a 6 byte blob, I am unsure of the efficiency of compilestring() and wanted to see if anyone had anything better?

function stringToBlob(str) { local length = str.len()/2; local ret = blob(length); for (local idx = 0; idx < length; idx+=1) { local part = compilestring("return 0x"+str.slice(idx*2, (idx*2+2))+";")(); ret.writen(part, 'b'); } return ret; }

Thanks.


#2

Yeah, I was going to benchmark both methods but then didn’t :slight_smile:

compilestring is both a security issue and also a resource issue as I remember.


#3

Is it documented anywhere what is removed from squirrel? Would help for those of us programming and testing locally then pushing to production since there isn’t a great staging methodology for the imp.


#4

Well, compilestring() doesn’t exist on the imp or agent, for starters…

Assuming well formed input, this works well:

function stringToBlob(str) { local length = str.len()/2; local ret = blob(length); for (local idx = 0; idx < length; idx+=1) { local part = str.slice(idx*2, (idx*2+2)); ret.writen(((part[0]<='9'?(part[0]-48):(part[0]-87))<<4)+((part[1]<='9'?(part[1]-48):(part[1]-87))), 'b'); } return ret; }


#5

But we removed compilestring() as a security measure… didn’t we?

I’d be tempted to do something with “0123456789abcdef”.find(s.toupper()), though I agree it’s a shame that “0xAB”.tointeger() doesn’t work.

Peter


#6

http://devwiki.electricimp.com/doku.php?id=learningsquirrel

Peter


#7

Hugo, great function! however, there seems to be a small bug somewhere. See here:

Works:
Input string: 906000000000
binary: 90 60 00 00 00 00

Fails:
Input string: CA0000054141410F0E
binary: aa 00 00 05 41 41 41 ef ee
mind the aa, ef, ee

Fails:
Input string: 900A0000010000
binary: 90 ea 00 00 01 00 00
mind the ea

But again this one works:
Input string: 00A4040007D276000085010100
binary: 00 a4 04 00 07 d2 76 00 00 85 01 01 00


#8

As written, it only works correctly on “lower-case hex”; try adding a “str=str.tolower()”.

Peter


#9

The code that handles odd number of hex digits in the string would look like:

function hexStringToBlob(str) {
    if (str.len() % 2) {
        str = "0" + str;
    }
    local length = str.len() / 2;
    local ret = blob(length);
    for (local idx = 0; idx < length; idx += 1) { 
        local part = str.slice(idx * 2, (idx * 2 + 2)).tolower();
        ret.writen(
            (((part[0] <='9' ? (part[0] - '0') : (part[0] - 'a' + 10)) << 4) +
             ((part[1] <='9' ? (part[1] - '0') : (part[1] - 'a' + 10)))), 
        'b');
    }
    return ret;
}

#10

Here’s a version (eerily similar to ppetrosh) that I’ve been using for a long time.

function hexStrToBlob(hexStr){
    // pad string with leading 0 if not even in length
    if (hexStr.len() % 2 != 0 )
        hexStr = "0" + hexStr
        
    local length = hexStr.len()/2
    local result = blob(length)
    for (local i = 0; i < length; i+=1) { 
        local hi = hexStr[i*2] - '0'
        if (hi > 9 )
            hi = ((hi & 0x1f) - 7)
        local lo = hexStr[i*2+1] - '0'
        if (lo > 9 )
            lo = ((lo & 0x1f) - 7)
        result[i] = hi << 4 | lo
    }
    return result
}

From my experience, strings are much faster in Squirrel than blobs. The only exception is string.find, which seems to be unusually inefficient. If you are using blobs, you can using index references b[n] rather than b.writen() if you’ve allocated the appropriate size already.


#11

Hi im new to all of this, I tried the last function that @coverdriven posted, to write an array hex to a sector of spiflash and seems like wrote into the sector without problems, so is there another function to convert that “manual blob” that is wrote on the sector into an hex again?


#12

Something like:

function blobToHexString(b) {
    local s = "";
    for (local i = 0 ; i < b.len() ; i++) s += format("%02x", b[i]);
    
    // Comment out the following line if you don't want an '0x' prefix
    s = "0x" + s;

    return s;
}