UART problems using Dorji LoRa

Hardware
1

I’ve been going nuts trying to find out what’s going on. I’ve got some sensors sending data via a Dorji 1278DL LoRa radio. The sensors and the remote radio are controlled by a different processor and have been confirmed as sending good data. In fact, I have a second receiving radio connected to a PIC processor and that confirms the data being sent is good. I’m trying to implement an IMP003 to as my receiving module.

That radio on the IMP is picking up the transmission but it only gets 79 to 81 of the 90 bytes sent, AND the bytes are not the same as those sent (noise?). There appears to be no relationship between the value of the received bytes and the value of the transmitted bytes.

I’ve checked and double checked (many times over). The uart I’m using is uartFG
Radio <- hardware.uartFG
and is set to
Radio.configure(19200, 8, PARITY_NONE, 1, NO_CTSRTS, readSerial)

I’ve confirmed in several different ways that the radio itself is set to the same parameters. I even tried another radio, just in case I had a bad radio. Same results.

I’ve also tried increasing the receive buffer size, but that had no effect whatsoever.

My callback looks like this:

function readSerial() {
local d = Radio.readblob()

dataProbe.writeblob(d)

//show rcvd bytes
//server.log(d.len() + " bytes; dataProbe ptr @ " + dataProbe.tell())
/*
for (local i=0;i<d.len();i++) {
server.log(d[i])
}

*/
if (delay_timer) imp.cancelwakeup(delay_timer)

delay_timer = imp.wakeup(0.05,ProcessRecords)        //wait for more if none process

}

I’ve tried longer wakeup delays, but that makes no difference.

Can anyone suggest what I’m doing wrong or where I could look next?

2

80 bytes is suspiciously the size of the UART buffer, which implies that your code is blocking and you’re getting a buffer overflow - the logging could possibly do that (logging each byte individually is a lot of outbound network packets), but any imp.sleeps elsewhere could also cause that.

The simple fix is likely to be to increase your receive buffer. Add this before your Radio.configure() line:

Radio.setrxfifosize(500);

…to set a 500 byte FIFO size for that UART. See https://electricimp.com/docs/api/hardware/uart/setrxfifosize/ for more detail.

3

Thanks for the suggestion.

I’ve actually changed the logging to just showing me the contents of the data blob when it’s done being filled. So that bit of overhead was taken out of the picture … without an effect.

I have tried the receive buffer at 100 but not at 500. So I’ll give that a shot. And I have no other timers running.

I’m also dinking around with the uart baud rate (leaving the radio alone). I notice I get different “bytes received” when I vary that. Unfortunately, they’re still not the right bytes.

4

Nope. Buffer boost has no effect.

I did move to a different uart, uartDM. That’s where I’ve been experimenting with the different baud rates.

On DM at standard baud I seem to be consistently reading TOO MANY bytes on DM (110 or 111 instead of 90).

5

How often are the sensors sending data? Are the data terminated by a special character? If so, why not read each byte from Radio until you get that byte?

At the moment, your handler “readSerial” is queued for invocation as soon as the first byte is received on the UART. Depending on what your squirrel code is doing, it could be a few milliseconds before your handler is actually called. This means the number of bytes your call to uart.readblob returns will vary, depending on baud rate, and how busy your code is.

If you are going to use the optional callback, I recommend that you read single bytes from the uart until you get (-1) or the total number of bytes that you are expecting. There are lots of imp examples that show you how to do it.

6

I don’t quite get the comment about changing the baudrate without touching the radio - if you do this, then how is the radio going to send you data at the right speed? (or are you sending commands to the radio so that it knows your baudrate?)

It could be what @coverdriven says - ie that you are not getting the entire packet - but seems like the fragment you’ve attached will only trigger your data processing routine when there have been 50ms of silence on the RX line.

Can you attach the entire code?

7

My data burst has no terminating bytes and can vary in length (number of records sent) I can (and do) look for a constant byte pattern to determine a valid record and then I further run a crc check on that record before it gets processed. In my working application (without IMP) I send bursts of 10 to 12 45byte records and process them on the receiving end. In my IMP trial, I’m only sending 2 45byte records.

My callback only empties the FIFO and accumulates the bytes in a global blob that has a pre-allocated size. Only when there is a pause in the transmission do I do any processing. In this test, I’m doing nothing else and there is a two minute gap between transmissions.

My processing on the accumulated bytes will find valid records and process them for posting the data to Initial State. When the global blob has been processed, the pointer is set to zero so that it is ready for new data. In this test, the processing involves showing me the contents of the blob in the server log and, having found no valid records, resetting the pointer.

The baud rate experiment consists of setting the radio to 38400 and then varying the baud number in radio.configure(). I reload the code and see what happens at the new baud rate. I expected to see no communication unless the two rates matched. But I can set the IMP rate to almost any any value and still get bytes. Keeping the IMP withing a thousand or so seems to have no effect. ie. gives me the same (but incorrect) byte stream.

One question that occurred to me was if the IMP requires any circuitry on the uart pins (i.e. pullups) I’m using a direct connect at the moment.

8

The baud rates of your transmitter and receiver must match.

If they do not match, the receiver will end up with a mixture of (false) bytes and framing errors, as it misinterprets the data on the line.

Because the receiver and transmitter do not have a common clock source, they both need to agree on a rate at which to count off the data bits once a start bit is detected.

If the radio is set to 38400, make sure the imp is as well. As long as everything else is ok, you should get the data you expect.

9

As @coverdriven says, you will get junk if the rates don’t match. This is 100% expected and will happen with any UART - it’s not an imp thing. UARTs can deal with some mismatch in baudrates - async serial is intended to deal with mismatched clocks - note also that uart.configure returns the ACTUAL rate that was set, based on the clocks and divisors available within the silicon.

Once more, can you please attach your code? If you don’t want to, please PM it. Trying to debug a problem without the actual code is rather hard.

I very much doubt it’s an impOS problem; we have many many customers who rely on reliable UART operation (including @coverdriven) and some of them run at up to 1Mbit/s. 38400 is positively leisurely.

10

Responding to coverdriven, yes my radios are matched. The communicate with each other just fine. I’m working with several of them and have swapped them around to confirm. As long as I have them associated with my PIC processor, all is fine. Moved to the IMP, not so much. As Hugo suggests, it is probably something in my code that I’m overlooking. I’m new to the Squirrel version of C++, so that is very possible. I’ll upload the code for you to look at a little later. Also, being new to this forum, is there a preferred way to post code? (I noticed the gray boxes). Or do I just send it as a post?

11

I’ve attached my code to this post.

12

I’m not talking about the data rate between the modems, which sounds like it’s matched from your description. It’s essential that the external TX/RX on the radio module is matched to the imp. Using UARTs is generally pretty painless.

What output do you get when you try to run the code you’ve posted?

One way of checking if you’ve got the baud rate right is to looks at the uart flags for any errors.

When you initialise your port add this after Radio.configure…
Radio.flags() //to clear any initial flags
Then, when ProcessRecord() is called, add this code:

`local errors=Radio.flags()
if (errors&FRAME_ERROR)
  server.log("Framing Error detected")
if (errors&NOISE_ERROR)
  server.log("Noise Error detected")
if (errors&OVERRUN_ERROR)
  server.log("Buffer Overrun")
`
13

Is there a reason you commented out the rxfifo size setting?

14

I’ll check out the uart flags. My output is a string of bytes which does not match the number of bytes sent and the value of the bytes seems to have no relationship to the value of the bytes sent. i.e. I can not detect a pattern where I could say ‘this byte should be x and that byte should be y’. There seems to be not relationship between what was sent and what was received.

I commented out the buffer size because in earlier trials, that made no difference whatsoever. So I saw no reason to utilize the extra memory.

This is a frustrating problem because as coverdriven says, uart connections are usually trivial. I’ve gotten to like the capabilities of my IMP and worked it into more of my application. But without this key issue resolved, I won’t be able to use it. So I thank you guys for trying to help me out.

15

Okay. coverdriven’s suggestion tells me that there is a framing error. What does one do with that?

16

I tried fooling around with baud rates, parity, and stop bits, all to no avail. One thought I had was if RS232 vs RS422 was important. I see no mention of it in the docs. The radios are RS232 format.

17

Ahhhhh… ok I’ve worked it out. Should have googled the exact model number you posted originally, but your RS232/RS422 comment made me thing.

This is your module, right? http://www.dorji.com/products-detail.php?ProId=35

The problem is that it’s RS232. RS232 is NOT 3.3v “TTL” style serial, and in fact uses generally +/-12v levels, so connecting it to an imp directly may have destroyed the imp pins.

If it’s not blown up (you may be lucky as you’re still receiving gibberish) the main thing is that voltage levels are inverted on RS232 vs TTL serial. This means that depending on the data, you’ll receive junk of vaguely the right length as the imp will interpret an idle line as a start bit.

On TTL, an idle line is represented by logic 1, 3.3v. On RS232 it’s represented by -12v.
On TTL, a busy line is reporesneted by logic 0, 0v. On RS232 it’s represented by +12v.

One of these converters will get everything to the right level: https://www.sparkfun.com/products/11189?gclid=CP7pk-eFndECFc5bfgodtVwAXw

…or alternatively, you could just use the DRF1278F, which is almost certainly what is actually inside the 1278DL you’re using, but without the converter boosting the 3.3v levels to +/-12V.

http://www.dorji.com/products-detail.php?ProId=14

18

I certainly hope I’ve not blown the uart pins. The radio works on 3.3v so I’m hoping that the output voltage has not been boosted. But I’m glad to hear that we’re closhser to a solution. Buying new radios is not an option right now. I’ve got 30 and that runs the project cost up a bit. But I will look into a converter chip.

Just a suggestion. The RS422 requirement should be documented somewhere in the uart section.

Thanks Hugo. I’ll let you know how it goes once I get the converter.

19

The supply voltage to the radio is unrelated to the I/O - things like the MAX3232 have internal charge pumps. What is the other end hooked to? If it’s hooked to (eg) a 5v arduino with serial inverted (the ATMEGA can do this) then it’s still likely to work, but again will be slowly frying the attached CPU by putting DC current through the ESD diodes in the CPU.

The UARTs are absolutely only logic level. They are not RS422 either - RS422 is differential and can go up to +/- 25v. The imp is no different to any other low voltage UART - every arduino, raspberry pi and every CPU from every vendor (AMD to Zilog with Atmel, Intel, Microchip and ST inbetween) don’t talk RS232, RS422, or RS485 from the chip. Doing negative voltages requires different silicon design, and it’s hard to do on the same die as logic or memory.

MAX3232 boards are cheap and easily available. Search for “RS232 to TTL 3v” on eBay and Amazon and you get options (not shipping subsidised) at around $3.

We’ll see if we can add a note about UART not being RS232/etc somewhere in the intro section.

20

I can order the radio with a RS485 interface. Will that be compatible? Or does that still need conversion?

BTW: My PIC processor handles the RS232 signal directly, so I was surprised to run into this.

I can work with the MAX3232 boards, but will need to implement something on my own boards. If I need this kind of conversion, I’ll need to design it in.