Receiving serial data with no unique start/stop frame characters

Device Code
1

I’m working on a project where several imps are communicating with commercial A/V equipment; projectors, amplifiers, displays, etc. The imps communicate with the equipment over RS232.

Some manufacturers use protocols that have unique start frame and stop frame delimiting characters, and I’ve had no problem writing uart callbacks for these. ie. if the incoming byte is STX, clear my incoming packet array, append subsequent incoming bytes until I get an ETX byte, packet is complete, parse it…

The problem is some manufacturers use much less friendly framing, and I’m having trouble figuring out how to approach these in terms of the incoming serial receive callback.

For example, one manufacturer, for all of the serial packets the equipment transmits to the imp, uses a frame format that:

  • has no unique start character across all commands
  • has no unique stop character across all commands (last byte is a checksum)
  • the first incoming character can appear elsewhere in the frame
  • NAK messages are framed differently than ACK messages (ie. start character, length, etc.)

Here’s a concrete example. For a particular line of NEC projectors, a power on command:

Sent from Imp to projector:
02H 00H FFH F0H 00H CHECKSUM

ACK message received:
22H 00H 01H 01H 00H CHECKSUM

or NAK message received:
A2H 00H 01H 01H 02H DATA01 DATA02 CHECKSUM

where DATA01 could be 00H to 04H and DATA02 could be 00H to 0CH. Last byte is always a checksum byte.

So, how do I detect that a received frame is complete and ready to parse, and how do I detect the start of the frame so I can clear my packet buffer for the new frame?

Would appreciate any suggestions or reading material.

Cheers,
Ken C.

2

Instead of looking for particular characters, set up a timer using imp.wakeup when you transmit your request. The duration of the timer should allow ample time for the response to arrive (perhaps a few hundred ms). While the timer is running, simply capture all the received bytes with your handler and store them in a blob. Don’t try to interpret any structure to it at that stage. When the timer expires, look at the entire buffer you have collected. The first thing to check is whether you have any data at all. If there is no data, you’ll need to respond accordingly. If there is data, you should attempt to validate the checksum immediately. If and only if the check sum is valid can you look at the rest of the packet to determine what you received. Protocols that have no specific start or terminating character are quite common.

3

Slightly depends from received - termination character exist or not.
As I have used - read until ‘\n’ and then store data or execute other function. If there are no termination character (I have seen such kind of devices) - it is in a bit more complicated.
If the the length of received is the same size - one solution- count bytes, if not - as you say: checksum always is last byte - calculate received checksum and compare - the main idea why checksum are used at all. (But you must know the checksum calculation algorithm).

4

Modbus-RTU works somewhat how you describe – there’s no frame delimiter, you have to look out for a certain size gap in transmission to detect a frame boundary. The imp UART’s TIMING_ENABLED flag was added to help deal with this sort of regrettable protocol design, see https://developer.electricimp.com/api/hardware/uart/configure and https://developer.electricimp.com/api/hardware/uart/read

Peter