@smittytone, For the hobbyist, I think that the keys to making the environment more approachable to new adoptees are:
More worked examples (best way to learn a language is to see it in action)
increased availability of library classes
I think the API documentation is already pretty good and is positively brilliant compared to what existed 18 months ago. Unfortunately, there isn’t an active user base of Squirrel coders on StackOverflow which is a excellent source of advice for learners and experts in mainstream languages. There are pearls of wisdom tucked into the existing forums, but we all know that they are difficult to find (using forum search or google), as they get lost in all the false positives.
For the commercial developer, another key is the inclusion of frameworks into Squirrel that do much of the heavy lifting for users. Rocky and Bullwinkle represent a start, but more could be done to offer transaction, presentation and persistence frameworks. This stuff takes times, though.
@smittyoine, I completely agree with @codedriven that examples are the key. If explaining the sleep function, simple examples to show the different sleep functions & some comments, so we can try it would be ideal.
The issue is not just Psychological, as when a newbie wants to learn a new language, there are so much out there we can reference to and examples to try, e.g try searching for Java, Python or C coding examples, you will find many, trying searching for squirrel is a different story. I applaud the Imp staff taking time and effort to listen to customers/users and try to address this to make a difference and is a key success factor for user loyalty.
ken
Thanks for your thoughts, folks. For more worked examples, I’d point to our imp API Code Cookbook, which has commented code for achieving specific tasks under impOS. Feel free to suggest recipes we can add here. For longer, more project-oriented code which you can examine even if you’re not building the project, we have Learning Projects.
I absolutely hear what you’re saying about more example code. As I say, let me know what kind of tasks you’d like to perform and we can see about adding code here, StackOverflow fashion, and integrating it into the Code Cookbook too.
Where could we make suggestions for additional HW APIs that do under the hood processing of real time data ? I myself would be very much helped by a simple frame detection mechanism on the UART. Something that stores incoming bytes in the FIFO and fires a callback if a pause of x msecs is observed. In C, on bare metal Cortex Mx, this wouldn’t take me more than a couple of hrs to write and fully test so …
We’re now doing that in squirrel but both the granularity of the timers + the overhead it takes makes that we never got beyond 57600 in a reliable way. Maybe the increased FIFO in R32 will allow to avoid lost frames due to Squirrel not able to keep up under network activity load, let’s see.
Like it is for @vedecoid, The current uart arrangement is not perfect for me either. Data frames are often punctuated by a necessary idle period. At the moment I’m using a millisecond timer that I reset after receiving each byte. This is not ideal as the callback handler is not an interrupt handler and could be called many ms after the byte was actually received. In other event-driven, “near” realtime projects, I’ve used in-band signalling to allow my uart code to understand that there’s a measureable gap between frames.
For example, when I configure a uart, I can tell it to insert a “delta” into the received data stream. The delta represents a fixed amount of time that the line has been idle, say 1ms. This can help me determine whether there is separation between frames if the event code was too busy to attend to the data at the time it was received. This can naturally chew up buffer space, but provides more confidence over timing.
for the imp’s API, one possible way of doing it would be to allow uart.configure to support a uart rx_idle threshold. If the line is idle for more than the specified amount of time, it would invoke the uart callback. You could then adjust uart.read() so that it returns not just -1 but another negative number that represents that the threshold was met. The squirrel code can then process what’s already been received before reading in the next frame.
Since the imp has a decent amount of RAM, I would also suggest that electric imp implement an event queue for digitial inputs. The imp is currently not ideal for pulse counting because it will miss pulses or pin transitions when it’s busy doing something else. An event queue can capture all of the transitions (with some sw-configured filters on really short flicker/bounce), allowing the squirrel code to catch up with activity on that pin when it’s done with servicing other matters. It’s possible do pull this off using the sampler, but it’s heavy work having to sift through a full buffer when nothing actually happened on the pin and you have to use a pin that has an ADC.
The UART callback does get told about LINE_IDLE events (one character’s worth of idle), but if further bytes then appear after that (i.e., a new data frame starts before your handler runs) then you can’t tell which bytes belonged to which frame.
@peter, the LINE_IDLE bit may be useful in some circumstances. I have scenarios where it’s of less use. If I have an imp as a MODBUS slave, it needs to discern between frames on a half-duplex bus. A frame is terminated by an idle period of a minimum width of 3.5 characters. If the handler fires promptly, this is fine, but there are occasions where the imp is simply too busy doing other stuff to be 100% certain of attending to the frame before the next one arrives.
Since we will shortly be able to set the buffer size of our UARTS :-), a very useful diagnostic for us would be to be able to query the longest time between moments where the device OS is idle. By having access to that value, we can set our buffer sizes, identify code that hogs the cpu and choose which stuff to run asynchronously. I’m already aware that imp.scanwifinetworks() runs on the same thread and blocks for up to 1s. Thus, it must be used sparingly if there are other services running that require speedy attention.
Native Modbus handling was supposed to be on the API roadmap, but that was a comment from @hugo almost 2 yrs ago Is it still on ? Or just not enough demand for it…
For now I don’t have a problem because all the devices we want on the bus have been designed by us so we implement a much bigger idle period which allows the imp to properly deal with it. Same frame detection method as @coverdriven. This won’t work anymore guaranteed when introducing commercial devices on the bus, as they may only adhere to the 3,5chars standard.
We do work with 3rd party modbus devices and some of them sadly don’t follow the modbus standard to the letter. This means that even a standard implementation (if electric imp were to implement one) would need to be overridden with special behaviour to deal with with these rogues. Rather than full native support, some of these applications can be satisfied by an expanded UART API and the use of libraries for various buses.
You’re right. On top, the Modbus standard is seldomly used beyond some basic functionality, so probably overkill to fully support it natively… I’d say I would settle for a simple UART storing incoming bytes in FIFO with a callback triggered by a specified idle time(or max received bytes) with a resolution of say 100us… The asynchronous packet processing is often simple for most connected devices and can be done efficiently in squirrel (as there aren’t that many messages anyway, i’m just sending the received packets as a blob to the agent for processing to offload the imp)
I’ve filed a feature request for this; my suggestion (which is still up for discussion) would be to enable a timing mode where each FIFO entry would either be a byte or an idle time if the bytes were not back to back; a maximum idle time would be able to be set that would queue the timing entry when this was reached.
You’d then be able to make your own decisions about inter-packet gaps as necessary.
my experience is that the byte by byte processing is what takes most time (even if only a single check is done) and is what limits the reachable speed. Although more flexible, I’m not sure your suggestion would fully solve it, but it would already be a step in the right direction (as the accuracy of the gap measurement would be much better - now it’s 10msec at best).
The suggestion sounds good to me. I haven’t had such a problem with byte by byte processing so far. But, most of my applications have been dealing with data rates from 2400 to 57600, nothing higher than that.
it’s indeed above 57600 that you start to lose a lot of frames so that’s the speed I’m running at.
Question on the proposed approach, wouldn’t that consume 2 bytes per received byte then ? How else are you going to be able to distinguish between normal binary bytes in the stream and such a time slot entry ?
The handler coughs up an integer. This is how -1 is not confused with 0xFF. The idle time character could also be something that can be distinguished from an 8 bit char.
As coverdriven says, yes it would be an integer. Almost certainly the readblob/readstring methods would just ignore the timing bytes and only pull in real data.
@vedecoid have you tried setting bigger serial buffers on release 32? Does this help you? The release 30 size of 80 bytes is rather restrictive for many devices.
I guess I’m too used to the restrictive nature of very statically typed language such as C. Hadn’t though about methods returning different types (while I do it all the time myself in Squirrel to be honest
I haven’t tried the bigger FIFOs yet as we’ve now settled on the 57200 which works. Is more for future projects. Will do so as soon as I get a chance but I think it will already make the necessary difference because that deeper FIFO will allow to survive the gaps in available processing time. so there’s no frames lost. Only those cases where the inter-frame gap detection mechanism doesn’t fire because it lack of idle time would be lost, but that will be very rare.
