SPIFlash Libraries

Hey everyone!

I’m really pleased to announce a collection of new and useful libraries for managing storage on SPI Flash.

  • SPIFlash - This class mimics the built in hardware.spiflash interface and is designed to be used on the imp001 and imp002 (i.e. modules that don’t have built in / accessible SPI Flash)
  • SPIFlashLogger - The SPIFlashLogger library lets you really easily push data into the SPIFlash, and later retrieve it. It makes use of our Serializer library, which can automagically convert objects to blobs :slight_smile:
  • SPIFlashFileSystem - An actual filesystem for your SPI Flash. This library allows you to create, read, and delete files in SPIFlash. It’s not at memory efficient as it could be - but we spent a LONG time balancing speed, and memory efficiency, and we’re pretty happy with what we ended up with. Whether you’re storing config files, audio data, or something else - this is a pretty powerful and handy class to have around :slight_smile:

If you have any questions about using any of these classes let us know and we’ll be happy to help :slight_smile:

Edit - fixed links

Is this class for any SPI flash in general? Or a specific chip-set?

It’s general 4k erase page size SPI flash. Most chips should work, the ones in the imp003 datasheet definitely will.

Nice job. Is access to the physical EEPROM semaphore guarded under the hood (or, are read/write requests buffered when a previous operation isn’t finished yet), or is this something that needs to be handled in the application code ?

Both hardware.spiflash and the SPIFlash class (which was designed to operate in the same way was hardware.spiflash) can only perform synchronous operations (i.e. your code will block while interacting with the SPIFlash).

question : if I read the docs, then it seems that (for the time being) the only way to update an existing file with new content (replace or append) is to first read it, modify it , delete it and then write it to a new file with the same name. (doc says : “If you attempt to open an existing file with mode = “w”, a SPIFlashFileSystem.ERR_FILE_EXISTS error will be thrown.”) . I understand that the “a” file opening mode is under development, will this allow easy updates of existing files ?

second question : what would be the most optimal way of storing the content of a tables in a file (eg a backup of config table) :

  1. encode/decode them as json strings with http.jsonencode/decode (can only be done in the agent) and store the string
    2.use the serializer class (which can run also inthe device) and store the resulting blob

now playing with it intensively. I’m using a AT25SF081 8Mbit SPI flash. I’m sure it works fine as it is completely equivalent to some of the more popular types. Also .chipid() gives the expected result, but the .size() method return a value of 4194304 bytes or 4MB, which is about 4 times the amount I would expect from an 8Mbit device…

That library doesn’t work out the size by itself (there’s no standard command to read it, and the nefarious means employed by real imp003s aren’t applicable). As a user of the library, you tell it the size in the constructor – or else it defaults to 64*64kbytes, or 4Mbytes.

Peter

I already wondered how you guys did the detection :slight_smile:

@vedecoid - As you mentioned, the only way to (currently) append to a file is to open, read, delete, then create a new file. Appending is in out TODO, but we wanted to get something out quickly :slight_smile:

I would recommend using the serializer class for storing table table.

Also - if you’re looking for something simpler than a file system (for example, if you essentially want a single file that you append data to) you my want to take a look at the SPIFlashLogger which automagically encoded table data with the Serializer class, and allows you to easily append data to the end of the list.

I’m actually using both. File system for some few config files and the bulk of the EEPROM to log sampled data. Well done to allow partial usage of the EEPROM for these purposes. We have our own pretty robust Flash driver for quite some time, but never got to implementing this higher level layers, so you saved us a bunch of time :slight_smile: If you could add some asynchronicity to the TODO, that would be even better. For now I’ll try to add our own queueing + callback mechanism to agent read/write requests to avoid having to care about the availability of the HW from an agent point of view. Doesn’t matter for the device as the operations are blocking squirrel execution anyway.

@vedecoid In fact it’s not that nefarious. We only care about flash chips up to 32Mbytes, so we write some data at the top of the 32Mbyte region and then check how many times it’s aliased lower down. Which is slightly cheating, because it assumes that the flash just ignores higher-order address bits that it doesn’t have, and the datasheets don’t in general guarantee that – though it’s how every single actual flash chip we’ve ever seen works. In particular, it’s how all the flash chips recommended in the imp003 datasheet work :slight_smile:

Also, of course, you don’t get to use that method if the chip might already contain data that you care about.

Peter