Electric Imp and SD card storage/ retrieval

Hello, has anyone had experience with SD card storage using the electric imp? From my understanding the SPI interface and a digital pin (SS) can be used for the communication btw the imp and the SD. Does anyone have any tips/ examples? Thanks!

This uses the SD card in MMC mode; if you google the MMC card spec you’ll find more useful info.

We have got a SPI flash filesystem library coming up very soon, which helps if you need SPI flash amounts of storage (vs huge SD storage)

We just published our SPIFlashFileSystem library - more info in this post :slight_smile:

Cool! Looks awesome guys. Back on the SD note, my logic behind that was for products/applications where a device may be damaged and need to be replaced. As far as I understand, a new device cannot be associated with a previous agent. So if I was storing “config” data for a particular “installation” on the device/agent, then if the unit was replaced the entire unit would need to be reconfigured. Same with if the data is stored on a flash system.

I guess a better question to ask is, is it possible (or even right) to save imp related configs on an SD? Meaning the data from blinkup, wi-fi, Plan ID, etc (encrypted of course) can be stored on the SD and just transferred to another unit (swapping SD) to allow a “seamless” replacement? I know I could host the installation config data on some webpage and have the unit look for it automatically and download after a re-blinkup, but that’s just another step. I wasn’t sure if this is possible or not. I am sure there are security concerns of course.

In my application, all the installation config data needs to be stored locally on the device, and the agent has a “copy” that is sent to any webpage request. But if I have to replace the unit, and it only has embedded flash memory, the only other way to ensure the data can be transferred to a new device is if I host the data externally and pull it. But that requires re-blinkup and probably some initial “setting” to point it to the right location for the data. I am looking for a seamless, robust way to ensure a customer has local access to their data (SD card they could remove, copy, etc) without complicated steps.

Hopefully someone has some thoughts on this… Thanks!

So would the SPIFlashFileSystem library need to be modified for the 512 byte page format? This sounds like an ideal option and would like to make it work. Also, how can the “blink-up” related data be accessed and stored? Through the blinkup SDK data? I’m actually looking at the JS blinkup now, I think this is the route we will have to take for now… I have breakout boards on their way for micro SD, just need to get frame work going… Thanks again Hugo for your awesome support!

Yep, SD would work for that. Really you want the SPIFlashFileSystem but running on a SPI attached MMC card… hmmm. The MMC stuff is actually pretty easy to handle compared to “real” flash as it deals with a lot of the complexity as I remember (and has 512 byte pages).

Technically yes it could be modified to work with an SD card in MMC mode, but I don’t think we’ll get round to that ourselves… open to pull requests :wink:

The SSID can be retrieved, but there is intentionally no way to pull the password out of the imp, or the enrol token (which is a one-time token).

Will a SPI FRAM memory work with the spiflashfilesystem libraries?

Like this one: https://www.adafruit.com/products/1897

I am unsure if it will or not. Sorry for the noob question…

Great question Physicsnole! Based on prior research for a different system I took typical SPI EEPROMs to be interchangable with SPI FRAM. But, I never got to execute that project to completion. For my next Electric Imp design I’ve been assuming FRAM, but can’t say it works for sure either. Hopefully we can have a guru chime in.

I have one of the Adafruit FRAM boards, but never got round to using it. I’ll dig it out and see if I can get it to work with the spiflashfilesystem.

This is the first I’ve heard of FRAM… a few notes from the quick reading I’ve done:

  • I haven’t read enough (or dug into datasheets) to see if FRAM and Flash share a similar interface
  • FRAM looks like it’s a bit more expensive, but you get you get some big improvements:
    • No pages (i.e. you don’t need to erase large chunks of data at once)
    • Trillions of write cycles vs. tens of thousands of write cycles
    • Faster, and a bit lower power.

So what does this all mean…

  • FRAM may or may not work with the file system class
  • You typically aren’t going to see the benefits from FRAM using our filesystem because it was written specifically to address the challenges FRAM overcomes (specifically - pages, and wear leveling)

I would suggest that if you’re planning to use this particular filesystem, spi flash is probably the way to go because of cost (though if you already have FRAM kicking around, and it does have a similar interface to flash… try it out and let us know)!!

If FRAM becomes really popular in our community / with our customers I suspect we’d likely look into writing a filesystem that was designed specifically for FRAM that takes advantage of it’s benefits :slight_smile:

Turns out my FRAM board is I2C so not good for use with spiflashfilesystem as is. Will investigate and examine writing a comparable class.

Thanks for your time smittytone…

I just got my SPI FRAM board from Adafruit. I would like to test. Is there an example program I can just load and test? Do I need to modify any code in the library to accommodate for the lower storage space?

Thanks…

@physicsnole - Try using the SPIFlash class to read / write / erase some data. If that works, the SPIFlashFileSystem will as well.

Here’s some code you can test with:

`
#require “SPIFlash.class.nut:1.0.1”

// spi257 hooked up to flash
// pin8 hooked up to enable
spiFlash <- SPIFlash(hardware.spi257, hardware.pin8);
spiFlash.configure(30000);

// Enable the SPIFlash
spiFlash.enable();

// Write some data:
spiFlash.write(0x01, “\xDE\xAD\xBE\xEF”);

// Read the data:
local data = spiFlash.read(0x01, 4);

// Erase the sector
spiFlash.eraseSector(0x00);

// Read the data:
local newData = spiFlash.read(0x01, 4);
assert(data[0] == 0x00);
assert(data[1] == 0x00);
assert(data[2] == 0x00);
assert(data[3] == 0x00);

// disable the spiflash
spiFlash.disable();

server.log(“All tests passed!”);
`

I would strongly recommend against using/adapting the SPIFlashFileSystem with FRAM. It is really designed for SPI flashes which are at a minimum 64Kbytes but ideally at least 1Mbyte if you want to store any significant data. FRAMs by comparison are typically only a few Kbytes (the Adafruit one is 8Kbytes). The vast majority of the class is working around the shortcomings of NOR flash which is the exact reason that FRAM was invented.

I would suggest that you use the Serializer class and write your own FRAM class which just has read and write blob functions. Since serializer can store tables and a file system is really just a table, that you should get what you want provided the whole image can still fit in memory.

Beardedinventor… Got the system hooked up. Small error, spiFlash.eraseSector(0x00) threw an error, changed to spiFlash.erasesector(0x00). Then re-ran, and got:

2015-09-09 09:36:03 UTC-4 [Status] Device connected
2015-09-09 09:36:03 UTC-4 [Device] ERROR: assertion failed
2015-09-09 09:36:03 UTC-4 [Device] ERROR: at main:22

Looking at the assert documentation, it throws an error if the argument is false. So the code would throw an error if data[0] !=0, which should be the case if the data was written and stored on the FRAM, correct?. I commented out the assert part and logged data[0-3] and got:

2015-09-09 09:38:39 UTC-4 [Status] Device disconnected
2015-09-09 09:38:40 UTC-4 [Status] Downloading new code; 6.15% program storage used
2015-09-09 09:38:46 UTC-4 [Device] 222
2015-09-09 09:38:46 UTC-4 [Device] 173
2015-09-09 09:38:46 UTC-4 [Device] 190
2015-09-09 09:38:46 UTC-4 [Device] 239
2015-09-09 09:38:46 UTC-4 [Device] All tests passed!

So the data seems to be stored! Seems to work… DEAD BEEF… haha

@brandon, I completely agree with your assessment. I believe the community would appreciate a specific FRAM class. Unfortunately, I do not have the time to create atm. Just trying to get some nv storage for the imp (without using server) for worst case scenarios. I think both an SD library and a Fram library would help the community and customers in the future…

EDIT: Looking at the code, the sector is erased, then read again to ensure the sector was erased back to x00. Since the assertion failed, does this mean that its incompatible with the library? I am looking to see if the data is stored and able to be retrieved on power loss… The device does not seem to erase the stored data, but seems you can write and read… Ran this code once written with data:

`#require “SPIFlash.class.nut:1.0.1”

// spi257 hooked up to flash
// pin8 hooked up to enable
spiFlash <- SPIFlash(hardware.spi257, hardware.pin8);
spiFlash.configure(30000);

// Enable the SPIFlash
spiFlash.enable();

// Write some data:
//spiFlash.write(0x01, “\xDE\xAD\xBE\xEF”);

// Read the data:
//local data = spiFlash.read(0x01, 4);

// Erase the sector
spiFlash.erasesector(0x00);

// Read the data:
local newData = spiFlash.read(0x01, 4);
//assert(data[0] == 0x00);
//assert(data[1] == 0x00);
//assert(data[2] == 0x00);
//assert(data[3] == 0x00);

server.log(newData[0]);
server.log(newData[1]);
server.log(newData[2]);
server.log(newData[3]);

// disable the spiflash
spiFlash.disable();

server.log(“All tests passed!”);`

returns:

2015-09-09 10:37:18 UTC-4 [Status] Device disconnected
2015-09-09 10:37:19 UTC-4 [Status] Downloading new code; 6.08% program storage used
2015-09-09 10:37:23 UTC-4 [Device] 222
2015-09-09 10:37:23 UTC-4 [Device] 173
2015-09-09 10:37:23 UTC-4 [Device] 190
2015-09-09 10:37:23 UTC-4 [Device] 239
2015-09-09 10:37:23 UTC-4 [Device] All tests passed!

Also want to note that server.log(spiFlash.size()) returns 4194304. The board is a 8Kb chip, not too sure what to make of this…

Even though the erasesector doesn’t work, I can still read/write over the existing bytes. Set them to x00 or whatever… May be a workaround…

Might have better luck converting this arduino library to squirrel…

Whoops - that last set of asserts is incorrect. When you erase a sector on SPIFlash you’re actually reseting the bytes to 0xFF not 0x00, though based on the server.logs… it looks like it didn’t do that either.

There’s no (non-hacky) way to automagically determine the size of flash… You can pass a size parameter to the SPIFlash class constructor (checkout the repo for more info).

It was a bit of a shot in the dark in terms of whether the interface would be identical to SPIFlash… I suspect you’re correct in that converting the Arduino_FRAM_SPI class will be your best bet :slight_smile:

I encourage you take a look at the SPIFlash class for inspiration, a lot of work went into optimizing the class for performance… so let us know if you have any questions :slight_smile:

Folks, the FRAM libraries are now up. There’s an MB85RC library for controlling a Fujitsu FRAM chip over I²C, and a separate library, FramStore, for combining FRAM chips of various sizes into a single store with a single address space. The MB85RC librarysupports the same methods as the current hardware.spiflash object to make it hopefully easier to migrate code.