Design of a logging system

Discussion in 'Computing and Networks' started by snowsara, Jan 25, 2013.

  1. snowsara

    Thread Starter New Member

    Jan 25, 2013
    Hello experts !

    I am designing an embedded system whom goal is to log sensors state continuously and that can be powered off at any time.

    For specific reasons, I have currently to use Windows XP, and a Compact Flash storage media. Firsts prototypes use this configuration :
    * 1rst partition readonly for the system (NTFS),
    * 2nd partition for data logging (NTFS).
    First tests are not so bad, but I am not fully satisfied of this solution (time to time filesystem or Compact flash is dead).

    Do you have advices about a way to improve the solution that I am validating ? Which filesystem to use for a flash storage media ?
    Which solution would you use (HW is x86) if Windows XP and a compact flash were not a constraint (which storage media, which filesystem, which os configuration, etc.) ?

    Thank you for your advices
  2. panic mode

    Senior Member

    Oct 10, 2011
    the problem with flash is that it has limited number of writes. if you keep writing every single record, there is obviously going to be failure after failure. you need to log data into RAM and when you have enough to write one page to flash, write it in one pass (this should store bunch of record while using single write). another issue is power outage. you need to detect brownout and write whatever you have collected to flash before it is too late. also consider having battery backup. I hope you are just writing to flash using XP (and not booting XP from flash).
  3. davebee

    Well-Known Member

    Oct 22, 2008
    If your system can have unexpected power outages that I'd say to avoid a FAT filesystem, because they can be damaged by an unexpected shutdown. Journaling filesystems like NTFS have a better chance of surviving a powerdown because that is what journaling is designed for.

    Other than avoiding FAT filesystems, the best thing I could suggest would be to make up redundent loggers, because hardware failures will happen and no single OS or filesystem can overcome a completely failed hard disk or compact flash device.