File System Component  Version 6.3
MDK-Professional Middleware for Devices with Flash File System
 All Data Structures Files Functions Variables Enumerations Enumerator Macros Groups Pages
FAT File System

The FAT File System was initially used on PC operating systems such as MS-DOS and early versions of Microsoft Windows. Still, it is widely used on USB memory sticks or memory cards for file storage. It is simple, robust and offers good performance especially in embedded applications.

FAT is an acronym for File Allocation Table. This table provides the index of the files in the system and is statically allocated at the time of formatting the drive. It contains an entry for each cluster (a data storage area). The FAT's entries contain either the number of the next cluster in the file, or a marker indicating EOF (end of file), unused disk space, or other areas of the drive that are specially reserved. The drive's root directory contains the number of the first cluster of each file in the directory; the operating system traverses the FAT table, looking up the cluster number of each successive part of the file, until the end of the file is reached. Likewise, sub-directories are implemented as special files containing the directory entries of their respective files.

Since the first implementation, the maximum number of clusters of a drive has increased dramatically, and so the number of bits used to identify each cluster has grown. The versions of the FAT format are named after the number of table element bits: FAT12, FAT16, and FAT32. Each of these variants is is supported by the File System Component.

Layout

The actual FAT file system is made up of four different sections:

Section Description
Boot sector Contains machine startup code
File Allocation TableContains the data region map; multiple entries might exist for redundancy purposes
Root Directory Stores information about the files and directories located in the root directory (FAT12/FAT16 only)
File Data Region Actual stored file data

Volume name

Volume label for FAT12, FAT16 or FAT32 volume must be maximum 11 characters long and cannot contain the following characters: * ? / \ | , ; : + = < > [ ] " .

Path name handling

Path name format used to specify either a drive, directory or file:

  • [drive:][directory separator][path]

Depending on the usage case, each specifier can be optional:

  • drive specifier must consist of drive letter followed by optional drive number and a semicolon.
  • directory separator can be slash (/) or backslash (\)
  • path can consist of multiple directory names and a file name

Absolute and relative path

The table below explains how to specify either absolute or relative path.

Path name Type Description
\ Absolute Root directory
\ .. Absolute Root directory
. Relative Current directory
.. Relative Parent directory
M:\file.txt Absolute A file in root directory of the drive M0
M:file.txt Relative A file in current directory of the drive M0
\file.txt Absolute A file in the root directory of the current drive
file.txt Relative A file in the current directory of the current drive
folder1\ .. Relative Current directory

Long and Short File Name handling

File System component is provided with the library with long (LFN) and short (SFN or 8.3) filename support. Since there is no compulsory algorithm for creating the 8.3 name from an LFN, FileSystem component uses convention described below.

  • Uppercased or lowercased name which is 8.3 valid, is stored uppercased in a single SFN entry
    • Example: "TEXTFILE.TXT" is stored as "TEXTFILE.TXT" (SFN)
    • Example: "textfile.txt" is stored as "TEXTFILE.TXT" (SFN)
  • Mixedcase name which is 8.3 valid, is stored mixed-case in LFN entry and SFN entry is created, which is concluded with tilde and a numeric value.
    • Example: "TextFile.txt" is stored as "TextFile.txt" (LFN) and "TEXTFI~1.TXT" (SFN)
  • Name which is not 8.3 valid is stored in LFN entry and SFN entry. Name writen in SFN entry is uppercased, stripped of invalid 8.3 characters which are replaced with underscore "_" and concluded with tilde and a numeric value.
    • Example: "Tex+File.txt" is stored as "Tex+File.txt" (LFN) and "TEX_FI~1.TXT (SFN)"

Journaling

The FAT File System implementation supports Journaling which makes file I/O functions robust against system crashes or power failures. When FAT Journaling is enabled FAT (File Allocation Table) and Directory entries are protected.

FAT Journaling keeps track of the changes in the file system using a journal before committing them to the main file system. In the event of a system failure, file system does not become corrupted. Only content of files that are opened in writing mode may be lost. Other files are not affected.

Journaling can be enabled individually for each drive that is using the FAT File System. For information on how to enable Journaling refer to:

System Initialization and Drive Formatting

There requirements for creating a journal are:

  • the drive must be formatted with two FAT tables (two consecutive tables in the FAT section of the drive)
  • the drive must have at least 64 sectors (= 32kB) of contiguous free space at the end of a partition (in the last 1% space).

If journaling is enabled, mounting a drive triggers the File System to search for the journal.

When journal is found the journaling system is initialized and the journal content is checked for any incomplete data transactions that may be the result of a system crash. When incomplete data transactions are found the file system restores the drive to the last known valid state.

When no journal is found, the journal is automatically created. The required space (32kB) for journal is reserved by marking the adequate amount of clusters in the last 1% of the drive partition. This reduces the reported amount of free space on the drive accordingly.

No special drive formatting is required. When a drive is formatted on a different computer (running Window or Linux) free space is typical available at the end of a partition. Creation of a journal may fail on removable media when above requirements are not meet. This may happen if the drive is fragmented or there is not enough space. In this case the drive should be defragmented or some files should be deleted.

When using the fformat function of the File System component, the journal is automatically created on drives with enabled option "Use FAT Journal".

System failure behaviour

When File System component experiences a power loss or crashes before a file opened for writing is successfully closed the content of that file is lost. This results in lost data clusters which can be restored using a file system repair utility tool such as chkdsk on Windows or fsck on Linux.

NAND Flash Translation Layer

NFTL Features

NAND Flash Overview

NAND Flash memory is used in USB drives, secure digital (SD) cards, memory sticks, CompactFlash cards and multimedia cards (MMC). It is a non-volatile memory with 60 percent smaller memory cells comparing to NOR memory cells thus offers high data density at a lower cost. It is organized into a series of blocks, which are the smallest erasable entities in a NAND Flash device. The smallest entity that can be programmed is a byte. Due to a limited number of times NAND Flash blocks can be reliably programmed and erased (nominally, each NAND block will survive 100000 program/erase cycles) it is critical to implement a NAND Flash Translation Layer (NFTL) to maximize the lifespan of the device. This layer comprises of wear leveling algorithms, bad block management and error control algorithms and thus maximizes the lifespan of a device by redirecting new writes to different physical blocks. It only applies to NAND Flash devices directly attached to the microcontroller.

Note
External storage devices such as memory cards or USB sticks carry an integrated NAND Flash controller that is taking care of the NAND Flash in the device.

NAND Flash Architecture

A NAND Flash array is organized in blocks with multiple pages per block. Each page consist of user data area and spare data area. The spare data area is physically the same as user data area and is typically used for error correction information (ECC), wear leveling and other software system functions. The user data area is available to the user for data storage. The size of the user data area depends on the page size and is typically of one or multiple sector size.

Wear Leveling

Due to its architecture, data can be written to an address on a NAND flash memory only a finite number of times. NAND flash memory wears out if data is written too often to the same address, so wear leveling is used to help prolong the life of the NAND flash device. It ensures that data erasures and writes are distributed evenly across the NAND Flash storage medium, so that NAND memory blocks don't fail prematurely due to a high number of erase cycles.

Dynamic wear leveling addresses the issue of repeated writes to the same logical blocks by redirecting new writes to different physical blocks. Dynamic wear leveling works on data blocks that are being written to dynamically.

Static wear leveling addresses the issue of static data that is kept unchanged for a long time in content and location. Some static data can stay in the same data block until the NAND flash memory is broken. To evenly distribute write and erase cycles across NAND memory blocks, an algorithm is implemented to move the static data from one location to the other in the Flash.

The File System Component implements wear leveling for all kinds of NAND Flash devices (MLC, SLC, TLC).

Bad Block Management

It is normal for a NAND flash memory to contain bad or invalid blocks of memory. Invalid blocks are blocks that contain one or more invalid bits so that the reliability can no longer be guaranteed. Blocks can become invalid during erasing and programming at run-time or during the manufacturing process. Bad blocks generated during manufacturing process are marked bad by the manufacturer. Bad blocks detected at run-time are marked as bad by monitoring the status register in the NAND flash chip during an erase and program operation. A block that is marked as bad is skipped and no longer used.

The File System Component implements bad block management for all kinds of NAND Flash devices (MLC, SLC, TLC):

  • Skip factory marked bad blocks
  • Mark block as bad as soon as an erase operation fails on the block
  • Replace/mode data from blocks marked as bad to good blocks

Error Correction Codes (ECC)

Error detection and correction codes are used in flash memory to protect data from corruption. All types of error correction codes add redundant information to the data being stored to the flash memory. The redundant information is calculated before the data is written, added to the spare area of the page and then stored in flash memory. When the data is read, the redundant information is recalculated and compared to those stored in the flash.

Error correction codes (ECC) used in the NAND flash memory are block codes. This means that the redundant data bits are calculated for a fixed size block of used data. NTFL is implementing a Hamming ECC algorithm for a fixed size of one sector or 512 bytes for SLC NAND Flashes. The redundant information is calculated in a way that a balance of correction power and efficiency is achieved.

Warning
If you are using a TLC or MLC NAND Flash device, you need to disable the Software ECC in the FS_Config_NAND_0.h file, otherwise the File System will not function correctly.