Commit 0e8cc8bd authored by William Juul's avatar William Juul Committed by Scott Wood
Browse files

YAFFS2 import

Direct import of yaffs as a tarball as of 20071113 from their public
CVS-web at http://www.aleph1.co.uk/cgi-bin/viewcvs.cgi/yaffs2/



The code can also be imported on the command line with:
export CVSROOT=:pserver:anonymous@cvs.aleph1.co.uk:/home/aleph1/cvs cvs logon
(Hit return when asked for a password)
cvs checkout yaffs2
Signed-off-by: default avatarWilliam Juul <william.juul@tandberg.com>
Signed-off-by: default avatarStig Olsen <stig.olsen@tandberg.com>
parent 3043c045
#
# YAFFS file system configurations
#
config YAFFS_FS
tristate "YAFFS2 file system support"
default n
depends on MTD
select YAFFS_YAFFS1
select YAFFS_YAFFS2
help
YAFFS2, or Yet Another Flash Filing System, is a filing system
optimised for NAND Flash chips.
To compile the YAFFS2 file system support as a module, choose M
here: the module will be called yaffs2.
If unsure, say N.
Further information on YAFFS2 is available at
<http://www.aleph1.co.uk/yaffs/>.
config YAFFS_YAFFS1
bool "512 byte / page devices"
depends on YAFFS_FS
default y
help
Enable YAFFS1 support -- yaffs for 512 byte / page devices
Not needed for 2K-page devices.
If unsure, say Y.
config YAFFS_9BYTE_TAGS
bool "Use older-style on-NAND data format with pageStatus byte"
depends on YAFFS_YAFFS1
default n
help
Older-style on-NAND data format has a "pageStatus" byte to record
chunk/page state. This byte is zero when the page is discarded.
Choose this option if you have existing on-NAND data using this
format that you need to continue to support. New data written
also uses the older-style format. Note: Use of this option
generally requires that MTD's oob layout be adjusted to use the
older-style format. See notes on tags formats and MTD versions
in yaffs_mtdif1.c.
If unsure, say N.
config YAFFS_DOES_ECC
bool "Lets Yaffs do its own ECC"
depends on YAFFS_FS && YAFFS_YAFFS1 && !YAFFS_9BYTE_TAGS
default n
help
This enables Yaffs to use its own ECC functions instead of using
the ones from the generic MTD-NAND driver.
If unsure, say N.
config YAFFS_ECC_WRONG_ORDER
bool "Use the same ecc byte order as Steven Hill's nand_ecc.c"
depends on YAFFS_FS && YAFFS_DOES_ECC && !YAFFS_9BYTE_TAGS
default n
help
This makes yaffs_ecc.c use the same ecc byte order as Steven
Hill's nand_ecc.c. If not set, then you get the same ecc byte
order as SmartMedia.
If unsure, say N.
config YAFFS_YAFFS2
bool "2048 byte (or larger) / page devices"
depends on YAFFS_FS
default y
help
Enable YAFFS2 support -- yaffs for >= 2K bytes per page devices
If unsure, say Y.
config YAFFS_AUTO_YAFFS2
bool "Autoselect yaffs2 format"
depends on YAFFS_YAFFS2
default y
help
Without this, you need to explicitely use yaffs2 as the file
system type. With this, you can say "yaffs" and yaffs or yaffs2
will be used depending on the device page size (yaffs on
512-byte page devices, yaffs2 on 2K page devices).
If unsure, say Y.
config YAFFS_DISABLE_LAZY_LOAD
bool "Disable lazy loading"
depends on YAFFS_YAFFS2
default n
help
"Lazy loading" defers loading file details until they are
required. This saves mount time, but makes the first look-up
a bit longer.
Lazy loading will only happen if enabled by this option being 'n'
and if the appropriate tags are available, else yaffs2 will
automatically fall back to immediate loading and do the right
thing.
Lazy laoding will be required by checkpointing.
Setting this to 'y' will disable lazy loading.
If unsure, say N.
config YAFFS_CHECKPOINT_RESERVED_BLOCKS
int "Reserved blocks for checkpointing"
depends on YAFFS_YAFFS2
default 10
help
Give the number of Blocks to reserve for checkpointing.
Checkpointing saves the state at unmount so that mounting is
much faster as a scan of all the flash to regenerate this state
is not needed. These Blocks are reserved per partition, so if
you have very small partitions the default (10) may be a mess
for you. You can set this value to 0, but that does not mean
checkpointing is disabled at all. There only won't be any
specially reserved blocks for checkpointing, so if there is
enough free space on the filesystem, it will be used for
checkpointing.
If unsure, leave at default (10), but don't wonder if there are
always 2MB used on your large page device partition (10 x 2k
pagesize). When using small partitions or when being very small
on space, you probably want to set this to zero.
config YAFFS_DISABLE_WIDE_TNODES
bool "Turn off wide tnodes"
depends on YAFFS_FS
default n
help
Wide tnodes are only used for NAND arrays >=32MB for 512-byte
page devices and >=128MB for 2k page devices. They use slightly
more RAM but are faster since they eliminate chunk group
searching.
Setting this to 'y' will force tnode width to 16 bits and save
memory but make large arrays slower.
If unsure, say N.
config YAFFS_ALWAYS_CHECK_CHUNK_ERASED
bool "Force chunk erase check"
depends on YAFFS_FS
default n
help
Normally YAFFS only checks chunks before writing until an erased
chunk is found. This helps to detect any partially written
chunks that might have happened due to power loss.
Enabling this forces on the test that chunks are erased in flash
before writing to them. This takes more time but is potentially
a bit more secure.
Suggest setting Y during development and ironing out driver
issues etc. Suggest setting to N if you want faster writing.
If unsure, say Y.
config YAFFS_SHORT_NAMES_IN_RAM
bool "Cache short names in RAM"
depends on YAFFS_FS
default y
help
If this config is set, then short names are stored with the
yaffs_Object. This costs an extra 16 bytes of RAM per object,
but makes look-ups faster.
If unsure, say Y.
# Main Makefile for YAFFS
#
#
# YAFFS: Yet Another Flash File System. A NAND-flash specific file system.
#
# Copyright (C) 2002-2007 Aleph One Ltd.
# for Toby Churchill Ltd and Brightstar Engineering
#
# Created by Charles Manning <charles@aleph1.co.uk>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 2 as
# published by the Free Software Foundation.
ifneq ($(KERNELRELEASE),)
EXTRA_CFLAGS += -DYAFFS_OUT_OF_TREE
obj-m := yaffs2.o
yaffs2-objs := yaffs_mtdif.o yaffs_mtdif2.o
yaffs2-objs += yaffs_mtdif1.o yaffs_packedtags1.o
yaffs2-objs += yaffs_ecc.o yaffs_fs.o yaffs_guts.o
yaffs2-objs += yaffs_packedtags2.o yaffs_qsort.o
yaffs2-objs += yaffs_tagscompat.o yaffs_tagsvalidity.o
yaffs2-objs += yaffs_checkptrw.o yaffs_nand.o
else
KERNELDIR ?= /lib/modules/$(shell uname -r)/build
PWD := $(shell pwd)
modules default:
$(MAKE) -C $(KERNELDIR) M=$(PWD) modules
mi modules_install:
$(MAKE) -C $(KERNELDIR) M=$(PWD) modules_install
clean:
$(MAKE) -C $(KERNELDIR) M=$(PWD) clean
endif
#
# Makefile for the linux YAFFS filesystem routines.
#
obj-$(CONFIG_YAFFS_FS) += yaffs.o
yaffs-y := yaffs_ecc.o yaffs_fs.o yaffs_guts.o yaffs_checkptrw.o
yaffs-y += yaffs_packedtags1.o yaffs_packedtags2.o yaffs_nand.o yaffs_qsort.o
yaffs-y += yaffs_tagscompat.o yaffs_tagsvalidity.o
yaffs-y += yaffs_mtdif.o yaffs_mtdif1.o yaffs_mtdif2.o
Welcome to YAFFS, the first file system developed specifically for NAND flash.
It is now YAFFS2 - original YAFFS (AYFFS1) only supports 512-byte page
NAND and is now deprectated. YAFFS2 supports 512b page in 'YAFFS1
compatibility' mode (CONFIG_YAFFS_YAFFS1) and 2K or larger page NAND
in YAFFS2 mode (CONFIG_YAFFS_YAFFS2).
A note on licencing
-------------------
YAFFS is available under the GPL and via alternative licensing
arrangements with Aleph One. If you're using YAFFS as a Linux kernel
file system then it will be under the GPL. For use in other situations
you should discuss licensing issues with Aleph One.
Terminology
-----------
Page - NAND addressable unit (normally 512b or 2Kbyte size) - can
be read, written, marked bad. Has associated OOB.
Block - Eraseable unit. 64 Pages. (128K on 2K NAND, 32K on 512b NAND)
OOB - 'spare area' of each page for ECC, bad block marked and YAFFS
tags. 16 bytes per 512b - 64 bytes for 2K page size.
Chunk - Basic YAFFS addressable unit. Same size as Page.
Object - YAFFS Object: File, Directory, Link, Device etc.
YAFFS design
------------
YAFFS is a log-structured filesystem. It is designed particularly for
NAND (as opposed to NOR) flash, to be flash-friendly, robust due to
journalling, and to have low RAM and boot time overheads. File data is
stored in 'chunks'. Chunks are the same size as NAND pages. Each page
is marked with file id and chunk number. These marking 'tags' are
stored in the OOB (or 'spare') region of the flash. The chunk number
is determined by dividing the file position by the chunk size. Each
chunk has a number of valid bytes, which equals the page size for all
except the last chunk in a file.
File 'headers' are stored as the first page in a file, marked as a
different type to data pages. The same mechanism is used to store
directories, device files, links etc. The first page describes which
type of object it is.
YAFFS2 never re-writes a page, because the spec of NAND chips does not
allow it. (YAFFS1 used to mark a block 'deleted' in the OOB). Deletion
is managed by moving deleted objects to the special, hidden 'unlinked'
directory. These records are preserved until all the pages containing
the object have been erased (We know when this happen by keeping a
count of chunks remaining on the system for each object - when it
reaches zero the object really is gone).
When data in a file is overwritten, the relevant chunks are replaced
by writing new pages to flash containing the new data but the same
tags.
Pages are also marked with a short (2 bit) serial number that
increments each time the page at this position is incremented. The
reason for this is that if power loss/crash/other act of demonic
forces happens before the replaced page is marked as discarded, it is
possible to have two pages with the same tags. The serial number is
used to arbitrate.
A block containing only discarded pages (termed a dirty block) is an
obvious candidate for garbage collection. Otherwise valid pages can be
copied off a block thus rendering the whole block discarded and ready
for garbage collection.
In theory you don't need to hold the file structure in RAM... you
could just scan the whole flash looking for pages when you need them.
In practice though you'd want better file access times than that! The
mechanism proposed here is to have a list of __u16 page addresses
associated with each file. Since there are 2^18 pages in a 128MB NAND,
a __u16 is insufficient to uniquely identify a page but is does
identify a group of 4 pages - a small enough region to search
exhaustively. This mechanism is clearly expandable to larger NAND
devices - within reason. The RAM overhead with this approach is approx
2 bytes per page - 512kB of RAM for a whole 128MB NAND.
Boot-time scanning to build the file structure lists only requires
one pass reading NAND. If proper shutdowns happen the current RAM
summary of the filesystem status is saved to flash, called
'checkpointing'. This saves re-scanning the flash on startup, and gives
huge boot/mount time savings.
YAFFS regenerates its state by 'replaying the tape' - i.e. by
scanning the chunks in their allocation order (i.e. block sequence ID
order), which is usually different form the media block order. Each
block is still only read once - starting from the end of the media and
working back.
YAFFS tags in YAFFS1 mode:
18-bit Object ID (2^18 files, i.e. > 260,000 files). File id 0- is not
valid and indicates a deleted page. File od 0x3ffff is also not valid.
Synonymous with inode.
2-bit serial number
20-bit Chunk ID within file. Limit of 2^20 chunks/pages per file (i.e.
> 500MB max file size). Chunk ID 0 is the file header for the file.
10-bit counter of the number of bytes used in the page.
12 bit ECC on tags
YAFFS tags in YAFFS2 mode:
4 bytes 32-bit chunk ID
4 bytes 32-bit object ID
2 bytes Number of data bytes in this chunk
4 bytes Sequence number for this block
3 bytes ECC on tags
12 bytes ECC on data (3 bytes per 256 bytes of data)
Page allocation and garbage collection
Pages are allocated sequentially from the currently selected block.
When all the pages in the block are filled, another clean block is
selected for allocation. At least two or three clean blocks are
reserved for garbage collection purposes. If there are insufficient
clean blocks available, then a dirty block ( ie one containing only
discarded pages) is erased to free it up as a clean block. If no dirty
blocks are available, then the dirtiest block is selected for garbage
collection.
Garbage collection is performed by copying the valid data pages into
new data pages thus rendering all the pages in this block dirty and
freeing it up for erasure. I also like the idea of selecting a block
at random some small percentage of the time - thus reducing the chance
of wear differences.
YAFFS is single-threaded. Garbage-collection is done as a parasitic
task of writing data. So each time some data is written, a bit of
pending garbage collection is done. More pages are garbage-collected
when free space is tight.
Flash writing
YAFFS only ever writes each page once, complying with the requirements
of the most restricitve NAND devices.
Wear levelling
This comes as a side-effect of the block-allocation strategy. Data is
always written on the next free block, so they are all used equally.
Blocks containing data that is written but never erased will not get
back into the free list, so wear is levelled over only blocks which
are free or become free, not blocks which never change.
Some helpful info
-----------------
Formatting a YAFFS device is simply done by erasing it.
Making an initial filesystem can be tricky because YAFFS uses the OOB
and thus the bytes that get written depend on the YAFFS data (tags),
and the ECC bytes and bad block markers which are dictated by the
hardware and/or the MTD subsystem. The data layout also depends on the
device page size (512b or 2K). Because YAFFS is only responsible for
some of the OOB data, generating a filesystem offline requires
detailed knowledge of what the other parts (MTD and NAND
driver/hardware) are going to do.
To make a YAFFS filesystem you have 3 options:
1) Boot the system with an empty NAND device mounted as YAFFS and copy
stuff on.
2) Make a filesystem image offline, then boot the system and use
MTDutils to write an image to flash.
3) Make a filesystem image offline and use some tool like a bootloader to
write it to flash.
Option 1 avoids a lot of issues because all the parts
(YAFFS/MTD/hardware) all take care of their own bits and (if you have
put things together properly) it will 'just work'. YAFFS just needs to
know how many bytes of the OOB it can use. However sometimes it is not
practical.
Option 2 lets MTD/hardware take care of the ECC so the filesystem
image just had to know which bytes to use for YAFFS Tags.
Option 3 is hardest as the image creator needs to know exactly what
ECC bytes, endianness and algorithm to use as well as which bytes are
available to YAFFS.
mkyaffs2image creates an image suitable for option 3 for the
particular case of yaffs2 on 2K page NAND with default MTD layout.
mkyaffsimage creates an equivalent image for 512b page NAND (i.e.
yaffs1 format).
Bootloaders
-----------
A bootloader using YAFFS needs to know how MTD is laying out the OOB
so that it can skip bad blocks.
YAFFS Tracing
-------------
To build YAFFS in the Linux kernel tree you need to run the patch-ker.sh
script from the yaffs source directory, giving your choice as to whether
you wish to copy (c) or link (l) the code and the path to your kernel
sources, e.g:
./patch-ker.sh c /usr/src/linux
This will copy the yaffs files into fs/yaffs2 and modify the Kconfig
and Makefiles in the fs directory.
./patch-ker.sh l /usr/src/linux
This does the same as the above but makes symbolic links instead.
After you've run the script, go back to your normal kernel making procedure
and configure the yaffs settings you want.
Prolems? Contact the yaffs mailing list:
http://www.aleph1.co.uk/mailman/listinfo/yaffs
/*
* YAFFS: Yet another Flash File System . A NAND-flash specific file system.
*
* Copyright (C) 2002-2007 Aleph One Ltd.
* for Toby Churchill Ltd and Brightstar Engineering
*
* Created by Charles Manning <charles@aleph1.co.uk>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License version 2.1 as
* published by the Free Software Foundation.
*
* Note: Only YAFFS headers are LGPL, YAFFS C code is covered by GPL.
*/
/*
* This file is just holds extra declarations used during development.
* Most of these are from kernel includes placed here so we can use them in
* applications.
*
*/
#ifndef __EXTRAS_H__
#define __EXTRAS_H__
#if defined WIN32
#define __inline__ __inline
#define new newHack
#endif
#if !(defined __KERNEL__) || (defined WIN32)
/* User space defines */
typedef unsigned char __u8;
typedef unsigned short __u16;
typedef unsigned __u32;
/*
* Simple doubly linked list implementation.
*
* Some of the internal functions ("__xxx") are useful when
* manipulating whole lists rather than single entries, as
* sometimes we already know the next/prev entries and we can
* generate better code by using them directly rather than
* using the generic single-entry routines.
*/
#define prefetch(x) 1
struct list_head {
struct list_head *next, *prev;
};
#define LIST_HEAD_INIT(name) { &(name), &(name) }
#define LIST_HEAD(name) \
struct list_head name = LIST_HEAD_INIT(name)
#define INIT_LIST_HEAD(ptr) do { \
(ptr)->next = (ptr); (ptr)->prev = (ptr); \
} while (0)
/*
* Insert a new entry between two known consecutive entries.
*
* This is only for internal list manipulation where we know
* the prev/next entries already!
*/
static __inline__ void __list_add(struct list_head *new,
struct list_head *prev,
struct list_head *next)
{
next->prev = new;
new->next = next;
new->prev = prev;
prev->next = new;
}
/**
* list_add - add a new entry
* @new: new entry to be added
* @head: list head to add it after
*
* Insert a new entry after the specified head.
* This is good for implementing stacks.
*/
static __inline__ void list_add(struct list_head *new, struct list_head *head)
{
__list_add(new, head, head->next);
}
/**
* list_add_tail - add a new entry
* @new: new entry to be added
* @head: list head to add it before
*
* Insert a new entry before the specified head.
* This is useful for implementing queues.
*/
static __inline__ void list_add_tail(struct list_head *new,
struct list_head *head)
{
__list_add(new, head->prev, head);
}
/*
* Delete a list entry by making the prev/next entries
* point to each other.
*
* This is only for internal list manipulation where we know
* the prev/next entries already!
*/
static __inline__ void __list_del(struct list_head *prev,
struct list_head *next)
{
next->prev = prev;
prev->next = next;
}
/**
* list_del - deletes entry from list.
* @entry: the element to delete from the list.
* Note: list_empty on entry does not return true after this, the entry is
* in an undefined state.
*/
static __inline__ void list_del(struct list_head *entry)
{
__list_del(entry->prev, entry->next);
}
/**
* list_del_init - deletes entry from list and reinitialize it.
* @entry: the element to delete from the list.
*/
static __inline__ void list_del_init(struct list_head *entry)
{
__list_del(entry->prev, entry->next);
INIT_LIST_HEAD(entry);
}
/**
* list_empty - tests whether a list is empty
* @head: the list to test.
*/
static __inline__ int list_empty(struct list_head *head)
{
return head->next == head;
}
/**
* list_splice - join two lists
* @list: the new list to add.
* @head: the place to add it in the first list.
*/
static __inline__ void list_splice(struct list_head *list,
struct list_head *head)
{
struct list_head *first = list->next;
if (first != list) {
struct list_head *last = list->prev;
struct list_head *at = head->next;
first->prev = head;
head->next = first;
last->next = at;
at->prev = last;
}
}
/**
* list_entry - get the struct for this entry
* @ptr: the &struct list_head pointer.
* @type: the type of the struct this is embedded in.
* @member: the name of the list_struct within the struct.
*/
#define list_entry(ptr, type, member) \
((type *)((char *)(ptr)-(unsigned long)(&((type *)0)->member)))
/**
* list_for_each - iterate over a list
* @pos: the &struct list_head to use as a loop counter.
* @head: the head for your list.
*/
#define list_for_each(pos, head) \
for (pos = (head)->next, prefetch(pos->next); pos != (head); \
pos = pos->next, prefetch(pos->next))
/**