Commit 753ac610 authored by Charles Manning's avatar Charles Manning Committed by Wolfgang Denk

u-boot: Update yaffs2 file system

This patch updates the yaffs2 in u-boot to correspond to
git://www.aleph1.co.uk/yaffs2
commit id 9ee5d0643e559568dbe62215f76e0a7bd5a63d93
Signed-off-by: default avatarCharles Manning <cdhmanning@gmail.com>
parent 3874a377
This diff is collapsed.
......@@ -16,28 +16,43 @@
#
# $Id: Makefile,v 1.15 2007/07/18 19:40:38 charles Exp $
#EXTRA_COMPILE_FLAGS = -DYAFFS_IGNORE_TAGS_ECC
include $(TOPDIR)/config.mk
LIB = $(obj)libyaffs2.o
COBJS-$(CONFIG_YAFFS2) := \
yaffscfg.o yaffs_ecc.o yaffsfs.o yaffs_guts.o yaffs_packedtags1.o \
yaffs_tagscompat.o yaffs_packedtags2.o yaffs_tagsvalidity.o \
yaffs_nand.o yaffs_checkptrw.o yaffs_qsort.o yaffs_mtdif.o \
yaffs_mtdif2.o
yaffs_allocator.o yaffs_attribs.o yaffs_bitmap.o yaffs_uboot_glue.o\
yaffs_checkptrw.o yaffs_ecc.o yaffs_error.o \
yaffsfs.o yaffs_guts.o yaffs_hweight.o yaffs_nameval.o yaffs_nand.o\
yaffs_packedtags1.o yaffs_packedtags2.o yaffs_qsort.o \
yaffs_summary.o yaffs_tagscompat.o yaffs_verify.o yaffs_yaffs1.o \
yaffs_yaffs2.o yaffs_mtdif.o yaffs_mtdif2.o
SRCS := $(COBJS-y:.o=.c)
OBJS := $(addprefix $(obj),$(COBJS-y))
# -DCONFIG_YAFFS_NO_YAFFS1
CFLAGS += -DCONFIG_YAFFS_DIRECT -DCONFIG_YAFFS_SHORT_NAMES_IN_RAM -DCONFIG_YAFFS_YAFFS2 -DLINUX_VERSION_CODE=0x20622
YCFLAGS = -DCONFIG_YAFFS_DIRECT -DCONFIG_YAFFS_SHORT_NAMES_IN_RAM
YCFLAGS += -DCONFIG_YAFFS_YAFFS2 -DNO_Y_INLINE
YCFLAGS += -DCONFIG_YAFFS_PROVIDE_DEFS -DCONFIG_YAFFSFS_PROVIDE_VALUES
CFLAGS += $(YCFLAGS)
CPPFLAGS += $(YCFLAGS)
all: $(LIB)
$(LIB): $(obj).depend $(OBJS)
$(obj)libyaffs2.a: $(obj).depend $(OBJS)
$(AR) $(ARFLAGS) $@ $(OBJS)
$(obj)libyaffs2.o: $(obj).depend $(OBJS)
$(call cmd_link_o_target, $(OBJS))
.PHONY: clean distclean
clean:
rm -f $(OBJS)
distclean: clean
rm -f $(LIB) core *.bak .depend
#########################################################################
# defines $(obj).depend target
......
Welcome to YAFFS, the first file system developed specifically for NAND flash.
It is now YAFFS2 - original YAFFS (AYFFS1) only supports 512-byte page
NAND and is now deprectated. YAFFS2 supports 512b page in 'YAFFS1
compatibility' mode (CONFIG_YAFFS_YAFFS1) and 2K or larger page NAND
in YAFFS2 mode (CONFIG_YAFFS_YAFFS2).
A note on licencing
-------------------
YAFFS is available under the GPL and via alternative licensing
arrangements with Aleph One. If you're using YAFFS as a Linux kernel
file system then it will be under the GPL. For use in other situations
you should discuss licensing issues with Aleph One.
Terminology
-----------
Page - NAND addressable unit (normally 512b or 2Kbyte size) - can
be read, written, marked bad. Has associated OOB.
Block - Eraseable unit. 64 Pages. (128K on 2K NAND, 32K on 512b NAND)
OOB - 'spare area' of each page for ECC, bad block marked and YAFFS
tags. 16 bytes per 512b - 64 bytes for 2K page size.
Chunk - Basic YAFFS addressable unit. Same size as Page.
Object - YAFFS Object: File, Directory, Link, Device etc.
YAFFS design
------------
YAFFS is a log-structured filesystem. It is designed particularly for
NAND (as opposed to NOR) flash, to be flash-friendly, robust due to
journalling, and to have low RAM and boot time overheads. File data is
stored in 'chunks'. Chunks are the same size as NAND pages. Each page
is marked with file id and chunk number. These marking 'tags' are
stored in the OOB (or 'spare') region of the flash. The chunk number
is determined by dividing the file position by the chunk size. Each
chunk has a number of valid bytes, which equals the page size for all
except the last chunk in a file.
File 'headers' are stored as the first page in a file, marked as a
different type to data pages. The same mechanism is used to store
directories, device files, links etc. The first page describes which
type of object it is.
YAFFS2 never re-writes a page, because the spec of NAND chips does not
allow it. (YAFFS1 used to mark a block 'deleted' in the OOB). Deletion
is managed by moving deleted objects to the special, hidden 'unlinked'
directory. These records are preserved until all the pages containing
the object have been erased (We know when this happen by keeping a
count of chunks remaining on the system for each object - when it
reaches zero the object really is gone).
When data in a file is overwritten, the relevant chunks are replaced
by writing new pages to flash containing the new data but the same
tags.
Pages are also marked with a short (2 bit) serial number that
increments each time the page at this position is incremented. The
reason for this is that if power loss/crash/other act of demonic
forces happens before the replaced page is marked as discarded, it is
possible to have two pages with the same tags. The serial number is
used to arbitrate.
A block containing only discarded pages (termed a dirty block) is an
obvious candidate for garbage collection. Otherwise valid pages can be
copied off a block thus rendering the whole block discarded and ready
for garbage collection.
In theory you don't need to hold the file structure in RAM... you
could just scan the whole flash looking for pages when you need them.
In practice though you'd want better file access times than that! The
mechanism proposed here is to have a list of __u16 page addresses
associated with each file. Since there are 2^18 pages in a 128MB NAND,
a __u16 is insufficient to uniquely identify a page but is does
identify a group of 4 pages - a small enough region to search
exhaustively. This mechanism is clearly expandable to larger NAND
devices - within reason. The RAM overhead with this approach is approx
2 bytes per page - 512kB of RAM for a whole 128MB NAND.
Boot-time scanning to build the file structure lists only requires
one pass reading NAND. If proper shutdowns happen the current RAM
summary of the filesystem status is saved to flash, called
'checkpointing'. This saves re-scanning the flash on startup, and gives
huge boot/mount time savings.
YAFFS regenerates its state by 'replaying the tape' - i.e. by
scanning the chunks in their allocation order (i.e. block sequence ID
order), which is usually different form the media block order. Each
block is still only read once - starting from the end of the media and
working back.
YAFFS tags in YAFFS1 mode:
18-bit Object ID (2^18 files, i.e. > 260,000 files). File id 0- is not
valid and indicates a deleted page. File od 0x3ffff is also not valid.
Synonymous with inode.
2-bit serial number
20-bit Chunk ID within file. Limit of 2^20 chunks/pages per file (i.e.
> 500MB max file size). Chunk ID 0 is the file header for the file.
10-bit counter of the number of bytes used in the page.
12 bit ECC on tags
YAFFS tags in YAFFS2 mode:
4 bytes 32-bit chunk ID
4 bytes 32-bit object ID
2 bytes Number of data bytes in this chunk
4 bytes Sequence number for this block
3 bytes ECC on tags
12 bytes ECC on data (3 bytes per 256 bytes of data)
Page allocation and garbage collection
Pages are allocated sequentially from the currently selected block.
When all the pages in the block are filled, another clean block is
selected for allocation. At least two or three clean blocks are
reserved for garbage collection purposes. If there are insufficient
clean blocks available, then a dirty block ( ie one containing only
discarded pages) is erased to free it up as a clean block. If no dirty
blocks are available, then the dirtiest block is selected for garbage
collection.
Garbage collection is performed by copying the valid data pages into
new data pages thus rendering all the pages in this block dirty and
freeing it up for erasure. I also like the idea of selecting a block
at random some small percentage of the time - thus reducing the chance
of wear differences.
YAFFS is single-threaded. Garbage-collection is done as a parasitic
task of writing data. So each time some data is written, a bit of
pending garbage collection is done. More pages are garbage-collected
when free space is tight.
Flash writing
YAFFS only ever writes each page once, complying with the requirements
of the most restricitve NAND devices.
Wear levelling
This comes as a side-effect of the block-allocation strategy. Data is
always written on the next free block, so they are all used equally.
Blocks containing data that is written but never erased will not get
back into the free list, so wear is levelled over only blocks which
are free or become free, not blocks which never change.
Some helpful info
-----------------
Formatting a YAFFS device is simply done by erasing it.
Making an initial filesystem can be tricky because YAFFS uses the OOB
and thus the bytes that get written depend on the YAFFS data (tags),
and the ECC bytes and bad block markers which are dictated by the
hardware and/or the MTD subsystem. The data layout also depends on the
device page size (512b or 2K). Because YAFFS is only responsible for
some of the OOB data, generating a filesystem offline requires
detailed knowledge of what the other parts (MTD and NAND
driver/hardware) are going to do.
To make a YAFFS filesystem you have 3 options:
1) Boot the system with an empty NAND device mounted as YAFFS and copy
stuff on.
2) Make a filesystem image offline, then boot the system and use
MTDutils to write an image to flash.
3) Make a filesystem image offline and use some tool like a bootloader to
write it to flash.
Option 1 avoids a lot of issues because all the parts
(YAFFS/MTD/hardware) all take care of their own bits and (if you have
put things together properly) it will 'just work'. YAFFS just needs to
know how many bytes of the OOB it can use. However sometimes it is not
practical.
Option 2 lets MTD/hardware take care of the ECC so the filesystem
image just had to know which bytes to use for YAFFS Tags.
Option 3 is hardest as the image creator needs to know exactly what
ECC bytes, endianness and algorithm to use as well as which bytes are
available to YAFFS.
mkyaffs2image creates an image suitable for option 3 for the
particular case of yaffs2 on 2K page NAND with default MTD layout.
mkyaffsimage creates an equivalent image for 512b page NAND (i.e.
yaffs1 format).
Bootloaders
-----------
A bootloader using YAFFS needs to know how MTD is laying out the OOB
so that it can skip bad blocks.
YAFFS Tracing
-------------
/*
* YAFFS: Yet another Flash File System . A NAND-flash specific file system.
*
* Copyright (C) 2002-2007 Aleph One Ltd.
* for Toby Churchill Ltd and Brightstar Engineering
*
* Created by Charles Manning <charles@aleph1.co.uk>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License version 2.1 as
* published by the Free Software Foundation.
*
* Note: Only YAFFS headers are LGPL, YAFFS C code is covered by GPL.
*/
/*
* This file is just holds extra declarations used during development.
* Most of these are from kernel includes placed here so we can use them in
* applications.
*
*/
#ifndef __EXTRAS_H__
#define __EXTRAS_H__
#if defined WIN32
#define __inline__ __inline
#define new newHack
#endif
/* XXX U-BOOT XXX */
#if 1 /* !(defined __KERNEL__) || (defined WIN32) */
/* User space defines */
/* XXX U-BOOT XXX */
#if 0
typedef unsigned char __u8;
typedef unsigned short __u16;
typedef unsigned __u32;
#endif
#include <asm/types.h>
/*
* Simple doubly linked list implementation.
*
* Some of the internal functions ("__xxx") are useful when
* manipulating whole lists rather than single entries, as
* sometimes we already know the next/prev entries and we can
* generate better code by using them directly rather than
* using the generic single-entry routines.
*/
#define prefetch(x) 1
struct list_head {
struct list_head *next, *prev;
};
#define LIST_HEAD_INIT(name) { &(name), &(name) }
#define LIST_HEAD(name) \
struct list_head name = LIST_HEAD_INIT(name)
#define INIT_LIST_HEAD(ptr) do { \
(ptr)->next = (ptr); (ptr)->prev = (ptr); \
} while (0)
/*
* Insert a new entry between two known consecutive entries.
*
* This is only for internal list manipulation where we know
* the prev/next entries already!
*/
static __inline__ void __list_add(struct list_head *new,
struct list_head *prev,
struct list_head *next)
{
next->prev = new;
new->next = next;
new->prev = prev;
prev->next = new;
}
/**
* list_add - add a new entry
* @new: new entry to be added
* @head: list head to add it after
*
* Insert a new entry after the specified head.
* This is good for implementing stacks.
*/
static __inline__ void list_add(struct list_head *new, struct list_head *head)
{
__list_add(new, head, head->next);
}
/**
* list_add_tail - add a new entry
* @new: new entry to be added
* @head: list head to add it before
*
* Insert a new entry before the specified head.
* This is useful for implementing queues.
*/
static __inline__ void list_add_tail(struct list_head *new,
struct list_head *head)
{
__list_add(new, head->prev, head);
}
/*
* Delete a list entry by making the prev/next entries
* point to each other.
*
* This is only for internal list manipulation where we know
* the prev/next entries already!
*/
static __inline__ void __list_del(struct list_head *prev,
struct list_head *next)
{
next->prev = prev;
prev->next = next;
}
/**
* list_del - deletes entry from list.
* @entry: the element to delete from the list.
* Note: list_empty on entry does not return true after this, the entry is
* in an undefined state.
*/
static __inline__ void list_del(struct list_head *entry)
{
__list_del(entry->prev, entry->next);
}
/**
* list_del_init - deletes entry from list and reinitialize it.
* @entry: the element to delete from the list.
*/
static __inline__ void list_del_init(struct list_head *entry)
{
__list_del(entry->prev, entry->next);
INIT_LIST_HEAD(entry);
}
/**
* list_empty - tests whether a list is empty
* @head: the list to test.
*/
static __inline__ int list_empty(struct list_head *head)
{
return head->next == head;
}
/**
* list_splice - join two lists
* @list: the new list to add.
* @head: the place to add it in the first list.
*/
static __inline__ void list_splice(struct list_head *list,
struct list_head *head)
{
struct list_head *first = list->next;
if (first != list) {
struct list_head *last = list->prev;
struct list_head *at = head->next;
first->prev = head;
head->next = first;
last->next = at;
at->prev = last;
}
}
/**
* list_entry - get the struct for this entry
* @ptr: the &struct list_head pointer.
* @type: the type of the struct this is embedded in.
* @member: the name of the list_struct within the struct.
*/
#define list_entry(ptr, type, member) \
((type *)((char *)(ptr)-(unsigned long)(&((type *)0)->member)))
/**
* list_for_each - iterate over a list
* @pos: the &struct list_head to use as a loop counter.
* @head: the head for your list.
*/
#define list_for_each(pos, head) \
for (pos = (head)->next, prefetch(pos->next); pos != (head); \
pos = pos->next, prefetch(pos->next))
/**
* list_for_each_safe - iterate over a list safe against removal
* of list entry
* @pos: the &struct list_head to use as a loop counter.
* @n: another &struct list_head to use as temporary storage
* @head: the head for your list.
*/
#define list_for_each_safe(pos, n, head) \
for (pos = (head)->next, n = pos->next; pos != (head); \
pos = n, n = pos->next)
/*
* File types
*/
#define DT_UNKNOWN 0
#define DT_FIFO 1
#define DT_CHR 2
#define DT_DIR 4
#define DT_BLK 6
#define DT_REG 8
#define DT_LNK 10
#define DT_SOCK 12
#define DT_WHT 14
#ifndef WIN32
/* XXX U-BOOT XXX */
#if 0
#include <sys/stat.h>
#else
#include "common.h"
#endif
#endif
/*
* Attribute flags. These should be or-ed together to figure out what
* has been changed!
*/
#define ATTR_MODE 1
#define ATTR_UID 2
#define ATTR_GID 4
#define ATTR_SIZE 8
#define ATTR_ATIME 16
#define ATTR_MTIME 32
#define ATTR_CTIME 64
#define ATTR_ATIME_SET 128
#define ATTR_MTIME_SET 256
#define ATTR_FORCE 512 /* Not a change, but a change it */
#define ATTR_ATTR_FLAG 1024
struct iattr {
unsigned int ia_valid;
unsigned ia_mode;
unsigned ia_uid;
unsigned ia_gid;
unsigned ia_size;
unsigned ia_atime;
unsigned ia_mtime;
unsigned ia_ctime;
unsigned int ia_attr_flags;
};
#define KERN_DEBUG
#else
#ifndef WIN32
#include <linux/types.h>
#include <linux/list.h>
#include <linux/fs.h>
#include <linux/stat.h>
#endif
#endif
#if defined WIN32
#undef new
#endif
#endif
/* Dummy header for u-boot */
/* Dummy header for u-boot */
#include <linux/stddef.h>
#include <linux/string.h>
#include <linux/stat.h>
#include <common.h>
/*
* YAFFS: Yet Another Flash File System. A NAND-flash specific file system.
*
* Copyright (C) 2002-2011 Aleph One Ltd.
* for Toby Churchill Ltd and Brightstar Engineering
*
* Created by Charles Manning <charles@aleph1.co.uk>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*/
#include "yaffs_allocator.h"
#include "yaffs_guts.h"
#include "yaffs_trace.h"
#include "yportenv.h"
/*
* Each entry in yaffs_tnode_list and yaffs_obj_list hold blocks
* of approx 100 objects that are themn allocated singly.
* This is basically a simplified slab allocator.
*
* We don't use the Linux slab allocator because slab does not allow
* us to dump all the objects in one hit when we do a umount and tear
* down all the tnodes and objects. slab requires that we first free
* the individual objects.
*
* Once yaffs has been mainlined I shall try to motivate for a change
* to slab to provide the extra features we need here.
*/
struct yaffs_tnode_list {
struct yaffs_tnode_list *next;
struct yaffs_tnode *tnodes;
};
struct yaffs_obj_list {
struct yaffs_obj_list *next;
struct yaffs_obj *objects;
};
struct yaffs_allocator {
int n_tnodes_created;
struct yaffs_tnode *free_tnodes;
int n_free_tnodes;
struct yaffs_tnode_list *alloc_tnode_list;
int n_obj_created;
struct list_head free_objs;
int n_free_objects;
struct yaffs_obj_list *allocated_obj_list;
};
static void yaffs_deinit_raw_tnodes(struct yaffs_dev *dev)
{
struct yaffs_allocator *allocator =
(struct yaffs_allocator *)dev->allocator;
struct yaffs_tnode_list *tmp;
if (!allocator) {
BUG();
return;
}
while (allocator->alloc_tnode_list) {
tmp = allocator->alloc_tnode_list->next;
kfree(allocator->alloc_tnode_list->tnodes);
kfree(allocator->alloc_tnode_list);
allocator->alloc_tnode_list = tmp;
}
allocator->free_tnodes = NULL;
allocator->n_free_tnodes = 0;
allocator->n_tnodes_created = 0;
}
static void yaffs_init_raw_tnodes(struct yaffs_dev *dev)
{
struct yaffs_allocator *allocator = dev->allocator;
if (!allocator) {
BUG();
return;
}
allocator->alloc_tnode_list = NULL;
allocator->free_tnodes = NULL;
allocator->n_free_tnodes = 0;
allocator->n_tnodes_created = 0;
}
static int yaffs_create_tnodes(struct yaffs_dev *dev, int n_tnodes)
{
struct yaffs_allocator *allocator =
(struct yaffs_allocator *)dev->allocator;
int i;
struct yaffs_tnode *new_tnodes;
u8 *mem;
struct yaffs_tnode *curr;
struct yaffs_tnode *next;
struct yaffs_tnode_list *tnl;
if (!allocator) {
BUG();
return YAFFS_FAIL;
}
if (n_tnodes < 1)
return YAFFS_OK;
/* make these things */
new_tnodes = kmalloc(n_tnodes * dev->tnode_size, GFP_NOFS);
mem = (u8 *) new_tnodes;
if (!new_tnodes) {
yaffs_trace(YAFFS_TRACE_ERROR,
"yaffs: Could not allocate Tnodes");
return YAFFS_FAIL;
}
/* New hookup for wide tnodes */
for (i = 0; i < n_tnodes - 1; i++) {
curr = (struct yaffs_tnode *)&mem[i * dev->tnode_size];
next = (struct yaffs_tnode *)&mem[(i + 1) * dev->tnode_size];
curr->internal[0] = next;
}
curr = (struct yaffs_tnode *)&mem[(n_tnodes - 1) * dev->tnode_size];
curr->internal[0] = allocator->free_tnodes;
allocator->free_tnodes = (struct yaffs_tnode *)mem;
allocator->n_free_tnodes += n_tnodes;
allocator->n_tnodes_created += n_tnodes;
/* Now add this bunch of tnodes to a list for freeing up.
* NB If we can't add this to the management list it isn't fatal
* but it just means we can't free this bunch of tnodes later.
*/
tnl = kmalloc(sizeof(struct yaffs_tnode_list), GFP_NOFS);
if (!tnl) {
yaffs_trace(YAFFS_TRACE_ERROR,
"Could not add tnodes to management list");
return YAFFS_FAIL;
} else {
tnl->tnodes = new_tnodes;
tnl->next = allocator->alloc_tnode_list;
allocator->alloc_tnode_list = tnl;
}
yaffs_trace(YAFFS_TRACE_ALLOCATE, "Tnodes added");
return YAFFS_OK;
}
struct yaffs_tnode *yaffs_alloc_raw_tnode(struct yaffs_dev *dev)
{
struct yaffs_allocator *allocator =
(struct yaffs_allocator *)dev->allocator;
struct yaffs_tnode *tn = NULL;
if (!allocator) {
BUG();
return NULL;
}
/* If there are none left make more */
if (!allocator->free_tnodes)
yaffs_create_tnodes(dev, YAFFS_ALLOCATION_NTNODES);
if (allocator->free_tnodes) {
tn = allocator->free_tnodes;
allocator->free_tnodes = allocator->free_tnodes->internal[0];
allocator->n_free_tnodes--;
}
return tn;
}
/* FreeTnode frees up a tnode and puts it back on the free list */
void yaffs_free_raw_tnode(struct yaffs_dev *dev, struct yaffs_tnode *tn)
{
struct yaffs_allocator *allocator = dev->allocator;
if (!allocator) {
BUG();
return;
}
if (tn) {
tn->internal[0] = allocator->free_tnodes;
allocator->free_tnodes = tn;
allocator->n_free_tnodes++;
}
dev->checkpoint_blocks_required = 0; /* force recalculation */
}
/*--------------- yaffs_obj alloaction ------------------------