1. 14 Jun, 2010 1 commit
  2. 05 Jun, 2010 1 commit
  3. 21 May, 2010 1 commit
  4. 17 May, 2010 5 commits
  5. 16 May, 2010 6 commits
    • Theodore Ts'o's avatar
      ext4: Add new abstraction ext4_map_blocks() underneath ext4_get_blocks() · e35fd660
      Theodore Ts'o authored
      
      
      Jack up ext4_get_blocks() and add a new function, ext4_map_blocks()
      which uses a much smaller structure, struct ext4_map_blocks which is
      20 bytes, as opposed to a struct buffer_head, which nearly 5 times
      bigger on an x86_64 machine.  By switching things to use
      ext4_map_blocks(), we can save stack space by using ext4_map_blocks()
      since we can avoid allocating a struct buffer_head on the stack.
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      e35fd660
    • Theodore Ts'o's avatar
      ext4: Use our own write_cache_pages() · 8e48dcfb
      Theodore Ts'o authored
      
      
      Make a copy of write_cache_pages() for the benefit of
      ext4_da_writepages().  This allows us to simplify the code some, and
      will allow us to further customize the code in future patches.
      
      There are some nasty hacks in write_cache_pages(), which Linus has
      (correctly) characterized as vile.  I've just copied it into
      write_cache_pages_da(), without trying to clean those bits up lest I
      break something in the ext4's delalloc implementation, which is a bit
      fragile right now.  This will allow Dave Chinner to clean up
      write_cache_pages() in mm/page-writeback.c, without worrying about
      breaking ext4.  Eventually write_cache_pages_da() will go away when I
      rewrite ext4's delayed allocation and create a general
      ext4_writepages() which is used for all of ext4's writeback.  Until
      now this is the lowest risk way to clean up the core
      write_cache_pages() function.
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      Cc: Dave Chinner <david@fromorbit.com>
      8e48dcfb
    • Curt Wohlgemuth's avatar
      ext4: Remove extraneous newlines in ext4_msg() calls · fbe845dd
      Curt Wohlgemuth authored
      
      
      Addresses-Google-Bug: #2562325
      Signed-off-by: default avatarCurt Wohlgemuth <curtw@google.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      fbe845dd
    • Eric Sandeen's avatar
      ext4: don't use quota reservation for speculative metadata · 72b8ab9d
      Eric Sandeen authored
      
      
      Because we can badly over-reserve metadata when we
      calculate worst-case, it complicates things for quota, since
      we must reserve and then claim later, retry on EDQUOT, etc.
      Quota is also a generally smaller pool than fs free blocks,
      so this over-reservation hurts more, and more often.
      
      I'm of the opinion that it's not the worst thing to allow
      metadata to push a user slightly over quota.  This simplifies
      the code and avoids the false quota rejections that result
      from worst-case speculation.
      
      This patch stops the speculative quota-charging for
      worst-case metadata requirements, and just charges quota
      when the blocks are allocated at writeout.  It also is
      able to remove the try-again loop on EDQUOT.
      
      This patch has been tested indirectly by running the xfstests
      suite with a hack to mount & enable quota prior to the test.
      
      I also did a more specific test of fragmenting freespace
      and then doing a large delalloc write under quota; quota
      stopped me at the right amount of file IO, and then the
      writeout generated enough metadata (due to the fragmentation)
      that it put me slightly over quota, as expected.
      Signed-off-by: default avatarEric Sandeen <sandeen@redhat.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      72b8ab9d
    • Eric Sandeen's avatar
      ext4: don't scan/accumulate more pages than mballoc will allocate · c445e3e0
      Eric Sandeen authored
      
      
      There was a bug reported on RHEL5 that a 10G dd on a 12G box
      had a very, very slow sync after that.
      
      At issue was the loop in write_cache_pages scanning all the way
      to the end of the 10G file, even though the subsequent call
      to mpage_da_submit_io would only actually write a smallish amt; then
      we went back to the write_cache_pages loop ... wasting tons of time
      in calling __mpage_da_writepage for thousands of pages we would
      just revisit (many times) later.
      
      Upstream it's not such a big issue for sys_sync because we get
      to the loop with a much smaller nr_to_write, which limits the loop.
      
      However, talking with Aneesh he realized that fsync upstream still
      gets here with a very large nr_to_write and we face the same problem.
      
      This patch makes mpage_add_bh_to_extent stop the loop after we've
      accumulated 2048 pages, by setting mpd->io_done = 1; which ultimately
      causes the write_cache_pages loop to break.
      
      Repeating the test with a dirty_ratio of 80 (to leave something for
      fsync to do), I don't see huge IO performance gains, but the reduction
      in cpu usage is striking: 80% usage with stock, and 2% with the
      below patch.  Instrumenting the loop in write_cache_pages clearly
      shows that we are wasting time here.
      
      Eventually we need to change mpage_da_map_pages() also submit its I/O
      to the block layer, subsuming mpage_da_submit_io(), and then change it
      call ext4_get_blocks() multiple times.
      Signed-off-by: default avatarEric Sandeen <sandeen@redhat.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      c445e3e0
    • Dmitry Monakhov's avatar
      ext4: fix quota accounting in case of fallocate · 35121c98
      Dmitry Monakhov authored
      
      
      allocated_meta_data is already included in 'used' variable.
      Signed-off-by: default avatarDmitry Monakhov <dmonakhov@openvz.org>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      35121c98
  6. 03 Apr, 2010 2 commits
    • Curt Wohlgemuth's avatar
      ext4: Fix buffer head leaks after calls to ext4_get_inode_loc() · fd2dd9fb
      Curt Wohlgemuth authored
      
      
      Calls to ext4_get_inode_loc() returns with a reference to a buffer
      head in iloc->bh.  The callers of this function in ext4_write_inode()
      when in no journal mode and in ext4_xattr_fiemap() don't release the
      buffer head after using it.
      
      Addresses-Google-Bug: #2548165
      Signed-off-by: default avatarCurt Wohlgemuth <curtw@google.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      fd2dd9fb
    • Curt Wohlgemuth's avatar
      ext4: Fix possible lost inode write in no journal mode · 8b472d73
      Curt Wohlgemuth authored
      
      
      In the no-journal case, ext4_write_inode() will fetch the bh and call
      sync_dirty_buffer() on it.  However, if the bh has already been
      written and the bh reclaimed for some other purpose, AND if the inode
      is the only one in the inode table block in use, then
      ext4_get_inode_loc() will not read the inode table block from disk,
      but as an optimization, fill the block with zero's assuming that its
      caller will copy in the on-disk version of the inode.  This is not
      done by ext4_write_inode(), so the contents of the inode can simply
      get lost.  The fix is to use __ext4_get_inode_loc() with in_mem set to
      0, instead of ext4_get_inode_loc().  Long term the API needs to be
      fixed so it's obvious why latter is not safe.
      
      Addresses-Google-Bug: #2526446
      Signed-off-by: default avatarCurt Wohlgemuth <curtw@google.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      8b472d73
  7. 30 Mar, 2010 1 commit
    • Tejun Heo's avatar
      include cleanup: Update gfp.h and slab.h includes to prepare for breaking... · 5a0e3ad6
      Tejun Heo authored
      include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
      
      percpu.h is included by sched.h and module.h and thus ends up being
      included when building most .c files.  percpu.h includes slab.h which
      in turn includes gfp.h making everything defined by the two files
      universally available and complicating inclusion dependencies.
      
      percpu.h -> slab.h dependency is about to be removed.  Prepare for
      this change by updating users of gfp and slab facilities include those
      headers directly instead of assuming availability.  As this conversion
      needs to touch large number of source files, the following script is
      used as the basis of conversion.
      
        http://userweb.kernel.org/~tj/misc/slabh-sweep.py
      
      The script does the followings.
      
      * Scan files for gfp and slab usages and update includes such that
        only the necessary includes are there.  ie. if only gfp is used,
        gfp.h, if slab is used, slab.h.
      
      * When the script inserts a new include, it looks at the include
        blocks and try to put the new include such that its order conforms
        to its surrounding.  It's put in the include block which contains
        core kernel includes, in the same order that the rest are ordered -
        alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
        doesn't seem to be any matching order.
      
      * If the script can't find a place to put a new include (mostly
        because the file doesn't have fitting include block), it prints out
        an error message indicating which .h file needs to be added to the
        file.
      
      The conversion was done in the following steps.
      
      1. The initial automatic conversion of all .c files updated slightly
         over 4000 files, deleting around 700 includes and adding ~480 gfp.h
         and ~3000 slab.h inclusions.  The script emitted errors for ~400
         files.
      
      2. Each error was manually checked.  Some didn't need the inclusion,
         some needed manual addition while adding it to implementation .h or
         embedding .c file was more appropriate for others.  This step added
         inclusions to around 150 files.
      
      3. The script was run again and the output was compared to the edits
         from #2
      
       to make sure no file was left behind.
      
      4. Several build tests were done and a couple of problems were fixed.
         e.g. lib/decompress_*.c used malloc/free() wrappers around slab
         APIs requiring slab.h to be added manually.
      
      5. The script was run on all .h files but without automatically
         editing them as sprinkling gfp.h and slab.h inclusions around .h
         files could easily lead to inclusion dependency hell.  Most gfp.h
         inclusion directives were ignored as stuff from gfp.h was usually
         wildly available and often used in preprocessor macros.  Each
         slab.h inclusion directive was examined and added manually as
         necessary.
      
      6. percpu.h was updated not to include slab.h.
      
      7. Build test were done on the following configurations and failures
         were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
         distributed build env didn't work with gcov compiles) and a few
         more options had to be turned off depending on archs to make things
         build (like ipr on powerpc/64 which failed due to missing writeq).
      
         * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
         * powerpc and powerpc64 SMP allmodconfig
         * sparc and sparc64 SMP allmodconfig
         * ia64 SMP allmodconfig
         * s390 SMP allmodconfig
         * alpha SMP allmodconfig
         * um on x86_64 SMP allmodconfig
      
      8. percpu.h modifications were reverted so that it could be applied as
         a separate patch and serve as bisection point.
      
      Given the fact that I had only a couple of failures from tests on step
      6, I'm fairly confident about the coverage of this conversion patch.
      If there is a breakage, it's likely to be something in one of the arch
      headers which should be easily discoverable easily on most builds of
      the specific arch.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Guess-its-ok-by: default avatarChristoph Lameter <cl@linux-foundation.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
      5a0e3ad6
  8. 14 Mar, 2010 1 commit
  9. 05 Mar, 2010 1 commit
  10. 04 Mar, 2010 4 commits
    • Christoph Hellwig's avatar
      dquot: cleanup dquot initialize routine · 871a2931
      Christoph Hellwig authored
      
      
      Get rid of the initialize dquot operation - it is now always called from
      the filesystem and if a filesystem really needs it's own (which none
      currently does) it can just call into it's own routine directly.
      
      Rename the now static low-level dquot_initialize helper to __dquot_initialize
      and vfs_dq_init to dquot_initialize to have a consistent namespace.
      Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
      Signed-off-by: default avatarJan Kara <jack@suse.cz>
      871a2931
    • Christoph Hellwig's avatar
      dquot: move dquot initialization responsibility into the filesystem · 907f4554
      Christoph Hellwig authored
      
      
      Currently various places in the VFS call vfs_dq_init directly.  This means
      we tie the quota code into the VFS.  Get rid of that and make the
      filesystem responsible for the initialization.   For most metadata operations
      this is a straight forward move into the methods, but for truncate and
      open it's a bit more complicated.
      
      For truncate we currently only call vfs_dq_init for the sys_truncate case
      because open already takes care of it for ftruncate and open(O_TRUNC) - the
      new code causes an additional vfs_dq_init for those which is harmless.
      
      For open the initialization is moved from do_filp_open into the open method,
      which means it happens slightly earlier now, and only for regular files.
      The latter is fine because we don't need to initialize it for operations
      on special files, and we already do it as part of the namespace operations
      for directories.
      
      Add a dquot_file_open helper that filesystems that support generic quotas
      can use to fill in ->open.
      Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
      Signed-off-by: default avatarJan Kara <jack@suse.cz>
      907f4554
    • Christoph Hellwig's avatar
      dquot: cleanup dquot transfer routine · b43fa828
      Christoph Hellwig authored
      
      
      Get rid of the transfer dquot operation - it is now always called from
      the filesystem and if a filesystem really needs it's own (which none
      currently does) it can just call into it's own routine directly.
      
      Rename the now static low-level dquot_transfer helper to __dquot_transfer
      and vfs_dq_transfer to dquot_transfer to have a consistent namespace,
      and make the new dquot_transfer return a normal negative errno value
      which all callers expect.
      Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
      Signed-off-by: default avatarJan Kara <jack@suse.cz>
      b43fa828
    • Christoph Hellwig's avatar
      dquot: cleanup space allocation / freeing routines · 5dd4056d
      Christoph Hellwig authored
      
      
      Get rid of the alloc_space, free_space, reserve_space, claim_space and
      release_rsv dquot operations - they are always called from the filesystem
      and if a filesystem really needs their own (which none currently does)
      it can just call into it's own routine directly.
      
      Move shared logic into the common __dquot_alloc_space,
      dquot_claim_space_nodirty and __dquot_free_space low-level methods,
      and rationalize the wrappers around it to move as much as possible
      code into the common block for CONFIG_QUOTA vs not.  Also rename
      all these helpers to be named dquot_* instead of vfs_dq_*.
      Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
      Signed-off-by: default avatarJan Kara <jack@suse.cz>
      5dd4056d
  11. 03 Mar, 2010 1 commit
  12. 02 Mar, 2010 2 commits
  13. 04 Mar, 2010 1 commit
    • Jiaying Zhang's avatar
      ext4: use ext4_get_block_write in buffer write · 744692dc
      Jiaying Zhang authored
      
      
      Allocate uninitialized extent before ext4 buffer write and
      convert the extent to initialized after io completes.
      The purpose is to make sure an extent can only be marked
      initialized after it has been written with new data so
      we can safely drop the i_mutex lock in ext4 DIO read without
      exposing stale data. This helps to improve multi-thread DIO
      read performance on high-speed disks.
      
      Skip the nobh and data=journal mount cases to make things simple for now.
      Signed-off-by: default avatarJiaying Zhang <jiayingz@google.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      744692dc
  14. 02 Mar, 2010 2 commits
  15. 24 Feb, 2010 1 commit
  16. 16 Feb, 2010 1 commit
    • Curt Wohlgemuth's avatar
      ext4: Fix BUG_ON at fs/buffer.c:652 in no journal mode · 73b50c1c
      Curt Wohlgemuth authored
      
      
      Calls to ext4_handle_dirty_metadata should only pass in an inode
      pointer for inode-specific metadata, and not for shared metadata
      blocks such as inode table blocks, block group descriptors, the
      superblock, etc.
      
      The BUG_ON can get tripped when updating a special device (such as a
      block device) that is opened (so that i_mapping is set in
      fs/block_dev.c) and the file system is mounted in no journal mode.
      
      Addresses-Google-Bug: #2404870
      Signed-off-by: default avatarCurt Wohlgemuth <curtw@google.com>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      73b50c1c
  17. 15 Feb, 2010 1 commit
  18. 24 Jan, 2010 1 commit
    • Theodore Ts'o's avatar
      ext4: Use bitops to read/modify EXT4_I(inode)->i_state · 19f5fb7a
      Theodore Ts'o authored
      
      
      At several places we modify EXT4_I(inode)->i_state without holding
      i_mutex (ext4_release_file, ext4_bmap, ext4_journalled_writepage,
      ext4_do_update_inode, ...). These modifications are racy and we can
      lose updates to i_state. So convert handling of i_state to use bitops
      which are atomic.
      
      Cc: Jan Kara <jack@suse.cz>
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      19f5fb7a
  19. 15 Jan, 2010 1 commit
  20. 25 Jan, 2010 1 commit
  21. 22 Jan, 2010 1 commit
  22. 01 Jan, 2010 2 commits
    • Theodore Ts'o's avatar
      ext4: Calculate metadata requirements more accurately · 9d0be502
      Theodore Ts'o authored
      In the past, ext4_calc_metadata_amount(), and its sub-functions
      ext4_ext_calc_metadata_amount() and ext4_indirect_calc_metadata_amount()
      badly over-estimated the number of metadata blocks that might be
      required for delayed allocation blocks.  This didn't matter as much
      when functions which managed the reserved metadata blocks were more
      aggressive about dropping reserved metadata blocks as delayed
      allocation blocks were written, but unfortunately they were too
      aggressive.  This was fixed in commit 0637c6f4
      
      , but as a result the
      over-estimation by ext4_calc_metadata_amount() would lead to reserving
      2-3 times the number of pending delayed allocation blocks as
      potentially required metadata blocks.  So if there are 1 megabytes of
      blocks which have been not yet been allocation, up to 3 megabytes of
      space would get reserved out of the user's quota and from the file
      system free space pool until all of the inode's data blocks have been
      allocated.
      
      This commit addresses this problem by much more accurately estimating
      the number of metadata blocks that will be required.  It will still
      somewhat over-estimate the number of blocks needed, since it must make
      a worst case estimate not knowing which physical blocks will be
      needed, but it is much more accurate than before.
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      9d0be502
    • Theodore Ts'o's avatar
      ext4: Fix accounting of reserved metadata blocks · ee5f4d9c
      Theodore Ts'o authored
      Commit 0637c6f4
      
       had a typo which caused the reserved metadata blocks to
      not be released correctly.   Fix this.
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      ee5f4d9c
  23. 30 Dec, 2009 1 commit
    • Theodore Ts'o's avatar
      ext4: Patch up how we claim metadata blocks for quota purposes · 0637c6f4
      Theodore Ts'o authored
      As reported in Kernel Bugzilla #14936, commit d21cd8f1 triggered a BUG
      in the function ext4_da_update_reserve_space() found in
      fs/ext4/inode.c.  The root cause of this BUG() was caused by the fact
      that ext4_calc_metadata_amount() can severely over-estimate how many
      metadata blocks will be needed, especially when using direct
      block-mapped files.
      
      In addition, it can also badly *under* estimate how much space is
      needed, since ext4_calc_metadata_amount() assumes that the blocks are
      contiguous, and this is not always true.  If the application is
      writing blocks to a sparse file, the number of metadata blocks
      necessary can be severly underestimated by the functions
      ext4_da_reserve_space(), ext4_da_update_reserve_space() and
      ext4_da_release_space().  This was the cause of the dq_claim_space
      reports found on kerneloops.org.
      
      Unfortunately, doing this right means that we need to massively
      over-estimate the amount of free space needed.  So in some cases we
      may need to force the inode to be written to disk asynchronously in
      to avoid spurious quota failures.
      
      http://bugzilla.kernel.org/show_bug.cgi?id=14936
      
      Signed-off-by: default avatar"Theodore Ts'o" <tytso@mit.edu>
      0637c6f4
  24. 25 Dec, 2009 1 commit