1. 15 Aug, 2013 2 commits
    • Yan, Zheng's avatar
      ceph: introduce i_truncate_mutex · b0d7c223
      Yan, Zheng authored
      I encountered below deadlock when running fsstress
      
      wmtruncate work      truncate                 MDS
      ---------------  ------------------  --------------------------
                         lock i_mutex
                                            <- truncate file
      lock i_mutex (blocked)
                                            <- revoking Fcb (filelock to MIX)
                         send request ->
                                               handle request (xlock filelock)
      
      At the initial time, there are some dirty pages in the page cache.
      When the kclient receives the truncate message, it reduces inode size
      and creates some 'out of i_size' dirty pages. wmtruncate work can't
      truncate these dirty pages because it's blocked by the i_mutex. Later
      when the kclient receives the cap message that revokes Fcb caps, It
      can't flush all dirty pages because writepages() only flushes dirty
      pages within the inode size.
      
      When the MDS handles the 'truncate' request from kclient, it waits
      for the filelock to become stable. But the filelock is stuck in
      unstable state because it can't finish revoking kclient's Fcb caps.
      
      The truncate pagecache locking has already caused lots of trouble
      for use. I think it's time simplify it by introducing a new mutex.
      We use the new mutex to prevent concurrent truncate_inode_pages().
      There is no need to worry about race between buffered write and
      truncate_inode_pages(), because our "get caps" mechanism prevents
      them from concurrent execution.
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
      Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      b0d7c223
    • Milosz Tanski's avatar
      ceph: cleanup the logic in ceph_invalidatepage · b150f5c1
      Milosz Tanski authored
      The invalidatepage code bails if it encounters a non-zero page offset. The
      current logic that does is non-obvious with multiple if statements.
      
      This should be logically and functionally equivalent.
      Signed-off-by: default avatarMilosz Tanski <milosz@adfin.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
      b150f5c1
  2. 10 Aug, 2013 12 commits
  3. 05 Jul, 2013 1 commit
  4. 03 Jul, 2013 16 commits
    • Yan, Zheng's avatar
      ceph: fix race between cap issue and revoke · 6ee6b953
      Yan, Zheng authored
      If we receive new caps from the auth MDS and the non-auth MDS is
      revoking the newly issued caps, we should release the caps from
      the non-auth MDS. The scenario is filelock's state changes from
      SYNC to LOCK. Non-auth MDS revokes Fc cap, the client gets Fc cap
      from the auth MDS at the same time.
      Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
      6ee6b953
    • Yan, Zheng's avatar
      ceph: fix cap revoke race · b1530f57
      Yan, Zheng authored
      If caps are been revoking by the auth MDS, don't consider them as
      issued even they are still issued by non-auth MDS. The non-auth
      MDS should also be revoking/exporting these caps, the client just
      hasn't received the cap revoke/export message.
      
      The race I encountered is: When caps are exporting to new MDS, the
      client receives cap import message and cap revoke message from the
      new MDS, then receives cap export message from the old MDS. When
      the client receives cap revoke message from the new MDS, the revoking
      caps are still issued by the old MDS, so the client does nothing.
      Later when the cap export message is received, the client removes
      the caps issued by the old MDS. (Another way to fix the race is
      calling ceph_check_caps() in handle_cap_export())
      Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
      b1530f57
    • Yan, Zheng's avatar
      ceph: fix pending vmtruncate race · b415bf4f
      Yan, Zheng authored
      The locking order for pending vmtruncate is wrong, it can lead to
      following race:
      
              write                  wmtruncate work
      ------------------------    ----------------------
      lock i_mutex
      check i_truncate_pending   check i_truncate_pending
      truncate_inode_pages()     lock i_mutex (blocked)
      copy data to page cache
      unlock i_mutex
                                 truncate_inode_pages()
      
      The fix is take i_mutex before calling __ceph_do_pending_vmtruncate()
      
      Fixes: http://tracker.ceph.com/issues/5453Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
      b415bf4f
    • Sasha Levin's avatar
      ceph: avoid accessing invalid memory · 54464296
      Sasha Levin authored
      when mounting ceph with a dev name that starts with a slash, ceph
      would attempt to access the character before that slash. Since we
      don't actually own that byte of memory, we would trigger an
      invalid access:
      
      [   43.499934] BUG: unable to handle kernel paging request at ffff880fa3a97fff
      [   43.500984] IP: [<ffffffff818f3884>] parse_mount_options+0x1a4/0x300
      [   43.501491] PGD 743b067 PUD 10283c4067 PMD 10282a6067 PTE 8000000fa3a97060
      [   43.502301] Oops: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
      [   43.503006] Dumping ftrace buffer:
      [   43.503596]    (ftrace buffer empty)
      [   43.504046] CPU: 0 PID: 10879 Comm: mount Tainted: G        W    3.10.0-sasha #1129
      [   43.504851] task: ffff880fa625b000 ti: ffff880fa3412000 task.ti: ffff880fa3412000
      [   43.505608] RIP: 0010:[<ffffffff818f3884>]  [<ffffffff818f3884>] parse_mount_options$
      [   43.506552] RSP: 0018:ffff880fa3413d08  EFLAGS: 00010286
      [   43.507133] RAX: ffff880fa3a98000 RBX: ffff880fa3a98000 RCX: 0000000000000000
      [   43.507893] RDX: ffff880fa3a98001 RSI: 000000000000002f RDI: ffff880fa3a98000
      [   43.508610] RBP: ffff880fa3413d58 R08: 0000000000001f99 R09: ffff880fa3fe64c0
      [   43.509426] R10: ffff880fa3413d98 R11: ffff880fa38710d8 R12: ffff880fa3413da0
      [   43.509792] R13: ffff880fa3a97fff R14: 0000000000000000 R15: ffff880fa3413d90
      [   43.509792] FS:  00007fa9c48757e0(0000) GS:ffff880fd2600000(0000) knlGS:000000000000$
      [   43.509792] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
      [   43.509792] CR2: ffff880fa3a97fff CR3: 0000000fa3bb9000 CR4: 00000000000006b0
      [   43.509792] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      [   43.509792] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      [   43.509792] Stack:
      [   43.509792]  0000e5180000000e ffffffff85ca1900 ffff880fa38710d8 ffff880fa3413d98
      [   43.509792]  0000000000000120 0000000000000000 ffff880fa3a98000 0000000000000000
      [   43.509792]  ffffffff85cf32a0 0000000000000000 ffff880fa3413dc8 ffffffff818f3c72
      [   43.509792] Call Trace:
      [   43.509792]  [<ffffffff818f3c72>] ceph_mount+0xa2/0x390
      [   43.509792]  [<ffffffff81226314>] ? pcpu_alloc+0x334/0x3c0
      [   43.509792]  [<ffffffff81282f8d>] mount_fs+0x8d/0x1a0
      [   43.509792]  [<ffffffff812263d0>] ? __alloc_percpu+0x10/0x20
      [   43.509792]  [<ffffffff8129f799>] vfs_kern_mount+0x79/0x100
      [   43.509792]  [<ffffffff812a224d>] do_new_mount+0xcd/0x1c0
      [   43.509792]  [<ffffffff812a2e8d>] do_mount+0x15d/0x210
      [   43.509792]  [<ffffffff81220e55>] ? strndup_user+0x45/0x60
      [   43.509792]  [<ffffffff812a2fdd>] SyS_mount+0x9d/0xe0
      [   43.509792]  [<ffffffff83fd816c>] tracesys+0xdd/0xe2
      [   43.509792] Code: 4c 8b 5d c0 74 0a 48 8d 50 01 49 89 14 24 eb 17 31 c0 48 83 c9 ff $
      [   43.509792] RIP  [<ffffffff818f3884>] parse_mount_options+0x1a4/0x300
      [   43.509792]  RSP <ffff880fa3413d08>
      [   43.509792] CR2: ffff880fa3a97fff
      [   43.509792] ---[ end trace 22469cd81e93af51 ]---
      Signed-off-by: default avatarSasha Levin <sasha.levin@oracle.com>
      Reviewed-by: default avatarSage Weil <sage@inktan.com>
      54464296
    • majianpeng's avatar
      ceph: Reconstruct the func ceph_reserve_caps. · 93faca6e
      majianpeng authored
      Drop ignored return value.  Fix allocation failure case to not leak.
      Signed-off-by: default avatarJianpeng Ma <majianpeng@gmail.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
      93faca6e
    • majianpeng's avatar
      fb3101b6
    • Jianpeng Ma's avatar
      ceph: remove sb_start/end_write in ceph_aio_write. · 0405a149
      Jianpeng Ma authored
      Either in vfs_write or io_submit,it call file_start/end_write.
      The different between file_start/end_write and sb_start/end_write is
      file_ only handle regular file.But i think in ceph_aio_write,it only
      for regular file.
      Signed-off-by: default avatarJianpeng Ma <majianpeng@gmail.com>
      Acked-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      0405a149
    • majianpeng's avatar
    • majianpeng's avatar
      ceph: fix sleeping function called from invalid context. · a1dc1937
      majianpeng authored
      [ 1121.231883] BUG: sleeping function called from invalid context at kernel/rwsem.c:20
      [ 1121.231935] in_atomic(): 1, irqs_disabled(): 0, pid: 9831, name: mv
      [ 1121.231971] 1 lock held by mv/9831:
      [ 1121.231973]  #0:  (&(&ci->i_ceph_lock)->rlock){+.+...},at:[<ffffffffa02bbd38>] ceph_getxattr+0x58/0x1d0 [ceph]
      [ 1121.231998] CPU: 3 PID: 9831 Comm: mv Not tainted 3.10.0-rc6+ #215
      [ 1121.232000] Hardware name: To Be Filled By O.E.M. To Be Filled By
      O.E.M./To be filled by O.E.M., BIOS 080015  11/09/2011
      [ 1121.232027]  ffff88006d355a80 ffff880092f69ce0 ffffffff8168348c ffff880092f69cf8
      [ 1121.232045]  ffffffff81070435 ffff88006d355a20 ffff880092f69d20 ffffffff816899ba
      [ 1121.232052]  0000000300000004 ffff8800b76911d0 ffff88006d355a20 ffff880092f69d68
      [ 1121.232056] Call Trace:
      [ 1121.232062]  [<ffffffff8168348c>] dump_stack+0x19/0x1b
      [ 1121.232067]  [<ffffffff81070435>] __might_sleep+0xe5/0x110
      [ 1121.232071]  [<ffffffff816899ba>] down_read+0x2a/0x98
      [ 1121.232080]  [<ffffffffa02baf70>] ceph_vxattrcb_layout+0x60/0xf0 [ceph]
      [ 1121.232088]  [<ffffffffa02bbd7f>] ceph_getxattr+0x9f/0x1d0 [ceph]
      [ 1121.232093]  [<ffffffff81188d28>] vfs_getxattr+0xa8/0xd0
      [ 1121.232097]  [<ffffffff8118900b>] getxattr+0xab/0x1c0
      [ 1121.232100]  [<ffffffff811704f2>] ? final_putname+0x22/0x50
      [ 1121.232104]  [<ffffffff81155f80>] ? kmem_cache_free+0xb0/0x260
      [ 1121.232107]  [<ffffffff811704f2>] ? final_putname+0x22/0x50
      [ 1121.232110]  [<ffffffff8109e63d>] ? trace_hardirqs_on+0xd/0x10
      [ 1121.232114]  [<ffffffff816957a7>] ? sysret_check+0x1b/0x56
      [ 1121.232120]  [<ffffffff81189c9c>] SyS_fgetxattr+0x6c/0xc0
      [ 1121.232125]  [<ffffffff81695782>] system_call_fastpath+0x16/0x1b
      [ 1121.232129] BUG: scheduling while atomic: mv/9831/0x10000002
      [ 1121.232154] 1 lock held by mv/9831:
      [ 1121.232156]  #0:  (&(&ci->i_ceph_lock)->rlock){+.+...}, at:
      [<ffffffffa02bbd38>] ceph_getxattr+0x58/0x1d0 [ceph]
      
      I think move the ci->i_ceph_lock down is safe because we can't free
      ceph_inode_info at there.
      
      CC: stable@vger.kernel.org  # 3.8+
      Signed-off-by: default avatarJianpeng Ma <majianpeng@gmail.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
      a1dc1937
    • Yan, Zheng's avatar
    • Yan, Zheng's avatar
      ceph: clear migrate seq when MDS restarts · 667ca05c
      Yan, Zheng authored
      Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
      667ca05c
    • Yan, Zheng's avatar
      ceph: check migrate seq before changing auth cap · b8c2f3ae
      Yan, Zheng authored
      We may receive old request reply from the exporter MDS after receiving
      the importer MDS' cap import message.
      Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
      b8c2f3ae
    • Yan, Zheng's avatar
      ceph: fix race between page writeback and truncate · fc2744aa
      Yan, Zheng authored
      The client can receive truncate request from MDS at any time.
      So the page writeback code need to get i_size, truncate_seq and
      truncate_size atomically
      Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
      fc2744aa
    • Yan, Zheng's avatar
      3803da49
    • Yan, Zheng's avatar
      ceph: fix cap release race · bb137f84
      Yan, Zheng authored
      ceph_encode_inode_release() can race with ceph_open() and release
      caps wanted by open files. So it should call __ceph_caps_wanted()
      to get the wanted caps.
      Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
      bb137f84
    • Jie Liu's avatar
      vfs: export lseek_execute() to modules · 46a1c2c7
      Jie Liu authored
      For those file systems(btrfs/ext4/ocfs2/tmpfs) that support
      SEEK_DATA/SEEK_HOLE functions, we end up handling the similar
      matter in lseek_execute() to update the current file offset
      to the desired offset if it is valid, ceph also does the
      simliar things at ceph_llseek().
      
      To reduce the duplications, this patch make lseek_execute()
      public accessible so that we can call it directly from the
      underlying file systems.
      
      Thanks Dave Chinner for this suggestion.
      
      [AV: call it vfs_setpos(), don't bring the removed 'inode' argument back]
      
      v2->v1:
      - Add kernel-doc comments for lseek_execute()
      - Call lseek_execute() in ceph->llseek()
      Signed-off-by: default avatarJie Liu <jeff.liu@oracle.com>
      Cc: Dave Chinner <dchinner@redhat.com>
      Cc: Al Viro <viro@zeniv.linux.org.uk>
      Cc: Andi Kleen <andi@firstfloor.org>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Christoph Hellwig <hch@lst.de>
      Cc: Chris Mason <chris.mason@fusionio.com>
      Cc: Josef Bacik <jbacik@fusionio.com>
      Cc: Ben Myers <bpm@sgi.com>
      Cc: Ted Tso <tytso@mit.edu>
      Cc: Hugh Dickins <hughd@google.com>
      Cc: Mark Fasheh <mfasheh@suse.com>
      Cc: Joel Becker <jlbec@evilplan.org>
      Cc: Sage Weil <sage@inktank.com>
      Signed-off-by: default avatarAl Viro <viro@zeniv.linux.org.uk>
      46a1c2c7
  5. 01 Jul, 2013 3 commits
  6. 29 Jun, 2013 2 commits
    • Jeff Layton's avatar
      locks: protect most of the file_lock handling with i_lock · 1c8c601a
      Jeff Layton authored
      Having a global lock that protects all of this code is a clear
      scalability problem. Instead of doing that, move most of the code to be
      protected by the i_lock instead. The exceptions are the global lists
      that the ->fl_link sits on, and the ->fl_block list.
      
      ->fl_link is what connects these structures to the
      global lists, so we must ensure that we hold those locks when iterating
      over or updating these lists.
      
      Furthermore, sound deadlock detection requires that we hold the
      blocked_list state steady while checking for loops. We also must ensure
      that the search and update to the list are atomic.
      
      For the checking and insertion side of the blocked_list, push the
      acquisition of the global lock into __posix_lock_file and ensure that
      checking and update of the  blocked_list is done without dropping the
      lock in between.
      
      On the removal side, when waking up blocked lock waiters, take the
      global lock before walking the blocked list and dequeue the waiters from
      the global list prior to removal from the fl_block list.
      
      With this, deadlock detection should be race free while we minimize
      excessive file_lock_lock thrashing.
      
      Finally, in order to avoid a lock inversion problem when handling
      /proc/locks output we must ensure that manipulations of the fl_block
      list are also protected by the file_lock_lock.
      Signed-off-by: default avatarJeff Layton <jlayton@redhat.com>
      Signed-off-by: default avatarAl Viro <viro@zeniv.linux.org.uk>
      1c8c601a
    • Al Viro's avatar
      [readdir] convert ceph · 77acfa29
      Al Viro authored
      Signed-off-by: default avatarAl Viro <viro@zeniv.linux.org.uk>
      77acfa29
  7. 22 May, 2013 2 commits
    • Lukas Czerner's avatar
      ceph: use ->invalidatepage() length argument · 569d39fc
      Lukas Czerner authored
      ->invalidatepage() aop now accepts range to invalidate so we can make
      use of it in ceph_invalidatepage().
      Signed-off-by: default avatarLukas Czerner <lczerner@redhat.com>
      Acked-by: default avatarSage Weil <sage@inktank.com>
      Cc: ceph-devel@vger.kernel.org
      569d39fc
    • Lukas Czerner's avatar
      mm: change invalidatepage prototype to accept length · d47992f8
      Lukas Czerner authored
      Currently there is no way to truncate partial page where the end
      truncate point is not at the end of the page. This is because it was not
      needed and the functionality was enough for file system truncate
      operation to work properly. However more file systems now support punch
      hole feature and it can benefit from mm supporting truncating page just
      up to the certain point.
      
      Specifically, with this functionality truncate_inode_pages_range() can
      be changed so it supports truncating partial page at the end of the
      range (currently it will BUG_ON() if 'end' is not at the end of the
      page).
      
      This commit changes the invalidatepage() address space operation
      prototype to accept range to be invalidated and update all the instances
      for it.
      
      We also change the block_invalidatepage() in the same way and actually
      make a use of the new length argument implementing range invalidation.
      
      Actual file system implementations will follow except the file systems
      where the changes are really simple and should not change the behaviour
      in any way .Implementation for truncate_page_range() which will be able
      to accept page unaligned ranges will follow as well.
      Signed-off-by: default avatarLukas Czerner <lczerner@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Hugh Dickins <hughd@google.com>
      d47992f8
  8. 17 May, 2013 2 commits
    • Jim Schutt's avatar
      ceph: ceph_pagelist_append might sleep while atomic · 39be95e9
      Jim Schutt authored
      Ceph's encode_caps_cb() worked hard to not call __page_cache_alloc()
      while holding a lock, but it's spoiled because ceph_pagelist_addpage()
      always calls kmap(), which might sleep.  Here's the result:
      
      [13439.295457] ceph: mds0 reconnect start
      [13439.300572] BUG: sleeping function called from invalid context at include/linux/highmem.h:58
      [13439.309243] in_atomic(): 1, irqs_disabled(): 0, pid: 12059, name: kworker/1:1
          . . .
      [13439.376225] Call Trace:
      [13439.378757]  [<ffffffff81076f4c>] __might_sleep+0xfc/0x110
      [13439.384353]  [<ffffffffa03f4ce0>] ceph_pagelist_append+0x120/0x1b0 [libceph]
      [13439.391491]  [<ffffffffa0448fe9>] ceph_encode_locks+0x89/0x190 [ceph]
      [13439.398035]  [<ffffffff814ee849>] ? _raw_spin_lock+0x49/0x50
      [13439.403775]  [<ffffffff811cadf5>] ? lock_flocks+0x15/0x20
      [13439.409277]  [<ffffffffa045e2af>] encode_caps_cb+0x41f/0x4a0 [ceph]
      [13439.415622]  [<ffffffff81196748>] ? igrab+0x28/0x70
      [13439.420610]  [<ffffffffa045e9f8>] ? iterate_session_caps+0xe8/0x250 [ceph]
      [13439.427584]  [<ffffffffa045ea25>] iterate_session_caps+0x115/0x250 [ceph]
      [13439.434499]  [<ffffffffa045de90>] ? set_request_path_attr+0x2d0/0x2d0 [ceph]
      [13439.441646]  [<ffffffffa0462888>] send_mds_reconnect+0x238/0x450 [ceph]
      [13439.448363]  [<ffffffffa0464542>] ? ceph_mdsmap_decode+0x5e2/0x770 [ceph]
      [13439.455250]  [<ffffffffa0462e42>] check_new_map+0x352/0x500 [ceph]
      [13439.461534]  [<ffffffffa04631ad>] ceph_mdsc_handle_map+0x1bd/0x260 [ceph]
      [13439.468432]  [<ffffffff814ebc7e>] ? mutex_unlock+0xe/0x10
      [13439.473934]  [<ffffffffa043c612>] extra_mon_dispatch+0x22/0x30 [ceph]
      [13439.480464]  [<ffffffffa03f6c2c>] dispatch+0xbc/0x110 [libceph]
      [13439.486492]  [<ffffffffa03eec3d>] process_message+0x1ad/0x1d0 [libceph]
      [13439.493190]  [<ffffffffa03f1498>] ? read_partial_message+0x3e8/0x520 [libceph]
          . . .
      [13439.587132] ceph: mds0 reconnect success
      [13490.720032] ceph: mds0 caps stale
      [13501.235257] ceph: mds0 recovery completed
      [13501.300419] ceph: mds0 caps renewed
      
      Fix it up by encoding locks into a buffer first, and when the number
      of encoded locks is stable, copy that into a ceph_pagelist.
      
      [elder@inktank.com: abbreviated the stack info a bit.]
      
      Cc: stable@vger.kernel.org # 3.4+
      Signed-off-by: default avatarJim Schutt <jaschut@sandia.gov>
      Reviewed-by: default avatarAlex Elder <elder@inktank.com>
      39be95e9
    • Jim Schutt's avatar
      ceph: add cpu_to_le32() calls when encoding a reconnect capability · c420276a
      Jim Schutt authored
      In his review, Alex Elder mentioned that he hadn't checked that
      num_fcntl_locks and num_flock_locks were properly decoded on the
      server side, from a le32 over-the-wire type to a cpu type.
      I checked, and AFAICS it is done; those interested can consult
          Locker::_do_cap_update()
      in src/mds/Locker.cc and src/include/encoding.h in the Ceph server
      code (git://github.com/ceph/ceph).
      
      I also checked the server side for flock_len decoding, and I believe
      that also happens correctly, by virtue of having been declared
      __le32 in struct ceph_mds_cap_reconnect, in src/include/ceph_fs.h.
      
      Cc: stable@vger.kernel.org # 3.4+
      Signed-off-by: default avatarJim Schutt <jaschut@sandia.gov>
      Reviewed-by: default avatarAlex Elder <elder@inktank.com>
      c420276a