1. 10 Aug, 2013 9 commits
  2. 03 Jul, 2013 15 commits
    • Yan, Zheng's avatar
      ceph: fix race between cap issue and revoke · 6ee6b953
      Yan, Zheng authored
      If we receive new caps from the auth MDS and the non-auth MDS is
      revoking the newly issued caps, we should release the caps from
      the non-auth MDS. The scenario is filelock's state changes from
      SYNC to LOCK. Non-auth MDS revokes Fc cap, the client gets Fc cap
      from the auth MDS at the same time.
      Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
    • Yan, Zheng's avatar
      ceph: fix cap revoke race · b1530f57
      Yan, Zheng authored
      If caps are been revoking by the auth MDS, don't consider them as
      issued even they are still issued by non-auth MDS. The non-auth
      MDS should also be revoking/exporting these caps, the client just
      hasn't received the cap revoke/export message.
      The race I encountered is: When caps are exporting to new MDS, the
      client receives cap import message and cap revoke message from the
      new MDS, then receives cap export message from the old MDS. When
      the client receives cap revoke message from the new MDS, the revoking
      caps are still issued by the old MDS, so the client does nothing.
      Later when the cap export message is received, the client removes
      the caps issued by the old MDS. (Another way to fix the race is
      calling ceph_check_caps() in handle_cap_export())
      Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
    • Yan, Zheng's avatar
      ceph: fix pending vmtruncate race · b415bf4f
      Yan, Zheng authored
      The locking order for pending vmtruncate is wrong, it can lead to
      following race:
              write                  wmtruncate work
      ------------------------    ----------------------
      lock i_mutex
      check i_truncate_pending   check i_truncate_pending
      truncate_inode_pages()     lock i_mutex (blocked)
      copy data to page cache
      unlock i_mutex
      The fix is take i_mutex before calling __ceph_do_pending_vmtruncate()
      Fixes: http://tracker.ceph.com/issues/5453Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
    • Sasha Levin's avatar
      ceph: avoid accessing invalid memory · 54464296
      Sasha Levin authored
      when mounting ceph with a dev name that starts with a slash, ceph
      would attempt to access the character before that slash. Since we
      don't actually own that byte of memory, we would trigger an
      invalid access:
      [   43.499934] BUG: unable to handle kernel paging request at ffff880fa3a97fff
      [   43.500984] IP: [<ffffffff818f3884>] parse_mount_options+0x1a4/0x300
      [   43.501491] PGD 743b067 PUD 10283c4067 PMD 10282a6067 PTE 8000000fa3a97060
      [   43.502301] Oops: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
      [   43.503006] Dumping ftrace buffer:
      [   43.503596]    (ftrace buffer empty)
      [   43.504046] CPU: 0 PID: 10879 Comm: mount Tainted: G        W    3.10.0-sasha #1129
      [   43.504851] task: ffff880fa625b000 ti: ffff880fa3412000 task.ti: ffff880fa3412000
      [   43.505608] RIP: 0010:[<ffffffff818f3884>]  [<ffffffff818f3884>] parse_mount_options$
      [   43.506552] RSP: 0018:ffff880fa3413d08  EFLAGS: 00010286
      [   43.507133] RAX: ffff880fa3a98000 RBX: ffff880fa3a98000 RCX: 0000000000000000
      [   43.507893] RDX: ffff880fa3a98001 RSI: 000000000000002f RDI: ffff880fa3a98000
      [   43.508610] RBP: ffff880fa3413d58 R08: 0000000000001f99 R09: ffff880fa3fe64c0
      [   43.509426] R10: ffff880fa3413d98 R11: ffff880fa38710d8 R12: ffff880fa3413da0
      [   43.509792] R13: ffff880fa3a97fff R14: 0000000000000000 R15: ffff880fa3413d90
      [   43.509792] FS:  00007fa9c48757e0(0000) GS:ffff880fd2600000(0000) knlGS:000000000000$
      [   43.509792] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
      [   43.509792] CR2: ffff880fa3a97fff CR3: 0000000fa3bb9000 CR4: 00000000000006b0
      [   43.509792] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      [   43.509792] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      [   43.509792] Stack:
      [   43.509792]  0000e5180000000e ffffffff85ca1900 ffff880fa38710d8 ffff880fa3413d98
      [   43.509792]  0000000000000120 0000000000000000 ffff880fa3a98000 0000000000000000
      [   43.509792]  ffffffff85cf32a0 0000000000000000 ffff880fa3413dc8 ffffffff818f3c72
      [   43.509792] Call Trace:
      [   43.509792]  [<ffffffff818f3c72>] ceph_mount+0xa2/0x390
      [   43.509792]  [<ffffffff81226314>] ? pcpu_alloc+0x334/0x3c0
      [   43.509792]  [<ffffffff81282f8d>] mount_fs+0x8d/0x1a0
      [   43.509792]  [<ffffffff812263d0>] ? __alloc_percpu+0x10/0x20
      [   43.509792]  [<ffffffff8129f799>] vfs_kern_mount+0x79/0x100
      [   43.509792]  [<ffffffff812a224d>] do_new_mount+0xcd/0x1c0
      [   43.509792]  [<ffffffff812a2e8d>] do_mount+0x15d/0x210
      [   43.509792]  [<ffffffff81220e55>] ? strndup_user+0x45/0x60
      [   43.509792]  [<ffffffff812a2fdd>] SyS_mount+0x9d/0xe0
      [   43.509792]  [<ffffffff83fd816c>] tracesys+0xdd/0xe2
      [   43.509792] Code: 4c 8b 5d c0 74 0a 48 8d 50 01 49 89 14 24 eb 17 31 c0 48 83 c9 ff $
      [   43.509792] RIP  [<ffffffff818f3884>] parse_mount_options+0x1a4/0x300
      [   43.509792]  RSP <ffff880fa3413d08>
      [   43.509792] CR2: ffff880fa3a97fff
      [   43.509792] ---[ end trace 22469cd81e93af51 ]---
      Signed-off-by: default avatarSasha Levin <sasha.levin@oracle.com>
      Reviewed-by: default avatarSage Weil <sage@inktan.com>
    • majianpeng's avatar
      ceph: Reconstruct the func ceph_reserve_caps. · 93faca6e
      majianpeng authored
      Drop ignored return value.  Fix allocation failure case to not leak.
      Signed-off-by: default avatarJianpeng Ma <majianpeng@gmail.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
    • majianpeng's avatar
    • Jianpeng Ma's avatar
      ceph: remove sb_start/end_write in ceph_aio_write. · 0405a149
      Jianpeng Ma authored
      Either in vfs_write or io_submit,it call file_start/end_write.
      The different between file_start/end_write and sb_start/end_write is
      file_ only handle regular file.But i think in ceph_aio_write,it only
      for regular file.
      Signed-off-by: default avatarJianpeng Ma <majianpeng@gmail.com>
      Acked-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
    • majianpeng's avatar
    • majianpeng's avatar
      ceph: fix sleeping function called from invalid context. · a1dc1937
      majianpeng authored
      [ 1121.231883] BUG: sleeping function called from invalid context at kernel/rwsem.c:20
      [ 1121.231935] in_atomic(): 1, irqs_disabled(): 0, pid: 9831, name: mv
      [ 1121.231971] 1 lock held by mv/9831:
      [ 1121.231973]  #0:  (&(&ci->i_ceph_lock)->rlock){+.+...},at:[<ffffffffa02bbd38>] ceph_getxattr+0x58/0x1d0 [ceph]
      [ 1121.231998] CPU: 3 PID: 9831 Comm: mv Not tainted 3.10.0-rc6+ #215
      [ 1121.232000] Hardware name: To Be Filled By O.E.M. To Be Filled By
      O.E.M./To be filled by O.E.M., BIOS 080015  11/09/2011
      [ 1121.232027]  ffff88006d355a80 ffff880092f69ce0 ffffffff8168348c ffff880092f69cf8
      [ 1121.232045]  ffffffff81070435 ffff88006d355a20 ffff880092f69d20 ffffffff816899ba
      [ 1121.232052]  0000000300000004 ffff8800b76911d0 ffff88006d355a20 ffff880092f69d68
      [ 1121.232056] Call Trace:
      [ 1121.232062]  [<ffffffff8168348c>] dump_stack+0x19/0x1b
      [ 1121.232067]  [<ffffffff81070435>] __might_sleep+0xe5/0x110
      [ 1121.232071]  [<ffffffff816899ba>] down_read+0x2a/0x98
      [ 1121.232080]  [<ffffffffa02baf70>] ceph_vxattrcb_layout+0x60/0xf0 [ceph]
      [ 1121.232088]  [<ffffffffa02bbd7f>] ceph_getxattr+0x9f/0x1d0 [ceph]
      [ 1121.232093]  [<ffffffff81188d28>] vfs_getxattr+0xa8/0xd0
      [ 1121.232097]  [<ffffffff8118900b>] getxattr+0xab/0x1c0
      [ 1121.232100]  [<ffffffff811704f2>] ? final_putname+0x22/0x50
      [ 1121.232104]  [<ffffffff81155f80>] ? kmem_cache_free+0xb0/0x260
      [ 1121.232107]  [<ffffffff811704f2>] ? final_putname+0x22/0x50
      [ 1121.232110]  [<ffffffff8109e63d>] ? trace_hardirqs_on+0xd/0x10
      [ 1121.232114]  [<ffffffff816957a7>] ? sysret_check+0x1b/0x56
      [ 1121.232120]  [<ffffffff81189c9c>] SyS_fgetxattr+0x6c/0xc0
      [ 1121.232125]  [<ffffffff81695782>] system_call_fastpath+0x16/0x1b
      [ 1121.232129] BUG: scheduling while atomic: mv/9831/0x10000002
      [ 1121.232154] 1 lock held by mv/9831:
      [ 1121.232156]  #0:  (&(&ci->i_ceph_lock)->rlock){+.+...}, at:
      [<ffffffffa02bbd38>] ceph_getxattr+0x58/0x1d0 [ceph]
      I think move the ci->i_ceph_lock down is safe because we can't free
      ceph_inode_info at there.
      CC: stable@vger.kernel.org  # 3.8+
      Signed-off-by: default avatarJianpeng Ma <majianpeng@gmail.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
    • Yan, Zheng's avatar
    • Yan, Zheng's avatar
      ceph: clear migrate seq when MDS restarts · 667ca05c
      Yan, Zheng authored
      Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
    • Yan, Zheng's avatar
      ceph: check migrate seq before changing auth cap · b8c2f3ae
      Yan, Zheng authored
      We may receive old request reply from the exporter MDS after receiving
      the importer MDS' cap import message.
      Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
    • Yan, Zheng's avatar
      ceph: fix race between page writeback and truncate · fc2744aa
      Yan, Zheng authored
      The client can receive truncate request from MDS at any time.
      So the page writeback code need to get i_size, truncate_seq and
      truncate_size atomically
      Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
    • Yan, Zheng's avatar
    • Yan, Zheng's avatar
      ceph: fix cap release race · bb137f84
      Yan, Zheng authored
      ceph_encode_inode_release() can race with ceph_open() and release
      caps wanted by open files. So it should call __ceph_caps_wanted()
      to get the wanted caps.
      Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
  3. 01 Jul, 2013 3 commits
  4. 17 May, 2013 2 commits
    • Jim Schutt's avatar
      ceph: ceph_pagelist_append might sleep while atomic · 39be95e9
      Jim Schutt authored
      Ceph's encode_caps_cb() worked hard to not call __page_cache_alloc()
      while holding a lock, but it's spoiled because ceph_pagelist_addpage()
      always calls kmap(), which might sleep.  Here's the result:
      [13439.295457] ceph: mds0 reconnect start
      [13439.300572] BUG: sleeping function called from invalid context at include/linux/highmem.h:58
      [13439.309243] in_atomic(): 1, irqs_disabled(): 0, pid: 12059, name: kworker/1:1
          . . .
      [13439.376225] Call Trace:
      [13439.378757]  [<ffffffff81076f4c>] __might_sleep+0xfc/0x110
      [13439.384353]  [<ffffffffa03f4ce0>] ceph_pagelist_append+0x120/0x1b0 [libceph]
      [13439.391491]  [<ffffffffa0448fe9>] ceph_encode_locks+0x89/0x190 [ceph]
      [13439.398035]  [<ffffffff814ee849>] ? _raw_spin_lock+0x49/0x50
      [13439.403775]  [<ffffffff811cadf5>] ? lock_flocks+0x15/0x20
      [13439.409277]  [<ffffffffa045e2af>] encode_caps_cb+0x41f/0x4a0 [ceph]
      [13439.415622]  [<ffffffff81196748>] ? igrab+0x28/0x70
      [13439.420610]  [<ffffffffa045e9f8>] ? iterate_session_caps+0xe8/0x250 [ceph]
      [13439.427584]  [<ffffffffa045ea25>] iterate_session_caps+0x115/0x250 [ceph]
      [13439.434499]  [<ffffffffa045de90>] ? set_request_path_attr+0x2d0/0x2d0 [ceph]
      [13439.441646]  [<ffffffffa0462888>] send_mds_reconnect+0x238/0x450 [ceph]
      [13439.448363]  [<ffffffffa0464542>] ? ceph_mdsmap_decode+0x5e2/0x770 [ceph]
      [13439.455250]  [<ffffffffa0462e42>] check_new_map+0x352/0x500 [ceph]
      [13439.461534]  [<ffffffffa04631ad>] ceph_mdsc_handle_map+0x1bd/0x260 [ceph]
      [13439.468432]  [<ffffffff814ebc7e>] ? mutex_unlock+0xe/0x10
      [13439.473934]  [<ffffffffa043c612>] extra_mon_dispatch+0x22/0x30 [ceph]
      [13439.480464]  [<ffffffffa03f6c2c>] dispatch+0xbc/0x110 [libceph]
      [13439.486492]  [<ffffffffa03eec3d>] process_message+0x1ad/0x1d0 [libceph]
      [13439.493190]  [<ffffffffa03f1498>] ? read_partial_message+0x3e8/0x520 [libceph]
          . . .
      [13439.587132] ceph: mds0 reconnect success
      [13490.720032] ceph: mds0 caps stale
      [13501.235257] ceph: mds0 recovery completed
      [13501.300419] ceph: mds0 caps renewed
      Fix it up by encoding locks into a buffer first, and when the number
      of encoded locks is stable, copy that into a ceph_pagelist.
      [elder@inktank.com: abbreviated the stack info a bit.]
      Cc: stable@vger.kernel.org # 3.4+
      Signed-off-by: default avatarJim Schutt <jaschut@sandia.gov>
      Reviewed-by: default avatarAlex Elder <elder@inktank.com>
    • Jim Schutt's avatar
      ceph: add cpu_to_le32() calls when encoding a reconnect capability · c420276a
      Jim Schutt authored
      In his review, Alex Elder mentioned that he hadn't checked that
      num_fcntl_locks and num_flock_locks were properly decoded on the
      server side, from a le32 over-the-wire type to a cpu type.
      I checked, and AFAICS it is done; those interested can consult
      in src/mds/Locker.cc and src/include/encoding.h in the Ceph server
      code (git://github.com/ceph/ceph).
      I also checked the server side for flock_len decoding, and I believe
      that also happens correctly, by virtue of having been declared
      __le32 in struct ceph_mds_cap_reconnect, in src/include/ceph_fs.h.
      Cc: stable@vger.kernel.org # 3.4+
      Signed-off-by: default avatarJim Schutt <jaschut@sandia.gov>
      Reviewed-by: default avatarAlex Elder <elder@inktank.com>
  5. 08 May, 2013 1 commit
  6. 02 May, 2013 10 commits
    • Alex Elder's avatar
      ceph: use ceph_create_snap_context() · 812164f8
      Alex Elder authored
      Now that we have a library routine to create snap contexts, use it.
      This is part of:
          http://tracker.ceph.com/issues/4857Signed-off-by: default avatarAlex Elder <elder@inktank.com>
      Reviewed-by: default avatarJosh Durgin <josh.durgin@inktank.com>
    • Alex Elder's avatar
      libceph: kill off osd data write_request parameters · 406e2c9f
      Alex Elder authored
      In the incremental move toward supporting distinct data items in an
      osd request some of the functions had "write_request" parameters to
      indicate, basically, whether the data belonged to in_data or the
      out_data.  Now that we maintain the data fields in the op structure
      there is no need to indicate the direction, so get rid of the
      "write_request" parameters.
      Signed-off-by: default avatarAlex Elder <elder@inktank.com>
      Reviewed-by: default avatarJosh Durgin <josh.durgin@inktank.com>
    • Randy Dunlap's avatar
      ceph: fix printk format warnings in file.c · ac7f29bf
      Randy Dunlap authored
      Fix printk format warnings by using %zd for 'ssize_t' variables:
      fs/ceph/file.c:751:2: warning: format '%ld' expects argument of type 'long int', but argument 11 has type 'ssize_t' [-Wformat]
      fs/ceph/file.c:762:2: warning: format '%ld' expects argument of type 'long int', but argument 11 has type 'ssize_t' [-Wformat]
      Signed-off-by: default avatarRandy Dunlap <rdunlap@infradead.org>
      Cc:	ceph-devel@vger.kernel.org
      Signed-off-by: default avatarSage Weil <sage@inktank.com>
    • Yan, Zheng's avatar
      ceph: fix race between writepages and truncate · 1ac0fc8a
      Yan, Zheng authored
      ceph_writepages_start() reads inode->i_size in two places. It can get
      different values between successive read, because truncate can change
      inode->i_size at any time. The race can lead to mismatch between data
      length of osd request and pages marked as writeback. When osd request
      finishes, it clear writeback page according to its data length. So
      some pages can be left in writeback state forever. The fix is only
      read inode->i_size once, save its value to a local variable and use
      the local variable when i_size is needed.
      Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarAlex Elder <elder@inktank.com>
    • Yan, Zheng's avatar
      ceph: apply write checks in ceph_aio_write · 03d254ed
      Yan, Zheng authored
      copy write checks in __generic_file_aio_write to ceph_aio_write.
      To make these checks cover sync write path.
      Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarAlex Elder <elder@inktank.com>
    • Yan, Zheng's avatar
      ceph: take i_mutex before getting Fw cap · 37505d57
      Yan, Zheng authored
      There is deadlock as illustrated bellow. The fix is taking i_mutex
      before getting Fw cap reference.
            write                    truncate                 MDS
      ---------------------     --------------------      --------------
      get Fw cap
                                lock i_mutex
      lock i_mutex (blocked)
                                request setattr.size  ->
                                                      <-   revoke Fw cap
      Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarAlex Elder <elder@inktank.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
    • Alex Elder's avatar
      libceph: change how "safe" callback is used · 26be8808
      Alex Elder authored
      An osd request currently has two callbacks.  They inform the
      initiator of the request when we've received confirmation for the
      target osd that a request was received, and when the osd indicates
      all changes described by the request are durable.
      The only time the second callback is used is in the ceph file system
      for a synchronous write.  There's a race that makes some handling of
      this case unsafe.  This patch addresses this problem.  The error
      handling for this callback is also kind of gross, and this patch
      changes that as well.
      In ceph_sync_write(), if a safe callback is requested we want to add
      the request on the ceph inode's unsafe items list.  Because items on
      this list must have their tid set (by ceph_osd_start_request()), the
      request added *after* the call to that function returns.  The
      problem with this is that there's a race between starting the
      request and adding it to the unsafe items list; the request may
      already be complete before ceph_sync_write() even begins to put it
      on the list.
      To address this, we change the way the "safe" callback is used.
      Rather than just calling it when the request is "safe", we use it to
      notify the initiator the bounds (start and end) of the period during
      which the request is *unsafe*.  So the initiator gets notified just
      before the request gets sent to the osd (when it is "unsafe"), and
      again when it's known the results are durable (it's no longer
      unsafe).  The first call will get made in __send_request(), just
      before the request message gets sent to the messenger for the first
      time.  That function is only called by __send_queued(), which is
      always called with the osd client's request mutex held.
      We then have this callback function insert the request on the ceph
      inode's unsafe list when we're told the request is unsafe.  This
      will avoid the race because this call will be made under protection
      of the osd client's request mutex.  It also nicely groups the setup
      and cleanup of the state associated with managing unsafe requests.
      The name of the "safe" callback field is changed to "unsafe" to
      better reflect its new purpose.  It has a Boolean "unsafe" parameter
      to indicate whether the request is becoming unsafe or is now safe.
      Because the "msg" parameter wasn't used, we drop that.
      This resolves the original problem reportedin:
          http://tracker.ceph.com/issues/4706Reported-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Signed-off-by: default avatarAlex Elder <elder@inktank.com>
      Reviewed-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarSage Weil <sage@inktank.com>
    • Alex Elder's avatar
      ceph: let osd client clean up for interrupted request · 7d7d51ce
      Alex Elder authored
      In ceph_sync_write(), if a safe callback is supplied with a request,
      and an error is returned by ceph_osdc_wait_request(), a block of
      code is executed to remove the request from the unsafe writes list
      and drop references to capabilities acquired just prior to a call to
      The only function used for this callback is sync_write_commit(),
      and it does *exactly* what that block of error handling code does.
      Now in ceph_osdc_wait_request(), if an error occurs (due to an
      interupt during a wait_for_completion_interruptible() call),
      complete_request() gets called, and that calls the request's
      safe_callback method if it's defined.
      So this means that this cleanup activity gets called twice in this
      case, which is erroneous (and in fact leads to a crash).
      Fix this by just letting the osd client handle the cleanup in
      the event of an interrupt.
      This resolves one problem mentioned in:
          http://tracker.ceph.com/issues/4706Signed-off-by: default avatarAlex Elder <elder@inktank.com>
      Reviewed-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
    • Yan, Zheng's avatar
      ceph: fix symlink inode operations · 0b932672
      Yan, Zheng authored
      add getattr/setattr and xattrs related methods.
      Signed-off-by: default avatarYan, Zheng <zheng.z.yan@intel.com>
      Reviewed-by: default avatarGreg Farnum <greg@inktank.com>
    • Sam Lang's avatar
      ceph: Use pseudo-random numbers to choose mds · a84cd293
      Sam Lang authored
      We don't need to use up entropy to choose an mds,
      so use prandom_u32() to get a pseudo-random number.
      Also, we don't need to choose a random mds if only
      one mds is available, so add special casing for the
      common case.
      Fixes http://tracker.ceph.com/issues/3579Signed-off-by: default avatarSam Lang <sam.lang@inktank.com>
      Reviewed-by: default avatarGreg Farnum <greg@inktank.com>
      Reviewed-by: default avatarAlex Elder <elder@inktank.com>