1. 02 Mar, 2017 2 commits
  2. 07 Nov, 2015 1 commit
  3. 13 Feb, 2015 1 commit
  4. 30 Jan, 2014 1 commit
    • Shaohua Li's avatar
      percpu_ida: fix a live lock · d835502f
      Shaohua Li authored
      steal_tags only happens when free tags is more than half of the total
      tags.  This is too strict and can cause live lock. I found that if one
      cpu has free tags, but other cpu can't steal (thread is bound to
      specific cpus), threads which want to allocate tags are always
      sleeping. I found this when I run next patch, but this could happen
      without it I think.
      
      I did performance test too with null_blk. Two cases (each cpu has enough
      percpu tags, or total tags are limited), no performance changes were
      observed.
      Signed-off-by: default avatarShaohua Li <shli@fusionio.com>
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      d835502f
  5. 25 Jan, 2014 1 commit
  6. 23 Jan, 2014 1 commit
    • Kent Overstreet's avatar
      percpu_ida: Make percpu_ida_alloc + callers accept task state bitmask · 6f6b5d1e
      Kent Overstreet authored
      This patch changes percpu_ida_alloc() + callers to accept task state
      bitmask for prepare_to_wait() for code like target/iscsi that needs
      it for interruptible sleep, that is provided in a subsequent patch.
      
      It now expects TASK_UNINTERRUPTIBLE when the caller is able to sleep
      waiting for a new tag, or TASK_RUNNING when the caller cannot sleep,
      and is forced to return a negative value when no tags are available.
      
      v2 changes:
        - Include blk-mq + tcm_fc + vhost/scsi + target/iscsi changes
        - Drop signal_pending_state() call
      v3 changes:
        - Only call prepare_to_wait() + finish_wait() when != TASK_RUNNING
          (PeterZ)
      Reported-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Jens Axboe <axboe@kernel.dk>
      Signed-off-by: default avatarKent Overstreet <kmo@daterainc.com>
      Cc: <stable@vger.kernel.org> #3.12+
      Signed-off-by: default avatarNicholas Bellinger <nab@linux-iscsi.org>
      6f6b5d1e
  7. 25 Oct, 2013 3 commits
    • Shaohua Li's avatar
      percpu_ida: add an API to return free tags · 1dddc01a
      Shaohua Li authored
      Add an API to return free tags, blk-mq-tag will use it.
      
      Note, this just returns a snapshot of free tags number. blk-mq-tag has
      two usages of it. One is for info output for diagnosis. The other is to
      quickly check if there are free tags for request dispatch checking.
      Neither requires very precise.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarShaohua Li <shli@fusionio.com>
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      1dddc01a
    • Shaohua Li's avatar
      percpu_ida: add percpu_ida_for_each_free · 7fc2ba17
      Shaohua Li authored
      Add a new API to iterate free ids. blk-mq-tag will use it.
      
      Note, this doesn't guarantee to iterate all free ids restrictly. Caller
      should be aware of this. blk-mq uses it to do sanity check for request
      timedout, so can tolerate the limitation.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarShaohua Li <shli@fusionio.com>
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      7fc2ba17
    • Shaohua Li's avatar
      percpu_ida: make percpu_ida percpu size/batch configurable · e26b53d0
      Shaohua Li authored
      Make percpu_ida percpu size/batch configurable. The block-mq-tag will
      use it.
      
      After block-mq uses percpu_ida to manage tags, performance is improved.
      My test is done in a 2 sockets machine, 12 process cross the 2 sockets.
      So if there is lock contention or ipi, should be stressed heavily.
      Testing is done for null-blk.
      
      hw_queue_depth	nopatch iops	patch iops
      64		~800k/s		~1470k/s
      2048		~4470k/s	~4340k/s
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarShaohua Li <shli@fusionio.com>
      Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
      e26b53d0
  8. 03 Oct, 2013 1 commit
  9. 09 Sep, 2013 1 commit
    • Kent Overstreet's avatar
      idr: Percpu ida · 798ab48e
      Kent Overstreet authored
      Percpu frontend for allocating ids. With percpu allocation (that works),
      it's impossible to guarantee it will always be possible to allocate all
      nr_tags - typically, some will be stuck on a remote percpu freelist
      where the current job can't get to them.
      
      We do guarantee that it will always be possible to allocate at least
      (nr_tags / 2) tags - this is done by keeping track of which and how many
      cpus have tags on their percpu freelists. On allocation failure if
      enough cpus have tags that there could potentially be (nr_tags / 2) tags
      stuck on remote percpu freelists, we then pick a remote cpu at random to
      steal from.
      
      Note that there's no cpu hotplug notifier - we don't care, because
      steal_tags() will eventually get the down cpu's tags. We _could_ satisfy
      more allocations if we had a notifier - but we'll still meet our
      guarantees and it's absolutely not a correctness issue, so I don't think
      it's worth the extra code.
      
      From akpm:
      
          "It looks OK to me (that's as close as I get to an ack :))
      
      v6 changes:
        - Add #include <linux/cpumask.h> to include/linux/percpu_ida.h to
          make alpha/arc builds happy (Fengguang)
        - Move second (cpu >= nr_cpu_ids) check inside of first check scope
          in steal_tags() (akpm + nab)
      
      v5 changes:
        - Change percpu_ida->cpus_have_tags to cpumask_t (kmo + akpm)
        - Add comment for percpu_ida_cpu->lock + ->nr_free (kmo + akpm)
        - Convert steal_tags() to use cpumask_weight() + cpumask_next() +
          cpumask_first() + cpumask_clear_cpu() (kmo + akpm)
        - Add comment for alloc_global_tags() (kmo + akpm)
        - Convert percpu_ida_alloc() to use cpumask_set_cpu() (kmo + akpm)
        - Convert percpu_ida_free() to use cpumask_set_cpu() (kmo + akpm)
        - Drop percpu_ida->cpus_have_tags allocation in percpu_ida_init()
          (kmo + akpm)
        - Drop percpu_ida->cpus_have_tags kfree in percpu_ida_destroy()
          (kmo + akpm)
        - Add comment for percpu_ida_alloc @ gfp (kmo + akpm)
        - Move to percpu_ida.c + percpu_ida.h (kmo + akpm + nab)
      
      v4 changes:
      
        - Fix tags.c reference in percpu_ida_init (akpm)
      Signed-off-by: default avatarKent Overstreet <kmo@daterainc.com>
      Cc: Tejun Heo <tj@kernel.org>
      Cc: Oleg Nesterov <oleg@redhat.com>
      Cc: Christoph Lameter <cl@linux-foundation.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Andi Kleen <andi@firstfloor.org>
      Cc: Jens Axboe <axboe@kernel.dk>
      Cc: "Nicholas A. Bellinger" <nab@linux-iscsi.org>
      Signed-off-by: default avatarNicholas Bellinger <nab@linux-iscsi.org>
      798ab48e