1. 04 Apr, 2016 2 commits
    • Kirill A. Shutemov's avatar
      mm, fs: remove remaining PAGE_CACHE_* and page_cache_{get,release} usage · ea1754a0
      Kirill A. Shutemov authored
      Mostly direct substitution with occasional adjustment or removing
      outdated comments.
      Signed-off-by: 's avatarKirill A. Shutemov <kirill.shutemov@linux.intel.com>
      Acked-by: 's avatarMichal Hocko <mhocko@suse.com>
      Signed-off-by: 's avatarLinus Torvalds <torvalds@linux-foundation.org>
      ea1754a0
    • Kirill A. Shutemov's avatar
      mm, fs: get rid of PAGE_CACHE_* and page_cache_{get,release} macros · 09cbfeaf
      Kirill A. Shutemov authored
      PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} macros were introduced *long* time
      ago with promise that one day it will be possible to implement page
      cache with bigger chunks than PAGE_SIZE.
      
      This promise never materialized.  And unlikely will.
      
      We have many places where PAGE_CACHE_SIZE assumed to be equal to
      PAGE_SIZE.  And it's constant source of confusion on whether
      PAGE_CACHE_* or PAGE_* constant should be used in a particular case,
      especially on the border between fs and mm.
      
      Global switching to PAGE_CACHE_SIZE != PAGE_SIZE would cause to much
      breakage to be doable.
      
      Let's stop pretending that pages in page cache are special.  They are
      not.
      
      The changes are pretty straight-forward:
      
       - <foo> << (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> <foo>;
      
       - <foo> >> (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> <foo>;
      
       - PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} -> PAGE_{SIZE,SHIFT,MASK,ALIGN};
      
       - page_cache_get() -> get_page();
      
       - page_cache_release() -> put_page();
      
      This patch contains automated changes generated with coccinelle using
      script below.  For some reason, coccinelle doesn't patch header files.
      I've called spatch for them manually.
      
      The only adjustment after coccinelle is revert of changes to
      PAGE_CAHCE_ALIGN definition: we are going to drop it later.
      
      There are few places in the code where coccinelle didn't reach.  I'll
      fix them manually in a separate patch.  Comments and documentation also
      will be addressed with the separate patch.
      
      virtual patch
      
      @@
      expression E;
      @@
      - E << (PAGE_CACHE_SHIFT - PAGE_SHIFT)
      + E
      
      @@
      expression E;
      @@
      - E >> (PAGE_CACHE_SHIFT - PAGE_SHIFT)
      + E
      
      @@
      @@
      - PAGE_CACHE_SHIFT
      + PAGE_SHIFT
      
      @@
      @@
      - PAGE_CACHE_SIZE
      + PAGE_SIZE
      
      @@
      @@
      - PAGE_CACHE_MASK
      + PAGE_MASK
      
      @@
      expression E;
      @@
      - PAGE_CACHE_ALIGN(E)
      + PAGE_ALIGN(E)
      
      @@
      expression E;
      @@
      - page_cache_get(E)
      + get_page(E)
      
      @@
      expression E;
      @@
      - page_cache_release(E)
      + put_page(E)
      Signed-off-by: 's avatarKirill A. Shutemov <kirill.shutemov@linux.intel.com>
      Acked-by: 's avatarMichal Hocko <mhocko@suse.com>
      Signed-off-by: 's avatarLinus Torvalds <torvalds@linux-foundation.org>
      09cbfeaf
  2. 02 Mar, 2016 2 commits
  3. 26 Feb, 2016 2 commits
  4. 20 Feb, 2016 2 commits
  5. 21 Dec, 2015 1 commit
  6. 13 Oct, 2015 2 commits
  7. 08 Oct, 2015 2 commits
  8. 04 Oct, 2015 3 commits
  9. 29 Sep, 2015 9 commits
  10. 17 Sep, 2015 1 commit
  11. 15 Sep, 2015 3 commits
    • Olaf Weber's avatar
      staging/lustre/ptlrpc: make ptlrpcd threads cpt-aware · c5c4c6fa
      Olaf Weber authored
      On NUMA systems, the placement of worker threads relative to the
      memory they use greatly affects performance. The CPT mechanism can be
      used to constrain a number of Lustre thread types, and this change
      makes it possible to configure the placement of ptlrpcd threads in a
      similar manner.
      
      To simplify the code changes, the global structures used to manage
      ptlrpcd threads are changed to one per CPT. In particular this means
      there will be one ptlrpcd recovery thread per CPT.
      
      To prevent ptlrpcd threads from wandering all over the system, all
      ptlrpcd thread are bound to a CPT. Note that some CPT configuration
      is always created, but the defaults are not likely to be correct for
      a NUMA system. After discussing the options with Liang Zhen we
      decided that we would not bind ptlrpcd threads to specific CPUs,
      and rather trust the kernel scheduler to migrate ptlrpcd threads.
      
      With all ptlrpcd threads bound to a CPT, but not to specific CPUs,
      the load policy mechanism can be radically simplified:
      
      - PDL_POLICY_LOCAL and PDL_POLICY_ROUND are currently identical.
      - PDL_POLICY_ROUND, if fully implemented, would cost us the locality
        we are trying to achieve, so most or all calls using this policy
        would have to be changed to PDL_POLICY_LOCAL.
      - PDL_POLICY_PREFERRED is not used, and cannot be implemented without
        binding ptlrpcd threads to individual CPUs.
      - PDL_POLICY_SAME is rarely used, and cannot be implemented without
        binding ptlrpcd threads to individual CPUs.
      
      The partner mechanism is also updated, because now all ptlrpcd
      threads are "bound" threads. The only difference between the various
      bind policies, PDB_POLICY_NONE, PDB_POLICY_FULL, PDB_POLICY_PAIR, and
      PDB_POLICY_NEIGHBOR, is the number of partner threads. The bind
      policy is replaced with a tunable that directly specifies the size of
      the groups of ptlrpcd partner threads.
      
      Ensure that the ptlrpc_request_set for a ptlrpcd thread is created on
      the same CPT that the thread will work on. When threads are bound to
      specific nodes and/or CPUs in a NUMA system, it pays to ensure that
      the datastructures used by these threads are also on the same node.
      
      Visible changes:
      
      * ptlrpcd thread names include the CPT number, for example
        "ptlrpcd_02_07". In this case the "07" is relative to the CPT, and
        not a CPU number.
      
      Tunables added:
      
      * ptlrpcd_cpts (string): A CPT string describing the CPU partitions
        that ptlrpcd threads should run on. Used to make ptlrpcd threads
        run on a subset of all CPTs.
      
      * ptlrpcd_per_cpt_max (int): The maximum number of ptlrpcd threads
        to run in a CPT.
      
      * ptlrpcd_partner_group_size (int): The desired number of threads
        in each ptlrpcd partner thread group. Default is 2, corresponding
        to the old PDB_POLICY_PAIR. A negative value makes all ptlrpcd
        threads in a CPT partners of each other.
      
      Tunables obsoleted:
      
      * max_ptlrpcds: The new ptlrcpd_per_cpt_max can be used to obtain the
        same effect.
      
      * ptlrpcd_bind_policy: The new ptlrpcd_partner_group_size can be used
        to obtain the same effect.
      
      Internal interface changes:
      
      * pdb_policy_t and related code have been removed. Groups of partner
        ptlrpcd threads are still created, and all threads in a partner
        group are bound on the same CPT. The ptlrpcd threads bound to a
        CPT are typically divided into several partner groups. The partner
        groups on a CPT all have an equal number of ptlrpcd threads.
      
      * pdl_policy_t and related code have been removed. Since ptlrpcd
        threads are not bound to a specific CPU, all the code that avoids
        scheduling on the current CPU (or attempts to do so) has been
        removed as non-functional. A simplified form of PDL_POLICY_LOCAL
        is kept as the only load policy.
      
      * LIOD_BIND and related code have been removed. All ptlrpcd threads
        are now bound to a CPT, and no additional binding policy is
        implemented.
      
      * ptlrpc_prep_set(): Changed to allocate a ptlrpc_request_set
        on the current CPT.
      
      * ptlrpcd(): If an error is encountered before entering the main loop
        store the error in pc_error before exiting.
      
      * ptlrpcd_start(): Check pc_error to verify that the ptlrpcd thread
        has successfully entered its main loop.
      
      * ptlrpcd_init(): Initialize the struct ptlrpcd_ctl for all threads
        for a CPT before starting any of them. This closes a race during
        startup where a partner thread could reference a non-initialized
        struct ptlrpcd_ctl.
      Signed-off-by: 's avatarOlaf Weber <olaf@sgi.com>
      Reviewed-on: http://review.whamcloud.com/13972
      Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6325Reviewed-by: 's avatarGrégoire Pichon <gregoire.pichon@bull.net>
      Reviewed-by: 's avatarStephen Champion <schamp@sgi.com>
      Reviewed-by: 's avatarJames Simmons <uja.ornl@yahoo.com>
      Reviewed-by: 's avatarJinshan Xiong <jinshan.xiong@intel.com>
      Signed-off-by: 's avatarOleg Drokin <oleg.drokin@intel.com>
      Signed-off-by: 's avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      c5c4c6fa
    • Li Xi's avatar
      staging/lustre/osc: use global osc_rq_pool to reduce memory usage · aefd9d71
      Li Xi authored
      The per-osc request pools consume a lot of memory if there are
      hundreds of OSCs on one client. This will be a critical problem
      if the client doesn't have sufficient memory for both OSCs and
      applications.
      
      This patch replaces per-osc request pools with a global pool
      osc_rq_pool. The total memory usage is 5MB by default. And it
      can be set by a module parameter of OSC:
      "options osc osc_reqpool_mem_max=POOL_SIZE". The unit of POOL_SIZE
      is MB. If cl_max_rpcs_in_flight is the same for all OSCs, the
      memory usage of the OSC pool can be calculated as:
      Min(POOL_SIZE * 1M,
          (cl_max_rpcs_in_flight + 2) * OSC number * OST_MAXREQSIZE)
      
      Also, this patch changes the allocation logic of OSC write requests.
      The allocation from osc_rq_pool will only be tried after normal
      allocation failed.
      Signed-off-by: 's avatarWu Libin <lwu@ddn.com>
      Signed-off-by: 's avatarWang Shilong <wshilong@ddn.com>
      Signed-off-by: 's avatarLi Xi <lixi@ddn.com>
      Reviewed-on: http://review.whamcloud.com/15422
      Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6770Reviewed-by: 's avatarJinshan Xiong <jinshan.xiong@intel.com>
      Reviewed-by: 's avatarAndreas Dilger <andreas.dilger@intel.com>
      Signed-off-by: 's avatarOleg Drokin <oleg.drokin@intel.com>
      Signed-off-by: 's avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      aefd9d71
    • Andreas Dilger's avatar
      staging/lustre/ptlrpc: remove LUSTRE_MSG_MAGIC_V1 support · 2e4fe2bd
      Andreas Dilger authored
      Remove the remains of LUSTRE_MSG_MAGIC_V1 support from ptlrpc.
      It has not been supported since 1.8 and is not functional since 2.0.
      
      In lustre_msg_check_version(), return an error for unsupported RPC
      versions so that the server will reject such RPCs early.  Otherwise
      the server only prints an error message and continue on.
      Signed-off-by: 's avatarAndreas Dilger <andreas.dilger@intel.com>
      Reviewed-on: http://review.whamcloud.com/14007
      Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6349Reviewed-by: 's avatarJames Simmons <uja.ornl@yahoo.com>
      Reviewed-by: 's avatarDmitry Eremin <dmitry.eremin@intel.com>
      Reviewed-by: 's avatarJohn L. Hammond <john.hammond@intel.com>
      Signed-off-by: 's avatarOleg Drokin <oleg.drokin@intel.com>
      Signed-off-by: 's avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      2e4fe2bd
  12. 13 Sep, 2015 1 commit
  13. 15 Aug, 2015 1 commit
  14. 05 Aug, 2015 1 commit
  15. 12 Jun, 2015 1 commit
  16. 31 May, 2015 3 commits
  17. 18 Mar, 2015 1 commit
  18. 29 Oct, 2014 1 commit
  19. 20 Oct, 2014 1 commit
  20. 30 Aug, 2014 1 commit