• Pavel Tatashin's avatar
    mm/memory_hotplug: optimize memory hotplug · d0dc12e8
    Pavel Tatashin authored
    During memory hotplugging we traverse struct pages three times:
    1. memset(0) in sparse_add_one_section()
    2. loop in __add_section() to set do: set_page_node(page, nid); and
    3. loop in memmap_init_zone() to call __init_single_pfn()
    This patch removes the first two loops, and leaves only loop 3.  All
    struct pages are initialized in one place, the same as it is done during
    The benefits:
     - We improve memory hotplug performance because we are not evicting the
       cache several times and also reduce loop branching overhead.
     - Remove condition from hotpath in __init_single_pfn(), that was added
       in order to fix the problem that was reported by Bharata in the above
       email thread, thus also improve performance during normal boot.
     - Make memory hotplug more similar to the boot memory initialization
       path because we zero and initialize struct pages only in one
     - Simplifies memory hotplug struct page initialization code, and thus
       enables future improvements, such as multi-threading the
       initialization of struct pages in order to improve hotplug
       performance even further on larger machines.
    [pasha.tatashin@oracle.com: v5]
      Link: http://lkml.kernel.org/r/20180228030308.1116-7-pasha.tatashin@oracle.com
    Link: http://lkml.kernel.org/r/20180215165920.8570-7-pasha.tatashin@oracle.comSigned-off-by: default avatarPavel Tatashin <pasha.tatashin@oracle.com>
    Reviewed-by: default avatarIngo Molnar <mingo@kernel.org>
    Cc: Michal Hocko <mhocko@suse.com>
    Cc: Baoquan He <bhe@redhat.com>
    Cc: Bharata B Rao <bharata@linux.vnet.ibm.com>
    Cc: Daniel Jordan <daniel.m.jordan@oracle.com>
    Cc: Dan Williams <dan.j.williams@intel.com>
    Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
    Cc: "H. Peter Anvin" <hpa@zytor.com>
    Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
    Cc: Mel Gorman <mgorman@techsingularity.net>
    Cc: Steven Sistare <steven.sistare@oracle.com>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Vlastimil Babka <vbabka@suse.cz>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
sparse.c 22.8 KB