1. 20 Apr, 2012 1 commit
  2. 01 Mar, 2012 1 commit
    • Tejun Heo's avatar
      memblock: Fix size aligning of memblock_alloc_base_nid() · 847854f5
      Tejun Heo authored
      memblock allocator aligns @size to @align to reduce the amount
      of fragmentation.  Commit:
      
       7bd0b0f0 ("memblock: Reimplement memblock allocation using reverse free area iterator")
      
      Broke it by incorrectly relocating @size aligning to
      memblock_find_in_range_node().  As the aligned size is not
      propagated back to memblock_alloc_base_nid(), the actually
      reserved size isn't aligned.
      
      While this increases memory use for memblock reserved array,
      this shouldn't cause any critical failure; however, it seems
      that the size aligning was hiding a use-beyond-allocation bug in
      sparc64 and losing the aligning causes boot failure.
      
      The underlying problem is currently being debugged but this is a
      proper fix in itself, it's already pretty late in -rc cycle for
      boot failures and reverting the change for debugging isn't
      difficult. Restore the size aligning moving it to
      memblock_alloc_base_nid().
      Reported-by: default avatarMeelis Roos <mroos@linux.ee>
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: David S. Miller <davem@davemloft.net>
      Cc: Grant Likely <grant.likely@secretlab.ca>
      Cc: Rob Herring <rob.herring@calxeda.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Link: http://lkml.kernel.org/r/20120228205621.GC3252@dhcp-172-17-108-109.mtv.corp.google.comSigned-off-by: default avatarIngo Molnar <mingo@elte.hu>
      LKML-Reference: <alpine.SOC.1.00.1202130942030.1488@math.ut.ee>
      847854f5
  3. 16 Jan, 2012 1 commit
  4. 08 Dec, 2011 14 commits
    • Tejun Heo's avatar
      memblock: Reimplement memblock allocation using reverse free area iterator · 7bd0b0f0
      Tejun Heo authored
      Now that all early memory information is in memblock when enabled, we
      can implement reverse free area iterator and use it to implement NUMA
      aware allocator which is then wrapped for simpler variants instead of
      the confusing and inefficient mending of information in separate NUMA
      aware allocator.
      
      Implement for_each_free_mem_range_reverse(), use it to reimplement
      memblock_find_in_range_node() which in turn is used by all allocators.
      
      The visible allocator interface is inconsistent and can probably use
      some cleanup too.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      7bd0b0f0
    • Tejun Heo's avatar
      memblock: Kill early_node_map[] · 0ee332c1
      Tejun Heo authored
      Now all ARCH_POPULATES_NODE_MAP archs select HAVE_MEBLOCK_NODE_MAP -
      there's no user of early_node_map[] left.  Kill early_node_map[] and
      replace ARCH_POPULATES_NODE_MAP with HAVE_MEMBLOCK_NODE_MAP.  Also,
      relocate for_each_mem_pfn_range() and helper from mm.h to memblock.h
      as page_alloc.c would no longer host an alternative implementation.
      
      This change is ultimately one to one mapping and shouldn't cause any
      observable difference; however, after the recent changes, there are
      some functions which now would fit memblock.c better than page_alloc.c
      and dependency on HAVE_MEMBLOCK_NODE_MAP instead of HAVE_MEMBLOCK
      doesn't make much sense on some of them.  Further cleanups for
      functions inside HAVE_MEMBLOCK_NODE_MAP in mm.h would be nice.
      
      -v2: Fix compile bug introduced by mis-spelling
       CONFIG_HAVE_MEMBLOCK_NODE_MAP to CONFIG_MEMBLOCK_HAVE_NODE_MAP in
       mmzone.h.  Reported by Stephen Rothwell.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Stephen Rothwell <sfr@canb.auug.org.au>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Tony Luck <tony.luck@intel.com>
      Cc: Ralf Baechle <ralf@linux-mips.org>
      Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
      Cc: Chen Liqin <liqin.chen@sunplusct.com>
      Cc: Paul Mundt <lethal@linux-sh.org>
      Cc: "David S. Miller" <davem@davemloft.net>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      0ee332c1
    • Tejun Heo's avatar
      memblock: Implement memblock_add_node() · 7fb0bc3f
      Tejun Heo authored
      Implement memblock_add_node() which can add a new memblock memory
      region with specific node ID.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      7fb0bc3f
    • Tejun Heo's avatar
      memblock: s/memblock_analyze()/memblock_allow_resize()/ and update users · 1aadc056
      Tejun Heo authored
      The only function of memblock_analyze() is now allowing resize of
      memblock region arrays.  Rename it to memblock_allow_resize() and
      update its users.
      
      * The following users remain the same other than renaming.
      
        arm/mm/init.c::arm_memblock_init()
        microblaze/kernel/prom.c::early_init_devtree()
        powerpc/kernel/prom.c::early_init_devtree()
        openrisc/kernel/prom.c::early_init_devtree()
        sh/mm/init.c::paging_init()
        sparc/mm/init_64.c::paging_init()
        unicore32/mm/init.c::uc32_memblock_init()
      
      * In the following users, analyze was used to update total size which
        is no longer necessary.
      
        powerpc/kernel/machine_kexec.c::reserve_crashkernel()
        powerpc/kernel/prom.c::early_init_devtree()
        powerpc/mm/init_32.c::MMU_init()
        powerpc/mm/tlb_nohash.c::__early_init_mmu()  
        powerpc/platforms/ps3/mm.c::ps3_mm_add_memory()
        powerpc/platforms/embedded6xx/wii.c::wii_memory_fixups()
        sh/kernel/machine_kexec.c::reserve_crashkernel()
      
      * x86/kernel/e820.c::memblock_x86_fill() was directly setting
        memblock_can_resize before populating memblock and calling analyze
        afterwards.  Call memblock_allow_resize() before start populating.
      
      memblock_can_resize is now static inside memblock.c.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Russell King <linux@arm.linux.org.uk>
      Cc: Michal Simek <monstr@monstr.eu>
      Cc: Paul Mundt <lethal@linux-sh.org>
      Cc: "David S. Miller" <davem@davemloft.net>
      Cc: Guan Xuetao <gxt@mprc.pku.edu.cn>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      1aadc056
    • Tejun Heo's avatar
      memblock: Track total size of regions automatically · 1440c4e2
      Tejun Heo authored
      Total size of memory regions was calculated by memblock_analyze()
      requiring explicitly calling the function between operations which can
      change memory regions and possible users of total size, which is
      cumbersome and fragile.
      
      This patch makes each memblock_type track total size automatically
      with minor modifications to memblock manipulation functions and remove
      requirements on calling memblock_analyze().  [__]memblock_dump_all()
      now also dumps the total size of reserved regions.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      1440c4e2
    • Tejun Heo's avatar
      memblock: Reimplement memblock_enforce_memory_limit() using __memblock_remove() · c0ce8fef
      Tejun Heo authored
      With recent updates, the basic memblock operations are robust enough
      that there's no reason for memblock_enfore_memory_limit() to directly
      manipulate memblock region arrays.  Reimplement it using
      __memblock_remove().
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      c0ce8fef
    • Tejun Heo's avatar
      memblock: Make memblock functions handle overflowing range @size · eb18f1b5
      Tejun Heo authored
      Allow memblock users to specify range where @base + @size overflows
      and automatically cap it at maximum.  This makes the interface more
      robust and specifying till-the-end-of-memory easier.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      eb18f1b5
    • Tejun Heo's avatar
      memblock: Reimplement __memblock_remove() using memblock_isolate_range() · 71936180
      Tejun Heo authored
      __memblock_remove()'s open coded region manipulation can be trivially
      replaced with memblock_islate_range().  This increases code sharing
      and eases improving region tracking.
      
      This pulls memblock_isolate_range() out of HAVE_MEMBLOCK_NODE_MAP.
      Make it use memblock_get_region_node() instead of assuming rgn->nid is
      available.
      
      -v2: Fixed build failure on !HAVE_MEMBLOCK_NODE_MAP caused by direct
           rgn->nid access.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      71936180
    • Tejun Heo's avatar
      memblock: Separate out memblock_isolate_range() from memblock_set_node() · 6a9ceb31
      Tejun Heo authored
      memblock_set_node() operates in three steps - break regions crossing
      boundaries, set nid and merge back regions.  This patch separates the
      first part into a separate function - memblock_isolate_range(), which
      breaks regions crossing range boundaries and returns range index range
      for regions properly contained in the specified memory range.
      
      This doesn't introduce any behavior change and will be used to further
      unify region handling.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      6a9ceb31
    • Tejun Heo's avatar
      memblock: Kill memblock_init() · fe091c20
      Tejun Heo authored
      memblock_init() initializes arrays for regions and memblock itself;
      however, all these can be done with struct initializers and
      memblock_init() can be removed.  This patch kills memblock_init() and
      initializes memblock with struct initializer.
      
      The only difference is that the first dummy entries don't have .nid
      set to MAX_NUMNODES initially.  This doesn't cause any behavior
      difference.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      Cc: Russell King <linux@arm.linux.org.uk>
      Cc: Michal Simek <monstr@monstr.eu>
      Cc: Paul Mundt <lethal@linux-sh.org>
      Cc: "David S. Miller" <davem@davemloft.net>
      Cc: Guan Xuetao <gxt@mprc.pku.edu.cn>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      fe091c20
    • Tejun Heo's avatar
      memblock: Kill sentinel entries at the end of static region arrays · c5a1cb28
      Tejun Heo authored
      memblock no longer depends on having one more entry at the end during
      addition making the sentinel entries at the end of region arrays not
      too useful.  Remove the sentinels.  This eases further updates.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      c5a1cb28
    • Tejun Heo's avatar
      memblock: Add __memblock_dump_all() · 4ff7b82f
      Tejun Heo authored
      Add __memblock_dump_all() which dumps memblock configuration whether
      memblock_debug is enabled or not.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      4ff7b82f
    • Tejun Heo's avatar
      memblock: Use memblock_reserve() in memblock internal functions · 9c8c27e2
      Tejun Heo authored
      Make memblock_double_array(), __memblock_alloc_base() and
      memblock_alloc_nid() use memblock_reserve() instead of calling
      memblock_add_region() with reserved array directly.  This eases
      debugging and updates to memblock_add_region().
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      9c8c27e2
    • Tejun Heo's avatar
      memblock: Make memblock_{add|remove|free|reserve}() return int and update prototypes · 581adcbe
      Tejun Heo authored
      memblock_{add|remove|free|reserve}() return either 0 or -errno but had
      long as return type.  Chage it to int.  Also, drop 'extern' from all
      prototypes in memblock.h - they are unnecessary and used
      inconsistently (especially if mm.h is included in the picture).
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Yinghai Lu <yinghai@kernel.org>
      581adcbe
  5. 01 Nov, 2011 3 commits
  6. 26 Jul, 2011 1 commit
  7. 14 Jul, 2011 11 commits
  8. 13 Jul, 2011 4 commits
  9. 23 Mar, 2011 1 commit
    • Benjamin Herrenschmidt's avatar
      mm/memblock: properly handle overlaps and fix error path · 8f7a6605
      Benjamin Herrenschmidt authored
      Currently memblock_reserve() or memblock_free() don't handle overlaps of
      any kind.  There is some special casing for coalescing exactly adjacent
      regions but that's about it.
      
      This is annoying because typically memblock_reserve() is used to mark
      regions passed by the firmware as reserved and we all know how much we can
      trust our firmwares...
      
      Also, with the current code, if we do something it doesn't handle right
      such as trying to memblock_reserve() a large range spanning multiple
      existing smaller reserved regions for example, or doing overlapping
      reservations, it can silently corrupt the internal region array, causing
      odd errors much later on, such as allocations returning reserved regions
      etc...
      
      This patch rewrites the underlying functions that add or remove a region
      to the arrays.  The new code is a lot more robust as it fully handles
      overlapping regions.  It's also, imho, simpler than the previous
      implementation.
      
      In addition, while doing so, I found a bug where if we fail to double the
      array while adding a region, we would remove the last region of the array
      rather than the region we just allocated.  This fixes it too.
      Signed-off-by: default avatarBenjamin Herrenschmidt <benh@kernel.crashing.org>
      Acked-by: default avatarYinghai Lu <yinghai@kernel.org>
      Cc: Ingo Molnar <mingo@elte.hu>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      8f7a6605
  10. 12 Feb, 2011 1 commit
    • Yinghai Lu's avatar
      memblock: don't adjust size in memblock_find_base() · e6d2e2b2
      Yinghai Lu authored
      While applying patch to use memblock to find aperture for 64bit x86.
      Ingo found system with 1g + force_iommu
      
      > No AGP bridge found
      > Node 0: aperture @ 38000000 size 32 MB
      > Aperture pointing to e820 RAM. Ignoring.
      > Your BIOS doesn't leave a aperture memory hole
      > Please enable the IOMMU option in the BIOS setup
      > This costs you 64 MB of RAM
      > Cannot allocate aperture memory hole (0,65536K)
      
      the corresponding code:
      
      	addr = memblock_find_in_range(0, 1ULL<<32, aper_size, 512ULL<<20);
      	if (addr == MEMBLOCK_ERROR || addr + aper_size > 0xffffffff) {
      		printk(KERN_ERR
      			"Cannot allocate aperture memory hole (%lx,%uK)\n",
      				addr, aper_size>>10);
      		return 0;
      	}
      	memblock_x86_reserve_range(addr, addr + aper_size, "aperture64")
      
      fails because memblock core code align the size with 512M.  That could
      make size way too big.
      
      So don't align the size in that case.
      
      actually __memblock_alloc_base, the another caller already align that
      before calling that function.
      
      BTW. x86 does not use __memblock_alloc_base...
      Signed-off-by: default avatarYinghai Lu <yinghai@kernel.org>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: David Miller <davem@davemloft.net>
      Cc: "H. Peter Anvin" <hpa@zytor.com>
      Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
      Cc: Dave Airlie <airlied@linux.ie>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      e6d2e2b2
  11. 21 Jan, 2011 1 commit
  12. 11 Oct, 2010 1 commit
    • Yinghai Lu's avatar
      memblock: Annotate memblock functions with __init_memblock · cd79481d
      Yinghai Lu authored
      Stephen found
      
      WARNING: mm/built-in.o(.text+0x25ab8): Section mismatch in reference from the function memblock_find_base() to the function .init.text:memblock_find_region()
      The function memblock_find_base() references
      the function __init memblock_find_region().
      This is often because memblock_find_base lacks a __init
      annotation or the annotation of memblock_find_region is wrong.
      
      So let memblock_find_region() to use __init_memblock instead of __init
      directly.
      
      Also fix one function that did not have __init* to be __init_memblock.
      Reported-by: default avatarStephen Rothwell <sfr@canb.auug.org.au>
      Signed-off-by: default avatarYinghai Lu <yinghai@kernel.org>
      LKML-Reference: <4CB366B1.40405@kernel.org>
      Signed-off-by: default avatarH. Peter Anvin <hpa@linux.intel.com>
      cd79481d