• Douglas Anderson's avatar
    kbuild: Add a cache for generated variables · 3298b690
    Douglas Anderson authored
    While timing a "no-op" build of the kernel (incrementally building the
    kernel even though nothing changed) in the Chrome OS build system I
    found that it was much slower than I expected.
    
    Digging into things a bit, I found that quite a bit of the time was
    spent invoking the C compiler even though we weren't actually building
    anything.  Currently in the Chrome OS build system the C compiler is
    called through a number of wrappers (one of which is written in
    python!) and can take upwards of 100 ms to invoke even if we're not
    doing anything difficult, so these invocations of the compiler were
    taking a lot of time.  Worse the invocations couldn't seem to take
    advantage of the multiple cores on my system.
    
    Certainly it seems like we could make the compiler invocations in the
    Chrome OS build system faster, but only to a point.  Inherently
    invoking a program as big as a C compiler is a fairly heavy
    operation.  Thus even if we can speed the compiler calls it made sense
    to track down what was happening.
    
    It turned out that all the compiler invocations were coming from
    usages like this in the kernel's Makefile:
    
    KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,)
    
    Due to the way cc-option and similar statements work the above
    contains an implicit call to the C compiler.  ...and due to the fact
    that we're storing the result in KBUILD_CFLAGS, a simply expanded
    variable, the call will happen every time the Makefile is parsed, even
    if there are no users of KBUILD_CFLAGS.
    
    Rather than redoing this computation every time, it makes a lot of
    sense to cache the result of all of the Makefile's compiler calls just
    like we do when we compile a ".c" file to a ".o" file.  Conceptually
    this is quite a simple idea.  ...and since the calls to invoke the
    compiler and similar tools are centrally located in the Kbuild.include
    file this doesn't even need to be super invasive.
    
    Implementing the cache in a simple-to-use and efficient way is not
    quite as simple as it first sounds, though.  To get maximum speed we
    really want the cache in a format that make can natively understand
    and make doesn't really have an ability to load/parse files. ...but
    make _can_ import other Makefiles, so the solution is to store the
    cache in Makefile format.  This requires coming up with a valid/unique
    Makefile variable name for each value to be cached, but that's
    solvable with some cleverness.
    
    After this change, we'll automatically create a ".cache.mk" file that
    will contain our cached variables.  We'll load this on each invocation
    of make and will avoid recomputing anything that's already in our
    cache.  The cache is stored in a format that it shouldn't need any
    invalidation since anything that might change should affect the "key"
    and any old cached value won't be used.
    Signed-off-by: 's avatarDouglas Anderson <dianders@chromium.org>
    Tested-by: 's avatarIngo Molnar <mingo@kernel.org>
    Tested-by: 's avatarGuenter Roeck <linux@roeck-us.net>
    Signed-off-by: 's avatarMasahiro Yamada <yamada.masahiro@socionext.com>
    3298b690
Makefile 57.9 KB