• Ying Huang's avatar
    smp: Avoid using two cache lines for struct call_single_data · 966a9671
    Ying Huang authored
    struct call_single_data is used in IPIs to transfer information between
    CPUs.  Its size is bigger than sizeof(unsigned long) and less than
    cache line size.  Currently it is not allocated with any explicit alignment
    requirements.  This makes it possible for allocated call_single_data to
    cross two cache lines, which results in double the number of the cache lines
    that need to be transferred among CPUs.
    
    This can be fixed by requiring call_single_data to be aligned with the
    size of call_single_data. Currently the size of call_single_data is the
    power of 2.  If we add new fields to call_single_data, we may need to
    add padding to make sure the size of new definition is the power of 2
    as well.
    
    Fortunately, this is enforced by GCC, which will report bad sizes.
    
    To set alignment requirements of call_single_data to the size of
    call_single_data, a struct definition and a typedef is used.
    
    To test the effect of the patch, I used the vm-scalability multiple
    thread swap test case (swap-w-seq-mt).  The test will create multiple
    threads and each thread will eat memory until all RAM and part of swap
    is used, so that huge number of IPIs are triggered when unmapping
    memory.  In the test, the throughput of memory writing improves ~5%
    compared with misaligned call_single_data, because of faster IPIs.
    Suggested-by: default avatarPeter Zijlstra <peterz@infradead.org>
    Signed-off-by: default avatarHuang, Ying <ying.huang@intel.com>
    [ Add call_single_data_t and align with size of call_single_data. ]
    Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
    Cc: Aaron Lu <aaron.lu@intel.com>
    Cc: Borislav Petkov <bp@suse.de>
    Cc: Eric Dumazet <eric.dumazet@gmail.com>
    Cc: Juergen Gross <jgross@suse.com>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Michael Ellerman <mpe@ellerman.id.au>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Link: http://lkml.kernel.org/r/87bmnqd6lz.fsf@yhuang-mobile.sh.intel.comSigned-off-by: default avatarIngo Molnar <mingo@kernel.org>
    966a9671