Skip to content
  • Max Gurtovoy's avatar
    blk-mq: map all HWQ also in hyperthreaded system · fe631457
    Max Gurtovoy authored
    
    
    This patch performs sequential mapping between CPUs and queues.
    In case the system has more CPUs than HWQs then there are still
    CPUs to map to HWQs. In hyperthreaded system, map the unmapped CPUs
    and their siblings to the same HWQ.
    This actually fixes a bug that found unmapped HWQs in a system with
    2 sockets, 18 cores per socket, 2 threads per core (total 72 CPUs)
    running NVMEoF (opens upto maximum of 64 HWQs).
    
    Performance results running fio (72 jobs, 128 iodepth)
    using null_blk (w/w.o patch):
    
    bs      IOPS(read submit_queues=72)   IOPS(write submit_queues=72)   IOPS(read submit_queues=24)  IOPS(write submit_queues=24)
    -----  ----------------------------  ------------------------------ ---------------------------- -----------------------------
    512    4890.4K/4723.5K                 4524.7K/4324.2K                   4280.2K/4264.3K               3902.4K/3909.5K
    1k     4910.1K/4715.2K                 4535.8K/4309.6K                   4296.7K/4269.1K               3906.8K/3914.9K
    2k     4906.3K/4739.7K                 4526.7K/4330.6K                   4301.1K/4262.4K               3890.8K/3900.1K
    4k     4918.6K/4730.7K                 4556.1K/4343.6K                   4297.6K/4264.5K               3886.9K/3893.9K
    8k     4906.4K/4748.9K                 4550.9K/4346.7K                   4283.2K/4268.8K               3863.4K/3858.2K
    16k    4903.8K/4782.6K                 4501.5K/4233.9K                   4292.3K/4282.3K               3773.1K/3773.5K
    32k    4885.8K/4782.4K                 4365.9K/4184.2K                   4307.5K/4289.4K               3780.3K/3687.3K
    64k    4822.5K/4762.7K                 2752.8K/2675.1K                   4308.8K/4312.3K               2651.5K/2655.7K
    128k   2388.5K/2313.8K                 1391.9K/1375.7K                   2142.8K/2152.2K               1395.5K/1374.2K
    
    Signed-off-by: default avatarMax Gurtovoy <maxg@mellanox.com>
    Reviewed-by: default avatarChristoph Hellwig <hch@lst.de>
    Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
    fe631457