• James Hogan's avatar
    lib/mpi: Fix umul_ppmm() for MIPS64r6 · bbc25bee
    James Hogan authored
    Current MIPS64r6 toolchains aren't able to generate efficient
    DMULU/DMUHU based code for the C implementation of umul_ppmm(), which
    performs an unsigned 64 x 64 bit multiply and returns the upper and
    lower 64-bit halves of the 128-bit result. Instead it widens the 64-bit
    inputs to 128-bits and emits a __multi3 intrinsic call to perform a 128
    x 128 multiply. This is both inefficient, and it results in a link error
    since we don't include __multi3 in MIPS linux.
    For example commit 90a53e44 ("cfg80211: implement regdb signature
    checking") merged in v4.15-rc1 recently broke the 64r6_defconfig and
    64r6el_defconfig builds by indirectly selecting MPILIB. The same build
    errors can be reproduced on older kernels by enabling e.g. CRYPTO_RSA:
    lib/mpi/generic_mpih-mul1.o: In function `mpihelp_mul_1':
    lib/mpi/generic_mpih-mul1.c:50: undefined reference to `__multi3'
    lib/mpi/generic_mpih-mul2.o: In function `mpihelp_addmul_1':
    lib/mpi/generic_mpih-mul2.c:49: undefined reference to `__multi3'
    lib/mpi/generic_mpih-mul3.o: In function `mpihelp_submul_1':
    lib/mpi/generic_mpih-mul3.c:49: undefined reference to `__multi3'
    lib/mpi/mpih-div.o In function `mpihelp_divrem':
    lib/mpi/mpih-div.c:205: undefined reference to `__multi3'
    lib/mpi/mpih-div.c:142: undefined reference to `__multi3'
    Therefore add an efficient MIPS64r6 implementation of umul_ppmm() using
    inline assembly and the DMULU/DMUHU instructions, to prevent __multi3
    calls being emitted.
    Fixes: 7fd08ca5 ("MIPS: Add build support for the MIPS R6 ISA")
    Signed-off-by: default avatarJames Hogan <jhogan@kernel.org>
    Cc: Ralf Baechle <ralf@linux-mips.org>
    Cc: Herbert Xu <herbert@gondor.apana.org.au>
    Cc: "David S. Miller" <davem@davemloft.net>
    Cc: linux-mips@linux-mips.org
    Cc: linux-crypto@vger.kernel.org
    Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>