Skip to content
  • Ting-Wei Lan's avatar
    configure: Don't declare functions in AC_TRY_COMPILE and AC_LANG_PROGRAM · 31f7b553
    Ting-Wei Lan authored
    Both AC_TRY_COMPILE and AC_LANG_PROGRAM put code passed to their second
    arguments to the body of the main function. This means that we cannot
    and should not declare functions there, or we end up checking whether
    the compiler support nested functions instead of whether a compiler or
    linker flag is supported.
    
    GCC supports nested functions and tests succeed. Clang doesn't support
    nested functions, so tests fail and -fvisibility=hidden won't be used.
    This means that functions which are not intended to be used by other
    programs, such as gtk_menu_tracker*, gtk_action_observ*,
    gtk_menu_muxer_*, become global symbols with default visibility.
    
    GNOME Shell has a private library libgnome-shell-menu.so, which also has
    symbols gtk_menu_tracker*, gtk_action_observ*, gtk_menu_muxer_* that are
    intended to be used by GNOME Shell itself. When GNOME Shell still used
    Autotools build system, the executable gnome-shell explicitly linked to
    libgnome-shell-menu.so, so the linker loaded libgnome-shell-menu.so
    before libgtk-3.so.0 and GNOME Shell used correct symbols from its
    private library.
    
    However, after GNOME Shell switched to Meson build system, gnome-shell
    executable no longer lists libgnome-shell-menu.so as its dependency.
    Even if we adds it to the build file, it won't be listed in DT_NEEDED of
    gnome-shell because Meson uses -Wl,--as-needed by default. This causes
    the runtime linker to load libgtk-3.so.0 before libgnome-shell-menu.so
    and symbols gtk_menu_tracker*, gtk_action_observ*, gtk_menu_muxer_* are
    bound to libgtk-3.so.0 instead of libgnome-shell-menu.so. GNOME Shell
    hangs when opening more than one window because it uses functions from
    the wrong library.
    
    This problem is already fixed in OpenBSD ports. The article describing
    it can be found on OpenBSD Journal with this link:
    https://undeadly.org/cgi?action=article;sid=20170930133438
    
    https://bugzilla.gnome.org/show_bug.cgi?id=791943
    31f7b553