build: Avoid testing some UI files

If UI files contain widgets from libhandy, tests seems to fail
8 jobs for wip/sadiq/various-ui-fixes in 11 minutes and 45 seconds (queued for 1 second)