Labels change when keyboard is clicked

Steps to reproduce:

0. Use Polish layout in gsettings
1. Fire up rootston
2. Fire up a non-text-input application (yad --entry works with GTK<3.24)
3. Fire up virtboard
4. Type a number

Expected result:

• letter is entered

Actual result:

• letter is entered
• labels on keys momentarily change