d/patches: Use newer libhandy and remove hacks

Fixes #21 (closed)

I can also take a look at the amber package, but I got confused by the different branches.

Please note that I could not get the tests to build on my host machine (engine tests timed out even if raising to 120s).

Edit: CI pipeline however seems happy