dwc3 gadget driver causes resume to fail

Feature: https://source.puri.sm/Librem5/use-cases/issues/75

If the dwc3 driver is load during suspend to mem then resume will fail

depending issue: https://source.puri.sm/Librem5/linux-next/issues/64