Birch locks up: mmc0: Timeout waiting for hardware interrupt after xhci not responding

(too often to work reliably) my Birch device locks up the following way. I see this from day 1 I got the device, on our 5.3 current kernels:

[   47.081762] xhci-hcd xhci-hcd.3.auto: xHCI host controller not responding, assume dead
[   47.089734] xhci-hcd xhci-hcd.3.auto: HC died; cleaning up
[   57.569654] mmc0: Timeout waiting for hardware interrupt.
[   57.575062] mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
[   57.581503] mmc0: sdhci: Sys addr:  0x00000068 | Version:  0x00000002
[   57.587943] mmc0: sdhci: Blk size:  0x00000200 | Blk cnt:  0x00000000
[   57.594384] mmc0: sdhci: Argument:  0x002de2c0 | Trn mode: 0x0000002b
[   57.600824] mmc0: sdhci: Present:   0x01f88008 | Host ctl: 0x00000031
[   57.607264] mmc0: sdhci: Power:     0x00000002 | Blk gap:  0x00000080
[   57.613704] mmc0: sdhci: Wake-up:   0x00000008 | Clock:    0x0000000f
[   57.620144] mmc0: sdhci: Timeout:   0x0000008f | Int stat: 0x00000003
[   57.626584] mmc0: sdhci: Int enab:  0x117f100b | Sig enab: 0x117f100b
[   57.633024] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00008402
[   57.639464] mmc0: sdhci: Caps:      0x07eb0000 | Caps_1:   0x8000b407
[   57.645904] mmc0: sdhci: Cmd:       0x0000193a | Max curr: 0x00ffffff
[   57.652344] mmc0: sdhci: Resp[0]:   0x00000900 | Resp[1]:  0xffffffff
[   57.658783] mmc0: sdhci: Resp[2]:   0x328f5903 | Resp[3]:  0x00000900
[   57.665223] mmc0: sdhci: Host ctl2: 0x00000088
[   57.669667] mmc0: sdhci: ADMA Err:  0x00000000 | ADMA Ptr: 0xfec45268
[   57.676106] mmc0: sdhci: ============================================
[   68.577644] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[   68.583743] rcu: 	0-...!: (14 GPs behind) idle=754/0/0x0 softirq=12186/12186 fqs=0 
[   68.591401] rcu: 	1-...!: (12 GPs behind) idle=d14/0/0x0 softirq=11271/11272 fqs=0 
[   68.599058] rcu: 	2-...!: (18 GPs behind) idle=8e0/0/0x0 softirq=10355/10355 fqs=0 
[   68.606800] rcu: rcu_preempt kthread starved for 5252 jiffies! g11953 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=3
[   68.617058] rcu: RCU grace-period kthread stack dump:

It doesn't happen in every session, and it also happens that I see the "mmc0 timeout" only, during startup already, which then does not fully hang the system. A bit confusing, I know.

Who else sees these lock-ups?

Sidenote: I've had the same mmc timeouts on mmc1 too, which I solved by removing and properly re-inserting the wifi card.