Ethernet issues
Since the switch to recent images with 5.x, I keep getting networking issues. The first kernel I used had great trouble with the network and was completely unusable. The current kernel I'm using, 5.2.0-rc5-gfd230181, has problems after a while:
[21509.093254] INFO: task kworker/2:0:3974 blocked for more than 120 seconds.
[21509.100204] Not tainted 5.2.0-rc5-gfd2301817 #1
[21509.105296] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[21509.113192] kworker/2:0 D 0 3974 2 0x00000028
[21509.128979] Workqueue: ipv6_addrconf addrconf_verify_work [ipv6]
[21509.145101] Call trace:
[21509.157728] __switch_to+0xac/0x1b0
[21509.171446] __schedule+0x1e0/0x488
[21509.185132] schedule+0x38/0xc8
[21509.198467] schedule_preempt_disabled+0x20/0x38
[21509.213240] __mutex_lock.isra.1+0x1a0/0x4c0
[21509.227734] __mutex_lock_slowpath+0x10/0x18
[21509.242200] mutex_lock+0x38/0x40
[21509.255714] rtnl_lock+0x14/0x20
[21509.259013] addrconf_verify_work+0xc/0x20 [ipv6]
[21509.263739] process_one_work+0x1dc/0x338
[21509.267831] worker_thread+0x48/0x430
[21509.271528] kthread+0xf4/0x120
[21509.274687] ret_from_fork+0x10/0x18
...
[21629.928046] INFO: task systemd-network:339 blocked for more than 120 seconds.
[21629.935277] Not tainted 5.2.0-rc5-gfd2301817 #1
[21629.940365] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[21629.948207] systemd-network D 0 339 1 0x00000801
[21629.948216] Call trace:
[21629.948233] __switch_to+0xac/0x1b0
[21629.948241] __schedule+0x1e0/0x488
[21629.948247] schedule+0x38/0xc8
[21629.948253] schedule_preempt_disabled+0x20/0x38
[21629.948260] __mutex_lock.isra.1+0x1a0/0x4c0
[21629.948265] __mutex_lock_slowpath+0x10/0x18
[21629.948271] mutex_lock+0x38/0x40
[21629.948282] rtnetlink_rcv_msg+0xe4/0x390
[21629.964028] netlink_rcv_skb+0xe8/0x130
[21629.969938] rtnetlink_rcv+0x14/0x20
[21629.976582] netlink_unicast+0x188/0x208
[21629.985514] netlink_sendmsg+0x26c/0x338
[21629.993086] sock_sendmsg+0x18/0x30
[21630.007312] __sys_sendto+0xb8/0x138
[21630.021343] __arm64_sys_sendto+0x20/0x30
[21630.021351] el0_svc_common.constprop.0+0xb0/0x168
[21630.021355] el0_svc_handler+0x18/0x20
[21630.021359] el0_svc+0x8/0xc
[21630.021393] INFO: task NetworkManager:541 blocked for more than 120 seconds.
[21630.035183] Not tainted 5.2.0-rc5-gfd2301817 #1
[21630.049268] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[21630.049272] NetworkManager D 0 541 1 0x00000000
[21630.049278] Call trace:
[21630.049288] __switch_to+0xac/0x1b0
[21630.049296] __schedule+0x1e0/0x488
[21630.161458] schedule+0x38/0xc8
[21630.164620] schedule_preempt_disabled+0x20/0x38
[21630.169249] __mutex_lock.isra.1+0x1a0/0x4c0
[21630.173575] __mutex_lock_slowpath+0x10/0x18
[21630.177867] mutex_lock+0x38/0x40
[21630.181195] rtnetlink_rcv_msg+0xe4/0x390
[21630.185216] netlink_rcv_skb+0xe8/0x130
[21630.189070] rtnetlink_rcv+0x14/0x20
[21630.192667] netlink_unicast+0x188/0x208
[21630.196626] netlink_sendmsg+0x26c/0x338
[21630.200562] sock_sendmsg+0x18/0x30
[21630.204059] ___sys_sendmsg+0x27c/0x2a8
[21630.207897] __sys_sendmsg+0x60/0xb8
[21630.211492] __arm64_sys_sendmsg+0x1c/0x28
[21630.215605] el0_svc_common.constprop.0+0xb0/0x168
[21630.220406] el0_svc_handler+0x18/0x20
[21630.224180] el0_svc+0x8/0xc
[21630.227129] INFO: task kworker/2:0:3974 blocked for more than 241 seconds.
[21630.234019] Not tainted 5.2.0-rc5-gfd2301817 #1
[21630.239080] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[21630.246929] kworker/2:0 D 0 3974 2 0x00000028
[21630.252511] Workqueue: ipv6_addrconf addrconf_verify_work [ipv6]
[21630.258548] Call trace:
[21630.261026] __switch_to+0xac/0x1b0
[21630.264530] __schedule+0x1e0/0x488
[21630.268028] schedule+0x38/0xc8
[21630.271169] schedule_preempt_disabled+0x20/0x38
[21630.275807] __mutex_lock.isra.1+0x1a0/0x4c0
[21630.280101] __mutex_lock_slowpath+0x10/0x18
[21630.284412] mutex_lock+0x38/0x40
[21630.287729] rtnl_lock+0x14/0x20
[21630.291029] addrconf_verify_work+0xc/0x20 [ipv6]
[21630.295788] process_one_work+0x1dc/0x338
[21630.299813] worker_thread+0x48/0x430
[21630.303487] kthread+0xf4/0x120
[21630.306663] ret_from_fork+0x10/0x18
...
[21750.769567] INFO: task systemd-network:339 blocked for more than 241 seconds.
[21750.776731] Not tainted 5.2.0-rc5-gfd2301817 #1
[21750.781820] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[21750.789667] systemd-network D 0 339 1 0x00000801
[21750.789675] Call trace:
[21750.789689] __switch_to+0xac/0x1b0
[21750.789698] __schedule+0x1e0/0x488
[21750.805374] schedule+0x38/0xc8
[21750.805381] schedule_preempt_disabled+0x20/0x38
[21750.811353] __mutex_lock.isra.1+0x1a0/0x4c0
[21750.825012] __mutex_lock_slowpath+0x10/0x18
[21750.825016] mutex_lock+0x38/0x40
[21750.825027] rtnetlink_rcv_msg+0xe4/0x390
[21750.832793] netlink_rcv_skb+0xe8/0x130
[21750.832801] rtnetlink_rcv+0x14/0x20
[21750.847255] netlink_unicast+0x188/0x208
[21750.847261] netlink_sendmsg+0x26c/0x338
[21750.861718] sock_sendmsg+0x18/0x30
[21750.861724] __sys_sendto+0xb8/0x138
[21750.869065] __arm64_sys_sendto+0x20/0x30
[21750.869078] el0_svc_common.constprop.0+0xb0/0x168
[21750.883089] el0_svc_handler+0x18/0x20
[21750.883095] el0_svc+0x8/0xc
[21750.896888] INFO: task NetworkManager:541 blocked for more than 241 seconds.
[21750.975029] Not tainted 5.2.0-rc5-gfd2301817 #1
[21750.980108] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[21750.987966] NetworkManager D 0 541 1 0x00000000
[21750.993526] Call trace:
[21750.996007] __switch_to+0xac/0x1b0
[21750.999529] __schedule+0x1e0/0x488
[21751.003030] schedule+0x38/0xc8
[21751.006204] schedule_preempt_disabled+0x20/0x38
[21751.010861] __mutex_lock.isra.1+0x1a0/0x4c0
[21751.015148] __mutex_lock_slowpath+0x10/0x18
[21751.019429] mutex_lock+0x38/0x40
[21751.022756] rtnetlink_rcv_msg+0xe4/0x390
[21751.026835] netlink_rcv_skb+0xe8/0x130
[21751.030699] rtnetlink_rcv+0x14/0x20
[21751.034286] netlink_unicast+0x188/0x208
[21751.038218] netlink_sendmsg+0x26c/0x338
[21751.042193] sock_sendmsg+0x18/0x30
[21751.045717] ___sys_sendmsg+0x27c/0x2a8
[21751.049566] __sys_sendmsg+0x60/0xb8
[21751.053141] __arm64_sys_sendmsg+0x1c/0x28
[21751.057269] el0_svc_common.constprop.0+0xb0/0x168
[21751.062130] el0_svc_handler+0x18/0x20
[21751.065909] el0_svc+0x8/0xc
[21751.068877] INFO: task kworker/2:0:3974 blocked for more than 362 seconds.
[21751.075784] Not tainted 5.2.0-rc5-gfd2301817 #1
[21751.080866] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[21751.088713] kworker/2:0 D 0 3974 2 0x00000028
[21751.094366] Workqueue: ipv6_addrconf addrconf_verify_work [ipv6]
[21751.100441] Call trace:
[21751.102914] __switch_to+0xac/0x1b0
[21751.106416] __schedule+0x1e0/0x488
[21751.109914] schedule+0x38/0xc8
[21751.113107] schedule_preempt_disabled+0x20/0x38
[21751.117756] __mutex_lock.isra.1+0x1a0/0x4c0
[21751.122040] __mutex_lock_slowpath+0x10/0x18
[21751.126319] mutex_lock+0x38/0x40
[21751.129696] rtnl_lock+0x14/0x20
[21751.132995] addrconf_verify_work+0xc/0x20 [ipv6]
[21751.137717] process_one_work+0x1dc/0x338
[21751.141736] worker_thread+0x48/0x430
[21751.145428] kthread+0xf4/0x120
[21751.148623] ret_from_fork+0x10/0x18
[21871.609162] INFO: task systemd-network:339 blocked for more than 362 seconds.
[21871.616374] Not tainted 5.2.0-rc5-gfd2301817 #1
[21871.621471] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[21871.629338] systemd-network D 0 339 1 0x00000801
[21871.634898] Call trace:
[21871.637398] __switch_to+0xac/0x1b0
[21871.640892] __schedule+0x1e0/0x488
[21871.644396] schedule+0x38/0xc8
[21871.647550] schedule_preempt_disabled+0x20/0x38
[21871.652266] __mutex_lock.isra.1+0x1a0/0x4c0
[21871.656577] __mutex_lock_slowpath+0x10/0x18
[21871.660861] mutex_lock+0x38/0x40
[21871.664203] rtnetlink_rcv_msg+0xe4/0x390
[21871.668260] netlink_rcv_skb+0xe8/0x130
[21871.672148] rtnetlink_rcv+0x14/0x20
[21871.675756] netlink_unicast+0x188/0x208
[21871.679690] netlink_sendmsg+0x26c/0x338
[21871.683625] sock_sendmsg+0x18/0x30
[21871.687212] __sys_sendto+0xb8/0x138
[21871.690820] __arm64_sys_sendto+0x20/0x30
[21871.694847] el0_svc_common.constprop.0+0xb0/0x168
[21871.699664] el0_svc_handler+0x18/0x20
[21871.703469] el0_svc+0x8/0xc
[21871.706412] INFO: task NetworkManager:541 blocked for more than 362 seconds.
[21871.713500] Not tainted 5.2.0-rc5-gfd2301817 #1
[21871.718560] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[21871.726450] NetworkManager D 0 541 1 0x00000000
[21871.731967] Call trace:
[21871.734434] __switch_to+0xac/0x1b0
[21871.737973] __schedule+0x1e0/0x488
[21871.741506] schedule+0x38/0xc8
[21871.744650] schedule_preempt_disabled+0x20/0x38
[21871.749300] __mutex_lock.isra.1+0x1a0/0x4c0
[21871.753582] __mutex_lock_slowpath+0x10/0x18
[21871.757923] mutex_lock+0x38/0x40
[21871.761280] rtnetlink_rcv_msg+0xe4/0x390
[21871.765304] netlink_rcv_skb+0xe8/0x130
[21871.769152] rtnetlink_rcv+0x14/0x20
[21871.772760] netlink_unicast+0x188/0x208
[21871.776730] netlink_sendmsg+0x26c/0x338
[21871.780668] sock_sendmsg+0x18/0x30
[21871.784188] ___sys_sendmsg+0x27c/0x2a8
[21871.788035] __sys_sendmsg+0x60/0xb8
[21871.791696] __arm64_sys_sendmsg+0x1c/0x28
[21871.795857] el0_svc_common.constprop.0+0xb0/0x168
[21871.800662] el0_svc_handler+0x18/0x20
[21871.804423] el0_svc+0x8/0xc
[21871.807436] INFO: task kworker/2:0:3974 blocked for more than 483 seconds.
[21871.814359] Not tainted 5.2.0-rc5-gfd2301817 #1
[21871.819427] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[21871.827341] kworker/2:0 D 0 3974 2 0x00000028
[21871.832920] Workqueue: ipv6_addrconf addrconf_verify_work [ipv6]
[21871.838968] Call trace:
[21871.841457] __switch_to+0xac/0x1b0
[21871.844975] __schedule+0x1e0/0x488
[21871.848493] schedule+0x38/0xc8
[21871.851647] schedule_preempt_disabled+0x20/0x38
[21871.856276] __mutex_lock.isra.1+0x1a0/0x4c0
[21871.860633] __mutex_lock_slowpath+0x10/0x18
[21871.864940] mutex_lock+0x38/0x40
[21871.868278] rtnl_lock+0x14/0x20
[21871.871582] addrconf_verify_work+0xc/0x20 [ipv6]
[21871.876332] process_one_work+0x1dc/0x338
[21871.880390] worker_thread+0x48/0x430
[21871.884092] kthread+0xf4/0x120
[21871.887248] ret_from_fork+0x10/0x18
In this instance, existing SSH connections were up and working but attempts to make new SSH connections failed. Sudo consistently froze immediately and refused to be backgrounded. The following appeared in the process list:
3974 ? D 0:00 \_ [kworker/2:0+ipv6_addrconf]
4911 ? D 0:06 \_ [kworker/u8:3+events_unbound]
339 ? Ds 0:00 /lib/systemd/systemd-networkd
541 ? Dsl 0:15 /usr/sbin/NetworkManager --no-daemon
4986 pts/0 D+ 0:00 | \_ sudo journalctl -b -k
5050 pts/1 D+ 0:00 | \_ sudo ls
4940 ? Ds 0:00 \_ /usr/sbin/sshd -D -R
4965 ? Ds 0:00 \_ /usr/sbin/sshd -D -R
4969 ? Ds 0:00 \_ /usr/sbin/sshd -D -R
4896 ? Ds 0:00 /lib/systemd/systemd-networkd
4910 ? Ds 0:00 /lib/systemd/systemd-networkd
4944 ? Ds 0:00 /lib/systemd/systemd-networkd
4958 ? Ds 0:00 /lib/systemd/systemd-networkd
4972 ? Ds 0:00 /lib/systemd/systemd-networkd
5023 ? Ds 0:00 /lib/systemd/systemd-networkd
5036 ? Ds 0:00 /lib/systemd/systemd-networkd