Systemd network-online issues

8 April, 2025 - Categories: systemd, linux, homelab

Had a fun time troubleshooting a systemd issue today. All I wanted to do was to only start a service once a network share is mounted.

Turns out that's harder than expected when you're trying to mount the share as a user. Permissions-wise, at least using sshfs (a FUSE filesystem) you can completely use it from a non-root account. so that's what I did - and since it's a Systemd/Linux system I put the mount file in the usual ~/.config/systemd/user location. A mount file there will attempt to mount the specified share on boot.

Systemd docs also say that if it is a network share, it will get an automatic dependency on network.target and network-online.target. In theory, this will make systemd only attempt to mount the share once network is online.

Well - turns out USER units cannot depend on SYSTEM units, and with network-online.target being a system kinda thing, no matter how explicitly you specify you want to mount AFTER it, systemd tries to mount it ASAP, far before any networking is available, and at least SSHFS does not have some kind of "timeout" configured - it immediately fails, failing the mount, and in turn failing whatever service depends on the mount, never retrying.

I've tried simply adding some retry and timeout statements, but those did not work in this case - maybe they don't work for mounts, maybe some other issue, cba.

SOLUTION: either make a custom service that simply executes /usr/lib/sysystemd-networkd-wait-online --any (the ANY is important if you have more than one network adapter - if even one of your adapters isn't online, the entire thing will fail, despite networking working perfectly fine.) OR if you do use podman (which i do), there is a user service called "podman-user-wait-network-online.service" that gives you the same functionality just available as a user. this is the one i went with for today's issue, since the service i want to start is a Jellyfin server relying on a network share that stores all the media.