Tech Journal - Docker failed to mount drive

Problem

I have a container (lissy93/dashy) and another one (jellyfin) that is running normally using docker-compose. The docker-compose file looks similar to this

// Welcome to Dashy! To get started, run `docker compose up -d`
// You can configure your container here, by modifying this file
version: "3.8"
services:
  dashy:
    container_name: Dashy

    # Pull latest image from DockerHub
    image: lissy93/dashy

    # To build from source, replace 'image: lissy93/dashy' with 'build: .'
    # build: .

    # Or, to use a Dockerfile for your archtecture, uncomment the following
    # context: .
    # dockerfile: ./docker/Dockerfile-arm32v7

    # You can also use an image with a different tag, or pull from a different registry, e.g:
    # image: ghcr.io/lissy93/dashy or image: lissy93/dashy:arm64v8

    # Pass in your config file below, by specifying the path on your host machine
    volumes:
      - /home/username/dashy/config.yml:/app/public/conf.yml


    # Set port that web service will be served on. Keep container port as 80
    ports:
      - 2280:80

    # Set any environmental variables
    environment:
      - NODE_ENV=production
    # Specify your user ID and group ID. You can find this by running `id -u` and `id -g`
    #  - UID=1000
    #  - GID=1000

    # Specify restart policy
    restart: always

    # Configure healthchecks
    healthcheck:
      test: ['CMD', 'node', '/app/services/healthcheck']
      interval: 1m30s
      timeout: 10s
      retries: 3
      start_period: 40s

and this (jellyfin)

version: "3.9"
services:

  jellyfin:
    image: linuxserver/jellyfin:10.8.11
    container_name: jellyfin
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=UTC
    volumes:
      - jellyfin-config:/config
      - torrent-downloads:/data
    ports:
      - 2284:8096
      - 7359:7359/udp
      - 8920:8920
    restart: always
    networks:
      - media-stack-network

volumes:
  torrent-downloads: 
    driver: local
    driver_opts:
      o: bind
      type: none
      device: /media/username/E/TorrentDownloads


  jellyfin-config: 
    driver: local
    driver_opts:
      o: bind
      type: none
      device: ./jellyfin.config

  qbittorrent-config: 
    driver: local
    driver_opts:
      o: bind
      type: none
      device: ./qbittorrent.config


networks:
  media-stack-network:
    name: media-stack-network

The container works extremely well after I manually run docker-compose up. However, when the machine restart, Docker engine failed to start my container. According to the log from sudo journalctl -n docker.service, the error is

(for dashy)

failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting \"/home/username/dashy/config.yml\" to rootfs at \"/app/public/conf.yml\": mount /home/username//dashy/config.yml:/app/public/conf.yml (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type"

(for jellyfin)

ockerd[1397]: time="2024-02-18T14:55:05.354714568+07:00" level=error msg="failed to start container" container=eae3dd9446da55032f568dc0993efbeeb7c01b096206babce37b22f933e63d38 error="error while mounting volume '/var/lib/docker/volumes/media-stack_jellyfin-config/_data': failed to mount local volume: mount /home/username/media-stack/jellyfin.config:/var/lib/docker/volumes/media-stack_jellyfin-config/_data, flags: 0x1000: no such file or directory"
My first assumption is that the docker start before file/folder mounted successfully, so I edited the docker.service file to be like this

[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target docker.socket firewalld.service containerd.service time-set.target
Wants=network-online.target containerd.service
Requires=docker.socket

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
RequiresMountsFor= /media/username/E /home/username/
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutStartSec=0
RestartSec=2
Restart=always

(...)

Then I run system daemon-reload and “systemctl reload docker.service to trigger the change. But it’s not working. Also, I assume that the Docker version I am running is too old, but when runnng docker info the output is contradictory to my assumption

Client: Docker Engine - Community
 Version:    25.0.3
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.24.5
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Also, the check from docker-compose -v show that my version is 1.29

I attempted to use Portainer (from a different machine) to start the Dashy before loggin to the Linux OS. The request was failed with return code 400. The same with Jellyfin with Error code 500

Solution

Thanks for the help of my brilliant older brother, Linh Nguyen Vinh, he suggests that this is due to the fact that the home folder has not be mounted before I logging to the OS. Hence, he suggests two solutions:

1. To mount the home/username folder before logging.
2. To move the whole docker files and mounted drive to /opts/ folder.

I decided to go with the second method and it works.

Conclusion

Always deploy to opts folder.