Skip to content

Storage & Download Reliability Troubleshooting Guide ​

This guide describes how to diagnose storage issues, understand YTMDL's resilient storage state machine, and resolve operational problems.


1. Storage Status Codes & Diagnostics ​

YTMDL v0.11.0 monitors both the destination music library and the local persistent staging directory. You can inspect live diagnostics via GET /api/v1/storage/status (Admin only) or the Settings/Storage UI in the frontend.

Status CodeMeaningRoot CauseResolution
healthyStorage is verified and writableMount is online, identity marker matches, free space is above limit.No action required.
guard_missingMarker file .ytmdl-storage-id is absentThe network share is unmounted (pointing to an empty host directory) or marker was deleted.Verify host mount with mountpoint /srv/music. Ensure .ytmdl-storage-id is present on the NAS root.
guard_mismatchMarker file UUID does not match configA different volume is mounted, or storage_guard_id was misconfigured.Update storage_guard_id in config to match the UUID in .ytmdl-storage-id.
read_onlyShare cannot be written toNFS export is read-only, CIFS credentials lack write permissions, or host folder ownership is wrong.Check permissions on NAS/host. Ensure UID/GID 10001 has write access.
low_spaceAvailable space is below min_free_bytesStorage volume is nearly full.Free up disk space or adjust min_free_bytes safety threshold.
unavailablePath does not exist or statfs failsHost mount point is missing or NAS is offline.Check NAS network connectivity and remount on host.

2. Storage Outage Behavior & Safe States ​

When a network storage outage occurs during unattended 24/7 operation:

  1. Zero Data Loss & No False Fails:
    • YTMDL does not fail the job or spend retry attempts when storage drops.
    • Active download items waiting for finalization transition to waiting_for_storage or waiting_for_space.
    • Completed audio tracks remain fully preserved in /data/staging/<item-id>/.
  2. Automatic Background Recovery:
    • The background storage monitor probes the storage guard every 30 seconds.
    • When the NAS comes back online and passes guard verification, YTMDL wakes the queue and automatically commits all staged tracks.
  3. Container Health Independence:
    • The Podman container health check remains healthy during storage outages. This prevents Podman restart loops from killing active staging downloads.

3. Persistent Staging & Resume ​

  • Local staging is stored under /data/staging/<item-id>/.
  • Each download attempt works in its own .ytdm-attempt-* directory inside the item's staging directory. It is removed when the attempt ends, and one left by an interrupted process is removed by the item's next attempt, so a partial download is not resumed; the attempt starts again. The staging status still counts partial files inside running attempts.
  • Corrupted audio files (detected by duration or ffprobe verification) are automatically cleaned and re-downloaded up to 2 times.
  • Staged artifacts are cleaned up only after the database transaction commits.

Staging lifecycle ​

An item's staging directory lives exactly as long as the item can still use it:

Item stateStaging
pending, matching, downloading, tagging, finalizingkept — the work is running or will resume
retry_wait, waiting_for_storage, waiting_for_spacekept — the item is not finished; a verified file staged before the wait stays in place
completed, skipped, cancelled, failedremoved once that final state is stored

If the final state cannot be stored, the directory is kept for the next start to judge. A final failure that you retry later simply downloads again.

At start, after interrupted work has been returned to the queue and before any worker runs, YTMDL removes staging directories that are no longer needed. A directory is removed only when all of this holds:

  • its name is an item id (32 lower-case hex characters);
  • it is a real directory directly below the staging root — symbolic links are never followed or removed, and nothing outside the root is touched;
  • the database reports its item in a final state.

Directories of active, waiting or retryable items are kept, and so is everything the database does not know or that is not an item directory. If the item states cannot be read, nothing is removed. The backend log reports each run as staging pruned with the counts of removed and kept entries.


4. Idempotent Finalization (Crash Recovery) ​

If the server crashes or loses power after writing a track to /music/Artist/Album/01 - Song.opus but before the database records it:

  • On restart, YTMDL re-verifies the target file's SHA-256 checksum and file size against staged_sha256.
  • If the checksum matches, YTMDL recognizes the file was already successfully placed, skips copying, and updates the database without triggering a PATH_CONFLICT.
  • If a foreign, different file exists at that path, YTMDL strictly refuses to overwrite it and reports PATH_CONFLICT.

5. Storage Identity Guard Verification Command ​

To manually test guard verification on the host:

bash
# Check marker content
cat /srv/music/.ytmdl-storage-id

# Expected format:
# ytmdl-storage:12345678-1234-1234-1234-123456789abc

# Test write probe as UID 10001
sudo -u '#10001' touch /srv/music/.ytmdl-health-test && sudo rm -f /srv/music/.ytmdl-health-test

6. Library Reconciliation & Legacy File Audit Note ​

When comparing the total count of physical audio files on disk with the database files table:

  • Historical development versions prior to v0.9 used unstandardized folder naming (such as [Single] folder tags or single-artist folder names instead of collaborative album artists).
  • In modernized versions (v0.10.0+), tracks are catalogued and placed strictly according to the Plex/Jellyfin/Emby compatible path standard (e.g. Album Artist/Year - Title/Track - Title.opus).
  • Legacy files remaining in older directory locations are treated as unmanaged files. They represent no data corruption and are never automatically deleted or overwritten by YTMDL.
  • Operators can identify and manage unreferenced files via the Library Scan API (GET /api/v1/library/scan) and safely remove specific orphan issues via DELETE /api/v1/library/scan/issues/{id} if desired.

Released under the Apache 2.0 License.