Immich: Machine Learning, GPU Acceleration, and Storage Management (Part 2)

Hardware GPU Acceleration Configuration

Hardware acceleration significantly accelerates machine learning tasks (object detection, facial recognition, CLIP search) and transcoding.

NVIDIA GPU Passthrough (CUDA)

To leverage NVIDIA GPUs for the immich-machine-learning and immich-server containers, the host must have the NVIDIA driver and nvidia-container-toolkit installed.

Ensure the Docker daemon is configured to use the NVIDIA runtime by editing /etc/docker/daemon.json:

{
  "runtimes": {
    "nvidia": {
      "path": "nvidia-container-runtime",
      "runtimeArgs": []
    }
  }
}

In your docker-compose.yml, pass the GPU through to the immich-machine-learning service:

version: '3.8'

services:
  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:release-cuda
    volumes:
      - model-cache:/cache
    environment:
      - IMMICH_MEDIA_LOCATION=/usr/src/app/upload
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    restart: always

volumes:
  model-cache:

For hardware-accelerated video transcoding, configure the immich-server container under the environment and deploy blocks:

  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:release
    # ... other settings ...
    environment:
      - IMMICH_MEDIA_LOCATION=/usr/src/app/upload
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

Intel GPU Passthrough (QuickSync/VAAPI)

Intel GPUs use VAAPI (Video Acceleration API) and QuickSync for transcoding, and OpenVINO for machine learning inference.

To expose the Intel GPU, map the direct rendering infrastructure devices (/dev/dri) from the host into the containers.

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:release
    devices:
      - /dev/dri:/dev/dri
    # ... other settings ...

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:release
    devices:
      - /dev/dri:/dev/dri
    # Optional environment variables to force OpenVINO device selection
    environment:
      - IMMICH_MACHINE_LEARNING_DEVICE=openvino
      - OPENVINO_DEVICE=GPU

Verify device permissions on the host. The container user needs read/write access to /dev/dri/renderD128. Run ls -l /dev/dri on your host to check the group ID (often render or video), and ensure your Docker container runs with the correct group membership:

    group_add:
      - "105" # Match host's 'render' or 'video' group GID

Machine Learning Container Tuning

The immich-machine-learning container handles computationally heavy tasks like facial detection (InsightFace) and semantic search (CLIP). Tuning this service prevents CPU/GPU bottlenecks.

Device Selection & Acceleration Backend

Configure the inference engine via the environment variables inside immich-machine-learning:

• IMMICH_MACHINE_LEARNING_DEVICE: Controls the execution device. Options are cpu, cuda, or openvino.
• MACHINE_LEARNING_GPU_ACCELERATION: Set to true to enable GPU acceleration for models.

    environment:
      - IMMICH_MACHINE_LEARNING_DEVICE=cuda # or openvino / cpu
      - MACHINE_LEARNING_GPU_ACCELERATION=true

Model Cache Persistent Storage

Avoid downloading the ML models every time the container restarts by mounting a dedicated cache volume. The default cache directory is /cache.

    volumes:
      - model-cache:/cache

CPU-Only Tuning (Thread/Concurrency Optimization)

If running without a dedicated GPU, limit thread utilization to prevent the container from starving the host OS or immich-server.

Set thread counts explicitly using PyTorch and ONNX Runtime environment variables:

    environment:
      - OMP_NUM_THREADS=4
      - MKL_NUM_THREADS=4
      - OPENBLAS_NUM_THREADS=4
      - VECLIB_MAXIMUM_THREADS=4
      - NUMEXPR_NUM_THREADS=4
      - ONNXRUNTIME_CPU_THREAD_LIMIT=4

Align these limits with the physical cores available on your system, leaving headroom for PostgreSQL and Node.js server operations.

PostgreSQL & Pgvector Tuning

Immich stores high-dimensional image embeddings generated by CLIP in a PostgreSQL database using the pgvector extension. Large library sizes require optimizing memory configurations and indexing strategies.

Core Database Memory Configuration

Modify your PostgreSQL configuration (postgresql.conf or using Docker command-line overrides) to optimize for the vector search workload:

# Memory Configuration for a 16GB RAM System
shared_buffers = 4GB                  # 25% of total system memory
work_mem = 64MB                       # Memory per-operation for sorting and hashing
maintenance_work_mem = 2GB            # High allocation to speed up HNSW index creation
effective_cache_size = 12GB           # Estimate of total memory available for caching

For docker-compose setups, pass configuration flags directly via the command block:

  database:
    container_name: immich_postgres
    image: tensorchord/pgvecto-rs:pg16-v0.2.0
    environment:
      - POSTGRES_PASSWORD=X7z9UsI4nuar0YwnucxpUG77
      - POSTGRES_USER=postgres
      - POSTGRES_DB=immich
    volumes:
      - pgdata:/var/lib/postgresql/data
    command: >
      postgres
      -c shared_buffers=4GB
      -c work_mem=64MB
      -c maintenance_work_mem=2GB
      -c effective_cache_size=12GB
    restart: always

Vector Index Optimizations

Immich utilizes Hierarchical Navigable Small World (HNSW) indexes for efficient approximate nearest neighbor (ANN) vector search.

During initial bulk importing, index creation can consume substantial CPU and RAM. Adjust maintenance parameters to prevent out-of-memory (OOM) crashes:

• Index Build Tuning: Increase maintenance_work_mem to prevent swapping to disk during indexing.
• pgvector HNSW Parameters: Ensure database tables do not timeout during query operations by adjusting the index build options.

You can verify the current indexes by logging into the PostgreSQL container:

docker exec -it immich_postgres psql -U postgres -d immich

Check existing vector indexes:

SELECT relname, indexrelname, amname FROM pg_stat_all_indexes WHERE schemaname = 'public';

Mounting External Storage Safely

By default, Immich manages all uploads in its internal upload directory. To index pre-existing photo libraries without moving or copying files, configure external libraries.

Host Path Bind Mounts

Mount the external directories into the immich-server and immich-machine-learning containers as read-only. This protects your primary archives from accidental deletions or modifications by Immich.

services:
  immich-server:
    # ...
    volumes:
      - immich-data:/usr/src/app/upload
      - /mnt/nas/photos/archive:/usr/src/app/external/archive:ro

  immich-machine-learning:
    # ...
    volumes:
      - model-cache:/cache
      - /mnt/nas/photos/archive:/usr/src/app/external/archive:ro

[!IMPORTANT] The paths mounted under /usr/src/app/external/ must match exactly between the immich-server and immich-machine-learning containers. Discrepancies will cause machine learning jobs to fail because the path references saved in the database won't resolve correctly inside the ML container.

User and Group Permission Alignment

Ensure the user inside the container has read access to the mounted host directories. Immich containers typically run as user node (UID 1000) or root, but can be forced to run as specific system users.

Check the ownership of the files on the host:

ls -ld /mnt/nas/photos/archive

If the host folder is owned by UID 1024 and GID 100, specify the environment variable in your .env and map it in the compose file if your container image supports user mapping, or adjust host permissions to grant read/execute permissions:

# Allow read-only access for others on the host directory
chmod -R o+rX /mnt/nas/photos/archive

Adding the Library in the Immich UI

Once the paths are mounted: 1. Navigate to Administration -> External Libraries in the Immich Web UI. 2. Click Create Library. 3. Under Import Paths, add the path mapped inside the container (e.g., /usr/src/app/external/archive). 4. Run the Scan Library job to index the assets safely.