TechSetupGuides
Intermediateaillmn8nollamaqdrantpostgresqldockerworkflowautomationself-hosted

n8n Self-Hosted AI Starter Kit

Open-source Docker Compose template that rapidly sets up local AI infrastructure with n8n workflows, Ollama LLMs, and Qdrant vector database.

  1. Step 1

    What is the Self-Hosted AI Starter Kit?

    The n8n Self-Hosted AI Starter Kit is an open-source Docker Compose template that provides a complete, production-ready AI development environment running entirely on your own hardware. It combines low-code workflow automation with local language models, enabling you to build AI agents, chatbots, document processors, and automation workflows without sending data to external services.

    With 14,868+ stars on GitHub, this kit has become a go-to solution for developers who want the power of modern AI tools while maintaining full control over their data and infrastructure.

  2. Step 2

    Technology Stack

    The starter kit integrates four core open-source technologies into a cohesive AI development platform:

    n8n (Workflow Automation)

    • Low-code workflow automation platform
    • 400+ integrations with popular services
    • Advanced AI nodes for agents, classification, and extraction
    • Visual workflow builder with debugging capabilities
    • Image: n8nio/n8n:latest
    • Web interface on port 5678

    Ollama (Local LLM Runtime)

    • Runs large language models locally
    • Supports Llama, Mistral, Gemma, and many other models
    • GPU acceleration support (NVIDIA, AMD) or CPU fallback
    • Image: ollama/ollama:latest
    • API on port 11434

    Qdrant (Vector Database)

    • Open-source vector similarity search engine
    • Essential for RAG (Retrieval-Augmented Generation) workflows
    • Comprehensive REST and gRPC API
    • Image: qdrant/qdrant
    • API on port 6333

    PostgreSQL (Data Persistence)

    • Reliable relational database for n8n's workflow data
    • Stores credentials, execution history, and configurations
    • Image: postgres:16-alpine
    • Internal only (not exposed externally)

    All components communicate through Docker's internal networking, with persistent volumes for data storage.

    Core Stack:
├── n8n (latest) - Workflow automation on :5678
├── Ollama (latest) - Local LLMs on :11434
├── Qdrant (latest) - Vector DB on :6333
└── PostgreSQL (16-alpine) - Data persistence

Docker Profiles:
├── gpu-nvidia - NVIDIA GPU acceleration
├── gpu-amd - AMD GPU acceleration (Linux)
├── cpu - CPU-only execution
└── default - Mac/standard deployment
  3. Step 3

    Prerequisites

    Before installing the Self-Hosted AI Starter Kit, ensure your system meets these requirements:

    Required Software:

    • Docker and Docker Compose - Container runtime and orchestration
    • Git - To clone the repository

    Hardware Considerations:

    For CPU-only deployment, 8GB RAM minimum is recommended. For GPU acceleration, you'll need:

    • NVIDIA GPU: CUDA-capable GPU with appropriate drivers. If you haven't used your NVIDIA GPU with Docker before, follow the Ollama Docker NVIDIA setup guide
    • AMD GPU (Linux only): ROCm-compatible GPU with drivers installed
    • Mac/Apple Silicon: Note that GPUs cannot be exposed to Docker containers on Mac. You can run Ollama locally on macOS and connect it to the Dockerized n8n, or use the CPU-only profile

    Note on GPU Support: GPU acceleration significantly improves LLM inference speed. If you have a compatible GPU, it's worth configuring Docker for GPU access before proceeding.

    # Verify Docker installation
docker --version
docker compose version

# Verify Git installation
git --version

# For NVIDIA GPU users, verify GPU is accessible to Docker
docker run --rm --gpus all nvidia/cuda:12.0.0-base-ubuntu22.04 nvidia-smi
  4. Step 4

    Installation - Standard Setup

    The installation process is straightforward and takes just a few minutes. These steps work for all hardware configurations:

    # 1. Clone the repository
git clone https://github.com/n8n-io/self-hosted-ai-starter-kit.git
cd self-hosted-ai-starter-kit

# 2. Create environment configuration
cp .env.example .env

# 3. Edit .env file and update passwords/secrets
# Important: Change POSTGRES_PASSWORD, N8N_ENCRYPTION_KEY, etc.
nano .env  # or use your preferred editor

# 4. Launch with appropriate profile (see next steps for profile selection)
  5. Step 5

    Choosing Your Docker Compose Profile

    The starter kit provides different Docker Compose profiles optimized for your hardware. Choose the appropriate launch command based on your system:

    For NVIDIA GPU: Uses the gpu-nvidia profile with CUDA acceleration for maximum performance.

    For AMD GPU (Linux only): Uses the gpu-amd profile with ROCm acceleration.

    For CPU-only or Mac (Docker Ollama): Uses the cpu profile or default profile without GPU passthrough.

    For Mac with local Ollama: If you prefer to run Ollama natively on macOS (which can utilize Apple Silicon GPU), you'll need to update the .env file to point to your local Ollama instance.

    # NVIDIA GPU
docker compose --profile gpu-nvidia up -d

# AMD GPU (Linux)
docker compose --profile gpu-amd up -d

# CPU-only or standard Mac
docker compose --profile cpu up -d

# Default profile (Mac without GPU)
docker compose up -d
  6. Step 6

    Mac Users: Local Ollama Configuration

    macOS doesn't support GPU passthrough to Docker containers. For better performance on Apple Silicon, run Ollama natively on macOS and configure the Docker stack to use it:

    Step 1: Install Ollama locally on your Mac from https://ollama.ai

    Step 2: Update the .env file to point to your local Ollama instance:

    Step 3: Start Ollama on your Mac and pull the model:

    Step 4: Launch the Docker stack (Ollama service will be excluded):

    Step 5: After n8n starts, update the Ollama credentials in the n8n web interface to use the local instance.

    # Step 2: Edit .env file
# Change this line:
OLLAMA_HOST=ollama:11434
# To:
OLLAMA_HOST=host.docker.internal:11434

# Step 3: Start Ollama and pull model
ollama serve  # If not already running as a service
ollama pull llama3.2

# Step 4: Launch Docker stack without Ollama
docker compose up -d
  7. Step 7

    First-Time Setup

    After launching the Docker stack, the services will initialize and Ollama will automatically download the Llama 3.2 model (this may take several minutes depending on your internet connection).

    Once all services are running, access the n8n web interface to complete setup:

    1. Open http://localhost:5678/ in your browser
    2. Complete the one-time n8n configuration (create admin account)
    3. The starter kit includes a demo workflow at http://localhost:5678/workflow/srOnR8PAY3u4RSwb
    4. Click the Chat button in the workflow to test the AI setup

    The demo workflow demonstrates a basic chatbot that uses your local Llama 3.2 model through Ollama, with conversation context stored in Qdrant's vector database.

    # Check service status
docker compose ps

# View logs to monitor startup
docker compose logs -f

# Check if Ollama model is downloaded
docker compose exec ollama ollama list

# Test Ollama API directly
curl http://localhost:11434/api/tags
  8. Step 8

    Environment Configuration Reference

    The .env file contains critical configuration for all services. Here are the key variables you should review and customize:

    Database Configuration:

    • POSTGRES_USER - PostgreSQL username (default: postgres)
    • POSTGRES_PASSWORD - Change this! Database password
    • POSTGRES_DB - Database name (default: n8n)

    n8n Configuration:

    • N8N_ENCRYPTION_KEY - Change this! Encrypts credentials in database
    • N8N_USER_MANAGEMENT_JWT_SECRET - Change this! JWT signing key
    • OLLAMA_HOST - Ollama API endpoint (ollama:11434 or host.docker.internal:11434 for Mac)

    Important Security Note: Always change the default passwords and encryption keys before deploying to production or exposing the services beyond localhost.

    # Example .env configuration (update these values!)
POSTGRES_USER=postgres
POSTGRES_PASSWORD=your_secure_password_here
POSTGRES_DB=n8n

N8N_ENCRYPTION_KEY=your_encryption_key_here
N8N_USER_MANAGEMENT_JWT_SECRET=your_jwt_secret_here

OLLAMA_HOST=ollama:11434  # or host.docker.internal:11434 for Mac local Ollama

# Generate secure random strings for secrets:
openssl rand -base64 32
    ⚠ Heads up: Never commit your .env file to version control. The .env.example file should contain placeholder values only. Production deployments should use secrets management services.
  9. Step 9

    Working with Ollama Models

    The starter kit automatically downloads Llama 3.2, but Ollama supports dozens of other models. You can manage models through the Ollama CLI:

    List available models to see what's installed:

    Pull additional models for different use cases:

    Remove models you no longer need to save disk space:

    Each model has different resource requirements and capabilities. Smaller models (1B-7B parameters) run on modest hardware, while larger models (13B-70B) require significant RAM and benefit from GPU acceleration.

    # List installed models
docker compose exec ollama ollama list

# Pull popular models
docker compose exec ollama ollama pull mistral          # 7B params, fast and capable
docker compose exec ollama ollama pull codellama        # Code-specialized
docker compose exec ollama ollama pull llama3.1:70b     # Larger, more capable (requires lots of RAM)

# Remove a model
docker compose exec ollama ollama rm llama3.2

# For Mac users running Ollama locally, omit 'docker compose exec ollama'
ollama pull mistral
ollama list
  10. Step 10

    Building Your First Workflow

    n8n's visual workflow builder makes it easy to create AI-powered automation. Here's how to build a simple workflow:

    1. Create a new workflow from the n8n dashboard

    2. Add nodes by clicking the + button:

    • Webhook or Manual Trigger to start the workflow
    • Ollama Chat Model to connect to your local LLM
    • AI Agent for autonomous task execution
    • Qdrant Vector Store for document retrieval (RAG)

    3. Configure credentials for Ollama and Qdrant (both use localhost endpoints)

    4. Connect nodes by dragging between their connection points

    5. Test execution using the workflow's test button

    The included demo workflow is an excellent starting point to understand the node structure and data flow patterns.

    Common Workflow Patterns:

1. Simple Chatbot:
   Webhook → Ollama Chat → Respond

2. RAG Document QA:
   Webhook → Qdrant Search → Ollama Chat (with context) → Respond

3. AI Agent with Tools:
   Trigger → AI Agent (with tool nodes) → Process Results

4. Document Processor:
   File Upload → Text Extraction → Qdrant Insert → Confirm
  11. Step 11

    Qdrant Vector Database Setup

    Qdrant runs automatically with the starter kit and is accessible at http://localhost:6333. To use it for RAG (Retrieval-Augmented Generation) workflows:

    1. Create a collection in Qdrant through n8n's Qdrant Vector Store node or via API

    2. Insert vectors from your documents using n8n's document processing nodes

    3. Query vectors in your workflows to provide context to LLM responses

    Qdrant's web dashboard at http://localhost:6333/dashboard provides a visual interface to inspect collections, view vectors, and test queries.

    # Check Qdrant is running
curl http://localhost:6333/

# Create a collection via API (or use n8n nodes)
curl -X PUT 'http://localhost:6333/collections/documents' \
  -H 'Content-Type: application/json' \
  -d '{
    "vectors": {
      "size": 384,
      "distance": "Cosine"
    }
  }'

# List collections
curl http://localhost:6333/collections
  12. Step 12

    Data Persistence and Backups

    All data is stored in Docker volumes, which persist across container restarts. The starter kit creates four named volumes:

    • postgres_storage - n8n workflow data, credentials, execution history
    • n8n_storage - n8n configuration and settings
    • ollama_storage - Downloaded LLM models (~4-40GB depending on models)
    • qdrant_storage - Vector database collections and indexes

    To backup your data, create exports of these volumes or use database-specific backup tools:

    Regular backups are essential if you're building production workflows or storing important data.

    # List volumes
docker volume ls | grep self-hosted-ai-starter-kit

# Backup PostgreSQL database
docker compose exec postgres pg_dump -U postgres n8n > n8n_backup.sql

# Restore PostgreSQL backup
cat n8n_backup.sql | docker compose exec -T postgres psql -U postgres -d n8n

# Backup entire volume (generic approach)
docker run --rm -v self-hosted-ai-starter-kit_n8n_storage:/data \
  -v $(pwd):/backup alpine tar czf /backup/n8n_backup.tar.gz /data
  13. Step 13

    Troubleshooting Common Issues

    Service won't start or crashes: Check logs for the specific service to identify the issue:

    Ollama model download is slow or stuck: Ollama downloads models on first use. Large models can take 10-30 minutes. Monitor progress:

    n8n can't connect to Ollama: Verify the OLLAMA_HOST environment variable matches your setup. For Docker Ollama use ollama:11434, for Mac local Ollama use host.docker.internal:11434. You may also need to update credentials in the n8n web interface.

    Out of disk space: Ollama models consume significant storage (4-40GB each). Use ollama list to see installed models and ollama rm <model> to remove unused ones.

    GPU not being used: Verify Docker has GPU access with docker run --rm --gpus all nvidia/cuda:12.0.0-base-ubuntu22.04 nvidia-smi (NVIDIA) or check your ROCm installation (AMD).

    # View all service logs
docker compose logs

# View specific service logs (follow mode)
docker compose logs -f ollama
docker compose logs -f n8n

# Check service health
docker compose ps

# Restart a specific service
docker compose restart n8n

# Full restart
docker compose down && docker compose --profile gpu-nvidia up -d
  14. Step 14

    Production Deployment Considerations

    The starter kit is designed for local development and testing. For production deployment, consider these additional steps:

    Security:

    • Use strong, unique passwords for all services
    • Deploy behind a reverse proxy (nginx, Traefik) with HTTPS/TLS
    • Configure n8n authentication and user management
    • Restrict network access with firewall rules
    • Consider VPN or IP allowlisting for admin access

    Scaling:

    • Move PostgreSQL to a managed database service for reliability
    • Use external storage for volumes (NFS, cloud block storage)
    • Consider n8n's queue mode for high-volume workflows
    • Deploy Qdrant in cluster mode for high availability

    Monitoring:

    • Set up log aggregation (ELK, Grafana Loki)
    • Monitor resource usage (Prometheus, Datadog)
    • Configure health checks and alerting
    • Track workflow execution metrics in n8n
    ⚠ Heads up: The default configuration exposes services on localhost only. Never expose Docker services directly to the internet without proper authentication, encryption, and security hardening.
  15. Step 15

    Useful Commands

    Here's a quick reference for common operations:

    # Start services (choose appropriate profile)
docker compose --profile gpu-nvidia up -d

# Stop all services
docker compose down

# Stop and remove volumes (deletes all data!)
docker compose down -v

# View running services
docker compose ps

# View logs (all services)
docker compose logs -f

# View logs (specific service)
docker compose logs -f n8n

# Restart a service
docker compose restart ollama

# Execute command in service container
docker compose exec ollama ollama list
docker compose exec postgres psql -U postgres -d n8n

# Update to latest images
docker compose pull
docker compose --profile gpu-nvidia up -d

# Check disk usage
docker system df
  16. Step 16

    Next Steps and Resources

    Now that you have the Self-Hosted AI Starter Kit running, explore these resources to build more sophisticated workflows:

    Official Documentation:

    • n8n Documentation: https://docs.n8n.io/
    • Ollama Documentation: https://github.com/ollama/ollama/tree/main/docs
    • Qdrant Documentation: https://qdrant.tech/documentation/

    GitHub Repositories:

    • Starter Kit: https://github.com/n8n-io/self-hosted-ai-starter-kit
    • n8n: https://github.com/n8n-io/n8n
    • Ollama: https://github.com/ollama/ollama
    • Qdrant: https://github.com/qdrant/qdrant

    Community:

    • n8n Community Forum: https://community.n8n.io/
    • n8n Discord: https://discord.gg/n8n

    Workflow Ideas:

    • Document summarization with RAG
    • Automated email processing and response
    • Code review and analysis
    • Data extraction from PDFs and images
    • Multi-step AI agents for complex tasks
    • Chatbots with custom knowledge bases
    Access URLs (default):
├── n8n Workflows: http://localhost:5678/
├── Ollama API: http://localhost:11434/
├── Qdrant Dashboard: http://localhost:6333/dashboard
└── PostgreSQL: localhost:5432 (not exposed by default)

Starter Kit: https://github.com/n8n-io/self-hosted-ai-starter-kit
n8n Docs: https://docs.n8n.io/
Ollama Docs: https://github.com/ollama/ollama
Qdrant Docs: https://qdrant.tech/documentation/

Feature requests

Sign in to suggest features or vote on existing ones.

No feature requests yet.

Discussion

0 people marked this as worked·Sign in to mark your own.

Sign in to join the discussion.

No comments yet.