Build Your Own Agentic AI Platform

 

Build Your Own Agentic AI Platform at Home with Containers, Ollama, n8n, and Vector Databases

YouTube on Pi:    https://youtu.be/__Led8X0bOk

                            https://youtu.be/huaCfx1My6M

                            https://youtu.be/JrhRwU2TkEA

                            https://youtu.be/NCXgHdqjxI0

YouTube on VM: https://youtu.be/qar4KA47HIM

Artificial Intelligence is changing rapidly.

We are moving beyond simple chatbots into systems that can:

  • Search documents
  • Remember context
  • Use tools
  • Trigger workflows
  • Query databases
  • Make decisions
  • Automate tasks

These are often called agentic AI systems.

And the exciting part is this:

You can now build a surprisingly capable AI platform from:

  • A Raspberry Pi 5
  • A small VM on your laptop
  • Or a tiny home server

Using entirely open-source tooling.

That’s remarkable.

Because only a few years ago, this kind of infrastructure required expensive cloud platforms and enterprise hardware.

What Are We Building?

This home lab creates a complete mini AI platform containing:

  • Linux
  • Containers
  • Local AI models
  • Workflow automation
  • Relational databases
  • NoSQL databases
  • Vector databases
  • Web servers
  • Retrieval-Augmented Generation (RAG)

The end goal is a local AI customer service chatbot capable of:

  • Reading documents
  • Searching manuals
  • Understanding context
  • Answering questions
  • Automating workflows

All running privately on your own infrastructure.

What Is RAG?

RAG stands for:

Retrieval-Augmented Generation

Instead of an AI model relying only on training data, RAG systems:

  1. Search documents
  2. Retrieve relevant information
  3. Inject that context into the AI prompt
  4. Generate better answers

This is one of the most important modern AI patterns.

Because it allows AI systems to:

  • Use company knowledge
  • Search PDFs
  • Read manuals
  • Query databases
  • Access realtime information

Without retraining models.

Why Run Everything Locally?

Running locally gives you:

  • Privacy
  • Full control
  • No cloud costs
  • Faster experimentation
  • A better learning experience

And perhaps most importantly:
you learn how the systems underneath modern AI platforms actually work.

Why a Small VM Is Enough

This setup uses:

  • 12GB RAM
  • 8 CPU cores

That sounds large…
…but compared to enterprise AI systems, it’s tiny.

Yet modern lightweight models and containers make surprisingly powerful home AI platforms possible.

That’s one of the most exciting developments in AI infrastructure today.

Understanding the Vagrantfile

Vagrant creates the entire environment automatically.

This is Infrastructure-as-Code.

Instead of manually installing:

  • Linux
  • Databases
  • Containers
  • AI tooling

Everything becomes reproducible and automated.

Step 1: Create Ubuntu

config.vm.box = "bento/ubuntu-24.04"

This downloads a clean Ubuntu Linux image.

Every rebuild starts from a predictable baseline.

Step 2: Allocate Resources

v.memory = 12288
v.cpus = 8

The VM receives:

  • 12GB RAM
  • 8 CPU cores

This is ideal for:

  • Local LLMs
  • Databases
  • Vector search
  • Workflow engines
  • Multiple containers

Step 3: Put the VM on Your Network

config.vm.network "public_network",
  ip: "192.168.1.253"

Now the VM behaves like a real server on your home network.

You can access services from:

  • Your laptop
  • Tablet
  • Phone
  • Browser

This is fantastic for learning real networking concepts.

Podman Instead of Docker

This environment uses:

  • Podman
  • Podman Compose

Podman is similar to Docker but focuses heavily on:

  • Security
  • Rootless containers
  • Linux-native workflows

Many modern Linux engineers are increasingly exploring Podman-based infrastructure.

Ollama — Local AI Models Made Easy

Ollama is one of the most exciting AI tools available today.

Ollama makes running local LLMs incredibly approachable.

Instead of complicated GPU infrastructure, you can launch models with:

ollama run llama3.2

That simplicity changed local AI dramatically.

The Ollama Systemd Service

The Vagrantfile creates a dedicated system service:

ExecStart=/usr/local/bin/ollama serve

This ensures Ollama:

  • Starts automatically
  • Runs continuously
  • Accepts network requests
  • Behaves like a proper AI platform service

The line:

Environment="OLLAMA_HOST=0.0.0.0"

Allows remote API access across the network.

That’s important for workflow automation and integrations.

Why Multiple Models Matter

The setup downloads several AI models.

Each has different strengths.

DeepSeek-R1

DeepSeek-R1 focuses heavily on reasoning and problem solving.

Even the smaller 1.5B version is surprisingly capable on home hardware.

Llama 3.2

Llama models from Meta became hugely influential in open AI ecosystems.

They provide strong general-purpose conversational capability.

Gemma

Gemma from Google focuses heavily on lightweight efficient local AI.

Perfect for smaller home systems.

Nomic Embed Text

ollama pull nomic-embed-text

This model is especially important for RAG.

Embedding models convert text into vectors so semantic search becomes possible.

This is foundational for vector databases.

Understanding the Databases

This environment intentionally includes multiple database styles because modern AI systems rarely rely on just one database type.

MongoDB — NoSQL Documents

MongoDB stores flexible JSON-style documents.

Excellent for:

  • AI metadata
  • Workflow state
  • Flexible schemas
  • Rapid development

PostgreSQL — Relational Power

PostgreSQL provides:

  • Structured storage
  • Transactions
  • SQL querying
  • Reliability

Many AI platforms still rely heavily on PostgreSQL underneath.

MariaDB

MariaDB offers another relational database option with strong MySQL compatibility.

This helps learners compare ecosystems directly.

Vector Databases Are the Heart of RAG

Traditional databases search exact values.

Vector databases search meaning.

That’s a huge difference.

Weaviate

Weaviate provides:

  • Semantic search
  • Embedding storage
  • AI-native APIs
  • Vector retrieval

Perfect for AI knowledge systems.

Qdrant

Qdrant focuses heavily on:

  • Fast vector search
  • AI retrieval systems
  • Recommendation engines

Qdrant has become very popular in open-source AI projects.

n8n — The Workflow Brain

n8n orchestrates the platform.

This is where the “agentic” behaviour begins.

n8n can:

  • Trigger workflows
  • Process files
  • Query databases
  • Call Ollama APIs
  • Manage automations
  • Build AI pipelines

Instead of writing huge amounts of code, you visually connect systems together.

That’s extremely powerful for learning.

The Ford PDF Documents

The Vagrantfile downloads real PDF manuals:

curl ... ford_capri.pdf

These become the knowledge base for the chatbot.

This is the foundation of RAG:

  • Ingest documents
  • Chunk text
  • Generate embeddings
  • Store vectors
  • Retrieve context
  • Generate answers

Now the chatbot can answer questions about:

  • Vehicle features
  • Maintenance
  • Controls
  • Specifications

Using real manuals.

The Role of NGINX

NGINX provides lightweight web serving.

Even a simple:

hello world

Page introduces:

  • Web hosting
  • HTTP
  • Networking
  • Static content serving

These concepts become important later for APIs and frontend systems.

Why This Is Such a Good Learning Platform

This tiny home lab teaches:

  • Linux
  • Containers
  • AI models
  • APIs
  • Databases
  • Vector search
  • Workflow orchestration
  • Infrastructure-as-Code
  • Semantic search
  • Agentic AI patterns

That’s an enormous amount of modern infrastructure knowledge in one environment.

The Real Magic: Combining Everything

The exciting part is how these pieces work together.

Example workflow:

  1. User asks a question
  2. n8n receives request
  3. PDFs are searched via vector database
  4. Relevant context retrieved
  5. Ollama model generates response
  6. AI answers using retrieved knowledge

That is essentially a miniature enterprise AI platform.

Running locally.

On a home lab.

Why Experimentation Matters

One of the best parts of home labs is freedom.

You can:

  • Break systems
  • Rebuild containers
  • Swap databases
  • Try different models
  • Compare embeddings
  • Experiment safely

That hands-on experimentation builds real understanding quickly.

A Raspberry Pi Can Go Surprisingly Far

Even smaller systems like the Raspberry Pi 5 can run:

  • Lightweight models
  • Containers
  • Vector databases
  • APIs
  • Workflow engines

Modern software efficiency is genuinely impressive.

The Future Is Local + Hybrid AI

Cloud AI is powerful…
…but local AI is becoming increasingly important.

Local AI provides:

  • Privacy
  • Lower cost
  • Offline capability
  • Faster iteration
  • Home experimentation

And hybrid systems combining local + cloud models are becoming very common.

Final Thoughts

What makes modern AI infrastructure so exciting is accessibility.

You no longer need:

  • Enterprise budgets
  • Massive GPU clusters
  • Dedicated server rooms

A small VM or Raspberry Pi can now teach:

  • Agentic workflows
  • RAG architecture
  • Containers
  • Vector databases
  • AI orchestration
  • Local language models

That’s an extraordinary amount of technology sitting inside a tiny home lab.


Appendix

Here is the Vagrantfile used in the YouTube video.

Vagrant.configure("2") do |config|

 config.vm.box = "bento/ubuntu-24.04"

 config.vm.provider "VirtualBox" do |v|

   v.memory = 12288

   v.cpus = 8

 end

 config.vm.synced_folder ".", "/vagrant", disabled: true

 config.vm.network "public_network", ip: "192.168.1.253", use_dhcp_assigned_default_route: true

 config.vm.provision "shell", inline: <<-SHELL

      sudo apt update -y

      # sudo apt upgrade -y # openssh server upgrade stops build in 1 step

      sudo ufw disable

      sudo systemctl stop apparmor

      sudo systemctl disable apparmor

      sudo sed -i '/swap/d' /etc/fstab

      sudo swapoff -a

      echo "192.168.1.253 aionpi aionpi.local" >> /etc/hosts

      mkdir /home/vagrant/html

      groupadd docker

      usermod -aG docker vagrant

      apt-get -y install podman

      apt-get -y install podman-compose

      curl -fsSL https://ollama.com/install.sh | sh

      cat <<-'EOF' > /etc/systemd/system/ollama.service

[Unit]

Description=Ollama Service

After=network-online.target


[Service]

ExecStart=/usr/local/bin/ollama serve

User=ollama

Group=ollama

Restart=always

RestartSec=3

Environment="OLLAMA_HOST=0.0.0.0"


[Install]

WantedBy=default.target

EOF

      cat <<-'EOF' > /home/vagrant/html/index.html

hello world

EOF

      cat <<-'EOF' > /home/vagrant/.env

DOMAIN_NAME=local

SUBDOMAIN=aionpi

GENERIC_TIMEZONE=Europe/London

SSL_EMAIL=admin@secretsbox.org

EOF

      cat <<-'EOF' > /home/vagrant/compose.yaml

services:

 nginx:

   container_name: nginx

   image: docker.io/library/nginx

   ports:

     - 8090:80

   volumes:

    - /home/vagrant/html/:/usr/share/nginx/html

 mongo:

   container_name: mongo

   image: docker.io/library/mongo

   restart: always

   ports:

     - "27017:27017"

   volumes:

     - /home/vagrant/mongo_data:/data/db

 postgres:

   container_name: postgres

   image: docker.io/library/postgres:latest

   restart: always

   volumes:

     - /home/vagrant/pgdata:/var/lib/postgresql/data

   environment:

     POSTGRES_USER: vagrant

     POSTGRES_PASSWORD: 1234

     POSTGRES_DB: vagrant

 mariadb:

   container_name: maria

   image: docker.io/library/mariadb

   command: '--default-authentication-plugin=mysql_native_password'

   volumes:

     - /home/vagrant/maria_data:/var/lib/mysql

   restart: always

   environment:

     - MYSQL_ROOT_PASSWORD=vagrant

     - MYSQL_DATABASE=vagrant

     - MYSQL_USER=vagrant

     - MYSQL_PASSWORD=vagrant

   expose:

     - 3306

 weaviate:

   container_name: weaviate

   command:

   - --host

   - 0.0.0.0

   - --port

   - '8080'

   - --scheme

   - http

   image: cr.weaviate.io/semitechnologies/weaviate

   ports:

   - 8080:8080

   - 50051:50051

   volumes:

   - /home/vagrant/weaviate_data:/var/lib/weaviate

   restart: on-failure:0

   environment:

     QUERY_DEFAULTS_LIMIT: 25

     AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'

     PERSISTENCE_DATA_PATH: '/var/lib/weaviate'

     ENABLE_API_BASED_MODULES: 'true'

     CLUSTER_HOSTNAME: 'node1'

 qdrant:

   container_name: qdrant

   image: docker.io/qdrant/qdrant:latest

   restart: always

   container_name: qdrant

   ports:

     - 6333:6333

     - 6334:6334

   expose:

     - 6333

     - 6334

     - 6335

   configs:

     - source: qdrant_config

       target: /qdrant/config/production.yaml

   volumes:

     - /home/vagrant/qdrant_data:/qdrant/storage

 n8nmcp:

   container_name: n8nmcp

   image: ghcr.io/czlonkowski/n8n-mcp

 n8n:

   container_name: n8n

   image: docker.n8n.io/n8nio/n8n

   restart: always

   ports:

     - "0.0.0.0:5678:5678"

   environment:

     - N8N_HOST=aionpi.local

     - N8N_PORT=5678

     - N8N_PROTOCOL=http

     - NODE_ENV=production

     - WEBHOOK_URL=http://aionpi.local:5678/

     - GENERIC_TIMEZONE=Europe/London

     - N8N_SECURE_COOKIE=false

   volumes:

     - /home/vagrant/n8n_files:/files


configs:

 qdrant_config:

   content: |

     log_level: INFO

EOF

      sudo chown -R vagrant:vagrant /home/vagrant/html

      sudo systemctl stop ollama

      sudo systemctl daemon-reload

      sudo systemctl enable ollama

      sudo systemctl start ollama

      sudo systemctl status ollama

      ollama pull deepseek-r1:1.5b

      ollama pull nomic-embed-text

      ollama pull llama3.2:1b

      ollama pull gemma3n:e2b

      chmod -R 755 /home/vagrant/.env

      su - vagrant -c "podman compose -f /home/vagrant/compose.yaml up -d"

      su - vagrant -c "mkdir /home/vagrant/n8n_files/ford"

      curl -fsSl https://www.fordservicecontent.com/Ford_Content/Catalog/owner_information/CG4012enUSA-202408-20250505091915.pdf -o /home/vagrant/n8n_files/ford/ford_capri.pdf

      curl -fsSl https://www.fordservicecontent.com/Ford_Content/Catalog/owner_information/CG3929en-202309-20231024095529.pdf -o /home/vagrant/n8n_files/ford/ford_focus.pdf

      curl -fsSl https://www.fordservicecontent.com/Ford_Content/Catalog/owner_information/CG3980en-202405-20250307130903.pdf -o /home/vagrant/n8n_files/ford/ford_puma.pdf

      curl -fsSl https://www.fordservicecontent.com/Ford_Content/Catalog/owner_information/CG3920en-202410-20241030024645.pdf -o /home/vagrant/n8n_files/ford/ford_mustang.pdf

      curl -fsSl https://www.fordservicecontent.com/Ford_Content/Catalog/owner_information/CG3964en-202402-20240716133324.pdf -o /home/vagrant/n8n_files/ford/ford_transit.pdf

      chmod -R 755 /home/vagrant/n8n_files/ford

  SHELL

end


Comments

Post a Comment

Popular posts from this blog

Don’t Be Afraid to Experiment

Image
  Don’t Be Afraid to Experiment: Building Your Own Home Lab with Linux, Containers, and Tiny Servers YouTube: https://youtu.be/ePHyb73RRLE One of the biggest misconceptions in technology is this: “I might break something.” Here’s the truth: Breaking things safely is one of the best ways to learn. And today, learning infrastructure, Linux, containers, and web servers is easier than ever because you can experiment from: A Raspberry Pi 5 A tiny VM on your laptop An old mini PC Or even a spare desktop machine You no longer need: Enterprise hardware Expensive cloud accounts Massive servers Complex corporate environments A small home lab is enough to learn real modern infrastructure skills. And honestly, that’s incredibly exciting. Why Home Labs Are So Powerful A home lab gives you freedom. You can: Install software Break systems Rebuild environments Try new ideas Learn networking Deploy containers Explore Linux Without worrying about damaging production systems. That freedom remove...
Read more

Learning Databases and Big Data

Image
  Learning Databases and Big Data at Home with a Raspberry Pi or Tiny VM YouTube: https://youtu.be/04g-2O7e6Qc Modern technology runs on data. Every: Website AI model Mobile app Online shop Cloud platform Banking system Streaming service Depends on databases underneath. And the exciting part is this: You can now learn many of these technologies from: A Raspberry Pi 5 A small virtual machine on your laptop Or a tiny home server That’s incredibly powerful. Because databases are one of the most important foundations of modern computing. What Is a Database? A database stores and organises information. Think of it as a highly advanced digital filing cabinet. Databases help systems: Save data Retrieve data quickly Search efficiently Handle many users simultaneously Protect information safely Without databases, modern applications simply would not function. A Short History of Databases Early computer systems stored information in flat files. That quickly became difficult to manage. A...
Read more