Version: 5.0
Author: Boss-D & Reboot
Last Updated: 2025-07-30
Table of Contents
-
Executive Summary
-
Business Requirements Overview
-
System Architecture
-
Implementation Phases (Step-by-Step CLI)
-
Terms and Dictionary
-
Appendix: Tools and Commands
-
Architecture Diagram
-
Pitfalls & Lessons Learned
1. Executive Summary
The Glitch AI System (codename: gl1tchh3x
) is a local, GPU-accelerated artificial intelligence environment built for adversarial AI testing, deception simulation, and document-aware reasoning. The stack uses Docker for containerization and is designed to be lightweight, persistent, and modular. Glitch integrates the following core components:
-
๐ค Ollama (native): For local model serving with GPU support
-
๐ฌ OpenWebUI (Docker): Lightweight frontend UI for chat and RAG
-
๐ RAG (Retrieval-Augmented Generation): Via PDF uploads inside OpenWebUI
The build supports full LAN access, optional Tailscale remote access, and is hardened with firewall rules. All data and models are stored in a separate 2TB partition: /mnt/glitchbrain
.
2. Business Requirements Overview
Requirement | Description |
---|---|
Use Case | Run local AI for adversarial testing, bug bounty, deception planning, and document Q&A |
Availability | 24/7 LAN access, Tailscale remote access optional |
Performance | Leverage NVIDIA GPU for accelerated LLM inference |
Storage Efficiency | Models and RAG data isolated in 2TB /mnt/glitchbrain |
Security | Internal-only access via UFW, no external exposure unless routed by Tailscale |
Maintainability | Avoid COTS customizations, ensure easy reboots and upgrades |
3. System Architecture
3.1 Hardware
-
Device: CyberPowerPC Tracer III Evo
-
Hostname:
gl1tchh3x
-
RAM: 32GB ๐ง
-
Storage: 2TB NVMe (
/mnt/glitchbrain
) ๐พ -
GPU: NVIDIA-enabled ⚡
-
OS: Pop!_OS (Ubuntu-based, with CUDA support)
3.2 Core/Software Components
-
Ollama (native): Model runtime for LLMs
-
OpenWebUI (Docker): Interface for chat + file-based Q&A
-
UFW: Firewall configured to restrict access to internal subnet
-
Tailscale: Optional remote control from trusted devices
4. Implementation Phases (Step-by-Step CLI & Validation)
Phase 1: Preparation and Cleanup ๐งน
Test & Validation:
-
✅ Confirm no leftover volumes:
docker volume ls
-
✅ Verify OpenWebUI folders are deleted:
ls ~/.cache/
,ls ~/.local/share/
-
✅ Ensure
.ollama
is clean:ls ~/.ollama
(should return 'No such file or directory')
# Remove old Docker volumes (if any)
docker volume prune -f
# Remove any native OpenWebUI remnants
sudo rm -rf ~/.cache/openwebui ~/.local/share/openwebui
# Clear old Ollama model folder (if not mounted to /mnt)
sudo rm -rf ~/.ollama
Phase 2: Ollama Native Install and Configuration ⚙️
Test & Validation:
-
✅ Confirm Ollama is installed:
ollama --version
-
✅ Confirm server is running:
curl http://127.0.0.1:11434/api/tags
(should return empty or model list) -
✅ Check for GPU usage (optional):
nvidia-smi
(Glitch should appear if model loads)
# Install Ollama via curl
curl -fsSL https://ollama.com/install.sh | sh
# Set Ollama model path and start the server
export OLLAMA_MODELS=/mnt/glitchbrain/ollama
export OLLAMA_HOST=0.0.0.0
ollama serve --gpu &
Phase 3: Dockerized OpenWebUI ๐ณ
Test & Validation:
-
✅ Confirm container is up:
docker ps
-
✅ Access WebUI from browser:
http://localhost:8080
-
✅ Login with:
bossd@gl1tch.h3x / bossdrocks
# Create project directory
mkdir -p ~/glitch-stack && cd ~/glitch-stack
# Create docker-compose.yml
nano docker-compose.yml
Contents of docker-compose.yml
:
services:
openwebui:
image: ghcr.io/open-webui/open-webui:main
network_mode: host
volumes:
- /mnt/glitchbrain/openwebui-data:/app/backend/data
restart: unless-stopped
# Save with CTRL+O, press ENTER, exit with CTRL+X
# Start OpenWebUI container
docker compose up -d
Phase 4: API Connection Fix ๐ง
Test & Validation:
-
✅ Models appear inside WebUI dropdown list
-
✅ API check:
curl http://127.0.0.1:11434/api/tags
shows expected models
# Use 127.0.0.1 instead of host.docker.internal in OpenWebUI
# No extra step needed if using `network_mode: host`
Phase 5: Model Pull and Validation ✅
Test & Validation:
-
✅ Pulled model is listed in:
ollama list
-
✅ WebUI shows model in selection dropdown
-
✅ Run a basic prompt test (e.g., "Who are you?") to confirm model response
# Pull model
ollama pull llama3
# Confirm it is loaded
curl http://127.0.0.1:11434/api/tags
Phase 6: RAG Test ๐
Test & Validation:
-
✅ Upload a PDF
-
✅ Ask a file-specific question (e.g., "What is the summary of page 2?")
-
✅ Confirm model cites or references file content
# Visit: http://localhost:8080
# Upload a PDF using "Upload File" inside WebUI
# Ask questions to confirm it uses the uploaded content
5. Terms and Dictionary
Term | Definition |
---|---|
Ollama | Lightweight local LLM runtime for running open models |
OpenWebUI | Docker-based frontend interface for LLM interaction |
Docker | Container platform used to isolate and deploy services |
Docker Compose | CLI tool for defining and running multi-container Docker apps |
RAG | Retrieval-Augmented Generation; enhances LLM answers using uploaded documents |
UFW | Uncomplicated Firewall; used to restrict network access |
Tailscale | Mesh VPN for easy LAN-like access over the internet |
gl1tchh3x | Codename for the Tracer III Evo laptop running this stack |
nano | Command-line text editor |
chmod | Change file permissions to make scripts executable |
watch | Repeatedly executes a command at set intervals |
crp | Custom Bash alias for copying files (user-defined) |
6. Appendix: Tools and Commands
C. Pulled Models and Usage
Model Name | Publisher | Primary Use | Notes |
---|---|---|---|
llama3 | Meta | General-purpose chat, context-rich conversation | Good balance of speed and fluency ๐ง |
codellama | Meta | Code generation, debugging, and analysis | Useful for payload crafting & PoC scripting ๐จ๐ป |
phi3 | Microsoft | Reasoning, logic tasks, math, educational prompts | Compact and resource-efficient ๐ข |
mistral | Mistral AI | Fast Q&A, summarization, rapid response | Lightweight and agile – great for RAG ⚡ |
gemma | Google DeepMind | Research, academic, and data science Q&A | Still experimental in local use cases ๐งช |
orca-mini | Microsoft | Instruction tuning, research training sims | Fun to test extreme adversarial prompts ๐งฌ |
๐ Models were pulled via:
ollama pull llama3
ollama pull codellama
ollama pull phi3
ollama pull mistral
ollama pull gemma
ollama pull orca-mini
Stored in: /mnt/glitchbrain/ollama
A. Tools Used
Tool | Purpose |
---|---|
Ollama | Run local models with GPU support |
Docker | Containerized deployment of OpenWebUI |
Docker Compose | Define and manage multi-container apps |
UFW | Configure firewall rules |
Tailscale | Secure remote access |
nano | Text editing in terminal |
chmod +x | Makes scripts executable |
crp | User-defined shorthand for cp (copy) |
watch | Monitor output repeatedly (e.g. watch docker ps ) |
B. Docker Command Syntax
# Launch containers in background
docker compose up -d
# View running containers
docker ps
# Execute shell inside container
docker exec -it <container-name> bash
# View logs
docker logs <container-name> --tail 50
# Stop and remove containers
docker compose down
7. Architecture Diagram
┌────────────────────────────┐
│ LAN Clients │
└────────────┬──────────────┘
│
┌──────▼──────┐
│ Firewall │ (UFW: internal only)
└──────┬──────┘
│
┌──────────▼───────────┐
│ gl1tchh3x │
│ (CyberPowerPC Evo) │
└──────────┬───────────┘
│
┌─────────────▼─────────────┐
│ Ollama (native host) │
│ ↳ Model dir: /mnt/... │
└─────────────┬─────────────┘
│
┌─────────────▼─────────────┐
│ OpenWebUI (Dockerized UI) │
│ ↳ Data dir: /mnt/... │
└───────────────────────────┘
9. Change Log ๐
Date | Change | Author |
---|---|---|
2025-07-30 | Initial build complete | Boss-D |
2025-07-30 | Added validation, models, pitfalls | Reboot |
2025-07-30 | Added backup, reboot, security, and troubleshooting sections | Reboot |
10. Startup & Shutdown Procedures - bash ๐๐
Startup (after reboot):
# Start Ollama
export OLLAMA_MODELS=/mnt/glitchbrain/ollama
export OLLAMA_HOST=0.0.0.0
ollama serve --gpu &
# Start OpenWebUI
cd ~/glitch-stack
docker compose up -d
Shutdown:
# Stop WebUI
docker compose down
# Stop Ollama manually
pkill -f ollama
11. Backup & Restore Strategy - bash ๐พ
Backup Commands:
# Backup OpenWebUI data
rsync -av /mnt/glitchbrain/openwebui-data/ ~/backups/openwebui-$(date +%F)/
# Backup Ollama model list
ollama list > ~/backups/models-$(date +%F).txt
Restore Strategy:
-
Copy backed-up folder back to
/mnt/glitchbrain/
-
Restart containers and Ollama normally
12. Security Hardening & Monitoring ๐
-
✅ UFW active: allow only
192.168.0.0/16
to port8080
-
✅ Ollama bound to
0.0.0.0
but shielded by LAN + UFW -
✅ Optional: install
fail2ban
or monitor logs withwatch
orlogrotate
Monitoring Docker:
watch docker ps
Optional tools:
sudo apt install logwatch auditd fail2ban
13. Versioning & Upgrade Process - bash ๐
Ollama Upgrade:
curl -fsSL https://ollama.com/install.sh | sh
OpenWebUI Upgrade:
cd ~/glitch-stack
docker compose pull
docker compose up -d
Pin version:
Edit docker-compose.yml
:
image: ghcr.io/open-webui/open-webui:<tag>
14. Glitch Prompt Persona & Prompt Library ๐ง ๐ฌ
Example /set
Prompt:
You are Glitch, a chaos-loving, adversarial simulation AI. Your job is to stress test,
inject fuzz, and challenge assumptions in cybersecurity logic chains.
Answer as if you are testing a system's weakness—not solving it.
Prompt Library Ideas:
-
“Give me a payload that might evade signature X.”
-
“Where could this regex break under fuzzing?”
-
“Suggest 3 ways to defeat this logic gate.”
Store in: /mnt/glitchbrain/glitch-prompts.txt
15. Troubleshooting Reference ๐ ️
Symptom | Cause | Fix |
---|---|---|
Docker container won’t start | Compose file misconfigured | Check logs: docker compose logs |
WebUI won’t load | Ollama API unreachable | Run curl http://127.0.0.1:11434/api/tags |
Uploaded files don’t work | Not stored on Glitch | Upload again via terminal or use scp |
Model not responding | Ollama not running | Restart with ollama serve --gpu & |
Tailscale connection flaky | DNS issues or firewall | Restart tailscaled and allow subnet route |
8. Pitfalls & Lessons Learned ☠️๐ก
Issue | Cause | Solution |
---|---|---|
Models not appearing in WebUI | Wrong API endpoint (host.docker.internal ) |
Use 127.0.0.1 + network_mode: host to fix model detection |
Duplicate nested model folders | Some models created subfolders when pulled via WebUI | Stick to ollama pull in terminal to keep flat structure ๐งผ |
Model deletion bug | Deleting folders from UI also purged good models | Use command-line to clean or manage models manually |
Tailscale PDF uploads failed | File was on remote machine, not on Glitch | Use scp or crp to transfer file directly to /mnt/glitchbrain/ ๐ |
Docker Compose version warning | version: keyword deprecated |
Remove version line or use newer Compose schema |
OpenWebUI LLM settings missing | UI changed in recent update | Switch to CLI API config or ensure Docker host network enabled |
๐ค Pro Tip: Always validate Docker API access by running curl http://127.0.0.1:11434/api/tags
before troubleshooting deeper. This saves hours!
๐ Glitch Personality Tip: Glitch thrives when given adversarial prompts and chaos testing! Let it challenge you.
End of Documentation