2025-07-30

Project Title: Glitch AI System Build (RC 1) - in Alpha Test

 

Version: 5.0
Author: Boss-D & Reboot
Last Updated: 2025-07-30

Table of Contents

  1. Executive Summary

  2. Business Requirements Overview

  3. System Architecture

  4. Implementation Phases (Step-by-Step CLI)

  5. Terms and Dictionary

  6. Appendix: Tools and Commands

  7. Architecture Diagram

  8. Pitfalls & Lessons Learned

1. Executive Summary

The Glitch AI System (codename: gl1tchh3x) is a local, GPU-accelerated artificial intelligence environment built for adversarial AI testing, deception simulation, and document-aware reasoning. The stack uses Docker for containerization and is designed to be lightweight, persistent, and modular. Glitch integrates the following core components:

  • 🤖 Ollama (native): For local model serving with GPU support

  • 💬 OpenWebUI (Docker): Lightweight frontend UI for chat and RAG

  • 📄 RAG (Retrieval-Augmented Generation): Via PDF uploads inside OpenWebUI

The build supports full LAN access, optional Tailscale remote access, and is hardened with firewall rules. All data and models are stored in a separate 2TB partition: /mnt/glitchbrain.

2. Business Requirements Overview

Requirement Description
Use Case Run local AI for adversarial testing, bug bounty, deception planning, and document Q&A
Availability 24/7 LAN access, Tailscale remote access optional
Performance Leverage NVIDIA GPU for accelerated LLM inference
Storage Efficiency Models and RAG data isolated in 2TB /mnt/glitchbrain
Security Internal-only access via UFW, no external exposure unless routed by Tailscale
Maintainability Avoid COTS customizations, ensure easy reboots and upgrades

3. System Architecture

3.1 Hardware

  • Device: CyberPowerPC Tracer III Evo

  • Hostname: gl1tchh3x

  • RAM: 32GB 🧠

  • Storage: 2TB NVMe (/mnt/glitchbrain) 💾

  • GPU: NVIDIA-enabled ⚡

  • OS: Pop!_OS (Ubuntu-based, with CUDA support)

3.2 Core/Software Components

  • Ollama (native): Model runtime for LLMs

  • OpenWebUI (Docker): Interface for chat + file-based Q&A

  • UFW: Firewall configured to restrict access to internal subnet

  • Tailscale: Optional remote control from trusted devices

4. Implementation Phases (Step-by-Step CLI & Validation)

Phase 1: Preparation and Cleanup 🧹

Test & Validation:

  • ✅ Confirm no leftover volumes: docker volume ls

  • ✅ Verify OpenWebUI folders are deleted: ls ~/.cache/, ls ~/.local/share/

  • ✅ Ensure .ollama is clean: ls ~/.ollama (should return 'No such file or directory')

# Remove old Docker volumes (if any)
docker volume prune -f

# Remove any native OpenWebUI remnants
sudo rm -rf ~/.cache/openwebui ~/.local/share/openwebui

# Clear old Ollama model folder (if not mounted to /mnt)
sudo rm -rf ~/.ollama

Phase 2: Ollama Native Install and Configuration ⚙️

Test & Validation:

  • ✅ Confirm Ollama is installed: ollama --version

  • ✅ Confirm server is running: curl http://127.0.0.1:11434/api/tags (should return empty or model list)

  • ✅ Check for GPU usage (optional): nvidia-smi (Glitch should appear if model loads)

# Install Ollama via curl
curl -fsSL https://ollama.com/install.sh | sh

# Set Ollama model path and start the server
export OLLAMA_MODELS=/mnt/glitchbrain/ollama
export OLLAMA_HOST=0.0.0.0
ollama serve --gpu &

Phase 3: Dockerized OpenWebUI 🐳

Test & Validation:

  • ✅ Confirm container is up: docker ps

  • ✅ Access WebUI from browser: http://localhost:8080

  • ✅ Login with: bossd@gl1tch.h3x / bossdrocks

# Create project directory
mkdir -p ~/glitch-stack && cd ~/glitch-stack

# Create docker-compose.yml
nano docker-compose.yml

Contents of docker-compose.yml:

services:
  openwebui:
    image: ghcr.io/open-webui/open-webui:main
    network_mode: host
    volumes:
      - /mnt/glitchbrain/openwebui-data:/app/backend/data
    restart: unless-stopped

# Save with CTRL+O, press ENTER, exit with CTRL+X

# Start OpenWebUI container
docker compose up -d

Phase 4: API Connection Fix 🔧

Test & Validation:

  • ✅ Models appear inside WebUI dropdown list

  • ✅ API check: curl http://127.0.0.1:11434/api/tags shows expected models

# Use 127.0.0.1 instead of host.docker.internal in OpenWebUI
# No extra step needed if using `network_mode: host`

Phase 5: Model Pull and Validation ✅

Test & Validation:

  • ✅ Pulled model is listed in: ollama list

  • ✅ WebUI shows model in selection dropdown

  • ✅ Run a basic prompt test (e.g., "Who are you?") to confirm model response

# Pull model
ollama pull llama3

# Confirm it is loaded
curl http://127.0.0.1:11434/api/tags

Phase 6: RAG Test 📑

Test & Validation:

  • ✅ Upload a PDF

  • ✅ Ask a file-specific question (e.g., "What is the summary of page 2?")

  • ✅ Confirm model cites or references file content

# Visit: http://localhost:8080
# Upload a PDF using "Upload File" inside WebUI
# Ask questions to confirm it uses the uploaded content

5. Terms and Dictionary

Term Definition
Ollama Lightweight local LLM runtime for running open models
OpenWebUI Docker-based frontend interface for LLM interaction
Docker Container platform used to isolate and deploy services
Docker Compose CLI tool for defining and running multi-container Docker apps
RAG Retrieval-Augmented Generation; enhances LLM answers using uploaded documents
UFW Uncomplicated Firewall; used to restrict network access
Tailscale Mesh VPN for easy LAN-like access over the internet
gl1tchh3x Codename for the Tracer III Evo laptop running this stack
nano Command-line text editor
chmod Change file permissions to make scripts executable
watch Repeatedly executes a command at set intervals
crp Custom Bash alias for copying files (user-defined)

6. Appendix: Tools and Commands

C. Pulled Models and Usage

Model Name Publisher Primary Use Notes
llama3 Meta General-purpose chat, context-rich conversation Good balance of speed and fluency 🧠
codellama Meta Code generation, debugging, and analysis Useful for payload crafting & PoC scripting 👨‍💻
phi3 Microsoft Reasoning, logic tasks, math, educational prompts Compact and resource-efficient 🔢
mistral Mistral AI Fast Q&A, summarization, rapid response Lightweight and agile – great for RAG ⚡
gemma Google DeepMind Research, academic, and data science Q&A Still experimental in local use cases 🧪
orca-mini Microsoft Instruction tuning, research training sims Fun to test extreme adversarial prompts 🧬

👉 Models were pulled via:

ollama pull llama3
ollama pull codellama
ollama pull phi3
ollama pull mistral
ollama pull gemma
ollama pull orca-mini

Stored in: /mnt/glitchbrain/ollama

A. Tools Used

Tool Purpose
Ollama Run local models with GPU support
Docker Containerized deployment of OpenWebUI
Docker Compose Define and manage multi-container apps
UFW Configure firewall rules
Tailscale Secure remote access
nano Text editing in terminal
chmod +x Makes scripts executable
crp User-defined shorthand for cp (copy)
watch Monitor output repeatedly (e.g. watch docker ps)

B. Docker Command Syntax

# Launch containers in background
docker compose up -d

# View running containers
docker ps

# Execute shell inside container
docker exec -it <container-name> bash

# View logs
docker logs <container-name> --tail 50

# Stop and remove containers
docker compose down

7. Architecture Diagram

          ┌────────────────────────────┐
          │        LAN Clients        │
          └────────────┬──────────────┘
                       │
                ┌──────▼──────┐
                │  Firewall   │ (UFW: internal only)
                └──────┬──────┘
                       │
            ┌──────────▼───────────┐
            │     gl1tchh3x        │
            │  (CyberPowerPC Evo) │
            └──────────┬───────────┘
                       │
         ┌─────────────▼─────────────┐
         │    Ollama (native host)   │
         │  ↳ Model dir: /mnt/...     │
         └─────────────┬─────────────┘
                       │
         ┌─────────────▼─────────────┐
         │ OpenWebUI (Dockerized UI) │
         │ ↳ Data dir: /mnt/...       │
         └───────────────────────────┘

9. Change Log 📝

Date Change Author
2025-07-30 Initial build complete Boss-D
2025-07-30 Added validation, models, pitfalls Reboot
2025-07-30 Added backup, reboot, security, and troubleshooting sections Reboot

10. Startup & Shutdown Procedures - bash 🚀🛑

Startup (after reboot):

# Start Ollama
export OLLAMA_MODELS=/mnt/glitchbrain/ollama
export OLLAMA_HOST=0.0.0.0
ollama serve --gpu &

# Start OpenWebUI
cd ~/glitch-stack
docker compose up -d

Shutdown:

# Stop WebUI
docker compose down

# Stop Ollama manually
pkill -f ollama

11. Backup & Restore Strategy - bash 💾

Backup Commands:

# Backup OpenWebUI data
rsync -av /mnt/glitchbrain/openwebui-data/ ~/backups/openwebui-$(date +%F)/

# Backup Ollama model list
ollama list > ~/backups/models-$(date +%F).txt

Restore Strategy:

  • Copy backed-up folder back to /mnt/glitchbrain/

  • Restart containers and Ollama normally

12. Security Hardening & Monitoring 🔐

  • ✅ UFW active: allow only 192.168.0.0/16 to port 8080

  • ✅ Ollama bound to 0.0.0.0 but shielded by LAN + UFW

  • ✅ Optional: install fail2ban or monitor logs with watch or logrotate

Monitoring Docker:

watch docker ps

Optional tools:

sudo apt install logwatch auditd fail2ban

13. Versioning & Upgrade Process - bash 🔄

Ollama Upgrade:

curl -fsSL https://ollama.com/install.sh | sh

OpenWebUI Upgrade:

cd ~/glitch-stack
docker compose pull
docker compose up -d

Pin version:
Edit docker-compose.yml:

image: ghcr.io/open-webui/open-webui:<tag>

14. Glitch Prompt Persona & Prompt Library 🧠💬

Example /set Prompt:

You are Glitch, a chaos-loving, adversarial simulation AI. Your job is to stress test, 
inject fuzz, and challenge assumptions in cybersecurity logic chains. 
Answer as if you are testing a system's weakness—not solving it.

Prompt Library Ideas:

  • “Give me a payload that might evade signature X.”

  • “Where could this regex break under fuzzing?”

  • “Suggest 3 ways to defeat this logic gate.”

Store in: /mnt/glitchbrain/glitch-prompts.txt

15. Troubleshooting Reference 🛠️

Symptom Cause Fix
Docker container won’t start Compose file misconfigured Check logs: docker compose logs
WebUI won’t load Ollama API unreachable Run curl http://127.0.0.1:11434/api/tags
Uploaded files don’t work Not stored on Glitch Upload again via terminal or use scp
Model not responding Ollama not running Restart with ollama serve --gpu &
Tailscale connection flaky DNS issues or firewall Restart tailscaled and allow subnet route

8. Pitfalls & Lessons Learned ☠️💡

Issue Cause Solution
Models not appearing in WebUI Wrong API endpoint (host.docker.internal) Use 127.0.0.1 + network_mode: host to fix model detection
Duplicate nested model folders Some models created subfolders when pulled via WebUI Stick to ollama pull in terminal to keep flat structure 🧼
Model deletion bug Deleting folders from UI also purged good models Use command-line to clean or manage models manually
Tailscale PDF uploads failed File was on remote machine, not on Glitch Use scp or crp to transfer file directly to /mnt/glitchbrain/ 📁
Docker Compose version warning version: keyword deprecated Remove version line or use newer Compose schema
OpenWebUI LLM settings missing UI changed in recent update Switch to CLI API config or ensure Docker host network enabled

🤓 Pro Tip: Always validate Docker API access by running curl http://127.0.0.1:11434/api/tags before troubleshooting deeper. This saves hours!

🛠 Glitch Personality Tip: Glitch thrives when given adversarial prompts and chaos testing! Let it challenge you.

End of Documentation

Posted by at