← Centra overview
Phases · Centra

Engineering phases

10 phases, scoped end to end — objective, approach, acceptance criteria, and risks.

23%complete
S0

Foundation & runtime

Done50%▸

Containers, repo-as-infra, and verified GPU access — the bedrock.

S1

Local AI stack

In progress0%▸

A local LLM, STT, and TTS running on the 4060 — fully offline.

Prove the core premise: capable AI on your own hardware with no internet. This is the fastest motivating win and the model backend every project will call.

In scope
  • Ollama serving a quantized 7–9B model on the GPU
  • Open WebUI as a chat front-end
  • whisper.cpp / faster-whisper for speech-to-text
  • Piper for text-to-speech
  • An embedding model for later RAG
Out of scope
  • Agentic tool-calling and orchestration (S4)
  • Cross-device voice pipeline (Blu's project)
Engineering approach
Qwen3.5-9B (or Qwen3-7B) at Q4_K_M via Ollama/GGUF2026 sweet spot for 8 GB VRAM — ~40–55 tok/s fully in-GPU, strong reasoning/coding scores, and near-lossless under KV-cache quantization for long context.
One large model resident at a time (OLLAMA_MAX_LOADED_MODELS=1)8 GB holds a single 7–9B Q4 model well; whisper/piper/embeddings coexist since they're small or CPU-bound.
whisper.cpp for STTFast, CPU/GPU flexible, no Python runtime bloat; tiny/base for realtime, larger models when accuracy matters.
Stack
OllamaOpen WebUIwhisper.cppPipernomic-embed-text
Tasks
  • Ollama container on the GPU
  • Pull Qwen3.5-9B / Qwen3-7B (Q4_K_M)
  • Open WebUI chat front-end
  • whisper.cpp (STT) service
  • Piper (TTS) service
  • Embedding model wired for RAG
Deliverables
  • A browser chat against a local LLM
  • An STT endpoint
  • A TTS endpoint
Acceptance criteria
  • Chat with the local LLM in the browser with Wi-Fi off
  • Transcribe a WAV file locally in near real-time
  • Synthesize a spoken sentence from text locally
  • Sustained tokens/sec measured and recorded
Risks & mitigations
VRAM exhaustion / spill to CPU — Stick to Q4_K_M; cap loaded models; watch KV-cache size and context length.
Thermal throttling on the laptop — Elevate/cool the chassis; monitor GPU temp; keep sustained load reasonable.
Dependencies
S0
References
Best Local LLMs for 8GB VRAM (2026) ↗Ollama VRAM requirements (2026) ↗Ollama ↗whisper.cpp ↗
S2

Data layer

To do60%▸

One Postgres for relational + vectors + time-series, plus cache and object store — with restores that actually work.

S3

Event backbone & API

To do75%▸

The event schema and the gateway every service and device speaks to.

S4

AI Orchestrator

To do0%▸

The differentiated core: events in, typed tool-calls to the LLM, actions out.

S5

Secure remote access

To do0%▸

Reach Centra from anywhere with zero open router ports.

S6

IoT & Home Assistant

To do0%▸

The physical world plugs into the bus via Home Assistant + ESPHome.

S7

Clients

To do0%▸

Your face on the system — a web dashboard and a phone client.

S8

Observability, backup & hardening

To do0%▸

The sleep-at-night layer: metrics, logs, offsite DR, and secrets.

S9

Expansion & NAS migration

To do0%▸

Move the brain to dedicated hardware — the payoff for containerizing everything.