← Blu overview
Phases · Blu

Engineering phases

7 phases, scoped end to end — objective, approach, acceptance criteria, and risks.

41%complete
B0

Define & scaffold

Done100%▸

Give Blu a home, a spec, and a device runtime built on the resilient-edge contract.

B1

Local capture + STT

In progress25%▸

Hear, transcribe, and respond locally — offline-first.

Capture speech and turn it into text on the device (or via Centra when online), buffering everything so nothing is lost offline.

In scope
  • Audio capture (push-to-talk first, VAD later)
  • whisper.cpp on-device transcription (tiny/base)
  • Emit reminder.capture events into the outbox
  • Offload heavy STT to Centra when connected
Out of scope
  • Always-on wake word (B5)
  • Intent parsing (B2)
Engineering approach
whisper.cpp tiny/base on-device, larger on CentraTiny/base fit constrained hardware for realtime; when online, stream audio to Centra's whisper for higher accuracy — best of both.
Push-to-talk before voice-activity detectionA button is deterministic and battery-cheap; VAD and wake word add cost and come later.
Stack
whisper.cppI2S audioEdgeAgent outbox
Tasks
  • Audio capture loop (push-to-talk)
  • whisper.cpp on-device transcription
  • ✓Emit reminder.capture events to the outbox
  • Online STT offload to Centra
Deliverables
  • A capture→transcript pipeline
  • Buffered transcript events
Acceptance criteria
  • Speak with Wi-Fi off → Blu transcribes and buffers locally
  • Events sync to Centra on reconnect with no loss
  • Online mode uses Centra's larger whisper for accuracy
Risks & mitigations
On-device STT too slow/inaccurate — Fall back to buffered audio + Centra-side transcription when online.
Dependencies
B0Centra S1
References
whisper.cpp ↗
B2

Understand: classify + schedule

Done100%▸

Classify each captured note by type (idea/project/daily/rant/log/alert/reminder) and schedule the time-based ones.

B3

Notify + sync

In progress67%▸

Fire reminders and confirm them across devices, losing nothing offline.

Deliver due reminders reliably, support acknowledge/snooze, and replay anything missed during an outage.

In scope
  • Notification fan-out (push / TTS / phone)
  • Acknowledge + snooze loop
  • Offline replay of missed reminders
Out of scope
  • Hardware TTS playback wiring (B4)
Engineering approach
Reminders fire from Centra, with local echo on BluCentra is the reliable scheduler; Blu echoes locally (TTS/LED) so it works even if Blu was offline when it fired.
Missed-while-offline reminders resurface on reconnectA reminder that fired while you were away must still reach you — the whole point of the resilient design.
Stack
MQTTPiper (TTS)Push (ntfy / web push)
Tasks
  • ✓Notification fan-out (device TTS echo + opt-in ntfy push)
  • ✓Acknowledge + snooze loop (/ack, /snooze)
  • Offline replay of missed reminders
Deliverables
  • Multi-channel notifications
  • Ack/snooze flow
  • Offline replay
Acceptance criteria
  • A due reminder reaches you across at least two channels
  • Snooze and done both work and persist
  • A reminder that fired during an outage surfaces on reconnect
Risks & mitigations
Duplicate notifications across channels/devices — Idempotent notification IDs; ack collapses the fan-out.
Dependencies
B2Centra S5
References
ntfy ↗
B4a

Firmware in a web simulator (Wokwi)

Blocked33%▸

Skipped — Wokwi is slow and paywalls compilation/features, not worth it. Going straight to real hardware (B4); the firmware sketch (projects/blu/firmware/wokwi) is reusable on a real ESP32-S3.

B4

Pocket device — T-Watch S3

To do0%▸

Bring Blu onto a real handheld: an ESP32-S3 board with a touch screen, mic, haptic and battery (LilyGO T-Watch S3), running the EdgeAgent.

B5

On-device UX + polish

To do0%▸

The handheld experience end to end: browse notes by category, offline cache + sync, wake word, all-day battery, and a pocketable body.