WhisperPad — Private, context-aware notebook (local-first)

Vision

• Build a private-by-architecture, always-on, context-aware notebook that runs on-device, understands your personal context, and never leaks your inner monologue.

What makes it different

• All capture, transcribe, embed, and summarize on-device
• E2EE with per-device keys sealed in Secure Enclave
• Accuracy-first speech stack with VAD, diarization, confidence-aware UI
• 20–30s earshot rolling buffer (tap to save), not surveillance
• Context engine with a local vector store (people, places, projects, decisions)
• Trust features: Vault Mode, forensic wipe, redaction-at-source

MVP milestones

1. Week 0–2: rolling audio buffer, on-device ASR, confidence UI
2. Week 3–4: local embeddings + “Today’s Mind” dashboard
3. Week 5–6: handwriting OCR + personal correction feedback
4. Week 7–8: Vault Mode, device-key sync prototype, export with E2EE

Structure

• apps/ios: SwiftUI app (earshot buffer, dashboards)
• packages/CoreCrypto: E2EE + Secure Enclave key management (stubs)
• packages/ContextEngine: local graph + vector-store interfaces (stubs)
• packages/Embeddings: vector store bindings (stubs)
• packages/Summarizer: local LLM-facing interfaces (stubs)
• docs: architecture, security, roadmap

Status

• Initial scaffold with modules and docs. Stubs compile-leaning but not complete.

Local Model Configuration

WhisperPad now refuses to fall back to heuristic or mock inference. Configure real local models before attempting capture or summarisation:

1. LLM (summaries, entity extraction).

   • Build llama.cpp locally and download an instruction-tuned GGUF model (for example, Meta Llama 3.1 8B Instruct).
   • Point the app at the binaries by exporting:

     export WHISPERPAD_LLAMA_BIN=/path/to/llama/main
     export WHISPERPAD_LLAMA_MODEL=/path/to/model.gguf
     

     Optional overrides: WHISPERPAD_LLAMA_CTX, WHISPERPAD_LLAMA_GPU_LAYERS, WHISPERPAD_LLAMA_THREADS, WHISPERPAD_LLAMA_SEED.
   • Alternatively, place the binary at ~/Library/Application Support/WhisperPad/llama/main and the model at ~/Library/Application Support/WhisperPad/models/base.gguf.

2. Whisper (audio transcription).

   • Compile or vendor a native Whisper inference layer for iOS (e.g. whisper.cpp Metal build or a custom Core ML model).
   • Copy the .ggml/.gguf checkpoint onto the device and select it inside the app’s Settings panel.
   • Until a real runtime is connected, Earshot capture will surface a configuration error instead of emitting synthetic transcripts.

3. Build tooling.

   • Full Xcode is required for xcodebuild or for launching the app. Command line tools alone are insufficient.

The app will surface clear errors whenever the runner or transcriber have not been configured, preventing silent fallback to mock data.