feat: Ollama local provider — auto-detect and use local models for ingest pipeline

[bizmindx]

Summary

Add Ollama as an alternative AI provider so Robin can run the full ingest pipeline (extraction, classification, embedding, wiki generation) against local models when Ollama is detected — eliminating the OpenRouter dependency for self-hosted / privacy-first deployments.

Why This Matters

Cost: OpenRouter charges per token. A heavy Robin user logging 20+ thoughts/day burns through API credits on extraction, classification, embedding, and wiki regeneration. Local models = $0 marginal cost after hardware.

Privacy: Right now every thought you log gets sent to OpenRouter → upstream provider. For personal knowledge (journals, relationship notes, business ideas), some users will never accept that. Ollama keeps everything on-device.

Offline / Air-gapped: Robin currently can't ingest anything without internet. With Ollama, the pipeline works on a plane, in a bunker, wherever.

Developer experience: Contributors can hack on the AI pipeline without needing an OpenRouter key. Lower barrier to entry.

Current Architecture (What We're Working With)

Robin's AI integration is cleanly layered but hardcoded to OpenRouter:

Layer	File	Coupling
Provider config	@robin/agent/openrouter-config.ts	OpenRouterConfig interface — apiKey + 4 model slots
Agent factory	@robin/agent/agent-factory.ts	createOpenRouter() from @openrouter/ai-sdk-provider
Embeddings	@robin/agent/embeddings.ts	Raw fetch() to https://openrouter.ai/api/v1/embeddings
Config loader	core/src/lib/openrouter-config.ts	Reads OPENROUTER_API_KEY env + DB model preferences
Boot probe	core/src/bootstrap/check-openrouter-key.ts	Gates worker startup on OpenRouter reachability
Model list	core/src/routes/ai-models.ts	Fetches model catalog from OpenRouter API
AI preferences	core/src/routes/ai-preferences.ts	UI for selecting OpenRouter models per pipeline role
Default models	@robin/shared/prompts/models.ts	claude-sonnet-4.6, gemini-2.5-pro, claude-haiku-4.5

Key constraint: Embedding vectors are 1536-dim (pgvector column). The SAFE_EMBEDDING_MODELS allowlist currently only permits openai/text-embedding-3-small and qwen/qwen3-embedding-8b. Ollama embedding models (e.g. nomic-embed-text, mxbai-embed-large) produce different dimensions — this needs handling.

Proposed Design

1. Provider Abstraction

Replace OpenRouterConfig with a provider-agnostic LlmProviderConfig:

type ProviderType = 'openrouter' | 'ollama'

interface LlmProviderConfig {
  provider: ProviderType
  baseUrl: string              // OpenRouter: https://openrouter.ai/api/v1, Ollama: http://localhost:11434
  apiKey?: string              // Required for OpenRouter, unused for Ollama
  models: {
    extraction: string         // e.g. 'llama3.1:8b' or 'anthropic/claude-sonnet-4.6'
    classification: string
    wikiGeneration: string
    embedding: string          // e.g. 'nomic-embed-text' or 'openai/text-embedding-3-small'
  }
  embeddingDimensions: number  // 1536 for OpenRouter, varies for Ollama
}

2. Auto-Detection at Boot

During bootstrap, before worker startup:

1. Check if OLLAMA_HOST or default http://localhost:11434 is reachable (GET /api/tags)
2. If reachable → list available models, match to pipeline roles
3. If OPENROUTER_API_KEY also exists → user chooses preferred provider (env var or DB config)
4. If neither available → log warning, workers don't start (existing behavior)

New env vars:

• OLLAMA_HOST — override Ollama URL (default: http://localhost:11434)
• AI_PROVIDER — force provider: openrouter | ollama | auto (default: auto)

3. Ollama-Aware Agent Factory

Ollama exposes an OpenAI-compatible API. The Vercel AI SDK has an ollama provider (ollama-ai-provider). The agent factory switches based on provider type:

function createIngestAgents(config: LlmProviderConfig): IngestAgents {
  const model = config.provider === 'ollama'
    ? createOllama({ baseURL: config.baseUrl })
    : createOpenRouter({ apiKey: config.apiKey! })

  return {
    fragmenter: new Agent({ model: model(config.models.extraction), ... }),
    // ...
  }
}

4. Embedding Dimension Handling

This is the trickiest part. Options:

Option A — Fixed 1536, pad/truncate Ollama embeddings:
Simplest but lossy. Padding with zeros degrades search quality.

Option B — Configurable column width, migration on provider switch:
Store embeddingDimensions in config. If user switches providers and dimensions change, re-embed everything. Correct but expensive migration.

Option C — Store dimension in config, validate at boot:
Refuse to start if existing embeddings have different dimensions than the configured model. User must re-embed (via a one-time job) or clear embeddings to switch.

Recommendation: Option C — fail-safe, no silent quality degradation, explicit re-embed command.

5. Model Auto-Mapping

When Ollama is detected, auto-suggest models for each pipeline role based on what's pulled locally:

Pipeline Role	Good Ollama Models	Minimum
Extraction (structured output)	llama3.1:8b, mistral:7b, qwen2.5:7b	7B+
Classification (fast scoring)	llama3.2:3b, phi3:mini, qwen2.5:3b	3B+
Wiki Generation (long-form)	llama3.1:8b, mistral:7b, command-r	7B+
Embedding	nomic-embed-text, mxbai-embed-large	Any embedding model

6. UI Changes (wiki frontend)

• AI preferences page shows detected provider + available local models
• Toggle between OpenRouter / Ollama if both available
• Show Ollama connection status indicator
• Model selector populated from Ollama model list when Ollama is active

Implementation Phases

Phase 1: Provider abstraction + Ollama detection

• Refactor OpenRouterConfig → LlmProviderConfig
• Add boot-time Ollama detection (GET /api/tags)
• New env vars: OLLAMA_HOST, AI_PROVIDER
• No behavior change when Ollama absent

Phase 2: Ollama agent factory + embeddings

• ollama-ai-provider integration in agent factory
• Ollama embedding endpoint (/api/embeddings)
• Dimension validation at boot
• Re-embed command for provider switching

Phase 3: UI + model management

• Ollama model listing route
• AI preferences page updates
• Provider toggle in settings
• Connection status indicator

Phase 4: Smart defaults + DX

• Auto-map pulled models to pipeline roles
• pnpm ollama:setup script that pulls recommended models
• Fallback chain: try Ollama → fall back to OpenRouter if local model fails
• Documentation

Out of Scope (For Now)

• Running Ollama inside the Railway deployment (server-side local inference)
• Fine-tuning / custom model training
• Multiple simultaneous providers (e.g. Ollama for extraction, OpenRouter for generation)
• vLLM, llama.cpp, or other local inference servers (Ollama first, others later)

Embedding Dimension Reference

Model	Dimensions	Provider
openai/text-embedding-3-small	1536	OpenRouter
qwen/qwen3-embedding-8b	1536 (MRL truncated)	OpenRouter
nomic-embed-text	768	Ollama
mxbai-embed-large	1024	Ollama
all-minilm	384	Ollama
snowflake-arctic-embed	1024	Ollama

Acceptance Criteria

• Robin detects local Ollama automatically at boot
• Full pipeline works end-to-end with Ollama models (log entry → fragments → wiki)
• Embedding dimension mismatch is caught and reported clearly
• Existing OpenRouter users see zero behavior change
• AI preferences UI reflects active provider and available models
• Works without internet when Ollama is the active provider