Letterman

Letterman reads scanned letters with an LLM and turns each one into a translation, summary, deadlines, amounts owed and next actions. Tasks, calendar entries and an archive assistant follow automatically, with any AI provider and your own API key.

Physical mail still carries invoices, tax notices and legal deadlines, yet it sits in piles: unsearchable, easy to misread, and often in a language the recipient does not read fluently. Invoice data gets re-typed by hand, deadlines are missed, and organisations handling sensitive letters cannot simply send them to any cloud AI.

Letterman is designed as a multi-tenant web app that captures scans, cleans and OCRs them, and runs a provider-agnostic LLM pipeline that extracts sender, importance, deadlines, financial effect, invoice details and suggested actions, each with a confidence score. Actions become tasks and calendar events, and a citation-grounded assistant answers questions over the whole archive. AI runs behind one interface with bring-your-own keys, policy and region guards, and an on-premise option with local models.

• Typed API foundation

  Implemented

  FastAPI app factory with Pydantic response models and environment-driven settings that keep every secret out of logs and responses.

• Responsive bilingual app shell

  Implemented

  React 19 SPA with desktop side navigation, mobile bottom tab bar, German/English toggle and semantic light/dark design tokens.

• Capture and OCR pipeline

  Planned

  Upload PDF, image or phone-camera scans; clean-up, Tesseract OCR and automatic splitting of batches into individual letters.

• Confidence-gated letter understanding

  Planned

  LLM extraction of summary, sender, importance, type, deadlines, amounts and invoice details; low-confidence values are flagged or left blank rather than guessed.

• Corrections that stick

  Planned

  Every extracted field is editable, and user corrections override the model and survive re-processing.

• Tasks, calendar and reminders

  Planned

  Each deadline produces exactly one task and one calendar event, with ICS feed, Google and Microsoft 365 two-way sync and e-mail reminders.

• Hybrid semantic search

  Planned

  pgvector embeddings fused with full-text search and structured filters, so a query in plain words finds letters by meaning.

• Archive and letter assistant

  Planned

  Text and voice agent that answers only from stored letters, cites every source, aggregates amounts and drafts replies.

• Bring-your-own-key AI routing

  Planned

  OpenAI, Anthropic, DeepSeek, Gemini or local Ollama per task, with user > organisation > platform key precedence, failover and cost attribution.

A React SPA talks to a FastAPI backend. Uploads are encrypted into S3-compatible storage and queued in Redis; an arq worker runs the letter pipeline (clean-up, OCR, split, extract, translate, embed) as separate retryable jobs. All AI calls go through one provider-agnostic router that resolves keys, enforces provider and region policy, fails over and logs usage. PostgreSQL with pgvector holds tenant-scoped records, full-text and vector indexes. Today the API factory, health endpoint and SPA shell exist; the rest is specified in the architecture plan.

frontend

React 19TypeScriptViteTailwind CSS 4React Router 7TanStack Queryi18nextVitest

backend

Python 3.12FastAPIPydantic v2SQLAlchemy 2 (async)Alembicarquvruffmypy --strictpytest

data

PostgreSQL 16pgvectorRedisS3 / MinIO

ai

OpenAIAnthropic ClaudeDeepSeekGoogle GeminiOllama (local)Tesseract OCR

infra

Docker ComposeCaddyGitHub ActionsEU cloud hosting (planned)

integrations

Google CalendarMicrosoft 365 / GraphICSSMTPOIDC SSO

• Structured letter extraction

  A schema-bound LLM step returns sender, importance, type, deadlines, financial effect, invoice details and suggested actions, each with a confidence value.

• Confidence gating

  Values below a threshold are flagged for review, and amounts or dates below 0.3 confidence are stored as unknown instead of guessed.

• Translation and summaries

  Each letter is translated into the reader's language side by side with the original, plus a two-to-three sentence plain-language summary.

• Grounded archive assistant

  A tool-using agent over hybrid search and aggregates that must cite source letters and say when it does not know.

• Provider-agnostic routing

  One AIProvider interface with a capability registry, per-task model routes, failover, region guard and per-call cost attribution.

1. 01

   Secrets that never leak

   All credentials and provider keys are typed as SecretStr in the settings model so they cannot surface in reprs, logs or API responses; key encryption with per-tenant data keys is specified next.

2. 02

   Tenant isolation by construction

   The data model plans a tenant-scoped session that refuses unscoped queries, with a two-tenant test harness covering every route.

3. 03

   No single-provider dependency

   Providers differ in capabilities (for example, embeddings), so a registry declares capabilities and the router refuses to route a task to a provider that cannot perform it.

4. 04

   Testable app composition

   Both the FastAPI factory and the React App accept injected settings, router and query client, so tests run against isolated instances without network access.

• Bring your own key

  Users and organisations plug in their own AI keys with clear precedence, revocation and visible cost per provider.

• Accuracy over speed

  Every AI value shows its confidence, and user corrections permanently override the model.

• GDPR-first and on-prem capable

  EU-only processing by default, envelope encryption, full export and delete, and a local-model deployment path.

• Invariants with named tests

  Seventeen documented critical rules each map to a specific guard test.

The source repository is private. A live walkthrough is available on request.