Every layer,
every wire.
Four diagrams covering the full path of a request — from the browser through the FastAPI gateway, into a stateful LangGraph agent, out to hybrid retrieval, and the parallel Celery pipeline that turns documents into searchable chunks.
↔ swipe diagrams horizontally to see full detail
From keystroke to streamed token.
A chat message travels through five guarded boundaries before it ever reaches the agent. The response streams back as Server-Sent Events — token by token, with citations and validator results following the same channel.
Stateless auth
Clerk-issued JWT verified per request via JWKS (1h cache). user_id from sub claim flows into the database for RLS.
Per-route throttle
Sliding-window counters in Redis. 10/h ingest, 50/h chat, 200/h docs. Fail-closed on outage.
Cache + invalidate
sha256(normalised query) keyed in Redis, 24h TTL. Ingest finalisation calls invalidate_user(user_id).
One classifier. Four routes. Zero wasted calls.
A stateful StateGraph compiled with PostgreSQL checkpointing. The classifier merges intent detection and routing into a single LLM call, dispatching with LangGraph's Command pattern. Validation runs out-of-graph as a fire-and-forget background task — tokens reach the user before quality scoring begins.
One LLM, four routes
The classifier returns simple_answer | query_rewriter | retriever | query_expander as a Command — no separate router node, no second LLM call.
Off-graph validator
Quality scoring runs as a sampled background task after the SSE stream closes. Result and grounding score are written to the validation_results table.
Two indexes. One ranker. Five chunks.
Vector and full-text search run in parallel via asyncio.gather. Reciprocal Rank Fusion merges them in rank space (no incompatible score averaging). FlashRank then re-ranks the top candidates with a cross-encoder, and parent-chunk swap gives the LLM richer context than what was actually embedded.
HNSW dense index
Hierarchical Navigable Small World on 1536d vectors. Approximate nearest neighbour. Sub-50ms recall on hundreds of thousands of chunks.
GIN sparse index
PostgreSQL ts_vector with English stemming. Catches exact-term matches that semantic search misses (acronyms, identifiers, code).
Cross-encoder rerank
FlashRank's TinyBERT scores each (query, chunk) pair. 4MB model, runs in-process, no extra service. Cohere is wired as an optional upgrade.
Off the request thread, into the worker.
POST /ingest returns 202 Accepted in milliseconds. The actual work — parse, chunk, embed, store — runs in a Celery worker behind a Redis broker, with Docling pre-baked into the worker image to avoid cold-load cost on first task.
MIME-validated upload
libmagic checks the actual file header, not the extension. Defends against masquerade uploads. Allowed types: pdf, md, txt.
At-least-once
task_acks_late + reject_on_worker_lost means a crashed worker re-queues the task. Dedup by SHA256 keeps re-runs idempotent.
Pre-baked Docling
Layout and TableFormer ML models are downloaded into the worker image at build time — first ingest doesn't pay a 300–500MB download cost.
The supporting cast.
External services that the system leans on. All credentials are loaded via pydantic-settings; LangSmith is optional and auto-disabled if no API key is present.
Frontend · App Router · SSE client
JWT auth · JWKS
API · uvicorn :8000
Stateful agent orchestrator
Postgres · pgvector · Storage
Broker · cache · rate limit
Background ingest worker
gpt-4o-mini · embeddings
Tracing · observability