Skip to content
NDNicolas Desjardins, back to overview
Resume
Live Demo & Open Source02 / Phase 2 · Shipped

Ask your manuals

Context-Aware Hybrid Compliance Retrieval Engine

Staff ask a question about your procedures or regulations and get an answer that quotes the exact passage and the page it came from. When the documents do not say it, the answer says so.

Project page(opens in a new tab)Live demo(opens in a new tab)Source on GitHub(opens in a new tab)

The problem it solves

Naive RAG cuts text by character count and ranks by cosine similarity alone. It blurs exact identifiers such as 21 CFR Part 11 or SOP-QA-017, strips a clause of the document it came from, and leaves the model free to misquote a number or cite a page it never saw. In a compliance setting that is not a quality issue, it is a liability.

What was measured

  • Measured
  • Target

  • 6 / 6

    recorded questions answered with verified citations

    measured

  • 1 / 1

    out-of-corpus question declined, not guessed

    measured

  • 441 MB

    peak memory with both local models, on a 512 MB instance

    measured

  • $0.00

    marginal cost per question on the free stack

    measured
How it worksfor engineers and technical reviewers

In technical terms

A retrieval engine for SOPs and regulatory manuals that answers only by quoting the source. Vector and keyword search run in one PostgreSQL statement and are fused, a cross-encoder keeps the three best passages, and every quote the model returns is checked against the stored document before anyone sees it. When the documents do not say it, the answer says so. The public demo runs at zero cost on free models.

Architecture

PyMuPDF extracts layout blocks per page, and a structural chunker packs them along headings into 400-token chunks with a 60-token overlap. Following Anthropic's Contextual Retrieval pattern, every chunk is stored with a situational preamble, and both its embedding and a generated tsvector column are built from that text. One SQL statement runs a pgvector cosine search over an HNSW index (m=16, ef_construction=64) and a full-text search over a GIN index, then fuses the two lists with Reciprocal Rank Fusion (k=60) in a CTE. A cross-encoder reads the question with each of up to 50 fused candidates and keeps 3. The model must return strict JSON with an answer, a confidence score, and verbatim quotes, or an explicit insufficient-context verdict. The server re-anchors every quote to the stored chunk text, drops what does not match, fills titles and page numbers from the database, and withholds any answer with no surviving quote. Generation runs on an ordered chain of free OpenRouter models with fallback; Claude, OpenAI embeddings, and Cohere Rerank v3.5 are drop-in settings.

Stack

  • Python 3.12
  • FastAPI
  • Pydantic v2
  • PostgreSQL
  • pgvector (HNSW)
  • tsvector (GIN)
  • Reciprocal Rank Fusion
  • fastembed (ONNX)
  • OpenRouter
  • PyMuPDF
  • Render

Proof points

  • Seven real runs recorded on 14 October 2026 and published as the service returned them. They show that the pipeline works, not how often it is right, and a test re-finds every recorded quote verbatim in the sample PDF, on the page it cites.
  • A citation cannot be invented. Quotes are matched against the stored chunk, never the generated preamble, after Unicode and typographic folding, and the model never supplies a page number.
  • The hermetic suite needs no network, database, or API key, and runs with ruff on every push. Opt-in live PostgreSQL tests assert that the SQL fusion matches its Python twin to 1e-9 and that EXPLAIN uses the HNSW index.
  • Fits a 512 MB free instance: three memory changes, including turning off ONNX Runtime's memory arena, took the service from 856 MB to 441 MB at peak with both local models loaded.
  • Closed by design for the open internet: uploads need the operator's key, questions are budgeted per visitor and per day, and the day's free allowance is read from OpenRouter itself, so a restart cannot erase it.
  • Median over the seven runs: 1.4 s retrieval (embed, SQL, re-rank) and 6.8 s end to end on free models. The SQL itself takes about 170 ms; generation is most of the rest.

Request pipeline

  1. 01

    Ingest

    PyMuPDF blocks, heading-aware 400-token chunks, contextual preamble

  2. 02

    Index

    pgvector HNSW and a generated tsvector GIN index, one table

  3. 03

    Retrieve

    Dense cosine and full-text search in one SQL statement

  4. 04

    Fuse

    Reciprocal Rank Fusion CTE, k=60, up to 50 candidates

  5. 05

    Rerank

    Cross-encoder reads question and passage together, keeps 3

  6. 06

    Verify

    Strict JSON answer, quotes re-anchored to stored text, pages from the database