Nicolas Desjardins

Proof points

• Ten failure classes exercised offline in the cascade tests: 429, 503, 404, timeout, unexpected exception, invalid JSON, schema mismatch, empty response, in-band upstream error, and truncated output.
• The test suite cannot reach the network by construction. Every outbound path is closed in conftest, and a test that reaches for one fails immediately.
• Dated benchmark snapshot (7 October 2026): 3 passes of 7 documents per model, every call saved as JSONL so the figures can be re-derived without calling a model.
• Every figure on the project page is checked against the recordings by a test. A sentence that is not derived from the data fails the build.
• Logging hygiene: the model and timing are logged, never the vendor name or invoice number above DEBUG.

Proof points

• Seven real runs recorded on 14 October 2026 and published as the service returned them. They show that the pipeline works, not how often it is right, and a test re-finds every recorded quote verbatim in the sample PDF, on the page it cites.
• A citation cannot be invented. Quotes are matched against the stored chunk, never the generated preamble, after Unicode and typographic folding, and the model never supplies a page number.
• The hermetic suite needs no network, database, or API key, and runs with ruff on every push. Opt-in live PostgreSQL tests assert that the SQL fusion matches its Python twin to 1e-9 and that EXPLAIN uses the HNSW index.
• Fits a 512 MB free instance: three memory changes, including turning off ONNX Runtime's memory arena, took the service from 856 MB to 441 MB at peak with both local models loaded.
• Closed by design for the open internet: uploads need the operator's key, questions are budgeted per visitor and per day, and the day's free allowance is read from OpenRouter itself, so a restart cannot erase it.
• Median over the seven runs: 1.4 s retrieval (embed, SQL, re-rank) and 6.8 s end to end on free models. The SQL itself takes about 170 ms; generation is most of the rest.

Proof points

• Six scenarios recorded on 15 October 2026 against the live stack and published as the API returned them: happy path, self-correction, escalation, edit, reject and out of stock. The five runs that drafted copy took 19 to 66 seconds on free models, all twelve model calls were answered by the first model of the chain, and each of the three dispatches sat behind a recorded human decision.
• In the escalation run the validator model called the draft compliant, and the deterministic dosage check caught three swapped probiotic amounts (10, 10 and 20 billion CFU where the licence says 20, 20 and 10). The draft went to a person instead of out the door, and the person rejected it.
• In the reject run the validator sent the first draft back for an unauthorized claim and passed the rewrite: one self-correction cycle, 66 seconds. The reviewer then rejected it, and the webhook node never ran.
• Dated model benchmark (15 October 2026): 27 calls across three free models on the pipeline's own cases, every answer saved as JSONL so the figures can be re-derived without calling a model. The Liquid model answered 9 of 9 correctly at a 9.1 s median and the NVIDIA model 8 of 9, which is the order the shipped chain uses.
• 504 hermetic tests: no network, no PostgreSQL, no API key. Both HTTP transports raise before a request can leave the process, the real SDK clients cannot be constructed inside the suite, and the README and the project page are held to the code by tests. Four more tests run the compiled graph over a real PostgreSQL checkpointer when a database is provided.
• The webhook is the only mutation. The node refuses to send unless the human review status is approved or edited, a rejection is checkpointed as if the node had run, the receiver verifies the signature in constant time, and every attempt is in the audit log.