Overview
A single model asked to research a question can produce fluent prose with fabricated citations, misattributed quotes, and no mechanism to check any of it. The standard fix of telling the model to "cite your sources" does not work, because the model that writes the claim also decides whether to trust the citation.
This engine separates the roles. A generative model writes claims and extracts exact quotes. A separate, pinned NLI entailment classifier decides whether each claim is supported by its own cited evidence. A publication gate withholds anything that does not pass. The connection from citation to exact quote to source URL is structural; the model cannot supply or forge it.
Live demo: agentic-research-engine-live.onrender.com (free tier, ~1 min cold start)
Pipeline architecture
The pipeline is a LangGraph state machine with three fan-out/fan-in stages and one budget-guarded back-edge:
START → analyze_query → plan_research → generate_queries
→ [parallel] search_worker → dedupe_sources (fan-in barrier)
→ [parallel] fetch_worker → register_sources (fan-in barrier)
→ [parallel] extract_worker → assess_coverage (fan-in barrier)
→ [if gaps + budget] generate_followups → generate_queries (back-edge)
→ [if sufficient] synthesize → verify_citations → finalize → END
Blue nodes are fan-in barriers. The only back-edge (the coverage loop) is budget-guarded to prevent infinite loops.
Answer contract
Before any retrieval begins, the engine builds a contract from the question. A comparison naming fewer than two entities produces a contract with no valid slots, so the engine refuses rather than guesses. This prevents a class of hallucination where the model fills in an implied entity it invented.
Two-model design
The separation between writing and verification is the central design decision:
Write claims and quotes: A generative model (local: qwen3:4b / cloud: OpenAI) composes prose, extracts text from pages, and references evidence IDs. It cannot supply citation links directly; it references IDs, which the engine resolves to registered sources.
Decide what to publish: A pinned NLI entailment classifier (separate from the generative model) scores whether each claim is entailed by its own cited quote. The model cannot influence this judgment.
Gate relevance: A critic model (separate from the synthesizer) judges whether an entailed claim actually answers the question. Relevance fails closed; if the judgment cannot be obtained, the claims it would cover are withheld.
Verification layer
Each claim goes through a two-stage gate:
- NLI entailment: Is this claim entailed by its own cited quote at a threshold of 0.996 confidence? If not, withheld.
- Relevance gate: Does an entailed claim actually answer something in the question? The critic judges this. A claim can be factually correct and still be withheld if it is irrelevant or if the entailment judgment is unavailable.
Security controls
SSRF: Deny-by-default on loopback, private, link-local, multicast, and reserved addresses.
DNS rebinding: Connects to validated address; every redirect hop is revalidated and re-pinned.
Spend: Ceilings reserved before each provider request; output-token ceilings exact.
Prompt injection: Retrieved text is framed as untrusted data; extractor has no tools.
Secrets: gitleaks runs over full history with a positive control.
Results
Blinded self-review across three recorded local qwen3:4b runs:
- 21 unique candidates checked; 11 published, 10 withheld (not entailed)
- Precision: 1.00 — zero unsupported or uncertain claims published
- Recall: 0.58 — limited by design; the engine withholds rather than guesses
Adversarial regression (11 cases, 17 claims, two releases): irrelevant published claims went from 6 to 0 across releases. Correctly published claims fell from 14 to 6 in the same window, reflecting the precision-recall tradeoff, which is visible and recorded.
Performance on frozen corpus (n=2, no significance claimed):
Cloud (OpenAI): 8.84 to 28.51 seconds per run.
Local (qwen3:4b): 54.50 to 118.21 seconds completed; 120 to 122 seconds timeout.
Modes
local: Ollama qwen3:4b, no API key or search provider needed. Around 15 minutes per run on an M-series laptop.
cloud: OpenAI, substantially faster, at metered cost.
hybrid: Extraction local, reasoning cloud. Extraction is the highest-volume role and is mechanical.
ALLOW_CLOUD_FALLBACK defaults to false. Replay mode (the default for the hosted demo) serves committed recorded runs with no API key needed.