Research Engine

Agent research should expose what was found, what failed, and what remains unanswered.

A bounded research runtime that routes questions, collects read-only evidence, checks coverage and claim quality, and returns a concise machine-readable conclusion with audit artifacts.

Problem

Research answers are difficult to trust when source failures disappear, coverage is implicit, and a fluent summary blurs eligible claims with unanswered questions.

My contribution

Implemented typed connector contracts, failure classification, normalization, coverage and claim gates, bounded repair, JSONL artifacts, and report linkage. The artifact shape below is derived from inspected repository fields, not a claim about a real organizational decision.

Key decision

Treat failure as data but never as evidence: a bounded repair can address missing facets without relaxing source, freshness, scope, or robots rules.

Observed result

The offline evaluator records claim eligibility next to facet coverage and evidence artifacts.

Evidence

Verified344 tests passed and 1 was skipped after installing the declared development and report dependencies.
VerifiedTyped connectors, classified failures, deterministic gates, JSONL artifacts, a report path, and bounded repair are implemented.
Not establishedThe real workflow or decision supported by the representative run is not established in the repository evidence.

Verified repository code, test output, or history Not established human use, deployment, or impact evidence.

System architecture

Follow the authority boundary from input to verified outcome.

01 / 06 · Source

Question pack

A question selects explicit facets, source families, and claim rules.

QuestionChoose the evidence contract
ConnectCollect bounded evidence
NormalizeMake observations comparable
GateCheck claims and coverage
RepairRecover without relaxing safety
SummarizeReturn the conclusion first

Reproduce / inspect

Start with the shortest proof path.

Before you start: Python environment with the declared extras; install with python -m pip install -e '.[dev,report]'. External connectors remain subject to their own access boundaries.
Path: make eval
Observed: Offline fixtures write eval-results/scorecard.json and preserve claim_review.json alongside the evidence artifacts.

Open the repository

Evaluation & test evidence

What the inspected evidence supports.

Test runVerified
344 passed and 1 skipped after declared dependencies were installed.
Artifact pathVerified
A compact summary is linked to deeper evidence and report artifacts.
Quality impactNot established
No before/after research-quality or decision-impact result is claimed.

Artifact shape

Representative artifact shape

Evidence basis · research-engine · source-inspected artifact shape; illustrative fields, not a captured run

{
  "compact_summary": {"status": "complete_with_warnings", "rows": "eligible / total"},
  "evidence_row": {"facet": "overview", "quality_tier": "high", "claim_eligible": true, "status": "eligible"},
  "withheld_row": {"source_class": "discovery-only", "claim_eligible": false, "status": "withheld"},
  "failure_disclosure": [
    "invalid evidence rows are excluded from claims",
    "discovery-only rows are excluded from claims"
  ],
  "missing_facets": ["current_evidence", "alternatives", "risks"],
  "deeper_report": {"status": "generated", "markdown": "research_report.md", "pdf": "research_report.pdf"}
}

Repository depth

Follow the implemented interfaces, behavioral tests, and history behind this case.

Implemented interfaceExecution pipeline
Implemented interface: bounded connector execution and run state.
Implemented interfaceOffline evaluator
Implemented interface: fixture evaluation and scorecard generation.
Behavioral testArtifact contract tests
Behavioral test: machine-readable evidence and claim artifacts.
Behavioral testReport-mode tests
Behavioral test: deeper report generation and linkage.

Engineering judgment

One decision, the alternative, and the trade-off.

Decision

Make claim eligibility a deterministic gate and keep the default artifact compact.

Alternative

Generate a long narrative report first and ask a downstream agent to infer source health and coverage.

Why

The compact contract keeps failure and coverage visible to the next consumer while deeper artifacts remain available for audit.

Failure modes & limits

Public repository · source inspected

Network and connector failures explain missing coverage but cannot support a claim about the world.
A single repair pass may improve missing facets but does not bypass freshness, robots, or scope rules.
No comparative evaluation is claimed for research quality, recall, or decision impact.

Integrated dependencies

Python
Typed connectors
JSONL artifacts
Deterministic evaluation
PDF reporting