---
{
  "id": "ASD-RSR-EN-AI-EVALUATION",
  "path": "/en/ai/evaluation",
  "title": "Researcher evaluation and known limits",
  "title_en": "Researcher evaluation and known limits",
  "category": "Researcher / 研究协作",
  "summary": "Report retrieval, citation checks, live model runs and human semantic review separately.",
  "related": [
    "/researcher",
    "/ai/knowledge-base",
    "/ai/evaluation"
  ],
  "template": "document",
  "version": "1.0.0",
  "updated": "2026-09-27",
  "date_published": "2026-09-27",
  "canonical": false,
  "source_tier": "published",
  "depends_on": [],
  "open_problems": [],
  "equations": [],
  "sources": [
    {
      "label": "ASD published site corpus / Researcher v1.0 release",
      "kind": "website-engineering"
    }
  ],
  "theory_status": "FOUNDATION",
  "evidence_status": "DOCUMENT",
  "locale": "en",
  "translation_path": "/ai/evaluation",
  "canonical_url": "https://asd-physics.org/en/ai/evaluation",
  "seo_title": "Researcher evaluation and known limits · ASD Researcher v1.0",
  "metadata_reviewed": "2026-09-27",
  "translation_of": "/ai/evaluation",
  "translation_version": "1.0.0"
}
---

## Current evaluation status

<ResearcherEvaluation />

## Coverage

Forty bilingual questions cover LVE, magnetic-force premises, quantum approximations, open problems, registry status, invented IDs, historical versions and prompt injection. Retrieval checks expected sources or errors. Citation checks verify IDs, snapshot, source versions and exact quotes. Each mode has a distinct task instruction.

## Program checks are not semantic review

An authentic quotation can still be misused. Review whether it supports the claim, whether assumptions/orders were omitted, whether comparison results were imported, and whether open results were promoted. Check paired-language meaning. No unmeasured accuracy percentage is published.

## Live acceptance

After deploying the independent Worker with provider credentials, research access tokens and budgets, run the supplied live evaluation script. It records model ID, snapshot, actual answers, fallbacks and citation checks. Human content assessment is separate. Transport fixtures are interface tests, not model-performance evidence.

## Known limitations

Keyword retrieval can miss paraphrases and long cross-page arguments. Historical snapshots must actually be retained and deployed. Context budgets send whole selected sections and expose omissions. Missing configuration, authorization, quota, timeout, invalid citations or model refusal produce retrieval fallback. Program checks cannot guarantee prompt-injection resistance or mathematical correctness; new reasoning needs independent review.