EVAL

AI Agent 评估与上线验收AI Agent Evaluation and Launch Acceptance

用 AI 工具辅助整理样本、生成测试、比较回答、检查工具轨迹和形成上线验收报告。Use AI tools to prepare samples, generate tests, compare answers, inspect tool traces, and produce launch acceptance reports.

所属层级Layer

Enterprise LayerEnterprise Layer

FDE 不能只说 Agent 看起来不错。评估要证明它是否用了正确工具、是否遵守上下文、是否能被人工复核、是否达到上线门槛。An FDE cannot say an agent merely looks good. Evaluation proves whether it used the right tools, followed context, supports human review, and meets launch thresholds.

学习产出Learning Outcomes

用 AI 工具生成任务级测试样本Use AI tools to generate task-level test samples

区分开发时试错和上线前验收Separate dev experimentation from pre-launch acceptance

检查工具调用路径和最终答案质量Check tool paths and final answer quality

输出上线验收报告Ship a launch acceptance report

扩展学习资源Resources

Anthropic: Evaluating AI Agents 学习 eval 思维：如何证明 Agent 可靠，而不是凭感觉试用。 Learn eval thinking: prove agent reliability instead of vibes-testing. OpenAI Evals 作为评估样本和自动化测试概念参考。 Reference eval examples and automated testing concepts. RAGAS 参考 RAG 指标，不要求掌握全部框架。 Reference RAG metrics without mastering the full framework.

检查清单Checklist

AI 与评估AI and Evaluation

有没有 Golden Dataset 或历史样本用于评估？Is there a golden dataset or historical sample for evaluation?
工具调用路径是否可追踪？失败时能否回放？Can tool calls be traced and replayed after failure?
上线前谁验收？上线后如何发现模型退化、幻觉或数据漂移？Who accepts before launch, and how do we detect degradation, hallucination, or drift after launch?