评估与追踪Evaluation & tracing · 单元Unit 12
可观测性与生产事故：从日志到定位Observability and production incidents: from logs to root cause
线上问题不是“模型变笨了”，而是链路某一段缺信息。学完本单元，你能用 trace、hook、事件和渠道回调指标定位模型、工具、状态和平台层故障。Production issues are rarely 'the model got dumber'; some segment lacks information. After this unit you can use traces, hooks, events and channel metrics to locate model, tool, state and platform failures.
学习目标Learning goals
- 用 trace id 串联模型、工具与子智能体Link models, tools and subagents with a trace id
- 选择 Hook、事件或中间件做观测Choose hooks, events or middleware for observability
- 处理渠道回调验签、去重与快速 ACKHandle channel callback signatures, deduplication and fast ACK
概念闪卡Concept cards
点击卡片翻面查看解释。Click a card to flip it and reveal the explanation.
关键 APIKey APIs
点击进入对应的 Python 或 Java 参考页。Open the exact Python or Java reference for each type.
问题巩固Check yourself
完成 10 道题；答错会自动进入错题本。Answer all ten questions. Wrong answers are added to your wrong-answer book automatically.
01Python 中用于追踪回复、模型调用和工具执行的中间件是？Which Python middleware traces replies, model calls and tool execution?
解析Why TracingMiddleware 在 on_reply、on_model_call 和 on_acting 处埋点。TracingMiddleware instruments on_reply, on_model_call and on_acting.
02Java 中扩展生命周期行为的接口是？Which Java interface extends lifecycle behavior?
解析Why Hook 通过 onEvent 接收生命周期事件，并可按 priority 排序。Hook receives lifecycle events through onEvent and orders by priority.
03Java 中区分推理、行动、总结等阶段的枚举是？Which Java enum distinguishes reasoning, acting and summary stages?
解析Why HookEventType 决定钩子覆盖哪些阶段；具体事件携带对应上下文。HookEventType determines which stages a hook covers; concrete events carry context.
04Python 中模型调用开始的事件是？Which Python event marks the start of a model call?
解析Why 成对使用 ModelCallStartEvent 与 ModelCallEndEvent 可以计算模型耗时和失败率。Pairing ModelCallStartEvent with ModelCallEndEvent yields model latency and failure rates.
05一次请求的 trace id 应该覆盖哪些范围？What should a request trace id span?
解析Why trace id 必须贯穿整条调用链，否则跨服务日志无法关联。The trace id must span the entire call chain, or cross-service logs cannot correlate.
06评测集应该覆盖哪些输入？Which inputs should an evaluation set cover?
解析Why 只测正常路径无法发现越权、注入和极端长度等生产问题。Testing only the happy path misses permission, injection and extreme-length issues.
07线上抽样评估相比一次性离线评测的优势是？Why is online sampled evaluation better than a one-off offline run?
解析Why 真实流量的分布会变化，持续抽样才能发现漂移。Real traffic distributions shift; continuous sampling exposes drift.
08渠道回调的正确处理顺序是？What is the correct order for channel callbacks?
解析Why 同步长耗时会导致平台超时重推，必须快速 ACK 并异步处理。Synchronous long work triggers platform retries; ACK fast and process asynchronously.
09Java 官方钉钉渠道扩展构件是？Which official Java extension integrates DingTalk?
解析Why 渠道扩展还包括 feishu、wecom、weixin、github 和 gitlab 等。Channel extensions also include feishu, wecom, weixin, github and gitlab.
10只记录最终答案的最大问题是什么？What is the biggest problem with logging only the final answer?
解析Why 必须记录模型调用、工具执行、重试和状态变更，才能还原失败路径。Record model calls, tool executions, retries and state changes to reconstruct failure paths.