feat(harness): 低分自动纠偏 —— poor 触发评语驱动重生成,取更优者(测温计→恒温器)
评测闭环延伸出自愈:当自动评测判定输出为 poor(综合<0.5),dispatcher 在热路径外 自动用「原问题+初版回答+评审短板(flags/评语)」(有来源则连来源一并喂回、要求严格基于来源) 让模型重写,重评后仅当新分严格更高才采纳(绝不退步);采纳的修订版落会话历史, 保证多轮上下文用的是好答案而非被判低分的初版。评测终值带 corrected 标记经 NATS→网关落库。 - maxRefineRounds=1:poor 稀少,1 轮重写+重评够用,防成本失控 - canRefine 门控:模型就绪且熔断未开才纠偏,避免后端抖时雪上加霜 - 单 goroutine 串 评测→纠偏→落历史,杜绝原两 goroutine 对答案版本的竞态 - 契约 EvalEvent / Eval 表 / upsert / GET /tasks/:id/eval 均加 corrected 字段 - refine_test.go:采纳更优 / 不退步 / 非低分不触发 三测;live 验证好答案不误触发 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -70,7 +70,7 @@ func main() {
|
||||
flags, _ := json.Marshal(ev.Flags)
|
||||
if err := db.SaveEval(context.Background(), &store.Eval{
|
||||
TaskID: ev.TaskID, Overall: ev.Overall, Rule: ev.Rule, LLM: ev.LLM, Faithful: ev.Faithful,
|
||||
Level: ev.Level, Flags: string(flags), Reason: ev.Reason, Sources: ev.Sources,
|
||||
Level: ev.Level, Flags: string(flags), Reason: ev.Reason, Sources: ev.Sources, Corrected: ev.Corrected,
|
||||
}); err != nil {
|
||||
log.Printf("[gateway] 落库评测 %s 失败: %v", ev.TaskID, err)
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user