feat(harness): 低分自动纠偏 —— poor 触发评语驱动重生成,取更优者(测温计→恒温器)

评测闭环延伸出自愈:当自动评测判定输出为 poor(综合<0.5),dispatcher 在热路径外
自动用「原问题+初版回答+评审短板(flags/评语)」(有来源则连来源一并喂回、要求严格基于来源)
让模型重写,重评后仅当新分严格更高才采纳(绝不退步);采纳的修订版落会话历史,
保证多轮上下文用的是好答案而非被判低分的初版。评测终值带 corrected 标记经 NATS→网关落库。

- maxRefineRounds=1:poor 稀少,1 轮重写+重评够用,防成本失控
- canRefine 门控:模型就绪且熔断未开才纠偏,避免后端抖时雪上加霜
- 单 goroutine 串 评测→纠偏→落历史,杜绝原两 goroutine 对答案版本的竞态
- 契约 EvalEvent / Eval 表 / upsert / GET /tasks/:id/eval 均加 corrected 字段
- refine_test.go:采纳更优 / 不退步 / 非低分不触发 三测;live 验证好答案不误触发

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Blizzard
2026-06-25 16:22:45 +08:00
parent 10f08ffb14
commit 93e9d3b195
8 changed files with 227 additions and 34 deletions
+1 -1
View File
@@ -153,7 +153,7 @@ func (p *Postgres) SaveEval(ctx context.Context, e *Eval) error {
return p.db.WithContext(ctx).Clauses(clause.OnConflict{
Columns: []clause.Column{{Name: "task_id"}},
DoUpdates: clause.AssignmentColumns([]string{
"overall", "rule", "llm", "faithful", "level", "flags", "reason", "sources", "updated_at",
"overall", "rule", "llm", "faithful", "level", "flags", "reason", "sources", "corrected", "updated_at",
}),
}).Create(e).Error
}