Skip to content

产出质量评估方法决策(抽样/LLM 评判/指标) #233

Description

@dommaker

Part of #85

Question

定产出质量的评估方法:抽样人工评 / LLM-as-judge 自动评 / 量化指标,或组合。要在方法票里回答:

  • 评什么维度(结果回写结构完整度、任务达标度、review 有效性)
  • 用什么评(judge prompt 谁写、人工抽样的比例与触发)
  • 评出来记在哪、怎么被改进决策消费

素材口径(2026-08-18 用户裁决):方法开发与校准可用历史样本(#230 导入)+ 归档器现有样本;但正式评估对象是 dogfood 新样本(#132 线),方法定稿时要对新样本格式验证过。

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions