Conversation
There was a problem hiding this comment.
Pull request overview
Updates transcript folder resolution to support the new default layout where transcripts live under p_.../conversations/*.txt, ensuring judge run folder naming (and derived metadata like user/agent names) stays aligned with the generation run id when users pass p_.../conversations as input.
Changes:
- Add special-case handling in
resolve_conversation_input()to “lift”gen_run_root/basename from the parent when the input folder is a nestedconversations/dir under a generation run folder. - Add unit tests covering the nested
conversations/-as-input behavior and the non-generation-parent fallback behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
utils/conversation_layout.py |
Adds logic to detect .../conversations inputs and attribute them to the parent p_... run folder for consistent judge output naming/metadata. |
tests/unit/utils/test_conversation_layout.py |
Adds unit tests validating the new nested conversations/ input resolution behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Hmm... running Stll getting 69 judge_unscoped files similar to the ones in the issue and an empty single_datetime_conv folder. |
Description
With the new output defaults, conversations are now stored under p_.../conversations/.txt instead of conversations/p_.../.txt
When passing p_.../conversations as the input to judge, it creates a judge folder called j_...conversations instead of j_...p_... which caused "Unknown" User name and Agent name 💀
This PR addresses looking one level up for the expected p_... dir name to preserve user and agent information