Skip to content

look for p_ convo output#151

Open
jgieringer wants to merge 2 commits intov1.1from
bug/fix-unknown-model
Open

look for p_ convo output#151
jgieringer wants to merge 2 commits intov1.1from
bug/fix-unknown-model

Conversation

@jgieringer
Copy link
Copy Markdown
Collaborator

Description

With the new output defaults, conversations are now stored under p_.../conversations/.txt instead of conversations/p_.../.txt

When passing p_.../conversations as the input to judge, it creates a judge folder called j_...conversations instead of j_...p_... which caused "Unknown" User name and Agent name 💀

This PR addresses looking one level up for the expected p_... dir name to preserve user and agent information

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates transcript folder resolution to support the new default layout where transcripts live under p_.../conversations/*.txt, ensuring judge run folder naming (and derived metadata like user/agent names) stays aligned with the generation run id when users pass p_.../conversations as input.

Changes:

  • Add special-case handling in resolve_conversation_input() to “lift” gen_run_root/basename from the parent when the input folder is a nested conversations/ dir under a generation run folder.
  • Add unit tests covering the nested conversations/-as-input behavior and the non-generation-parent fallback behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
utils/conversation_layout.py Adds logic to detect .../conversations inputs and attribute them to the parent p_... run folder for consistent judge output naming/metadata.
tests/unit/utils/test_conversation_layout.py Adds unit tests validating the new nested conversations/ input resolution behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread utils/conversation_layout.py Outdated
Comment thread utils/conversation_layout.py
Comment thread tests/unit/utils/test_conversation_layout.py
@emily-vanark
Copy link
Copy Markdown
Collaborator

emily-vanark commented Apr 24, 2026

Hmm... running uv run pytest -m "not live" on this branch still results in the mysterious adhoc outputs... (I guess I was wrong about it being the live tests? Updated the issue.)

Stll getting 69 judge_unscoped files similar to the ones in the issue and an empty single_datetime_conv folder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants