Conversation
DOsinga
left a comment
DOsinga
left a comment
There was a problem hiding this comment.
Ok, this is much better!
I'm still a bit confused. I don't think this does what it needs to do.
Docs say that the structure is like this:
bucket/
project-name/
attempt-20250326-184018/
pass/
file1.java
file2.java
fail/
file3.java
logs/
migration.log
I don't think it does, but more importantly, I don't think it should. I think it should be something like:
bucket/
project-name/
attempt-000.json
attempt-001.json
attempt-002.json
with the json just have the payload of the results not result. Maybe do a brief chat?
| timestamp: datetime = None, | ||
| ) -> None: | ||
| timestamp = timestamp or datetime.now() | ||
| await self.upload(project, [results_file], "manifest", timestamp) |
There was a problem hiding this comment.
not sure if I want to put the manifest into a folder?
There was a problem hiding this comment.
I think we basically want to create the same structure we have in evals inside of the project directories, then we can point run_eval at these
| timestamp: datetime = None, | ||
| ) -> None: | ||
| timestamp = timestamp or datetime.now() | ||
| await self.upload(project, [results_file], "manifest", timestamp) |
There was a problem hiding this comment.
I think we basically want to create the same structure we have in evals inside of the project directories, then we can point run_eval at these
Kvadratni
force-pushed
the
mnovich/run-results
branch
from
499e965 to
933fc2c
Compare
3 participants
Simplified approach to a result uploader using S3 specifically.
Check Readme updates on how it is setup
tested with local runs.
