[3/4] Add Step Functions Lambda handlers for orchestration#77
Closed
ericc59 wants to merge 5 commits into
Closed
Conversation
Implements Lambda handlers invoked by AWS Step Functions to orchestrate benchmark runs: - initialize: Creates run record in DynamoDB, initializes task records for all tasks to be processed, returns run_id and task_ids - handle_error: Handles task failures from Batch jobs, manages retry logic (up to MAX_RETRIES), marks tasks as FAILED after exhausting retries - aggregate: Queries all task records, calculates metrics (accuracy, cost, completion rate), stores aggregated results in S3 - complete: Marks run as COMPLETED/COMPLETED_WITH_ERRORS/FAILED based on task outcomes, publishes final metrics to CloudWatch Also includes: - Comprehensive test suite using moto (8 tests) - README with deployment instructions and state machine flow diagram - Support for LocalStack testing via AWS_ENDPOINT_URL
ericc59
changed the title
ericc59
changed the title
…ERMANENT handling
ericc59
force-pushed
the
aws-integration-pr3
branch
from
ca9e4e6 to
187f23a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the Lambda handlers invoked by AWS Step Functions to orchestrate benchmark runs. This is PR 3/3 in the AWS integration series, completing the distributed execution layer.
Lambda Handlers:
initializehandle_erroraggregatecompleteState Machine Flow:
Files Added:
src/arc_agi_benchmarking/lambdas/__init__.pysrc/arc_agi_benchmarking/lambdas/initialize.pysrc/arc_agi_benchmarking/lambdas/handle_error.pysrc/arc_agi_benchmarking/lambdas/aggregate.pysrc/arc_agi_benchmarking/lambdas/complete.pysrc/arc_agi_benchmarking/lambdas/README.md(deployment instructions)src/arc_agi_benchmarking/tests/test_lambdas.py(8 tests)Dependencies
infra-live-sandbox-teal(add-stepfunctions-module branch)Test plan