[FEATURE] Add --test-suite flag for automated testing

[jwesleye]

Feature Description

Run a suite of test queries against an agent with pass/fail reporting.

Problem/Motivation

Testing agent changes manually is tedious. Need automated regression testing to ensure agents work correctly.

Proposed Solution

chat_loop myagent --test-suite tests/agent_tests.yaml

# tests/agent_tests.yaml:
# tests:
#   - query: "What is 2+2?"
#     expected_contains: "4"
#   - query: "List Python frameworks"
#     expected_contains: ["Django", "Flask"]
#
# Output:
# ✓ Test 1/10 passed
# ✗ Test 2/10 failed: Expected 'Django' in response

Benefits

• CI/CD integration
• Regression testing
• Quality assurance
• Documents expected behavior

Priority

• Critical
• High
• Medium
• Low

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.

If this issue is still relevant, please:

• Comment to keep it open
• Remove the "status: stale" label
• Provide additional information or updates

Thank you for your contributions!

[jwesleye]

Closing as out of scope for this AWS Strands CLI tool.

Automated testing of agents should be handled at the agent level, not the CLI level. The chat loop is an interactive interface, not a testing framework.

For agent testing, we recommend:

• Write unit tests for your agents directly
• Use AWS Strands' built-in testing capabilities
• Test agent responses programmatically outside the chat loop

Keeping the scope focused on interactive chat sessions.

Thank you for the suggestion!