Add comprehensive edge case tests for LLMRequest conversions#92
Open
sarvanithin wants to merge 1 commit intowithmartian:mainfrom
Open
Add comprehensive edge case tests for LLMRequest conversions#92sarvanithin wants to merge 1 commit intowithmartian:mainfrom
sarvanithin wants to merge 1 commit intowithmartian:mainfrom
Conversation
…ithmartian#64) Add 47 new tests systematically covering edge cases across all LLMRequest conversions: Tool/Function Calling Edge Cases (8 tests): - Empty tools list, empty/minimal schemas, complex nested schemas - Tool choice variants (any, none, specific tool) - Special characters in tool names, required field handling Message Content Edge Cases (9 tests): - Empty/whitespace content, newlines, emoji, unicode (CJK, RTL, math symbols) - Combined emoji with skin tones/ZWJ, very long content (10k chars) Roundtrip Conversions (3 tests): - Messages ↔ Responses ↔ Messages preserves parameters - Chat ↔ Messages ↔ Chat preserves tool definitions - Responses ↔ Chat ↔ Responses preserves system_prompt Parameter Boundaries (8 tests): - Temperature edge values (0.0, 2.0, conversion clamping) - Top-p boundaries (0.0, 1.0) - Max output tokens minimal values - Stop sequences at/exceeding Chat API limit (4 items) - Empty strings in stop sequences Error Handling - Strict Mode (4 tests): - Service tier validation (standard_only filtered by Chat) - Top-k unsupported by Chat/Responses raises - Stop sequences unsupported by Responses raises - Non-alternating messages for Anthropic raises Error Handling - Non-Strict Mode (3 tests): - Top-k silently filtered when unsupported - Stop sequences filtered for Responses - Duplicate role messages dropped with warning API-Specific Conversions (4 tests): - Responses string input → user message - Chat system message extraction to system_prompt - Responses instructions → system_prompt - Anthropic max_tokens defaults to 1024 Metadata Edge Cases (4 tests): - Null values, nested dicts, array values - Empty metadata dict omitted from output Tool Choice Roundtrips (2 tests): - Auto choice preserved across all API pairs - Specific tool name preserved in roundtrips Multiple Messages (2 tests): - Multiple empty messages handled - Properly alternating messages preserved Total test coverage increased from 55 to 102 tests (+85%). Addresses ~47 of ~200 edge cases identified in Issue withmartian#64.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Addresses Issue #64 by adding 47 comprehensive edge case tests for LLMRequest conversions across all three APIs (OpenAI Chat, OpenAI Responses, Anthropic Messages).
Test Coverage Added
Tool/Function Calling (8 tests)
Message Content (9 tests)
Roundtrip Conversions (3 tests)
Parameter Boundaries (8 tests)
Error Handling (7 tests)
API-Specific (4 tests)
Metadata & Tool Choice (6 tests)
Impact
Addresses Issue #64