diff --git a/README.md b/README.md index 37c588c..29c446b 100644 --- a/README.md +++ b/README.md @@ -44,6 +44,7 @@ Supported and loaded modules: - parse_robot: Robot Framework XML Files - parse_openapi: OpenAPI YML Files - add_run: Create a new test run + - fields: Manage fields (list dynamic filter fields) - labels: Manage labels (add, update, delete, list) - results: Manage test results (list, update) - references: Manage references (cases and runs) @@ -97,6 +98,7 @@ Commands: add_run Add a new test run in TestRail cases Manage test cases in TestRail export_gherkin Export BDD test case from TestRail as .feature file + fields Manage fields in TestRail import_gherkin Upload Gherkin .feature file to TestRail labels Manage labels in TestRail parse_cucumber Parse Cucumber JSON results and upload to TestRail @@ -1949,6 +1951,18 @@ $ trcli -c config.yml plans add \ --name "Complex Test Plan" \ --entries-file entries.json +# Create a plan with dynamic filters in entries (inline) +$ trcli -c config.yml plans add \ + --name "Automated High Priority Tests" \ + --entries '[{"suite_id": 1, "name": "Smoke Tests", "runs": [{"dynamic_filters": {"cases:priority_id": {"values": [1,2]}}}]}]' + +# Create a plan with dynamic filters from file +# Dynamic filters are specified in the runs array within entries +$ trcli -c config.yml plans add \ + --name "Sprint 5 Test Plan" \ + --entries-file plan_with_filters.json \ + --dynamic-filters-mode 2 + # Get JSON output for programmatic use $ trcli -c config.yml plans add \ --name "Automated Plan" \ @@ -2050,6 +2064,7 @@ Each entry represents a test run (or group of runs for multi-config scenarios). - `assignedto_id` (optional): User ID to assign the run to - `config_ids` (optional): Configuration IDs for multi-config runs - `runs` (optional): Array of run configurations (required when using `config_ids`) +- `dynamic_filters` (optional): Dynamic filter criteria for auto-updating test runs (see Dynamic Filters section for details) **Important Notes for Multi-Configuration Entries:** @@ -2076,6 +2091,66 @@ Example for cross-browser testing: This creates one run that tests cases 100, 101, 102 across Chrome and Firefox configurations. +**Dynamic Filters in Plan Entries:** + +You can use dynamic filters in plan entries to create auto-updating test runs. Dynamic filters are specified in the `runs` array and follow the same syntax as the `add_run` command (see [Dynamic Filters for Auto-Updating Test Runs](#dynamic-filters-for-auto-updating-test-runs) for detailed documentation). + +Example entry with dynamic filters: +```json +{ + "suite_id": 1, + "name": "High Priority Automated Tests", + "runs": [ + { + "dynamic_filters": { + "mode": "1", + "filters": { + "cases:priority_id": {"values": [1, 2]}, + "cases:is_automated": {"value": true} + } + } + } + ] +} +``` + +**Using --dynamic-filters-mode flag:** + +The `--dynamic-filters-mode` flag controls the filter mode for all runs in the plan (same precedence as `add_run`): + +```bash +# Apply OR mode to all runs without explicit mode in JSON +trcli plans add \ + --name "Sprint Plan" \ + --entries-file plan.json \ + --dynamic-filters-mode 2 +``` + +**Mode Precedence:** +1. **JSON mode** (if explicitly specified in JSON) - Takes highest priority +2. **CLI mode** (`--dynamic-filters-mode`) - Applied to runs without explicit mode +3. **Default "1"** (AND) - Used if neither JSON nor CLI specifies mode + +**Simplified format auto-wrapping:** +```json +{ + "runs": [ + { + "dynamic_filters": { + "cases:priority_id": {"values": [1, 2]} + } + } + ] +} +``` +This automatically wraps to include `"mode": "1"` and the `"filters"` wrapper. + +**Validation rules:** +- `dynamic_filters` and `case_ids` are mutually exclusive +- `dynamic_filters` and `include_all: true` are mutually exclusive +- Filter criteria must match at least one test case in the suite +- All field names must use the `"cases:"` prefix + ##### Configuration File Support The `plans` command supports configuration files, allowing you to specify connection details and project information once: @@ -3967,6 +4042,641 @@ trcli -y -h https://example.testrail.io/ --project "My Project" \ --clear-run-end-date ``` +### Dynamic Filters for Auto-Updating Test Runs + +The `add_run` command supports **dynamic filters**, which enable you to create auto-updating test runs that continuously synchronize with your test case repository. Instead of manually selecting specific test cases, dynamic filters allow TestRail to automatically include cases that match your filter criteria, even as your test suite evolves. + +#### Why Use Dynamic Filters? + +Dynamic filters are ideal for scenarios where: +- You want test runs to automatically include new test cases that match certain criteria +- You need to maintain test runs for specific priorities, milestones, or custom fields +- Your test suite is actively growing and you want runs to stay current +- You want to avoid manually updating run case selections + +#### Discovering Available Filter Fields + +Before creating dynamic filters, use the `fields list-dynamic` command to discover which fields are available for filtering in your project: + +```bash +# List available dynamic filter fields +trcli -y -h https://example.testrail.io/ --project "My Project" \ + fields list-dynamic + +# Output as JSON for programmatic use +trcli -y -h https://example.testrail.io/ --project "My Project" \ + fields list-dynamic --json-output +``` + +This command shows: +- Available fields (system and custom) +- Field types (Dropdown, Checkbox, String, Date, etc.) +- Supported operators for each field +- Available options for dropdown fields + +#### Creating Dynamic Filter Files + +Dynamic filters are defined in JSON files. The TestRail CLI supports two formats: + +**Note:** JSON files can use any whitespace formatting (spaces, tabs, minified, etc.) - the parser handles all valid JSON formats. + +**Full Format** (recommended for complex filters): +```json +{ + "mode": "1", + "filters": { + "cases:priority_id": {"values": [1, 2]}, + "cases:is_automated": {"value": true}, + "cases:title": { + "mode": "2", + "filters": [ + {"op": 5, "value": "login"}, + {"op": 5, "value": "auth"} + ] + } + } +} +``` + +**Simplified Format** (auto-wrapped with mode="1"): +```json +{ + "cases:priority_id": {"values": [1, 2]}, + "cases:type_id": {"values": [1]} +} +``` + +#### Filter Modes - AND and OR modes + +Dynamic filters support a **two types of mode** for powerful and flexible case selection: + +- **Mode `"1"` (AND)**: Test cases must match **ALL** field conditions + - Example: Priority = P1 **AND** Automated = true **AND** Title contains "login" + - Only cases satisfying every single field filter are included + +- **Mode `"2"` (OR)**: Test cases must match **ANY** field condition + - Example: Priority = P1 **OR** Automated = true **OR** Title contains "login" + - Cases matching at least one field filter are included + +##### **Mode Precedence and Defaults** + +When creating test runs with dynamic filters, the mode is determined in this order: + +1. **JSON file mode** (if explicitly specified) - Takes highest priority +2. **`--dynamic-filters-mode` CLI flag** - Used only if JSON has no mode +3. **Default: `"1"` (AND)** - Applied if neither JSON nor CLI specifies mode (also the current default in Web UI) + +##### **Mode Examples** + +**Example 1: AND Mode (all conditions must match)** +```json +{ + "mode": "1", + "filters": { + "cases:priority_id": {"values": [1, 2]}, + "cases:is_automated": {"value": true}, + "cases:title": { + "mode": "2", + "filters": [ + {"op": 5, "value": "login"}, + {"op": 5, "value": "auth"} + ] + } + } +} +``` + +**Matches**: Test cases that are: +- Priority P1 **OR** P2 (from values array) +- **AND** Is Automated = true +- **AND** Title contains "login" **OR** "auth" (field-level mode "2") + +**Test Case Examples**: +``` +MATCH: "Test user login flow" | P1 | Automated: Yes +MATCH: "OAuth authentication" | P2 | Automated: Yes +NO MATCH: "Test user login flow" | P1 | Automated: No (fails automation check) +NO MATCH: "Test checkout process" | P1 | Automated: Yes (fails title check) +``` + +**Example 2: OR Mode (any condition matches)** +```json +{ + "mode": "2", + "filters": { + "cases:priority_id": {"values": [1]}, + "cases:label_id": { + "mode": "1", + "values": [4, 7] + } + } +} +``` + +**Matches**: Test cases that are: +- Priority P1 +- **OR** Have labels "Smoke" **AND** "Regression" (field-level mode "1") + +**Test Case Examples**: +``` +MATCH: "Any test" | P1 | Labels: [] +MATCH: "Any test" | P3 | Labels: [Smoke, Regression] +MATCH: "Any test" | P1 | Labels: [Smoke, Regression] +NO MATCH: "Any test" | P3 | Labels: [Smoke] (needs both labels or P1 priority) +``` + +**Example 3: Complex Nested Modes** +```json +{ + "mode": "1", + "filters": { + "cases:section_id": {"values": [10, 20]}, + "cases:title": { + "mode": "1", + "filters": [ + {"op": 5, "value": "API"}, + {"op": 6, "value": "deprecated"} + ] + }, + "cases:custom_automation_status": {"value": 1} + } +} +``` + +**Matches**: Test cases that are: +- In section 10 **OR** 20 +- **AND** Title contains "API" **AND** does not contain "deprecated" +- **AND** Automation Status = Ready (value 1) + +**Test Case Examples**: +``` +MATCH: "API endpoint validation" | Section 10 | Status: Ready +NO MATCH: "API endpoint deprecated" | Section 10 | Status: Ready (contains "deprecated") +NO MATCH: "API endpoint validation" | Section 30 | Status: Ready (wrong section) +``` + +##### **Mode Usage Guidelines** + +**Use AND Mode (`"1"`)** when: +- You need strict filtering (e.g., "P1 AND automated AND in sprint-5 label") +- All conditions are required for test case inclusion +- Default behavior is desired + +**Use OR Mode (`"2"`)** when: +- You want broader test case selection (e.g., "P1 cases OR smoke label cases") +- Any single condition qualifies a case for inclusion +- Creating comprehensive test runs + + +#### Supported Filter Types + +**1. Checkbox Fields** - Boolean fields like "Is Automated" +```json +{ + "cases:is_automated": {"value": true} +} +``` + +**2. Dropdown Fields** - Single-select fields like Priority, Type, Milestone, User + +Select a single option: +```json +{ + "cases:priority_id": {"value": 1} +} +``` + +Or select multiple options (OR logic): +```json +{ + "cases:priority_id": {"values": [1, 2, 3]}, + "cases:type_id": {"values": ["all"]} +} +``` + +**3. Multi-Select Fields** - Multiple selection fields with optional AND/OR mode +```json +{ + "cases:custom_tags": { + "mode": "2", + "values": [1, 2, 3] + } +} +``` + +**4. Operator-Based Fields** - Text, number, and date fields with operators +```json +{ + "cases:title": { + "mode": "1", + "filters": [ + {"op": 5, "value": "login"}, + {"op": 6, "value": "deprecated"} + ] + } +} +``` + +**Supported Operators:** +- **1**: Is (exact match) +- **2**: Is Not +- **3**: Is Before (dates) +- **4**: Is After (dates) +- **5**: Contains (text) +- **6**: Does not contain (text) +- **7**: Is Less (numbers) +- **8**: Is More (numbers) + +#### Filter Format Examples with Acceptance Criteria + +This section provides concrete examples showing which test cases would match each filter configuration. + +##### **Scenario 1: High Priority Smoke Tests** + +**Filter Configuration**: +```json +{ + "mode": "1", + "filters": { + "cases:priority_id": {"values": [1, 2]}, + "cases:label_id": {"values": [4]} + } +} +``` + +**Acceptance Criteria**: +- Priority must be P1 (ID: 1) **OR** P2 (ID: 2) +- **AND** Must have label "Smoke" (ID: 4) + +**Test Case Matching Table**: + +| Test Case | Priority | Labels | Match? | Reason | +|-----------|----------|--------|--------|--------| +| "User login validation" | P1 | [Smoke] | ✅ Yes | P1 priority AND has Smoke label | +| "Password reset flow" | P2 | [Smoke, Regression] | ✅ Yes | P2 priority AND has Smoke label | +| "User logout" | P3 | [Smoke] | ❌ No | Priority P3 doesn't match (needs P1 or P2) | +| "Database migration" | P1 | [Regression] | ❌ No | Missing Smoke label | +| "API health check" | P1 | [] | ❌ No | No labels (needs Smoke) | + +**Expected Result**: Only test cases 1 and 2 included in run. + +--- + +##### **Scenario 2: Authentication Module Tests (Any Title Match)** + +**Filter Configuration**: +```json +{ + "cases:title": { + "mode": "2", + "filters": [ + {"op": 5, "value": "login"}, + {"op": 5, "value": "authentication"}, + {"op": 5, "value": "password"}, + {"op": 5, "value": "OAuth"} + ] + } +} +``` + +**Acceptance Criteria**: +- Title must contain **ANY** of: "login", "authentication", "password", or "OAuth" +- No other filters applied (all priorities, types, etc.) + +**Test Case Matching Table**: + +| Test Case Title | Match? | Reason | +|-----------------|--------|--------| +| "Test user login with valid credentials" | ✅ Yes | Contains "login" | +| "OAuth2 authentication flow" | ✅ Yes | Contains "OAuth" and "authentication" | +| "Password strength validation" | ✅ Yes | Contains "password" | +| "Reset password via email" | ✅ Yes | Contains "password" | +| "User profile update" | ❌ No | Doesn't contain any keyword | +| "Session timeout handling" | ❌ No | Doesn't contain any keyword | + +**Expected Result**: First 4 test cases included in run. + +--- + +##### **Scenario 3: Automated Non-Deprecated API Tests** + +**Filter Configuration**: +```json +{ + "mode": "1", + "filters": { + "cases:is_automated": {"value": true}, + "cases:type_id": {"values": [3]}, + "cases:title": { + "mode": "1", + "filters": [ + {"op": 5, "value": "API"}, + {"op": 6, "value": "deprecated"} + ] + } + } +} +``` + +**Acceptance Criteria**: +- Must be automated (is_automated = true) +- **AND** Must be type "API Test" (ID: 3) +- **AND** Title must contain "API" +- **AND** Title must NOT contain "deprecated" + +**Test Case Matching Table**: + +| Test Case | Automated | Type | Match? | Reason | +|-----------|-----------|------|--------|--------| +| "API endpoint validation" | Yes | API Test | ✅ Yes | All conditions met | +| "API response time check" | Yes | API Test | ✅ Yes | All conditions met | +| "API v1 endpoint (deprecated)" | Yes | API Test | ❌ No | Contains "deprecated" | +| "API health check" | No | API Test | ❌ No | Not automated | +| "UI login test with API" | Yes | Functional | ❌ No | Wrong test type | + +**Expected Result**: Only first 2 test cases included in run. + +--- + +##### **Scenario 4: Date Range Filter (Recently Updated)** + +**Filter Configuration**: +```json +{ + "cases:updated_on": { + "mode": "1", + "filters": [ + {"op": 4, "value": 1704067200} + ] + } +} +``` + +**Note**: Unix timestamp 1704067200 = 2024-01-01 00:00:00 UTC + +**Acceptance Criteria**: +- Updated date must be **after** January 1, 2024 (operator 4 = "Is After") + +**Test Case Matching Table**: + +| Test Case | Updated On | Match? | Reason | +|-----------|------------|--------|--------| +| "New feature test" | 2024-06-15 | ✅ Yes | After 2024-01-01 | +| "Recently modified test" | 2024-03-20 | ✅ Yes | After 2024-01-01 | +| "Legacy test case" | 2023-12-15 | ❌ No | Before 2024-01-01 | +| "Old regression test" | 2022-05-10 | ❌ No | Before 2024-01-01 | + +**Expected Result**: First 2 test cases included in run. + +--- + +##### **Scenario 5: Integer Range (Estimated Duration)** + +**Filter Configuration**: +```json +{ + "cases:estimate": { + "mode": "1", + "filters": [ + {"op": 8, "value": 60}, + {"op": 7, "value": 300} + ] + } +} +``` + +**Acceptance Criteria**: +- Estimate must be **more than** 60 seconds (operator 8 = "Is More") +- **AND** Estimate must be **less than** 300 seconds (operator 7 = "Is Less") +- Result: Tests between 1-5 minutes duration + +**Test Case Matching Table**: + +| Test Case | Estimate (seconds) | Match? | Reason | +|-----------|-------------------|--------|--------| +| "Quick smoke test" | 30 | ❌ No | Less than 60s | +| "Standard API test" | 120 | ✅ Yes | Between 60-300s | +| "Login flow test" | 180 | ✅ Yes | Between 60-300s | +| "Full regression suite" | 600 | ❌ No | More than 300s | + +**Expected Result**: Test cases 2 and 3 included in run. + +--- + +##### **Scenario 6: Multi-Select with AND Logic (Multiple Labels Required)** + +**Filter Configuration**: +```json +{ + "cases:label_id": { + "mode": "1", + "values": [4, 7, 9] + } +} +``` + +**Acceptance Criteria**: +- Must have label "Smoke" (ID: 4) +- **AND** Must have label "Regression" (ID: 7) +- **AND** Must have label "Critical" (ID: 9) +- All three labels required + +**Test Case Matching Table**: + +| Test Case | Labels | Match? | Reason | +|-----------|--------|--------|--------| +| "Core functionality test" | [Smoke, Regression, Critical] | ✅ Yes | Has all 3 labels | +| "Login test" | [Smoke, Regression] | ❌ No | Missing "Critical" label | +| "Payment flow" | [Regression, Critical] | ❌ No | Missing "Smoke" label | +| "UI validation" | [Smoke] | ❌ No | Missing "Regression" and "Critical" | + +**Expected Result**: Only first test case included in run. + +--- + +##### **Scenario 7: Complex Real-World Filter (CI Pipeline)** + +**Filter Configuration**: +```json +{ + "mode": "1", + "filters": { + "cases:priority_id": {"values": [1, 2]}, + "cases:is_automated": {"value": true}, + "cases:milestone_id": {"values": [5]}, + "cases:title": { + "mode": "2", + "filters": [ + {"op": 5, "value": "smoke"}, + {"op": 5, "value": "critical"} + ] + } + } +} +``` + +**Acceptance Criteria**: +- Priority must be P1 or P2 +- **AND** Must be automated +- **AND** Must be in milestone "Release 2.0" (ID: 5) +- **AND** Title contains "smoke" **OR** "critical" + +**Test Case Matching Table**: + +| Test Case | Priority | Auto | Milestone | Title | Match? | +|-----------|----------|------|-----------|-------|--------| +| "Critical login smoke test" | P1 | Yes | Release 2.0 | Contains both | ✅ Yes | +| "Smoke test - API health" | P2 | Yes | Release 2.0 | Contains "smoke" | ✅ Yes | +| "Critical payment flow" | P1 | Yes | Release 2.0 | Contains "critical" | ✅ Yes | +| "Smoke test - dashboard" | P3 | Yes | Release 2.0 | Contains "smoke" | ❌ No (P3) | +| "Critical login test" | P1 | No | Release 2.0 | Contains "critical" | ❌ No (Not automated) | +| "Critical login test" | P1 | Yes | Release 1.0 | Contains "critical" | ❌ No (Wrong milestone) | + +**Expected Result**: First 3 test cases included in run. + +#### Using Dynamic Filters with add_run + +Create a new test run with dynamic filters: + +```bash +# Create auto-updating run with dynamic filters +trcli -y -h https://example.testrail.io/ --project "My Project" \ + add_run --title "Automated Tests - High Priority" \ + --suite-id 1 \ + --dynamic-filters ./filters/high_priority.json +``` + +Specify filter mode via command line (used only when JSON file doesn't specify mode): + +```bash +# Use OR mode for top-level filter combination +# This only applies if priorities.json doesn't have a "mode" key +trcli -y -h https://example.testrail.io/ --project "My Project" \ + add_run --title "P1 or P2 Tests" \ + --suite-id 1 \ + --dynamic-filters ./filters/priorities.json \ + --dynamic-filters-mode "2" +``` + +#### Case Selection Precedence + +When creating test runs, case selection follows this precedence (highest to lowest): + +1. **`--run-case-ids`** - Explicit case IDs (highest priority) +2. **`--dynamic-filters`** - Dynamic filter criteria +3. **`--run-include-all`** - Include all suite cases (lowest priority) + +**Important:** These options are mutually exclusive. You can only use one at a time. + +#### Complete Examples + +**Example 1: High Priority Automated Tests** + +Create `filters/high_priority_automated.json`: +```json +{ + "mode": "1", + "filters": { + "cases:priority_id": {"values": [1, 2]}, + "cases:is_automated": {"value": true} + } +} +``` + +```bash +trcli -y -h https://example.testrail.io/ --project "My Project" \ + add_run --title "P1/P2 Automated Tests" \ + --suite-id 1 \ + --dynamic-filters ./filters/high_priority_automated.json \ + --milestone-id 5 +``` + +**Example 2: Authentication Tests** + +Create `filters/auth_tests.json`: +```json +{ + "mode": "2", + "filters": { + "cases:title": { + "mode": "2", + "filters": [ + {"op": 5, "value": "login"}, + {"op": 5, "value": "authentication"}, + {"op": 5, "value": "password"} + ] + }, + "cases:section_id": {"values": [42]} + } +} +``` + +```bash +trcli -y -h https://example.testrail.io/ --project "My Project" \ + add_run --title "Auth & Security Tests" \ + --suite-id 1 \ + --dynamic-filters ./filters/auth_tests.json +``` + +**Example 3: Regression Test Run** + +Create `filters/regression.json`: +```json +{ + "mode": "1", + "filters": { + "cases:type_id": {"values": [1, 2]}, + "cases:priority_id": {"values": [1, 2, 3]}, + "cases:custom_automation_status": {"values": [1]} + } +} +``` + +```bash +trcli -y -h https://example.testrail.io/ --project "My Project" \ + add_run --title "Nightly Regression" \ + --suite-id 1 \ + --dynamic-filters ./filters/regression.json \ + --run-assigned-to-id 5 \ + --run-refs "SPRINT-42" +``` + +#### Benefits of Dynamic Filters + +1. **Automatic Updates**: Test runs automatically include new cases that match criteria +2. **Maintainability**: No manual case selection needed as suite grows +3. **Consistency**: Ensure runs always include the right test cases +4. **Flexibility**: Complex filter combinations for precise case selection +5. **Traceability**: Filter criteria stored in version-controlled JSON files + +#### Validation and Error Handling + +The CLI validates dynamic filter files before sending to TestRail: + +```bash +# Invalid JSON format +Error: Invalid JSON: Expecting property name enclosed in double quotes + +# Missing required keys +Error: Invalid filter for cases:priority_id: Field filter must have 'value', 'values', or 'filters' + +# Invalid field name +Error: Field name must start with 'cases:' prefix: priority_id + +# Empty filters +Error: 'filters' object cannot be empty - at least one filter field required +``` + +#### Tips and Best Practices + +1. **Start Simple**: Begin with basic filters and add complexity as needed +2. **Use Field Discovery**: Always check available fields with `fields list-dynamic` first +3. **Test Your Filters**: Create a test run to verify your filters select the expected cases +4. **Version Control**: Store filter JSON files in your repository alongside test code +5. **Document Filters**: Add comments in JSON describing the purpose of complex filters +6. **Combine with Other Options**: Use dynamic filters with milestones, assignees, refs, etc. +7. **Field Names**: Always prefix field names with `"cases:"` (e.g., `"cases:priority_id"`) + Generating test cases from OpenAPI specs ----------------- diff --git a/tests/test_cmd_plans.py b/tests/test_cmd_plans.py index f2cafdb..ebc0129 100644 --- a/tests/test_cmd_plans.py +++ b/tests/test_cmd_plans.py @@ -25,6 +25,10 @@ def _setup_project_client_mock(self, mock_project_client, project_id=1): mock_client_instance = MagicMock() mock_project_client.return_value = mock_client_instance mock_client_instance.project.project_id = project_id + # Mock validate_and_process_plan_entries to pass through entries unchanged + mock_client_instance.api_request_handler.plan_handler.validate_and_process_plan_entries.side_effect = ( + lambda entries, cli_mode=None: (entries, "") + ) return mock_client_instance @mock.patch("trcli.commands.cmd_plans.ProjectBasedClient") diff --git a/tests/test_data/cli_test_data.py b/tests/test_data/cli_test_data.py index ee9e367..3133f2b 100644 --- a/tests/test_data/cli_test_data.py +++ b/tests/test_data/cli_test_data.py @@ -71,6 +71,7 @@ " - parse_robot: Robot Framework XML Files\n" " - parse_openapi: OpenAPI YML Files\n" " - add_run: Create a new test run\n" + " - fields: Manage fields (list dynamic filter fields)\n" " - labels: Manage labels (projects, cases, and tests)\n" " - references: Manage references (cases and runs)\n" " - cases: Query test cases (get and list)\n" diff --git a/tests/test_dynamic_filters_utils.py b/tests/test_dynamic_filters_utils.py new file mode 100644 index 0000000..1f31d30 --- /dev/null +++ b/tests/test_dynamic_filters_utils.py @@ -0,0 +1,481 @@ +""" +Unit tests for dynamic_filters_utils module +""" + +import json +import os +import tempfile +import pytest + +from trcli.api.dynamic_filters_utils import ( + load_dynamic_filters_from_file, + validate_dynamic_filters_structure, + validate_field_filter, + format_filter_for_display, + FIELD_TYPES, + OPERATORS, +) + + +class TestLoadDynamicFiltersFromFile: + """Tests for load_dynamic_filters_from_file function""" + + def test_load_full_format_valid(self): + """Test loading filters in full format with top-level mode""" + with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f: + json.dump( + {"mode": "1", "filters": {"cases:priority_id": {"values": [1, 2]}}}, + f, + ) + f.flush() + filters, error = load_dynamic_filters_from_file(f.name) + os.unlink(f.name) + + assert error == "" + assert filters["mode"] == "1" + assert filters["filters"] == {"cases:priority_id": {"values": [1, 2]}} + assert filters["_mode_from_json"] is True + + def test_load_simplified_format_auto_wrapped(self): + """Test loading filters in simplified format (auto-wrapped with mode=1)""" + with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f: + json.dump({"cases:priority_id": {"values": [1, 2]}}, f) + f.flush() + filters, error = load_dynamic_filters_from_file(f.name) + os.unlink(f.name) + + assert error == "" + assert filters["mode"] == "1" + assert filters["filters"] == {"cases:priority_id": {"values": [1, 2]}} + assert filters["_mode_from_json"] is False + + def test_load_invalid_json(self): + """Test loading file with invalid JSON""" + with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f: + f.write("{invalid json}") + f.flush() + filters, error = load_dynamic_filters_from_file(f.name) + os.unlink(f.name) + + assert "Invalid JSON" in error + assert filters == {} + + def test_load_file_not_found(self): + """Test loading non-existent file""" + filters, error = load_dynamic_filters_from_file("/nonexistent/file.json") + assert "File not found" in error + assert filters == {} + + def test_load_empty_filters(self): + """Test loading file with empty filters object""" + with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f: + json.dump({"mode": "1", "filters": {}}, f) + f.flush() + filters, error = load_dynamic_filters_from_file(f.name) + os.unlink(f.name) + + assert "'filters' object cannot be empty" in error + assert filters == {} + + +class TestValidateDynamicFiltersStructure: + """Tests for validate_dynamic_filters_structure function""" + + def test_validate_valid_structure(self): + """Test validation of valid filter structure""" + filters = {"mode": "1", "filters": {"cases:priority_id": {"values": [1, 2]}}} + is_valid, error = validate_dynamic_filters_structure(filters) + assert is_valid is True + assert error == "" + + def test_validate_missing_mode_defaults_to_1(self): + """Test that missing mode defaults to '1'""" + filters = {"filters": {"cases:priority_id": {"values": [1, 2]}}} + is_valid, error = validate_dynamic_filters_structure(filters) + assert is_valid is True + assert filters["mode"] == "1" + + def test_validate_invalid_mode(self): + """Test validation rejects invalid mode""" + filters = {"mode": "3", "filters": {"cases:priority_id": {"values": [1, 2]}}} + is_valid, error = validate_dynamic_filters_structure(filters) + assert is_valid is False + assert "mode must be '1' (AND) or '2' (OR)" in error + + def test_validate_missing_filters_key(self): + """Test validation rejects missing filters key""" + filters = {"mode": "1"} + is_valid, error = validate_dynamic_filters_structure(filters) + assert is_valid is False + assert "Missing 'filters' key" in error + + def test_validate_filters_not_dict(self): + """Test validation rejects non-dict filters""" + filters = {"mode": "1", "filters": []} + is_valid, error = validate_dynamic_filters_structure(filters) + assert is_valid is False + assert "'filters' must be a JSON object" in error + + def test_validate_empty_filters(self): + """Test validation rejects empty filters object""" + filters = {"mode": "1", "filters": {}} + is_valid, error = validate_dynamic_filters_structure(filters) + assert is_valid is False + assert "'filters' object cannot be empty" in error + + def test_validate_field_without_cases_prefix(self): + """Test validation rejects field names without 'cases:' prefix""" + filters = {"mode": "1", "filters": {"priority_id": {"values": [1, 2]}}} + is_valid, error = validate_dynamic_filters_structure(filters) + assert is_valid is False + assert "must start with 'cases:' prefix" in error + + +class TestValidateFieldFilter: + """Tests for validate_field_filter function""" + + def test_validate_checkbox_format(self): + """Test validation of checkbox field filter""" + is_valid, error = validate_field_filter("cases:is_automated", {"value": True}) + assert is_valid is True + assert error == "" + + def test_validate_single_value_invalid_type(self): + """Test validation rejects invalid value types (arrays, objects)""" + is_valid, error = validate_field_filter("cases:priority_id", {"value": [1, 2]}) + assert is_valid is False + assert "Value must be boolean" in error or "number" in error or "string" in error + + def test_validate_dropdown_single_value(self): + """Test validation of dropdown with single value (not array)""" + is_valid, error = validate_field_filter("cases:priority_id", {"value": 1}) + assert is_valid is True + assert error == "" + + # Also test with string value + is_valid, error = validate_field_filter("cases:custom_field", {"value": "option1"}) + assert is_valid is True + assert error == "" + + def test_validate_dropdown_format(self): + """Test validation of dropdown field filter""" + is_valid, error = validate_field_filter("cases:priority_id", {"values": [1, 2, 3]}) + assert is_valid is True + assert error == "" + + def test_validate_dropdown_with_all(self): + """Test validation of dropdown with 'all' value""" + is_valid, error = validate_field_filter("cases:priority_id", {"values": ["all"]}) + assert is_valid is True + assert error == "" + + def test_validate_multi_select_with_mode(self): + """Test validation of multi-select with mode""" + is_valid, error = validate_field_filter("cases:custom_field", {"mode": "1", "values": [1, 2, 3]}) + assert is_valid is True + assert error == "" + + def test_validate_multi_select_invalid_mode(self): + """Test validation rejects invalid mode in multi-select""" + is_valid, error = validate_field_filter("cases:custom_field", {"mode": "3", "values": [1, 2, 3]}) + assert is_valid is False + assert "Mode must be '1' (AND) or '2' (OR)" in error + + def test_validate_operator_based_format(self): + """Test validation of operator-based field filter""" + field_filter = {"mode": "1", "filters": [{"op": 5, "value": "test"}]} + is_valid, error = validate_field_filter("cases:title", field_filter) + assert is_valid is True + assert error == "" + + def test_validate_operator_based_missing_mode(self): + """Test validation rejects operator-based filter without mode""" + field_filter = {"filters": [{"op": 5, "value": "test"}]} + is_valid, error = validate_field_filter("cases:title", field_filter) + assert is_valid is False + assert "Filters with operators require 'mode'" in error + + def test_validate_operator_based_invalid_mode(self): + """Test validation rejects invalid mode in operator-based filter""" + field_filter = {"mode": "3", "filters": [{"op": 5, "value": "test"}]} + is_valid, error = validate_field_filter("cases:title", field_filter) + assert is_valid is False + assert "Filter mode must be '1' (match all) or '2' (match any)" in error + + def test_validate_operator_based_empty_filters(self): + """Test validation rejects empty filters array""" + field_filter = {"mode": "1", "filters": []} + is_valid, error = validate_field_filter("cases:title", field_filter) + assert is_valid is False + assert "'filters' array cannot be empty" in error + + def test_validate_operator_based_invalid_operator(self): + """Test validation rejects invalid operator ID""" + field_filter = {"mode": "1", "filters": [{"op": 99, "value": "test"}]} + is_valid, error = validate_field_filter("cases:title", field_filter) + assert is_valid is False + assert "Invalid operator" in error + + def test_validate_operator_based_missing_op(self): + """Test validation rejects operator filter without 'op' key""" + field_filter = {"mode": "1", "filters": [{"value": "test"}]} + is_valid, error = validate_field_filter("cases:title", field_filter) + assert is_valid is False + assert "must have 'op' and 'value' keys" in error + + def test_validate_operator_based_missing_value(self): + """Test validation rejects operator filter without 'value' key""" + field_filter = {"mode": "1", "filters": [{"op": 5}]} + is_valid, error = validate_field_filter("cases:title", field_filter) + assert is_valid is False + assert "must have 'op' and 'value' keys" in error + + def test_validate_invalid_field_filter_format(self): + """Test validation rejects field filter without required keys""" + is_valid, error = validate_field_filter("cases:priority_id", {}) + assert is_valid is False + assert "must have 'value', 'values', or 'filters'" in error + + +class TestFormatFilterForDisplay: + """Tests for format_filter_for_display function""" + + def test_format_no_filters(self): + """Test formatting empty filters""" + result = format_filter_for_display({}) + assert result == "No filters" + + result = format_filter_for_display(None) + assert result == "No filters" + + def test_format_and_mode(self): + """Test formatting filters with AND mode""" + filters = {"mode": "1", "filters": {"cases:priority_id": {"values": [1, 2]}}} + result = format_filter_for_display(filters) + assert "Match ALL conditions (AND)" in result + assert "priority_id" in result + + def test_format_or_mode(self): + """Test formatting filters with OR mode""" + filters = {"mode": "2", "filters": {"cases:priority_id": {"values": [1, 2]}}} + result = format_filter_for_display(filters) + assert "Match ANY condition (OR)" in result + + def test_format_checkbox_filter(self): + """Test formatting checkbox filter""" + filters = {"mode": "1", "filters": {"cases:is_automated": {"value": True}}} + result = format_filter_for_display(filters) + assert "is_automated: true" in result + + def test_format_dropdown_filter(self): + """Test formatting dropdown filter""" + filters = {"mode": "1", "filters": {"cases:priority_id": {"values": [1, 2, 3]}}} + result = format_filter_for_display(filters) + assert "priority_id: 1, 2, 3" in result + + def test_format_dropdown_filter_with_all(self): + """Test formatting dropdown filter with 'all' value""" + filters = {"mode": "1", "filters": {"cases:priority_id": {"values": ["all", 1, 2]}}} + result = format_filter_for_display(filters) + assert "priority_id: all" in result + + def test_format_operator_based_filter(self): + """Test formatting operator-based filter""" + filters = { + "mode": "1", + "filters": {"cases:title": {"mode": "2", "filters": [{"op": 5, "value": "test"}]}}, + } + result = format_filter_for_display(filters) + assert "title (Match ANY):" in result + assert "Contains: test" in result + + def test_format_with_field_labels(self): + """Test formatting with custom field labels""" + filters = {"mode": "1", "filters": {"cases:priority_id": {"values": [1, 2]}}} + field_labels = {"cases:priority_id": "Priority"} + result = format_filter_for_display(filters, field_labels) + assert "Priority:" in result + + +class TestConstants: + """Tests for module constants""" + + def test_field_types_constant(self): + """Test FIELD_TYPES constant has expected values""" + assert FIELD_TYPES[1] == "String" + assert FIELD_TYPES[6] == "Dropdown" + assert FIELD_TYPES[12] == "Multi-select" + + def test_operators_constant(self): + """Test OPERATORS constant has expected values""" + assert OPERATORS[1] == "Is" + assert OPERATORS[5] == "Contains" + assert OPERATORS[8] == "Is More" + + +class TestModePrecedence: + """Tests for mode precedence behavior between JSON file and CLI flag""" + + def test_mode_explicitly_provided_in_full_format(self): + """Test that _mode_from_json flag is True when mode is in JSON (full format)""" + with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f: + json.dump( + {"mode": "2", "filters": {"cases:priority_id": {"values": [1, 2]}}}, + f, + ) + f.flush() + filters, error = load_dynamic_filters_from_file(f.name) + os.unlink(f.name) + + assert error == "" + assert filters["mode"] == "2" + assert filters["_mode_from_json"] is True + + def test_mode_not_provided_in_simplified_format(self): + """Test that _mode_from_json flag is False when using simplified format (no mode in JSON)""" + with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f: + json.dump({"cases:priority_id": {"values": [1, 2]}}, f) + f.flush() + filters, error = load_dynamic_filters_from_file(f.name) + os.unlink(f.name) + + assert error == "" + assert filters["mode"] == "1" # Default mode added by validation + assert filters["_mode_from_json"] is False # Mode was NOT in original JSON + + def test_mode_not_provided_in_full_format_without_mode_key(self): + """Test that _mode_from_json flag is False when full format used but no mode key""" + with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f: + json.dump( + {"filters": {"cases:priority_id": {"values": [1, 2]}}}, + f, + ) + f.flush() + filters, error = load_dynamic_filters_from_file(f.name) + os.unlink(f.name) + + assert error == "" + assert filters["mode"] == "1" # Default mode added by validation + assert filters["_mode_from_json"] is False # Mode was NOT in original JSON + + def test_mode_precedence_json_mode_1_provided(self): + """Test that mode '1' from JSON is preserved""" + with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f: + json.dump( + {"mode": "1", "filters": {"cases:priority_id": {"values": [1, 2]}}}, + f, + ) + f.flush() + filters, error = load_dynamic_filters_from_file(f.name) + os.unlink(f.name) + + assert error == "" + assert filters["mode"] == "1" + assert filters["_mode_from_json"] is True + + def test_mode_precedence_json_mode_2_provided(self): + """Test that mode '2' from JSON is preserved""" + with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f: + json.dump( + {"mode": "2", "filters": {"cases:priority_id": {"values": [1, 2]}}}, + f, + ) + f.flush() + filters, error = load_dynamic_filters_from_file(f.name) + os.unlink(f.name) + + assert error == "" + assert filters["mode"] == "2" + assert filters["_mode_from_json"] is True + + def test_mode_flag_allows_cli_override_for_simplified_format(self): + """Test that simplified format (no mode) allows CLI override via _mode_from_json=False""" + with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f: + json.dump({"cases:priority_id": {"values": [1, 2]}}, f) + f.flush() + filters, error = load_dynamic_filters_from_file(f.name) + os.unlink(f.name) + + assert error == "" + # Initially has default mode "1" + assert filters["mode"] == "1" + # But flag indicates it can be overridden + assert filters["_mode_from_json"] is False + + # Simulate CLI override (this would happen in project_based_client.py) + mode_from_json = filters.pop("_mode_from_json", True) + cli_mode = "2" + if cli_mode and not mode_from_json: + filters["mode"] = cli_mode + + assert filters["mode"] == "2" # Successfully overridden by CLI + + def test_mode_flag_prevents_cli_override_when_json_has_mode(self): + """Test that JSON mode takes precedence over CLI via _mode_from_json=True""" + with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f: + json.dump( + {"mode": "1", "filters": {"cases:priority_id": {"values": [1, 2]}}}, + f, + ) + f.flush() + filters, error = load_dynamic_filters_from_file(f.name) + os.unlink(f.name) + + assert error == "" + assert filters["mode"] == "1" + assert filters["_mode_from_json"] is True + + # Simulate CLI override attempt (this would happen in project_based_client.py) + mode_from_json = filters.pop("_mode_from_json", True) + cli_mode = "2" + if cli_mode and not mode_from_json: + filters["mode"] = cli_mode + + assert filters["mode"] == "1" # JSON mode preserved, CLI ignored + + def test_complex_filter_with_explicit_mode(self): + """Test complex filter with explicit mode in JSON""" + with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f: + json.dump( + { + "mode": "2", + "filters": { + "cases:priority_id": {"values": [1, 2]}, + "cases:is_automated": {"value": True}, + "cases:title": {"mode": "1", "filters": [{"op": 5, "value": "test"}]}, + }, + }, + f, + ) + f.flush() + filters, error = load_dynamic_filters_from_file(f.name) + os.unlink(f.name) + + assert error == "" + assert filters["mode"] == "2" + assert filters["_mode_from_json"] is True + # Field-level mode is independent + assert filters["filters"]["cases:title"]["mode"] == "1" + + def test_complex_filter_without_top_level_mode(self): + """Test complex filter without top-level mode (should allow CLI override)""" + with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f: + json.dump( + { + "filters": { + "cases:priority_id": {"values": [1, 2]}, + "cases:is_automated": {"value": True}, + "cases:title": {"mode": "2", "filters": [{"op": 5, "value": "test"}]}, + } + }, + f, + ) + f.flush() + filters, error = load_dynamic_filters_from_file(f.name) + os.unlink(f.name) + + assert error == "" + assert filters["mode"] == "1" # Default + assert filters["_mode_from_json"] is False # Can be overridden + # Field-level mode is independent + assert filters["filters"]["cases:title"]["mode"] == "2" diff --git a/tests/test_plans_dynamic_filters.py b/tests/test_plans_dynamic_filters.py new file mode 100644 index 0000000..9b597f4 --- /dev/null +++ b/tests/test_plans_dynamic_filters.py @@ -0,0 +1,445 @@ +""" +Unit tests for dynamic filters in plans add command +""" + +import pytest +from unittest.mock import Mock, MagicMock + +from trcli.api.plan_handler import PlanHandler +from trcli.api.api_client import APIClient +from trcli.cli import Environment + + +class TestPlansDynamicFilters: + """Tests for dynamic filters in plans add command""" + + @pytest.fixture + def plan_handler(self): + """Create a PlanHandler instance with mocked dependencies""" + mock_client = Mock(spec=APIClient) + mock_env = Mock(spec=Environment) + mock_env.log = MagicMock() + mock_env.elog = MagicMock() + return PlanHandler(client=mock_client, environment=mock_env) + + def test_validate_basic_plan_with_filters(self, plan_handler): + """Test validation of basic plan with dynamic filters""" + entries = [ + { + "suite_id": 1, + "name": "Smoke Tests", + "runs": [{"dynamic_filters": {"mode": "1", "filters": {"cases:priority_id": {"values": [1, 2]}}}}], + } + ] + + validated, error = plan_handler.validate_and_process_plan_entries(entries) + + assert error == "" + assert len(validated) == 1 + assert "dynamic_filters" in validated[0]["runs"][0] + # include_all should be set to false when using dynamic_filters + assert validated[0]["runs"][0]["include_all"] is False + + def test_validate_simplified_format_auto_wrapping(self, plan_handler): + """Test that simplified filter format is auto-wrapped""" + entries = [ + {"suite_id": 1, "name": "Entry 1", "runs": [{"dynamic_filters": {"cases:priority_id": {"values": [1, 2]}}}]} + ] + + validated, error = plan_handler.validate_and_process_plan_entries(entries) + + assert error == "" + # Check that filters were wrapped + assert "filters" in validated[0]["runs"][0]["dynamic_filters"] + assert validated[0]["runs"][0]["dynamic_filters"]["mode"] == "1" + assert "cases:priority_id" in validated[0]["runs"][0]["dynamic_filters"]["filters"] + + def test_reject_filters_with_case_ids(self, plan_handler): + """Test rejection of entries with both filters and case_ids""" + entries = [ + { + "suite_id": 1, + "name": "Invalid Entry", + "runs": [{"case_ids": [1, 2, 3], "dynamic_filters": {"cases:priority_id": {"values": [1]}}}], + } + ] + + validated, error = plan_handler.validate_and_process_plan_entries(entries) + + assert error != "" + assert "Entry 'Invalid Entry', Run 1" in error + assert "dynamic_filters and case_ids cannot be used together" in error + + def test_reject_filters_with_include_all_true(self, plan_handler): + """Test rejection of entries with filters and include_all=true""" + entries = [ + { + "suite_id": 1, + "name": "Test Entry", + "runs": [{"include_all": True, "dynamic_filters": {"cases:priority_id": {"values": [1]}}}], + } + ] + + validated, error = plan_handler.validate_and_process_plan_entries(entries) + + assert error != "" + assert "Entry 'Test Entry', Run 1" in error + assert "dynamic_filters and include_all=true cannot be used together" in error + + def test_allow_filters_with_include_all_false(self, plan_handler): + """Test that filters with include_all=false are allowed""" + entries = [ + { + "suite_id": 1, + "name": "Valid Entry", + "runs": [{"include_all": False, "dynamic_filters": {"cases:priority_id": {"values": [1]}}}], + } + ] + + validated, error = plan_handler.validate_and_process_plan_entries(entries) + + assert error == "" + # include_all should be set to false when using dynamic_filters + assert validated[0]["runs"][0]["include_all"] is False + + def test_multiple_entries_with_different_filters(self, plan_handler): + """Test validation of multiple entries with different filters""" + entries = [ + {"suite_id": 1, "name": "Smoke Tests", "runs": [{"dynamic_filters": {"cases:label_id": {"values": [4]}}}]}, + { + "suite_id": 1, + "name": "Regression Tests", + "runs": [{"dynamic_filters": {"cases:priority_id": {"values": [1, 2]}}}], + }, + ] + + validated, error = plan_handler.validate_and_process_plan_entries(entries) + + assert error == "" + assert len(validated) == 2 + assert "label_id" in str(validated[0]["runs"][0]["dynamic_filters"]) + assert "priority_id" in str(validated[1]["runs"][0]["dynamic_filters"]) + + def test_multiple_runs_per_entry_with_different_filters(self, plan_handler): + """Test multiple runs per entry with different filter configurations""" + entries = [ + { + "suite_id": 1, + "name": "Multi-Run Entry", + "runs": [ + {"dynamic_filters": {"cases:priority_id": {"values": [1]}}}, + {"case_ids": [10, 20, 30]}, + {"include_all": True}, + ], + } + ] + + validated, error = plan_handler.validate_and_process_plan_entries(entries) + + assert error == "" + assert len(validated[0]["runs"]) == 3 + # First run has dynamic filters + assert "dynamic_filters" in validated[0]["runs"][0] + # Second run has case_ids + assert validated[0]["runs"][1]["case_ids"] == [10, 20, 30] + # Third run has include_all + assert validated[0]["runs"][2]["include_all"] is True + + def test_reject_empty_filters_object(self, plan_handler): + """Test rejection of empty filters object""" + entries = [ + {"suite_id": 1, "name": "Invalid Entry", "runs": [{"dynamic_filters": {"mode": "1", "filters": {}}}]} + ] + + validated, error = plan_handler.validate_and_process_plan_entries(entries) + + assert error != "" + assert "Entry 'Invalid Entry', Run 1" in error + assert "'filters' object cannot be empty" in error + + def test_reject_invalid_filter_structure(self, plan_handler): + """Test rejection of invalid filter structure""" + entries = [ + { + "suite_id": 1, + "name": "Bad Entry", + "runs": [ + { + "dynamic_filters": { + "mode": "1", + "filters": {"priority_id": {"values": [1, 2]}}, # Missing "cases:" prefix + } + } + ], + } + ] + + validated, error = plan_handler.validate_and_process_plan_entries(entries) + + assert error != "" + assert "Entry 'Bad Entry', Run 1" in error + assert "must start with 'cases:' prefix" in error + + def test_error_context_includes_entry_and_run_info(self, plan_handler): + """Test that error messages include entry name and run index""" + entries = [ + { + "suite_id": 1, + "name": "Test Suite A", + "runs": [ + {"include_all": True}, # Run 1 - valid + {"case_ids": [1, 2], "dynamic_filters": {"cases:priority_id": {"values": [1]}}}, # Run 2 - invalid + ], + } + ] + + validated, error = plan_handler.validate_and_process_plan_entries(entries) + + assert error != "" + assert "Entry 'Test Suite A', Run 2" in error + + def test_entry_without_name_uses_index(self, plan_handler): + """Test that entries without names use index in error messages""" + entries = [ + { + "suite_id": 1, + # No "name" field + "runs": [{"case_ids": [1, 2], "dynamic_filters": {"cases:priority_id": {"values": [1]}}}], + } + ] + + validated, error = plan_handler.validate_and_process_plan_entries(entries) + + assert error != "" + assert "Entry 'Entry 1', Run 1" in error # Note: quotes around generated name + + def test_config_ids_with_filters_allowed(self, plan_handler): + """Test that config_ids can be used with dynamic filters""" + entries = [ + { + "suite_id": 1, + "name": "Multi-Config Entry", + "runs": [{"config_ids": [1, 2], "dynamic_filters": {"cases:priority_id": {"values": [1, 2]}}}], + } + ] + + validated, error = plan_handler.validate_and_process_plan_entries(entries) + + assert error == "" + assert validated[0]["runs"][0]["config_ids"] == [1, 2] + assert "dynamic_filters" in validated[0]["runs"][0] + + def test_empty_entries_list_returns_empty(self, plan_handler): + """Test that empty entries list is handled gracefully""" + entries = [] + + validated, error = plan_handler.validate_and_process_plan_entries(entries) + + assert error == "" + assert validated == [] + + def test_none_entries_returns_none(self, plan_handler): + """Test that None entries is handled gracefully""" + validated, error = plan_handler.validate_and_process_plan_entries(None) + + assert error == "" + assert validated is None + + def test_entry_with_no_runs_passes_through(self, plan_handler): + """Test that entries without runs are passed through unchanged""" + entries = [ + { + "suite_id": 1, + "name": "Empty Entry", + # No "runs" field + } + ] + + validated, error = plan_handler.validate_and_process_plan_entries(entries) + + assert error == "" + assert len(validated) == 1 + assert validated[0]["name"] == "Empty Entry" + + def test_complex_nested_mode_filters(self, plan_handler): + """Test complex filters with nested modes""" + entries = [ + { + "suite_id": 1, + "name": "Complex Entry", + "runs": [ + { + "dynamic_filters": { + "mode": "1", + "filters": { + "cases:priority_id": {"values": [1, 2]}, + "cases:title": { + "mode": "2", + "filters": [{"op": 5, "value": "login"}, {"op": 5, "value": "auth"}], + }, + }, + } + } + ], + } + ] + + validated, error = plan_handler.validate_and_process_plan_entries(entries) + + assert error == "" + filters = validated[0]["runs"][0]["dynamic_filters"] + assert filters["mode"] == "1" + assert "cases:title" in filters["filters"] + assert filters["filters"]["cases:title"]["mode"] == "2" + + def test_mode_from_json_flag_removed(self, plan_handler): + """Test that _mode_from_json internal flag is removed""" + entries = [ + { + "suite_id": 1, + "name": "Test Entry", + "runs": [ + { + "dynamic_filters": { + "mode": "1", + "filters": {"cases:priority_id": {"values": [1]}}, + "_mode_from_json": True, # Internal flag should be removed + } + } + ], + } + ] + + validated, error = plan_handler.validate_and_process_plan_entries(entries) + + assert error == "" + assert "_mode_from_json" not in validated[0]["runs"][0]["dynamic_filters"] + + def test_cli_mode_override_simplified_format(self, plan_handler): + """Test that CLI mode overrides default mode when using simplified format""" + # Without CLI mode - should default to "1" + entries1 = [ + { + "suite_id": 1, + "name": "Test Entry", + "runs": [ + { + "dynamic_filters": { + # Simplified format - no explicit mode + "cases:priority_id": {"values": [1, 2]} + } + } + ], + } + ] + validated, error = plan_handler.validate_and_process_plan_entries(entries1) + assert error == "" + assert validated[0]["runs"][0]["dynamic_filters"]["mode"] == "1" + + # With CLI mode "2" - should use CLI mode (use fresh entries) + entries2 = [ + { + "suite_id": 1, + "name": "Test Entry", + "runs": [ + { + "dynamic_filters": { + # Simplified format - no explicit mode + "cases:priority_id": {"values": [1, 2]} + } + } + ], + } + ] + validated, error = plan_handler.validate_and_process_plan_entries(entries2, cli_mode="2") + assert error == "" + assert validated[0]["runs"][0]["dynamic_filters"]["mode"] == "2" + + def test_cli_mode_does_not_override_explicit_json_mode(self, plan_handler): + """Test that CLI mode does NOT override explicit mode in JSON""" + entries = [ + { + "suite_id": 1, + "name": "Test Entry", + "runs": [ + { + "dynamic_filters": { + "mode": "2", # Explicit mode in JSON + "filters": {"cases:priority_id": {"values": [1, 2]}}, + } + } + ], + } + ] + + # CLI mode "1" should NOT override JSON mode "2" + validated, error = plan_handler.validate_and_process_plan_entries(entries, cli_mode="1") + assert error == "" + assert validated[0]["runs"][0]["dynamic_filters"]["mode"] == "2" # JSON mode preserved + + def test_cli_mode_applies_to_all_runs(self, plan_handler): + """Test that CLI mode applies to all runs in the plan""" + entries = [ + {"suite_id": 1, "name": "Entry 1", "runs": [{"dynamic_filters": {"cases:priority_id": {"values": [1]}}}]}, + {"suite_id": 2, "name": "Entry 2", "runs": [{"dynamic_filters": {"cases:label_id": {"values": [5]}}}]}, + ] + + # CLI mode "2" should apply to all runs + validated, error = plan_handler.validate_and_process_plan_entries(entries, cli_mode="2") + assert error == "" + assert validated[0]["runs"][0]["dynamic_filters"]["mode"] == "2" + assert validated[1]["runs"][0]["dynamic_filters"]["mode"] == "2" + + def test_cli_mode_respects_mixed_explicit_and_simplified(self, plan_handler): + """Test CLI mode with mix of explicit and simplified formats""" + entries = [ + { + "suite_id": 1, + "name": "Entry 1", + "runs": [ + { + "dynamic_filters": { + "mode": "1", # Explicit mode + "filters": {"cases:priority_id": {"values": [1]}}, + } + } + ], + }, + { + "suite_id": 2, + "name": "Entry 2", + "runs": [ + { + "dynamic_filters": { + # Simplified format - no explicit mode + "cases:label_id": {"values": [5]} + } + } + ], + }, + ] + + # CLI mode "2" should only apply to Entry 2 (simplified format) + validated, error = plan_handler.validate_and_process_plan_entries(entries, cli_mode="2") + assert error == "" + assert validated[0]["runs"][0]["dynamic_filters"]["mode"] == "1" # Explicit mode preserved + assert validated[1]["runs"][0]["dynamic_filters"]["mode"] == "2" # CLI mode applied + + def test_cli_mode_with_multiple_runs_per_entry(self, plan_handler): + """Test CLI mode with multiple runs per entry""" + entries = [ + { + "suite_id": 1, + "name": "Test Entry", + "runs": [ + {"dynamic_filters": {"cases:priority_id": {"values": [1]}}}, + {"dynamic_filters": {"cases:priority_id": {"values": [2]}}}, + ], + } + ] + + # CLI mode should apply to all runs + validated, error = plan_handler.validate_and_process_plan_entries(entries, cli_mode="2") + assert error == "" + assert validated[0]["runs"][0]["dynamic_filters"]["mode"] == "2" + assert validated[0]["runs"][1]["dynamic_filters"]["mode"] == "2" diff --git a/trcli/api/api_request_handler.py b/trcli/api/api_request_handler.py index 8d9200e..b955296 100644 --- a/trcli/api/api_request_handler.py +++ b/trcli/api/api_request_handler.py @@ -27,6 +27,7 @@ from trcli.api.project_handler import ProjectHandler from trcli.api.template_handler import TemplateHandler from trcli.api.test_handler import TestHandler +from trcli.api.dynamic_filter_handler import DynamicFilterHandler from trcli.cli import Environment from trcli.constants import ( ProjectErrors, @@ -108,6 +109,7 @@ def __init__( self.project_handler = ProjectHandler(api_client) self.template_handler = TemplateHandler(api_client) self.test_handler = TestHandler(api_client) + self.dynamic_filter_handler = DynamicFilterHandler(api_client, environment) # BDD case cache for feature name matching (shared by CucumberParser and JunitParser) # Structure: {"{project_id}_{suite_id}": {normalized_name: [case_dict, case_dict, ...]}} @@ -256,6 +258,7 @@ def add_run( include_all: bool = False, refs: str = None, case_ids: List[int] = None, + dynamic_filters: Dict = None, ) -> Tuple[int, str]: return self.run_handler.add_run( project_id, @@ -269,6 +272,7 @@ def add_run( include_all, refs, case_ids, + dynamic_filters, ) def update_run( @@ -596,6 +600,12 @@ def __get_all_entities(self, entity: str, link=None, entities=[]) -> Tuple[List[ if isinstance(response.response_text, str): error_msg = FAULT_MAPPING["invalid_api_response"].format(error_details=response.response_text[:200]) return [], error_msg + # Check if response is an empty dict or missing expected key (e.g., service unavailable) + if not isinstance(response.response_text, dict) or entity not in response.response_text: + error_msg = FAULT_MAPPING["invalid_api_response"].format( + error_details=f"Expected '{entity}' key in response but got: {str(response.response_text)[:200]}" + ) + return [], error_msg # Endpoints with pagination entities = entities + response.response_text[entity] if response.response_text["_links"]["next"] is not None: @@ -635,6 +645,13 @@ def __get_all_entities_parallel(self, entity: str, link: str) -> Tuple[List[Dict error_msg = FAULT_MAPPING["invalid_api_response"].format(error_details=response.response_text[:200]) return [], error_msg + # Check if response is an empty dict or missing expected key (e.g., service unavailable) + if not isinstance(response.response_text, dict) or entity not in response.response_text: + error_msg = FAULT_MAPPING["invalid_api_response"].format( + error_details=f"Expected '{entity}' key in response but got: {str(response.response_text)[:200]}" + ) + return [], error_msg + # Collect first page results all_entities = response.response_text[entity] first_page_count = len(all_entities) diff --git a/trcli/api/dynamic_filter_handler.py b/trcli/api/dynamic_filter_handler.py new file mode 100644 index 0000000..e4e18f9 --- /dev/null +++ b/trcli/api/dynamic_filter_handler.py @@ -0,0 +1,46 @@ +""" +DynamicFilterHandler - Handles dynamic filter field discovery operations for TestRail + +This handler manages the discovery of available fields for dynamic filtering, +which enables auto-updating test runs that continuously synchronize with test case repository. +""" + +from beartype.typing import Tuple, List, Dict, Any + +from trcli.api.api_client import APIClient +from trcli.cli import Environment + + +class DynamicFilterHandler: + """Handles dynamic filter field discovery operations for TestRail""" + + def __init__( + self, + client: APIClient, + environment: Environment, + ): + """ + Initialize the DynamicFilterHandler + + :param client: APIClient instance for making API calls + :param environment: Environment configuration + """ + self.client = client + self.environment = environment + + def get_dynamic_filter_fields(self, project_id: int) -> Tuple[List[Dict[str, Any]], str]: + """ + Retrieve available fields for dynamic filtering in a project. + + This endpoint returns the canonical list of fields available for dynamic filtering, + matching the TestRail UI exactly. Includes both system fields and active custom fields. + + :param project_id: TestRail project ID + :returns: Tuple with (fields_list, error_message) + fields_list contains dicts with keys: type_id, system_name, label, + and optionally sub_filters (operators) and options (for dropdowns) + """ + response = self.client.send_get(f"get_dynamic_filter_fields/{project_id}") + if response.error_message: + return [], response.error_message + return response.response_text, "" diff --git a/trcli/api/dynamic_filters_utils.py b/trcli/api/dynamic_filters_utils.py new file mode 100644 index 0000000..9ee2450 --- /dev/null +++ b/trcli/api/dynamic_filters_utils.py @@ -0,0 +1,253 @@ +""" +Dynamic Filters Utilities - Helper functions for loading, validating, and formatting dynamic filters + +Dynamic filters enable auto-updating test runs that continuously synchronize with test case repository. +This module provides utilities for working with dynamic filter JSON files. +""" + +from beartype.typing import Dict, Tuple, Any +import json +import builtins + +# Field type constants (from TestRail API) +FIELD_TYPES = { + 1: "String", + 2: "Integer", + 4: "URL", + 5: "Checkbox", + 6: "Dropdown", + 7: "User", + 8: "Date", + 9: "Milestone", + 12: "Multi-select", +} + +# Operator constants (from TestRail API) +OPERATORS = { + 1: "Is", + 2: "Is Not", + 3: "Is Before", + 4: "Is After", + 5: "Contains", + 6: "Does not contain", + 7: "Is Less", + 8: "Is More", +} + + +def load_dynamic_filters_from_file(file_path: str) -> Tuple[Dict[str, Any], str]: + """ + Load and validate dynamic filters from JSON file. + + Supports two formats: + 1. Full format with top-level mode: + {"mode": "1", "filters": {"cases:priority_id": {...}}} + + 2. Simplified format (auto-wrapped): + {"cases:priority_id": {...}} + (automatically wrapped with mode "1") + + :param file_path: Path to JSON file containing filter criteria + :returns: Tuple with (filters_dict, error_message) + filters_dict has structure: {"mode": "1/2", "filters": {...}} + """ + try: + with open(file_path, "r") as f: + filters = json.load(f) + + # Track if mode was explicitly provided in JSON file + mode_explicitly_provided = "mode" in filters + + # Normalize: if no top-level "filters" key, wrap it + if "filters" not in filters: + # Simplified format - wrap it (don't set mode yet - let validation handle it) + filters = {"filters": filters} + + # Validate structure (this will add default mode if not present) + is_valid, error = validate_dynamic_filters_structure(filters) + if not is_valid: + return {}, error + + # Mark whether mode was explicitly provided (for CLI override logic) + filters["_mode_from_json"] = mode_explicitly_provided + + return filters, "" + except json.JSONDecodeError as e: + return {}, f"Invalid JSON: {str(e)}" + except FileNotFoundError: + return {}, f"File not found: {file_path}" + except Exception as e: + return {}, f"Error loading filter file: {str(e)}" + + +def validate_dynamic_filters_structure(filters: dict) -> Tuple[bool, str]: + """ + Validate filter structure matches TestRail API requirements. + + Note: Client sends format WITHOUT type_id. + TestRail validates and enriches with type_id on server side. + + :param filters: The filters dictionary to validate + :returns: Tuple with (is_valid, error_message) + """ + if not isinstance(filters, dict): + return False, "Filters must be a JSON object" + + # Check top-level mode if present + if "mode" in filters: + if filters["mode"] not in ["1", "2"]: + return False, "Top-level mode must be '1' (AND) or '2' (OR)" + else: + # Default to AND mode if not specified + filters["mode"] = "1" + + # Check filters dict + if "filters" not in filters: + return False, "Missing 'filters' key in top-level object" + + filters_dict = filters["filters"] + if not isinstance(filters_dict, dict): + return False, "'filters' must be a JSON object" + + if len(filters_dict) == 0: + return False, "'filters' object cannot be empty - at least one filter field required" + + # Validate each field filter + for field_name, field_filter in filters_dict.items(): + if not field_name.startswith("cases:"): + return False, f"Field name must start with 'cases:' prefix: {field_name}" + + is_valid, error = validate_field_filter(field_name, field_filter) + if not is_valid: + return False, f"Invalid filter for {field_name}: {error}" + + return True, "" + + +def validate_field_filter(field_name: str, field_filter: dict) -> Tuple[bool, str]: + """ + Validate individual field filter structure. + + Supports multiple formats based on field type: + 1. Single value: {"value": true/false/1/"string"} + - Checkbox: {"value": true/false} + - Dropdown single select: {"value": 1} + 2. Multiple values: {"values": ["all", 1, 2]} + - Dropdown/User/Milestone: {"values": ["all", 1, 2]} + 3. String/Integer/Date with operators: {"mode": "1/2", "filters": [{"op": 1, "value": "..."}]} + 4. Multi-select: {"mode": "1/2", "values": [1, 2, 3]} + + :param field_name: Name of the field (e.g., "cases:priority_id") + :param field_filter: The filter configuration for this field + :returns: Tuple with (is_valid, error_message) + """ + if not isinstance(field_filter, dict): + return False, "Field filter must be a JSON object" + + # Check for valid structures based on which keys are present + has_value = "value" in field_filter + has_values = "values" in field_filter + has_filters = "filters" in field_filter + has_mode = "mode" in field_filter + + if has_value: + # Format 1: Single value selection + # - Checkbox: {"value": true/false} - boolean + # - Dropdown/Single select: {"value": 1} - number or string + value = field_filter["value"] + if not isinstance(value, (bool, int, str)): + return False, "Value must be boolean (for checkboxes), number, or string" + + elif has_values and not has_filters: + # Format 2: Dropdown/Multi-select {"values": [...]} + # or Format 4: Multi-select with mode {"mode": "1/2", "values": [...]} + if not isinstance(field_filter["values"], builtins.list): + return False, "'values' must be an array" + + if has_mode and field_filter["mode"] not in ["1", "2"]: + return False, "Mode must be '1' (AND) or '2' (OR)" + + elif has_filters: + # Format 3: Operator-based {"mode": "1/2", "filters": [...]} + if not has_mode: + return False, "Filters with operators require 'mode'" + + if field_filter["mode"] not in ["1", "2"]: + return False, "Filter mode must be '1' (match all) or '2' (match any)" + + if not isinstance(field_filter["filters"], builtins.list): + return False, "'filters' must be an array" + + if len(field_filter["filters"]) == 0: + return False, "'filters' array cannot be empty" + + # Validate each operator filter + for op_filter in field_filter["filters"]: + if not isinstance(op_filter, dict): + return False, "Operator filter must be a JSON object" + + if "op" not in op_filter or "value" not in op_filter: + return False, "Operator filter must have 'op' and 'value' keys" + + if not isinstance(op_filter["op"], int): + return False, "Operator 'op' must be an integer (1-8)" + + # Validate operator value range + if op_filter["op"] not in OPERATORS: + return False, f"Invalid operator: {op_filter['op']}. Must be 1-8" + else: + return False, "Field filter must have 'value', 'values', or 'filters'" + + return True, "" + + +def format_filter_for_display(filters: dict, field_labels: dict = None) -> str: + """ + Format filters for human-readable CLI display. + + :param filters: The dynamic_filters object with structure {"mode": "1/2", "filters": {...}} + :param field_labels: Optional dict mapping system_name to label for better readability + :returns: Formatted multi-line string for CLI output + """ + if not filters or "filters" not in filters: + return "No filters" + + mode = filters.get("mode", "1") + mode_text = "Match ALL conditions (AND)" if mode == "1" else "Match ANY condition (OR)" + + lines = [f"Mode: {mode_text}", "Filters:"] + + for field_name, field_filter in filters["filters"].items(): + # Remove "cases:" prefix for display + display_name = field_name.replace("cases:", "") + + # Use label if provided + if field_labels and field_name in field_labels: + display_name = field_labels[field_name] + + # Format based on filter type + if "value" in field_filter: + # Checkbox + value_text = "true" if field_filter["value"] else "false" + lines.append(f" - {display_name}: {value_text}") + + elif "values" in field_filter: + # Dropdown/Multi-select + values = field_filter["values"] + if "all" in values: + values_text = "all" + else: + values_text = ", ".join(str(v) for v in values) + lines.append(f" - {display_name}: {values_text}") + + elif "filters" in field_filter: + # Operator-based + filter_mode = "Match ALL" if field_filter.get("mode") == "1" else "Match ANY" + lines.append(f" - {display_name} ({filter_mode}):") + + for op_filter in field_filter["filters"]: + op_name = OPERATORS.get(op_filter["op"], f"Operator {op_filter['op']}") + value = op_filter["value"] + lines.append(f" {op_name}: {value}") + + return "\n".join(lines) diff --git a/trcli/api/plan_handler.py b/trcli/api/plan_handler.py index 84a1804..f57aeff 100644 --- a/trcli/api/plan_handler.py +++ b/trcli/api/plan_handler.py @@ -5,12 +5,14 @@ - Retrieving individual plans - Listing plans with pagination - Creating new plans +- Validating dynamic filters in plan entries """ -from beartype.typing import Tuple, Optional, Dict, Any +from beartype.typing import Tuple, Optional, Dict, Any, List from trcli.api.api_client import APIClient from trcli.cli import Environment +from trcli.api.dynamic_filters_utils import load_dynamic_filters_from_file, validate_dynamic_filters_structure class PlanHandler: @@ -88,6 +90,11 @@ def add_plan( """ Create a new test plan + TestRail's add_plan API doesn't support dynamic_filters in entries during plan creation. + If entries contain dynamic_filters, we: + 1. Create the plan without those entries + 2. Add each entry with dynamic_filters using add_plan_entry endpoint + :param project_id: TestRail project ID :param name: Name of the test plan (required) :param description: Description of the test plan @@ -97,14 +104,33 @@ def add_plan( :param due_on: Due date as UNIX timestamp :returns: Tuple with (created_plan_dict, error_message) """ + # Separate entries with dynamic_filters from regular entries + regular_entries = [] + dynamic_entries = [] + + if entries: + for entry in entries: + has_dynamic_filters = False + if "runs" in entry: + for run in entry["runs"]: + if "dynamic_filters" in run: + has_dynamic_filters = True + break + + if has_dynamic_filters: + dynamic_entries.append(entry) + else: + regular_entries.append(entry) + + # Create plan with regular entries only payload = {"name": name} if description is not None: payload["description"] = description if milestone_id is not None: payload["milestone_id"] = milestone_id - if entries is not None: - payload["entries"] = entries + if regular_entries: + payload["entries"] = regular_entries if start_on is not None: payload["start_on"] = start_on if due_on is not None: @@ -113,4 +139,186 @@ def add_plan( response = self.client.send_post(f"add_plan/{project_id}", payload) if response.error_message: return {}, response.error_message - return response.response_text, "" + + plan_data = response.response_text + plan_id = plan_data.get("id") + + # Add entries with dynamic_filters using add_plan_entry + if dynamic_entries and plan_id: + for entry in dynamic_entries: + # Check if entry or any run has config_ids (multi-config format) + has_config_ids = "config_ids" in entry and entry["config_ids"] + if not has_config_ids and "runs" in entry: + # Check if any run has config_ids + for run in entry["runs"]: + if "config_ids" in run and run["config_ids"]: + has_config_ids = True + break + + # For entries with runs array but no config_ids, flatten to single run format + if "runs" in entry and not has_config_ids and len(entry["runs"]) == 1: + # Single run without configs - use flat format like add_run does (line 127 in run_handler.py) + run = entry["runs"][0] + entry_payload = { + "suite_id": entry["suite_id"], + "name": entry.get("name", "Test Run"), + } + # Copy run fields to entry level + if "dynamic_filters" in run: + entry_payload["dynamic_filters"] = run["dynamic_filters"] + # add_plan_entry requires include_all field even with dynamic_filters + entry_payload["include_all"] = False + if "description" in entry: + entry_payload["description"] = entry["description"] + else: + # Multi-config format or multiple runs - use runs array + # According to TestRail API docs, dynamic_filters should be at ENTRY level, not in runs array + if "runs" in entry and len(entry["runs"]) == 1: + run = entry["runs"][0] + if "config_ids" in run: + # Move config_ids and dynamic_filters to entry level (per TestRail API spec) + entry_payload = { + "suite_id": entry["suite_id"], + "name": entry.get("name", "Test Run"), + "config_ids": run["config_ids"], + "include_all": False, + } + # Move dynamic_filters to entry level (not in runs array) + if "dynamic_filters" in run: + entry_payload["dynamic_filters"] = run["dynamic_filters"] + if "description" in entry: + entry_payload["description"] = entry["description"] + + # Create runs array + entry_payload["runs"] = [{"config_ids": run["config_ids"]}] + else: + entry_payload = entry + # For runs array format without config_ids, ensure include_all is set in runs + if "runs" in entry_payload: + for r in entry_payload["runs"]: + if "dynamic_filters" in r and "include_all" not in r: + r["include_all"] = False + else: + entry_payload = entry + # For runs array format without config_ids, ensure include_all is set in runs + if "runs" in entry_payload: + for run in entry_payload["runs"]: + if "dynamic_filters" in run and "include_all" not in run: + run["include_all"] = False + + entry_response = self.client.send_post(f"add_plan_entry/{plan_id}", entry_payload) + if entry_response.error_message: + # Provide more helpful error message for common issues + error_msg = entry_response.error_message + if "case_ids needs to include at least one test case" in error_msg: + entry_name = entry.get("name", "Unnamed entry") + return {}, ( + f"Plan created (ID: {plan_id}) but failed to add entry '{entry_name}': " + f"Dynamic filters matched zero test cases. " + f"Please verify that:\n" + f" - The suite contains test cases\n" + f" - The filter criteria matches at least one test case\n" + f" - All field names exist in the suite (e.g., 'cases:is_automated')\n" + f"Original error: {error_msg}" + ) + return {}, f"Plan created (ID: {plan_id}) but failed to add entry with dynamic_filters: {error_msg}" + + # Add the created entry to plan_data for return + if "entries" not in plan_data: + plan_data["entries"] = [] + plan_data["entries"].append(entry_response.response_text) + + return plan_data, "" + + def validate_and_process_plan_entries( + self, entries: List[Dict[str, Any]], cli_mode: Optional[str] = None + ) -> Tuple[List[Dict[str, Any]], str]: + """ + Validate and process plan entries with dynamic filters support. + + This method: + 1. Validates dynamic_filters structure in each run + 2. Checks mutual exclusivity (filters vs case_ids/include_all) + 3. Auto-wraps simplified filter format + 4. Removes internal _mode_from_json flag + 5. Applies CLI mode override if specified (CLI > JSON precedence) + + :param entries: List of plan entry dictionaries + :param cli_mode: Optional CLI mode parameter ("1" or "2") to override JSON mode for all runs + :returns: Tuple with (processed_entries, error_message) + """ + if not entries: + return entries, "" + + processed_entries = [] + + for entry_idx, entry in enumerate(entries): + entry_name = entry.get("name", f"Entry {entry_idx + 1}") + runs = entry.get("runs", []) + + if not runs: + # No runs, just pass through + processed_entries.append(entry) + continue + + processed_runs = [] + + for run_idx, run in enumerate(runs): + run_label = f"Entry '{entry_name}', Run {run_idx + 1}" + + # Check if run has dynamic_filters + if "dynamic_filters" in run: + dynamic_filters = run["dynamic_filters"] + + # Validation: Mutual exclusivity with case_ids + if "case_ids" in run and run["case_ids"]: + return ( + [], + f"{run_label}: dynamic_filters and case_ids cannot be used together. Choose one case selection method.", + ) + + # Validation: Mutual exclusivity with include_all=true + if run.get("include_all") is True: + return ( + [], + f"{run_label}: dynamic_filters and include_all=true cannot be used together. Set include_all=false or omit it when using dynamic filters.", + ) + + # Track if mode was explicitly provided in the JSON + mode_was_in_json = "mode" in dynamic_filters + + # Normalize: if simplified format (no "filters" wrapper), wrap it + if "filters" not in dynamic_filters: + dynamic_filters = {"mode": "1", "filters": dynamic_filters} + # Simplified format means no mode in JSON + mode_was_in_json = False + + # Apply CLI mode override if specified (CLI > JSON precedence) + # Only override if mode was NOT explicitly set in JSON + if cli_mode and not mode_was_in_json: + # No explicit mode in JSON, apply CLI mode + dynamic_filters["mode"] = cli_mode + + # Validate filter structure + is_valid, error = validate_dynamic_filters_structure(dynamic_filters) + if not is_valid: + return [], f"{run_label}: {error}" + + # Remove internal flag if present + if "_mode_from_json" in dynamic_filters: + del dynamic_filters["_mode_from_json"] + + # Update run with processed filters + run["dynamic_filters"] = dynamic_filters + + # Ensure include_all is false when using dynamic_filters (required by add_plan_entry API) + if "include_all" not in run: + run["include_all"] = False + + processed_runs.append(run) + + # Update entry with processed runs + entry["runs"] = processed_runs + processed_entries.append(entry) + + return processed_entries, "" diff --git a/trcli/api/project_based_client.py b/trcli/api/project_based_client.py index 99d99cb..0651611 100644 --- a/trcli/api/project_based_client.py +++ b/trcli/api/project_based_client.py @@ -194,6 +194,32 @@ def create_or_update_test_run(self) -> Tuple[int, str]: """ If a run_id is provided, update the test run; otherwise, add a new test run. """ + # Load dynamic filters from file if provided + dynamic_filters_data = None + dynamic_filters_file = getattr(self.environment, "dynamic_filters", None) + if dynamic_filters_file and isinstance(dynamic_filters_file, str): + from trcli.api.dynamic_filters_utils import load_dynamic_filters_from_file + + filters, error = load_dynamic_filters_from_file(dynamic_filters_file) + if error: + self.environment.elog(f"Error loading dynamic filters: {error}") + return None, error + + # Apply mode from --dynamic-filters-mode if not explicitly specified in JSON file + dynamic_filters_mode = getattr(self.environment, "dynamic_filters_mode", None) + mode_from_json = filters.pop("_mode_from_json", True) # Remove internal flag + + if dynamic_filters_mode and not mode_from_json: + # CLI flag overrides only when JSON file didn't specify mode + filters["mode"] = dynamic_filters_mode + self.environment.log(f"Using filter mode '{dynamic_filters_mode}' from --dynamic-filters-mode flag") + elif mode_from_json: + self.environment.log(f"Using filter mode '{filters.get('mode', '1')}' from JSON file") + else: + self.environment.log(f"Using default filter mode '1' (AND)") + + dynamic_filters_data = filters + if not self.environment.run_id: self.environment.log(f"Creating test run. ", new_line=False) added_run, error_message = self.api_request_handler.add_run( @@ -208,6 +234,7 @@ def create_or_update_test_run(self) -> Tuple[int, str]: include_all=bool(self.environment.run_include_all), refs=self.environment.run_refs, case_ids=self.environment.run_case_ids, + dynamic_filters=dynamic_filters_data, ) run_id = added_run else: diff --git a/trcli/api/run_handler.py b/trcli/api/run_handler.py index 161fd9b..7562b85 100644 --- a/trcli/api/run_handler.py +++ b/trcli/api/run_handler.py @@ -60,6 +60,7 @@ def add_run( include_all: bool = False, refs: str = None, case_ids: List[int] = None, + dynamic_filters: Dict = None, ) -> Tuple[int, str]: """ Creates a new test run. @@ -75,6 +76,7 @@ def add_run( :param include_all: include all cases :param refs: references :param case_ids: specific case ids + :param dynamic_filters: dynamic filters for auto-updating runs :returns: Tuple with run id and error string. """ add_run_data = self.data_provider.add_run( @@ -86,15 +88,18 @@ def add_run( assigned_to_id=assigned_to_id, include_all=include_all, refs=refs, + dynamic_filters=dynamic_filters, ) # Validate that we have test cases to include in the run - # Empty runs are not allowed for parse commands unless include_all is True + # Empty runs are not allowed for parse commands unless include_all is True or dynamic_filters is used # However, add_run command explicitly allows empty runs for later result uploads is_add_run_command = self.environment.cmd == "add_run" + has_dynamic_filters = add_run_data.get("dynamic_filters") is not None if ( not is_add_run_command and not include_all + and not has_dynamic_filters and (not add_run_data.get("case_ids") or len(add_run_data["case_ids"]) == 0) ): error_msg = ( diff --git a/trcli/commands/cmd_add_run.py b/trcli/commands/cmd_add_run.py index 5f702bb..5588da1 100644 --- a/trcli/commands/cmd_add_run.py +++ b/trcli/commands/cmd_add_run.py @@ -7,7 +7,7 @@ def print_config(env: Environment): - env.log( + config_msg = ( f"Parser Results Execution Parameters" f"\n> TestRail instance: {env.host} (user: {env.username})" f"\n> Project: {env.project if env.project else env.project_id}" @@ -23,6 +23,9 @@ def print_config(env: Environment): f"\n> Refs: {env.run_refs}" f"\n> Refs Action: {env.run_refs_action if hasattr(env, 'run_refs_action') else 'add'}" ) + if hasattr(env, "dynamic_filters") and env.dynamic_filters: + config_msg += f"\n> Dynamic Filters: {env.dynamic_filters}" + env.log(config_msg) def write_run_to_file(environment: Environment, run_id: int): @@ -141,6 +144,19 @@ def write_run_to_file(environment: Environment, run_id: int): metavar="", help="Action to perform on references: 'add' (default), 'update' (replace all), or 'delete' (remove all or specific)", ) +@click.option( + "--dynamic-filters", + type=click.Path(exists=True), + metavar="", + help="Path to JSON file containing dynamic filter criteria. Enables auto-updating runs that continuously sync with test case repository.", +) +@click.option( + "--dynamic-filters-mode", + type=click.Choice(["1", "2"], case_sensitive=False), + default="1", + metavar="", + help="Mode for combining dynamic filter conditions: '1' (AND - match all, default) or '2' (OR - match any). Only used if not specified in JSON file.", +) @click.option("-f", "--file", type=click.Path(), metavar="", help="Write run data to file.") @click.pass_context @pass_environment @@ -179,6 +195,22 @@ def cli(environment: Environment, context: click.Context, *args, **kwargs): environment.elog("Error: --run-case-ids and --run-include-all cannot be used together.") exit(1) + # Validation: dynamic filters are mutually exclusive with case_ids, include_all, and clear_run_case_ids + if hasattr(environment, "dynamic_filters") and environment.dynamic_filters: + if environment.run_case_ids: + environment.elog( + "Error: --dynamic-filters and --run-case-ids cannot be used together. Dynamic filters automatically determine which cases to include." + ) + exit(1) + if environment.run_include_all: + environment.elog( + "Error: --dynamic-filters and --run-include-all cannot be used together. Dynamic filters automatically determine which cases to include." + ) + exit(1) + if hasattr(environment, "clear_run_case_ids") and environment.clear_run_case_ids: + environment.elog("Error: --dynamic-filters and --clear-run-case-ids cannot be used together.") + exit(1) + # Validation: clear-run-description requires --run-id and is mutually exclusive if hasattr(environment, "clear_run_description") and environment.clear_run_description and not environment.run_id: environment.elog( diff --git a/trcli/commands/cmd_fields.py b/trcli/commands/cmd_fields.py new file mode 100644 index 0000000..704b81c --- /dev/null +++ b/trcli/commands/cmd_fields.py @@ -0,0 +1,131 @@ +import click +import json + +from trcli.api.project_based_client import ProjectBasedClient +from trcli.cli import pass_environment, CONTEXT_SETTINGS, Environment +from trcli.data_classes.dataclass_testrail import TestRailSuite +from trcli.api.dynamic_filters_utils import FIELD_TYPES + + +def print_config(env: Environment, action: str): + env.log( + f"Fields {action} Execution Parameters" + f"\n> TestRail instance: {env.host} (user: {env.username})" + f"\n> Project: {env.project if env.project else env.project_id}" + ) + + +@click.group(context_settings=CONTEXT_SETTINGS) +@click.pass_context +@pass_environment +def cli(environment: Environment, context: click.Context, *args, **kwargs): + """Manage fields in TestRail""" + environment.cmd = "fields" + environment.set_parameters(context) + + +@cli.command(name="list-dynamic") +@click.option("--json-output", is_flag=True, help="Output fields as raw JSON from API.") +@click.pass_context +@pass_environment +def list_dynamic( + environment: Environment, + context: click.Context, + json_output: bool, + *args, + **kwargs, +): + """List available dynamic filter fields for a project""" + environment.check_for_required_parameters() + + print_config(environment, "List-Dynamic") + + # Create ProjectBasedClient to resolve project + project_client = ProjectBasedClient( + environment=environment, + suite=TestRailSuite(name=environment.suite_name, suite_id=environment.suite_id), + ) + + # Resolve project (converts name to ID if needed) + project_client.resolve_project() + + environment.log(f"Retrieving dynamic filter fields for project ID {project_client.project.project_id}...") + + # Retrieve fields using DynamicFilterHandler + fields, error_message = project_client.api_request_handler.dynamic_filter_handler.get_dynamic_filter_fields( + project_id=project_client.project.project_id + ) + + if error_message: + environment.elog(f"Error: Failed to retrieve fields: {error_message}") + raise SystemExit(1) + + # Handle output format + if json_output: + # Output prettified JSON response + print(json.dumps(fields, indent=2)) + else: + # Display fields in human-readable format + if not fields: + environment.log("No dynamic filter fields found.") + else: + environment.log(f"\nFound {len(fields)} available field(s) for dynamic filtering:\n") + + for field in fields: + type_id = field.get("type_id", "Unknown") + type_name = FIELD_TYPES.get(type_id, f"Type {type_id}") + system_name = field.get("system_name", "N/A") + label = field.get("label", "N/A") + + environment.log(f"Field: {label}") + environment.log(f" System Name: {system_name}") + environment.log(f" Type: {type_name} (ID: {type_id})") + + # Show operators if available (for fields that support operator-based filtering) + sub_filters = field.get("sub_filters") + if sub_filters: + environment.log(f" Available Operators:") + for op in sub_filters: + op_id = op.get("op", "N/A") + op_label = op.get("label", "N/A") + environment.log(f" - {op_label} (op: {op_id})") + + # Show options if available (for dropdown/multi-select fields) + options = field.get("options") + if options: + # Parse options - can be a string format "ID,Label\nID,Label" or a list + parsed_options = [] + if isinstance(options, str): + # Parse string format: "1,GPT-5\n2,Gemini 3\n3,Sonnet 3.5" + for line in options.strip().split("\n"): + if line.strip(): + parts = line.split(",", 1) # Split on first comma only + if len(parts) == 2: + opt_id = parts[0].strip() + opt_label = parts[1].strip() + parsed_options.append({"id": opt_id, "label": opt_label}) + else: + # Fallback for malformed lines + parsed_options.append({"id": "N/A", "label": line.strip()}) + elif isinstance(options, list): + # Already a list of dicts or strings + parsed_options = options + + if parsed_options: + environment.log(f" Available Options ({len(parsed_options)}):") + # Show first 5 options, then indicate if there are more + display_options = parsed_options[:5] + for option in display_options: + if isinstance(option, dict): + opt_id = option.get("id", "N/A") + opt_label = option.get("label", "N/A") + environment.log(f" - {opt_label} (ID: {opt_id})") + else: + # Option is a plain string + environment.log(f" - {option}") + if len(parsed_options) > 5: + environment.log( + f" ... and {len(parsed_options) - 5} more (use --json-output to see all)" + ) + + environment.log("") # Blank line between fields diff --git a/trcli/commands/cmd_plans.py b/trcli/commands/cmd_plans.py index 7acdf7c..9f476fa 100644 --- a/trcli/commands/cmd_plans.py +++ b/trcli/commands/cmd_plans.py @@ -363,6 +363,13 @@ def list( metavar="", help="Path to JSON file containing plan entries. Use this for complex entries with multiple runs/configs. Mutually exclusive with --entries.", ) +@click.option( + "--dynamic-filters-mode", + type=click.Choice(["1", "2"], case_sensitive=False), + default="1", + metavar="", + help="Mode for combining dynamic filter conditions: '1' (AND - match all, default) or '2' (OR - match any). Applies to all runs in the plan if not specified in JSON file. Used when entries contain dynamic_filters.", +) @click.option( "--start-on", metavar="", @@ -388,6 +395,7 @@ def add( milestone_id: int, entries: str, entries_file: str, + dynamic_filters_mode: str, start_on: list, due_on: list, json_output: bool, @@ -509,6 +517,19 @@ def add( environment.elog("Error: Plan name is required (use --name or provide 'name' in JSON)") raise SystemExit(1) + # Validate and process plan entries (handles dynamic filters) + if parsed_entries: + # Pass CLI mode for dynamic_filters_mode override + validated_entries, validation_error = ( + project_client.api_request_handler.plan_handler.validate_and_process_plan_entries( + parsed_entries, cli_mode=dynamic_filters_mode + ) + ) + if validation_error: + environment.elog(f"Error: {validation_error}") + raise SystemExit(1) + parsed_entries = validated_entries + environment.log(f"Creating plan '{final_name}' in project ID {project_client.project.project_id}...") # Create the plan diff --git a/trcli/constants.py b/trcli/constants.py index 0460684..e36ac7b 100644 --- a/trcli/constants.py +++ b/trcli/constants.py @@ -106,6 +106,7 @@ projects=dict(**FAULT_MAPPING), templates=dict(**FAULT_MAPPING), tests=dict(**FAULT_MAPPING), + fields=dict(**FAULT_MAPPING), ) PROMPT_MESSAGES = dict( @@ -130,6 +131,7 @@ - parse_robot: Robot Framework XML Files - parse_openapi: OpenAPI YML Files - add_run: Create a new test run + - fields: Manage fields (list dynamic filter fields) - labels: Manage labels (projects, cases, and tests) - references: Manage references (cases and runs) - cases: Query test cases (get and list) diff --git a/trcli/data_providers/api_data_provider.py b/trcli/data_providers/api_data_provider.py index 787ba47..3cccfa6 100644 --- a/trcli/data_providers/api_data_provider.py +++ b/trcli/data_providers/api_data_provider.py @@ -92,8 +92,15 @@ def add_run( assigned_to_id=None, include_all=None, refs=None, + dynamic_filters=None, ): - """Return body for adding or updating a run.""" + """Return body for adding or updating a run. + + Case selection precedence (highest to lowest): + 1. case_ids - Explicit case IDs take highest priority + 2. dynamic_filters - Dynamic filters if no case_ids specified + 3. include_all - Include all cases if no case_ids or dynamic_filters + """ if case_ids is None: case_ids = [ int(case) for section in self.suites_input.testsections for case in section.testcases if int(case) > 0 @@ -106,7 +113,25 @@ def add_run( ] if self.run_description: properties.insert(0, f"{self.run_description}\n") - body = {"suite_id": self.suites_input.suite_id, "description": "\n".join(properties), "case_ids": case_ids} + body = {"suite_id": self.suites_input.suite_id, "description": "\n".join(properties)} + + # Apply case selection precedence logic + # Priority 1: dynamic_filters (auto-updating filter criteria) + if dynamic_filters: + body["dynamic_filters"] = dynamic_filters + # When dynamic_filters provided, don't include case_ids or include_all + # Priority 2: include_all (include all suite cases) + elif include_all is not None: + body["include_all"] = include_all + # When include_all is set, also include case_ids for compatibility + # TestRail ignores case_ids when include_all=True + if case_ids: + body["case_ids"] = case_ids + # Priority 3: case_ids (explicit case selection) - default behavior + else: + if case_ids: + body["case_ids"] = case_ids + if isinstance(start_date, list) and start_date is not None: try: dt = datetime(start_date[2], start_date[0], start_date[1], tzinfo=timezone.utc) @@ -119,8 +144,6 @@ def add_run( body["due_on"] = int(dt.timestamp()) except ValueError: body["due_on"] = None - if include_all is not None: - body["include_all"] = include_all if assigned_to_id is not None: body["assignedto_id"] = assigned_to_id if refs is not None: