Advanced Filtering: Database-style Query Operations

[anitnilay20]

Advanced Filtering: Database-style Query Operations

Problem Statement

Currently, Thoth supports text-based search and JSONPath queries, but lacks database-style filtering operations. Users working with numerical data, timestamps, or need to filter records based on comparison operators (>, <, >=, <=, !=) have to manually scan results.

Use Cases:

• DevOps: Filter logs where response_time > 500 or status_code != 200
• Analytics: Find records where price < 100 and stock > 0
• Monitoring: Identify events where timestamp > "2024-01-01" and error_level = "critical"
• API testing: Filter responses where user.age >= 18 or items.length < 10

Proposed Solution

Add a filtering query language that supports comparison operators, logical operators, and type-aware comparisons.

Syntax Examples

price > 100
age >= 18 AND country = "US"
status != "deleted" AND (priority = "high" OR urgent = true)
items.length >= 3
deleted_at = null

Implementation Approach

1. Query parser using recursive descent or nom crate
2. Type-aware evaluation (numbers, strings, booleans, null)
3. Parallel evaluation with Rayon
4. Integration with existing search UI

Acceptance Criteria

• Parse comparison operators (>, <, >=, <=, !=, =)
• Support logical operators (AND, OR) with proper precedence
• Handle parentheses for grouping
• Type-aware comparisons
• Field existence checks
• Maintain search performance
• Tests and documentation

[anitnilay20]

Related Issues:

• Sorting: Multi-Column Sort with Custom Comparators #54 - Sorting results (ORDER BY for filtered data)
• Aggregation: GROUP BY and Aggregate Functions #55 - Aggregation functions (GROUP BY for analytics)

These three features form a complete query system:

WHERE status = "active" AND price > 100    # Issue #53 (Filtering)
GROUP BY category SELECT COUNT(*), AVG(price)  # Issue #55 (Aggregation)
SORT BY avg_price DESC                         # Issue #54 (Sorting)