Add YAML Configuration Validation using POJO-Generated JSON Schemas #2046
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Added victools jsonschema-generator for POJO-to-schema conversion - Added jsonschema-module-jackson for Jackson integration - Added jsonschema-module-swagger-2 for Swagger annotation support - Added networknt json-schema-validator for YAML validation - Added jackson-dataformat-yaml for YAML processing These dependencies will enable automatic JSON schema generation from configuration DTOs and validation of YAML configuration files.
…emas This commit adds a comprehensive validation system for YAML configuration files that automatically generates JSON schemas from POJO configuration DTOs and validates all YAML files against them. Features: - Automatic JSON Schema generation from POJO/DTO classes - Validates all 15+ main configuration YAML files - Configurable validation timing (startup and/or runtime) - Configurable failure modes (FAIL_FAST, WARN, SKIP) - Schema caching for improved performance - Detailed error reporting with clear messages - Integration with existing ConfigurationManager Components Added: - ValidationConfig: Configuration class for validation behavior - JsonSchemaGenerator: Generates schemas from POJOs using victools library - YamlValidator: Validates YAML files against generated schemas - ConfigValidator: High-level integration with ConfigurationManager - YamlValidatorTest: Comprehensive unit tests Integration: - ConfigurationManager now calls validation at startup - Validation can be enabled/disabled via system properties - Supports all main config files: MessageDaemon, NetworkManager, AuthManager, DestinationManager, DeviceManager, etc. Configuration via system properties: - validation.startup=true/false (default: true) - validation.runtime=true/false (default: false) - validation.mode=FAIL_FAST/WARN/SKIP (default: FAIL_FAST) - validation.cache.schemas=true/false (default: true) - validation.verbose=true/false (default: false) Documentation: - Added comprehensive YAML_VALIDATION.md guide - Includes usage examples, configuration options, and best practices
The tests were failing because validation was too lenient with type coercion. This commit fixes the issue by: 1. JsonSchemaGenerator: - Added Option.STRICT_TYPE_INFO for strict type schemas - Added Option.FORBIDDEN_ADDITIONAL_PROPERTIES_BY_DEFAULT 2. YamlValidator: - Configured SchemaValidatorsConfig with setTypeLoose(false) - This prevents type coercion (e.g., string to integer) - Now correctly rejects invalid types like DelayedPublishInterval: "not a number" This ensures that YAML files with wrong types (e.g., string where int is expected) will fail validation as intended. Fixes: - testInvalidYamlFile: Now correctly fails on type mismatches - testValidationModes: Now correctly validates strict types
1. Fixed YamlValidator compilation error:
- Removed incorrect defaultConfig() method call on Builder
- Applied SchemaValidatorsConfig directly to getSchema() method
- Maintained strict type validation (setTypeLoose(false))
2. Added comprehensive schema storage documentation:
- docs/SCHEMA_STORAGE.md with complete guide
- Location: ${MAPS_HOME}/schemas/cache/<DTOClassName>.schema.json
- Explains caching, inspection, and configuration options
3. Created schema inspection utility:
- scripts/inspect-schemas.sh for easy schema examination
- Commands: list, view, count, search, validate, stats, clean
- Supports jq for pretty-printing JSON schemas
This fixes the compilation issue while maintaining strict type validation.
- Complete testing procedures from compilation to deployment - Schema inspection instructions with examples - Performance testing guidelines - Troubleshooting common issues - Verification checklist
The testValidYamlFile was failing because when validating a file directly,
the validator was using the temp filename as the config name instead of
deriving it from the DTO class name.
Changes:
- Added deriveConfigName() method to extract config name from class name
- Strips ConfigDTO, DTO, or Config suffixes
- Examples: MessageDaemonConfigDTO -> MessageDaemon
SecurityManagerDTO -> SecurityManager
- Now correctly extracts the root YAML key (e.g., 'MessageDaemon:')
This fixes the validation error:
'property MessageDaemon is not defined in the schema'
The validator now properly validates YAML files with root keys like:
MessageDaemon:
DelayedPublishInterval: 1000
...
The schema generator was creating property names in camelCase
(e.g., delayedPublishInterval) but YAML files use PascalCase
(e.g., DelayedPublishInterval).
Solution:
- Configured ObjectMapper with UPPER_CAMEL_CASE naming strategy
- This ensures generated schemas match YAML property names
- Schema now has: DelayedPublishInterval, SessionPipeLines, etc.
- Matches ConfigurationProperties naming convention
Before:
Schema: {"delayedPublishInterval": {...}}
YAML: DelayedPublishInterval: 1000 ❌ Mismatch
After:
Schema: {"DelayedPublishInterval": {...}}
YAML: DelayedPublishInterval: 1000 ✅ Match
This fixes validation errors where properties were not recognized.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR implements a comprehensive YAML configuration validation system that automatically generates JSON schemas from POJO/DTO classes and validates all YAML configuration files against them.
Features Implemented
Core Functionality
FAIL_FAST(default): Block startup/updates on validation failureWARN: Log warnings but continue with defaultsSKIP: Silently skip invalid configurationsConfiguration Options
All configurable via system properties:
Validated Configuration Files
Implementation Details
Components Added
Dependencies Added
Testing
Unit Tests
Run tests:
mvn test -Dtest=YamlValidatorTestDocumentation
Added Documentation Files
Code Metrics
Breaking Changes
None - This is a new feature with validation enabled by default in FAIL_FAST mode.
Checklist
🤖 Generated with Claude Code