Finalize Spring Kafka Migration, Code Cleanup, Docker Compose Updates, Reliability Fixes & Metrics Integration#574
Merged
dan-du-car merged 69 commits intousdot-jpo-ode:developfrom May 8, 2025
Conversation
* Replaced usages of MessageProducer with KafkaTemplate in TimDepositController Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Used `serialIdJ2735.getStreamId()` as key when sending data to topics in TimDepositController Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Revert "Used `serialIdJ2735.getStreamId()` as key when sending data to topics in TimDepositController" This reverts commit 73cf99d. * Revert "Replaced usages of MessageProducer with KafkaTemplate in TimDepositController" This reverts commit 136a654. * Refactored TimDepositControllerTest to use Spring annotations Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Removed unnecessary `@Capturing` variables from TimDepositControllerTest and formatted file Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Renamed fields in TimDepositControllerTest Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.messageWithNoRSUsOrSDWShouldReturnWarning() to verify that records are published to broadcast topics Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Formatted TimDepositControllerTest Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.failedObjectNodeConversionShouldReturnConvertingError to verify records get published to Kafka topics Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Formatted TimDepositControllerTest Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Added some TODOs Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.failedXmlConversionShouldReturnConversionError to verify records get published to Kafka Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Cleaned up consumer initialization & added consumer cleanup in TimDepositControllerTest Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Formatted TimDepositControllerTest Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest to set unique topic names for each test Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Formatted TimDepositControllerTest Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Started working on updating TimDepositControllerTest.testSuccessfulMessageReturnsSuccessMessagePost to verify records get deposited to Kafka topics Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Finished updating TimDepositControllerTest.testSuccessfulMessageReturnsSuccessMessagePost to verify records get deposited to Kafka topics Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.testSuccessfulSdwRequestMessageReturnsSuccessMessagePost to verify records get deposited to Kafka topics Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.testSuccessfulMessageReturnsSuccessMessagePostWithOde to verify records get deposited to Kafka topics Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.testSuccessfulMessageReturnsSuccessMessagePut to verify records get deposited to Kafka topics Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.testDepositingTimWithExtraProperties to verify records get deposited to Kafka topics Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.testSuccessfulTimIngestIsTracked to verify records get deposited to Kafka topics Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.testSuccessfulRsuMessageReturnsSuccessMessagePost to verify records get deposited to Kafka topics Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.messageWithNoRSUsOrSDWShouldReturnWarning to verify contents of json topic record Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.failedObjectNodeConversionShouldReturnConvertingError to verify contents of json topic record & updated non-null assertions to use record value Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.testSuccessfulMessageReturnsSuccessMessagePost to verify contents of published records Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Formatted TimDepositControllerTest Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.testSuccessfulSdwRequestMessageReturnsSuccessMessagePost to verify contents of published records Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.testSuccessfulMessageReturnsSuccessMessagePostWithOde to verify contents of published records Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.testSuccessfulMessageReturnsSuccessMessagePut to verify contents of published records Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.testDepositingTimWithExtraProperties to verify contents of published records Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.testSuccessfulTimIngestIsTracked to verify contents of published records Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.testSuccessfulRsuMessageReturnsSuccessMessagePost to verify contents of published records Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated TimDepositControllerTest.failedXmlConversionShouldReturnConversionError to mock return value of TimTransmogrifier.obfuscateRsuPassword() Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Formatted TimDepositControllerTest Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Replaced usages of MessageProducer with KafkaTemplate in TimDepositController" Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Used `serialIdJ2735.getStreamId()` as key when sending data to topics in TimDepositController * Updated TimDepositController to support usage of KafkaTemplate instead of MessageProducer Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Formatted TimDepositControllerTest Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Set key to stream id for pojo tim broadcast topic & updated unit tests Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Updated schema version in test data Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Removed nested json assertions from TimDepositControllerTest to improve readability & maintainability Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Addressed checkstyle comments for TimDepositControllerTest Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Removed nested xml assertions from TimDepositControllerTest to improve readability & maintainability Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> * Removed OdeTimSerializer.java Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com> --------- Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com>
PSM Schema Fix
* test: add canConvertPojoToJSON to ToJsonServiceControllerTest * refactor: rename ToJsonServiceController to BSMPojoToJSONListener Renamed the class and related test file to reflect its responsibility more accurately. Updated all references to align with the new class name. This improves clarity and consistency in the codebase. * refactor: BSMPojoToJSONListener consumption now using Spring Kafka The tests fail because I have not yet configured the correct KafkaTemplate for message production. This just proves we can consume the data without error * refactor: BSMPojoToJSONListener production now using Spring Kafka This uses a blend of Spring Kafka and our previous JsonUtils setup. The next step is to migrate to using the DI provided ObjectMapper to convert to the JSON representation during message production * refactor: BSMPojoToJSONListener production using DI-provided ObjectMapper * test: correctly close test consumer in BSMPojoToJSONListenerTest * chore: remove deprecated and unused classes Deleted `OdeBsmDeserializer`, `OdeBsmSerializer`, `OdeDataPublisher`, and related test files, as they were deprecated or no longer in use. Also marked `MessagingDeserializer` as `final` for better immutability and clarity. * test: properly close out test consumers in Spring Kafka tests * chore: delete unused ToJsonConverter.java * chore: delete unused AbstractSubPubTransformer.java * docs: add detailed JavaDocs to BSMPojoToJSONListener class Added comprehensive JavaDocs to the BSMPojoToJSONListener class, including descriptions for the class itself, its constructor, and the Kafka listener method. These improvements clarify the purpose, functionality, and expected behavior of the class and its components. * refactor: serialization to directly use Kryo API Removed `SerializationUtils` and its associated test class, replacing them with direct usage of the Kryo library in `MessagingSerializer` and `MessagingDeserializer`. Updated serialization and deserialization logic to include logging and streamlined test cases for improved clarity and functionality. Separating Serialization and Deserialization responsibilities in to their respective classes better indicates the intentions of the classes. * docs: correct references in javadocs * chore: remove dead comment from BSMPojoToJSONListenerTest * perf: update serializer to use Output.toBytes for accurate byte array Replaced `output.getBuffer()` with `output.toBytes` to ensure accurate serialization and avoid including extra byte padding produced to the topic. this will save significant space on the topic as BSM data is one of the most frequent data types produced/consumed by the ODE.
* refactor: replace consumption/production for STOMP messaging with Spring Kafka Removed `StompStringExporter` and its integration with the file upload controller, replacing it with Kafka-based message consumption using `@KafkaListener`. Updated related tests to reflect the changes, simplifying the architecture and removing unused components. * chore: delete unused classes and tests Removed the `Exporter` and `StompStringMessageDistributor` classes along with their associated test files. These components were not in use and their removal helps in reducing code clutter and maintenance overhead. * refactor: tests in StompStringExporterTest to use Mockito. Replaced JMockit with Mockito for cleaner and more modern testing. Added setup with mocked SimpMessagingTemplate and refactored test cases to verify behavior using Mockito assertions. Improved test clarity by focusing on specific functionalities like `listenUnfiltered` and `listenFiltered`. * docs: update FileUploadController with detailed Javadocs Added Javadoc comments to enhance code clarity and provide detailed explanations for methods and constructors. Updated formatting and variable alignment for better readability. Minor logging improvements for exception handling in the file not found scenario. * refactor: use Spring bean for ImporterDirectoryWatcher Replaced direct instantiation of `ImporterDirectoryWatcher` with a Spring-managed bean for improved configurability and dependency injection. Added a new configuration class `ImporterDirectoryWatcherConfig` to define the bean. Simplifies the `FileUploadController` constructor and enhances code maintainability. * refactor: FileUploadControllerTest to use Mockito framework. Replaced JMockit with Mockito for mocking and expectations in tests, improving test readability and consistency with modern testing practices. Simplified imports and removed unused dependencies to clean up the codebase. Updated setup and test methods to align with Mockito's syntax and functionality. * refactor: enable conditional loading for StompStringExporter class Introduce a conditional configuration to enable or disable the `StompStringExporter` based on the `ode.stomp-exporter.enabled` property. This change optimizes resource usage by allowing the exporter to be disabled in production environments while retaining functionality for the Demo Console. Updated configuration files and sample environment variables accordingly. * refactor: move file topic configuration to stomp-exporter Removed `FileTopics` class and its associated test. Updated configuration keys to align with the `stomp-exporter` naming convention. Adjusted YAML files and `StompStringExporter` constructor for the updated configuration structure. * chore: remove empty `file:` from application.yaml * chore: remove unnecessary mocking in FileUploadControllerTest
…s.setClock (#173) * refactor: return previous clock from DateTimeUtils.setClock This allows tests to reset the global DateTimeUtils.clock to its previous state. It is intended to prevent test executions from interfering with each other. The previous implementation didn't make it clear that the clock is global and needs to be reset after the test is completed. * test: update tests using DateTimeUtils.setClock to reset clock to previous state * style: reformat DateTimeUtils * chore: remove unused code from DateTimeUtils * docs: add missing javadoc to DateTimeUtils * chore: set clock back to previous in TimTransmogrifierTest
…igration to Spring Kafka (#162) * refactor: extract LogFileParserFactory and integrate it across modules Implemented LogFileParserFactory to streamline parser creation based on log file prefixes. Updated relevant classes and tests to use the new factory, ensuring consistent instantiation of log file parsers. This change improves maintainability and readability of the codebase. * test: switch to JUnit 5 in DistressMsgFileParserTest Replaced JUnit 4 imports and annotations with JUnit 5 equivalents. Updated test methods to follow JUnit 5 method signatures and added necessary setup using @beforeeach. This improves test consistency and ensures compatibility with modern testing practices. * test: switch to JUnit 5 in BsmFileParserTest Replaced JUnit 4 imports and annotations with JUnit 5 equivalents. Updated test methods to follow JUnit 5 method signatures and added necessary setup using @beforeeach. This improves test consistency and ensures compatibility with modern testing practices. * test: switch to JUnit 5 in DriverAlertFileParserTest Replaced JUnit 4 imports and annotations with JUnit 5 equivalents. Updated test methods to follow JUnit 5 method signatures and added necessary setup using @beforeeach. This improves test consistency and ensures compatibility with modern testing practices. * test: switch to JUnit 5 in PayloadParserTest Replaced JUnit 4 imports and annotations with JUnit 5 equivalents. Updated test methods to follow JUnit 5 method signatures and added necessary setup using @beforeeach. This improves test consistency and ensures compatibility with modern testing practices. * test: switch to JUnit 5 in RxMsgFileParserTest Replaced JUnit 4 imports and annotations with JUnit 5 equivalents. Updated test methods to follow JUnit 5 method signatures and added necessary setup using @beforeeach. This improves test consistency and ensures compatibility with modern testing practices. * style: update style and javadoc in Asn1CodecPublisher * refactor: rename factory method for clarity * test: add LogFileParserFactoryTest * docs: add javadocs to LogFileParserFactory * docs: add javadocs to FileParser * style: reformat and add javadocs to RxMsgFileParser * style: reformat and add javadocs to BsmLogFileParser * style: reformat and add javadocs to DistressMsgFileParser * style: reformat and add javadocs to DriverAlertFileParser * style: reformat and add javadocs to LogFileParser * chore: set getAlert to public until we can refactor * style: update style and javadoc in LogFileToAsn1CodecPublisher * test: update simple tests in LogFileToAsn1CodecPublisherTest * refactor: pass LogFileParser to LogFileToAsn1CodecPublisher.publish Updated the publish method to explicitly accept a LogFileParser instance, allowing for improved flexibility and better separation of concerns. Adjusted related methods and tests to accommodate the new parameter. * refactor: tests to use Mockito instead of JMockit Replaced JMockit with Mockito for mocking behavior in LogFileToAsn1CodecPublisherTest. Simplified dependency initialization and adjusted test methods for compatibility, improving test maintainability and consistency. * test: add assertion to testPublishDistressNotificationLogFile * chore: extract inline JSON to test resource files Moved inline JSON strings in LogFileToAsn1CodecPublisherTest to external test resource files for better readability and maintainability. Added a utility function to load JSON from files and updated test cases to use these resources. * chore: update static final constant name in LogFileToAsn1CodecPublisherTest * refactor: `LogFileParser` for cleaner code and enhanced logging. Updated method implementations to use modern Java features like pattern matching and inline casting for improved readability. Added `@Slf4j` for better logging support and streamlined `buildReceivedMessageDetails` for concise logic and debug messaging when `locationParser` is unavailable. * refactor: set ParserStatus variable without default initialization Removed the unnecessary default initialization of the `ParserStatus` variable in the `parseFile` method. This change ensures cleaner code and avoids redundant assignments during method execution. Functionality remains unaffected. * refactor: use assertInstanceOf and refine exceptions in LogFileParserTest Replaced assertTrue with assertInstanceOf to improve clarity and type-specific assertions in parser tests. Updated the factory exception test to assert the correct custom exception type, ensuring better alignment with expected behavior. * refactor: file parsers to remove redundant filename parameters Updated file parsers to pass filename and record type via constructor, eliminating the need to pass filenames repeatedly during parsing. Simplified method signatures and improved encapsulation for more streamlined and maintainable code. * style: address checkstyle warnings in TimeParserTest * style: address checkstyle warnings in LocationParserTest * style: address checkstyle warnings in SecResultCodeParserTest * test: refactor parser test constructors for direct initialization. Replaced the use of `@Tested` and `@Injectable` annotations with explicit constructors to initialize parser instances. This change improves clarity and aligns with best practices for test setup. * test: update tests to use JUnit 5 Assertions Replaced JUnit 4 assertions with JUnit 5 equivalents in test files. This modernizes the codebase for compatibility with JUnit 5 and aligns with updated testing standards. No functionality changes were introduced. * style: correct checkstyle violations in SpatLogFileParser Reorganize code to improve readability, addressing checkstyle violations such as indentation and formatting. Documented class details and parsing steps to enhance code clarity and maintainability. * style: correct checkstyle violations in PayloadParser Reorganize code to improve readability, addressing checkstyle violations such as indentation and formatting. Documented class details and parsing steps to enhance code clarity and maintainability. * style: correct checkstyle violations in IntersectionParser Reorganize code to improve readability, addressing checkstyle violations such as indentation and formatting. Documented class details and parsing steps to enhance code clarity and maintainability. * style: correct checkstyle violations in LocationParser Reorganize code to improve readability, addressing checkstyle violations such as indentation and formatting. Documented class details and parsing steps to enhance code clarity and maintainability. * style: correct checkstyle violations in SecurityResultCodeParser Reorganize code to improve readability, addressing checkstyle violations such as indentation and formatting. Documented class details and parsing steps to enhance code clarity and maintainability. * style: correct checkstyle violations in FileAsn1CodecPublisher Reorganize code to improve readability, addressing checkstyle violations such as indentation and formatting. Documented class details and parsing steps to enhance code clarity and maintainability. * style: correct checkstyle violations in TimeParser Reorganize code to improve readability, addressing checkstyle violations such as indentation and formatting. Documented class details and parsing steps to enhance code clarity and maintainability. * chore: set properties to private final in LogFileToAsn1CodecPublisher * chore: rename xmlFile to file for accuracy * chore: standardize casing of filename parameter * chore: simplify boolean in setCertPresent * docs: remove lingering '*' from javadoc
* chore: delete unused FormattedTimer.java * chore: delete unused WebSocketEndpoint.java and test * chore: delete unused WebSocketUtils.java * chore: delete unused WebSocketClient.java and interface and base class * chore: delete unused WebSocketMessageDecoder.java * chore: delete unused WebSocketMessageHandler.java * chore: delete unused SSLBuilder.java and test * style: reformat CommonUtils * chore: delete unused CommonUtils.java * chore: delete unused SerializableMessageProducerPool.java * chore: delete unused SerializableObjectPool.java * chore: port method from CommonUtils to TravelerMessageFromHumanToAsnConverter for enumToString * chore: delete unused exceptions * chore: delete unused fields in AppContext * chore: delete unused HttpClientFactory.java and related tests * chore: use enumToString in TravelerMessageFromHumanToAsnConverter * chore: delete disabled and unnecessary BrakeAppliedStatusNamesTest.java * refactor: remove custom thread management classes and use default executor Replaced `ServiceManager` and `UdpServiceThreadFactory` with `DefaultManagedTaskExecutor` to simplify thread management. Removed outdated test class `UdpServiceThreadFactoryTest` as it no longer applies to the updated implementation. This change reduces complexity and leverages Spring's built-in tools for better maintainability. * chore: remove obsolete plugin-related classes and tests Deleted legacy plugin framework, including interfaces, implementations, factory, and associated tests. These classes were no longer in use and their removal simplifies the codebase, reducing maintenance overhead. It also removes the compiler warnings that the Plugin implementation is deprecated and will not be supported in the future
[skip ci]
… from the ODE source code
Set up CI with Azure Pipelines
Adding Missing ODE Variables to Docker Compose
…nager project in `release_process_quarterly.md` Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com>
Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com>
Include Git History in Azure Artifacts
* refactor: replace consumption/production for STOMP messaging with Spring Kafka Removed `StompStringExporter` and its integration with the file upload controller, replacing it with Kafka-based message consumption using `@KafkaListener`. Updated related tests to reflect the changes, simplifying the architecture and removing unused components. * chore: delete unused classes and tests Removed the `Exporter` and `StompStringMessageDistributor` classes along with their associated test files. These components were not in use and their removal helps in reducing code clutter and maintenance overhead. * refactor: tests in StompStringExporterTest to use Mockito. Replaced JMockit with Mockito for cleaner and more modern testing. Added setup with mocked SimpMessagingTemplate and refactored test cases to verify behavior using Mockito assertions. Improved test clarity by focusing on specific functionalities like `listenUnfiltered` and `listenFiltered`. * docs: update FileUploadController with detailed Javadocs Added Javadoc comments to enhance code clarity and provide detailed explanations for methods and constructors. Updated formatting and variable alignment for better readability. Minor logging improvements for exception handling in the file not found scenario. * refactor: use Spring bean for ImporterDirectoryWatcher Replaced direct instantiation of `ImporterDirectoryWatcher` with a Spring-managed bean for improved configurability and dependency injection. Added a new configuration class `ImporterDirectoryWatcherConfig` to define the bean. Simplifies the `FileUploadController` constructor and enhances code maintainability. * refactor: FileUploadControllerTest to use Mockito framework. Replaced JMockit with Mockito for mocking and expectations in tests, improving test readability and consistency with modern testing practices. Simplified imports and removed unused dependencies to clean up the codebase. Updated setup and test methods to align with Mockito's syntax and functionality. * refactor: enable conditional loading for StompStringExporter class Introduce a conditional configuration to enable or disable the `StompStringExporter` based on the `ode.stomp-exporter.enabled` property. This change optimizes resource usage by allowing the exporter to be disabled in production environments while retaining functionality for the Demo Console. Updated configuration files and sample environment variables accordingly. * refactor: move file topic configuration to stomp-exporter Removed `FileTopics` class and its associated test. Updated configuration keys to align with the `stomp-exporter` naming convention. Adjusted YAML files and `StompStringExporter` constructor for the updated configuration structure. * refactor: extract LogFileParserFactory and integrate it across modules Implemented LogFileParserFactory to streamline parser creation based on log file prefixes. Updated relevant classes and tests to use the new factory, ensuring consistent instantiation of log file parsers. This change improves maintainability and readability of the codebase. * test: switch to JUnit 5 in DistressMsgFileParserTest Replaced JUnit 4 imports and annotations with JUnit 5 equivalents. Updated test methods to follow JUnit 5 method signatures and added necessary setup using @beforeeach. This improves test consistency and ensures compatibility with modern testing practices. * test: switch to JUnit 5 in BsmFileParserTest Replaced JUnit 4 imports and annotations with JUnit 5 equivalents. Updated test methods to follow JUnit 5 method signatures and added necessary setup using @beforeeach. This improves test consistency and ensures compatibility with modern testing practices. * test: switch to JUnit 5 in DriverAlertFileParserTest Replaced JUnit 4 imports and annotations with JUnit 5 equivalents. Updated test methods to follow JUnit 5 method signatures and added necessary setup using @beforeeach. This improves test consistency and ensures compatibility with modern testing practices. * test: switch to JUnit 5 in PayloadParserTest Replaced JUnit 4 imports and annotations with JUnit 5 equivalents. Updated test methods to follow JUnit 5 method signatures and added necessary setup using @beforeeach. This improves test consistency and ensures compatibility with modern testing practices. * test: switch to JUnit 5 in RxMsgFileParserTest Replaced JUnit 4 imports and annotations with JUnit 5 equivalents. Updated test methods to follow JUnit 5 method signatures and added necessary setup using @beforeeach. This improves test consistency and ensures compatibility with modern testing practices. * style: update style and javadoc in Asn1CodecPublisher * refactor: rename factory method for clarity * test: add LogFileParserFactoryTest * docs: add javadocs to LogFileParserFactory * docs: add javadocs to FileParser * style: reformat and add javadocs to RxMsgFileParser * style: reformat and add javadocs to BsmLogFileParser * style: reformat and add javadocs to DistressMsgFileParser * style: reformat and add javadocs to DriverAlertFileParser * style: reformat and add javadocs to LogFileParser * chore: set getAlert to public until we can refactor * style: update style and javadoc in LogFileToAsn1CodecPublisher * test: update simple tests in LogFileToAsn1CodecPublisherTest * refactor: pass LogFileParser to LogFileToAsn1CodecPublisher.publish Updated the publish method to explicitly accept a LogFileParser instance, allowing for improved flexibility and better separation of concerns. Adjusted related methods and tests to accommodate the new parameter. * refactor: tests to use Mockito instead of JMockit Replaced JMockit with Mockito for mocking behavior in LogFileToAsn1CodecPublisherTest. Simplified dependency initialization and adjusted test methods for compatibility, improving test maintainability and consistency. * test: add assertion to testPublishDistressNotificationLogFile * chore: extract inline JSON to test resource files Moved inline JSON strings in LogFileToAsn1CodecPublisherTest to external test resource files for better readability and maintainability. Added a utility function to load JSON from files and updated test cases to use these resources. * chore: update static final constant name in LogFileToAsn1CodecPublisherTest * refactor: `LogFileParser` for cleaner code and enhanced logging. Updated method implementations to use modern Java features like pattern matching and inline casting for improved readability. Added `@Slf4j` for better logging support and streamlined `buildReceivedMessageDetails` for concise logic and debug messaging when `locationParser` is unavailable. * refactor: set ParserStatus variable without default initialization Removed the unnecessary default initialization of the `ParserStatus` variable in the `parseFile` method. This change ensures cleaner code and avoids redundant assignments during method execution. Functionality remains unaffected. * refactor: use assertInstanceOf and refine exceptions in LogFileParserTest Replaced assertTrue with assertInstanceOf to improve clarity and type-specific assertions in parser tests. Updated the factory exception test to assert the correct custom exception type, ensuring better alignment with expected behavior. * refactor: file parsers to remove redundant filename parameters Updated file parsers to pass filename and record type via constructor, eliminating the need to pass filenames repeatedly during parsing. Simplified method signatures and improved encapsulation for more streamlined and maintainable code. * style: address checkstyle warnings in TimeParserTest * style: address checkstyle warnings in LocationParserTest * style: address checkstyle warnings in SecResultCodeParserTest * test: refactor parser test constructors for direct initialization. Replaced the use of `@Tested` and `@Injectable` annotations with explicit constructors to initialize parser instances. This change improves clarity and aligns with best practices for test setup. * test: update tests to use JUnit 5 Assertions Replaced JUnit 4 assertions with JUnit 5 equivalents in test files. This modernizes the codebase for compatibility with JUnit 5 and aligns with updated testing standards. No functionality changes were introduced. * style: correct checkstyle violations in SpatLogFileParser Reorganize code to improve readability, addressing checkstyle violations such as indentation and formatting. Documented class details and parsing steps to enhance code clarity and maintainability. * style: correct checkstyle violations in PayloadParser Reorganize code to improve readability, addressing checkstyle violations such as indentation and formatting. Documented class details and parsing steps to enhance code clarity and maintainability. * style: correct checkstyle violations in IntersectionParser Reorganize code to improve readability, addressing checkstyle violations such as indentation and formatting. Documented class details and parsing steps to enhance code clarity and maintainability. * style: correct checkstyle violations in LocationParser Reorganize code to improve readability, addressing checkstyle violations such as indentation and formatting. Documented class details and parsing steps to enhance code clarity and maintainability. * style: correct checkstyle violations in SecurityResultCodeParser Reorganize code to improve readability, addressing checkstyle violations such as indentation and formatting. Documented class details and parsing steps to enhance code clarity and maintainability. * style: correct checkstyle violations in FileAsn1CodecPublisher Reorganize code to improve readability, addressing checkstyle violations such as indentation and formatting. Documented class details and parsing steps to enhance code clarity and maintainability. * style: correct checkstyle violations in TimeParser Reorganize code to improve readability, addressing checkstyle violations such as indentation and formatting. Documented class details and parsing steps to enhance code clarity and maintainability. * chore: set properties to private final in LogFileToAsn1CodecPublisher * chore: rename xmlFile to file for accuracy * chore: standardize casing of filename parameter * chore: simplify boolean in setCertPresent * style: address checkstyle violations - excludes javadocs as changes are incoming * refactor: file processing logic in ImporterProcessor Decoupled file processing from backup/move operations for improved clarity and reusability. Introduced `FileType` enum and helper methods to streamline file type detection and input stream handling. Updated tests to reflect changes and removed deprecated method `processAndBackupFile`. * refactor: Change return type of processDirectory to void Updated processDirectory to no longer return a file count and replaced the return statement with a debug log for processed file count. Refined exception handling to catch IOException instead of a generic Exception. * chore: make patterns static final and fix minor formatting issue Updated `gZipPattern` and `zipPattern` to be `static final` for consistency and better clarity. Also corrected a small spacing issue in `publishFile` method to improve code readability. * refactor: file processing and remove unused code. Made the `processFile` method private to restrict its visibility and removed the disabled test for exception handling. Cleaned up unused imports and redundant logging to streamline code and improve maintainability. * test: define test cases for ImporterProcessorTest * chore: remove return value from LogFileToAsn1CodecPublisher.removeNextNewLineCharacter The method performs the operation directly on the input stream passed in. we don't create a new input stream, so it is not necessary (or accurate) to return the input stream. * refactor: parser status enums for clarity and consistency Updated `ParserStatus` enums to enhance clarity by differentiating between entry-level and file-level parsing states. Replaced ambiguous statuses like `COMPLETE` and `PARTIAL` with clear alternatives such as `ENTRY_PARSING_COMPLETE` and `ENTRY_PARTIALLY_PROCESSED` throughout the codebase, ensuring consistency across files and tests. * refactor: remove FileAsn1CodecPublisher and simplify usage Replaced FileAsn1CodecPublisher with direct usage of LogFileToAsn1CodecPublisher to simplify code structure and reduce unnecessary abstraction. Updated associated classes to accommodate this change and streamline functionality. * style: addressed checkstyle warnings in ImporterDirectoryWatcher * refactor: extract named method for clarity * refactor: replace StringPublisher with KafkaTemplate Switched from StringPublisher to Spring Kafka's KafkaTemplate for publishing messages, simplifying integration with Kafka. Updated constructors, tests, and relevant usage to reflect this change, ensuring compatibility and improved maintainability. * style: address checkstyle warnings in ImporterDirectoryWatcherTest * refactor: remove unused OdeKafkaProperties field from ImporterDirectoryWatcherConfig * refactor: ImporterDirectoryWatcher to use Spring Scheduling Replaced the manual executor-based scheduling with Spring's @scheduled annotation for directory polling. Removed the ImporterDirectoryWatcherConfig class to simplify configuration and shifted towards using Spring-managed components. This improves maintainability and aligns with Spring conventions. * chore: remove filetype param from test construction * chore: use formatted string for UperUtil.StartFlagNotFoundException * chore: remove implements Runnable from ImporterDirectoryWatcher * refactor: pull ImporterFileType into separate class Moved the `ImporterFileType` enum from `ImporterDirectoryWatcher` to a standalone class for better modularity and reusability. Updated all references to point to the new location. This change simplifies code dependencies and improves maintainability. * docs: add Javadocs for ImporterDirectoryWatcher * refactor: remove ImporterDirectoryWatcher from FileUploadController * style: reformat FileUploadController * refactor: ImporterFileType and update file type handling. Removed unused `watching` property in `ImporterDirectoryWatcher` for cleaner code. Replaced `JSON_FILE` type with `UNKNOWN` in `ImporterFileType` and updated test to reflect this change. Added detailed documentation to `ImporterFileType` enum for better clarity. * chore: delete unused DecoderPublisher.java * chore: rename FileType.java to FileFormat This will increase clarity of intent and further delineate the difference between this enum and the ImporterFileType enum * docs: added missing Javadocs to ImporterProcessor * chore: address warnings in ImporterDirectoryWatcherTest and remove unused variables * test: add suppression of unchecked warnings to LogFileToAsn1CodecPublisherTest * chore: delete unused classes and their tests * refactor: replace magic strings with LogFileType enum * style: address checkstyle warnings in FileSystemStorageService * refactor: logging to use Lombok's log instead of EventLogger Replaced EventLogger with Lombok's built-in log for consistency and simplicity. Updated all relevant logging calls and removed the unused EventLogger import. This change reduces dependency clutter and aligns with modern best practices. * refactor: remove unused loadAsResource method and related tests Removed the loadAsResource method from both the implementation and interface since it is no longer needed. Associated test cases and imports were also cleaned up to align with the updated functionality. * style: address checkstyle warnings in StorageService * refactor: remove unused loadAll method from storage services. The loadAll method and associated references were removed from FileSystemStorageService, StorageService, and related tests. This cleanup eliminates unnecessary code and improves maintainability by removing unused functionalities. * refactor: remove unused init method from storage services. The init method and associated references were removed from FileSystemStorageService, StorageService, and related tests. This cleanup eliminates unnecessary code and improves maintainability by removing unused functionalities. * refactor: remove unused deleteAll method from storage services. The deleteAll method and associated references were removed from FileSystemStorageService, StorageService, and related tests. This cleanup eliminates unnecessary code and improves maintainability by removing unused functionalities. * refactor: remove unused rootLocation property from FileSystemStorageService * style: address checkstyle violations in FileSystemStorageServiceTest * test: enable first FileSystemStorageServiceTest and upgrade to junit jupiter * test: convert to Mockito and update next test in FileSystemStorageServiceTest * test: update storeShouldRethrowDeleteException test in FileSystemStorageServiceTest * test: update storeShouldRethrowCopyException test in FileSystemStorageServiceTest * test: correct Mockito usage * refactor: align param type with expectations of ImporterProcessor.publishZipFiles * refactor: file format detection and processing logic. Moved file format detection logic to `FileFormat` enum for better modularity. Simplified and enhanced file processing by removing redundant methods and combining file movement directly in `processDirectory`. Improved code readability and maintainability. * chore: add a system-agnostic FileTypeDetector for test purposes Introduce a `SystemAgnosticFileTypeDetector` to identify MIME types based on file extensions, such as `.gz` and `.zip`. This utility is designed exclusively for testing and does not depend on OS-specific mechanisms. * test: unit tests for FileFormat.detectFileFormat method Introduces test cases to verify the detection of GZIP, ZIP, and UNKNOWN file formats based on file extensions. Ensures the functionality behaves as expected using mocked file paths. * style: remove exception class qualifiers * style: address checkstyle violations in OdeFileUtilsTest * test: update OdeFileUtilsTest to use Mockito and cover actual behaviors * test: ImporterProcessorTest.testProcessDirectoryGZIP * test: ImporterProcessorTest.testProcessDirectoryWithNestedDirectories * test: ImporterProcessorTest.testProcessDirectoryPlainText * test: ImporterProcessorTest.testProcessDirectoryZip * test: ImporterProcessorTest.testProcessWithAllFileTypes * chore: add zip file for live local testing * test: update OdeFileUtilsTest test cases for better isolation * docs: add missing javadoc to ImporterDirectoryWatcher * style: checkstyle violations LogFileToAsn1CodecPublisherTest * refactor: decouple tests and use MockitoExtension in LogFileToAsn1CodecPublisherTest * chore: rename test cases for clarity in LogFileToAsn1CodecPublisherTest * docs: add note that Distress Notifications may not be supported * refactor: rename stringPublisher to template for consistency * docs: improve accuracy of LogFileToAsn1CodecPublisher javadocs * docs: add missing javadocs to LogFileType * test: define mock value for backupDir.toString() in OdeFileUtilsTest.backupFileShouldThrowExceptionUnableToMoveFile
* style: reformat Asn1EncodedDataRouter and Asn1CommandManager * chore: delete disabled tests from Asn1EncodedDataRouterTest to start from clean slate * refactor: extract nested logic to named methods * test: add asn1-encoder-output-tim.xml Generated by sending tim to deposit tim endpoint and grabbing value produced to the `topic.Asn1EncoderOutput` topic via kafka-ui * refactor: encapsulate date setting logic for code clarity * chore: rename asn1-encoder-output-tim.xml to asn1-encoder-output-unsigned-tim.xml * refactor: pull object creation up to allow mocking of dependencies * test: initial processSNMPDeposit test setup. encoder not consuming * test: processEncodedTimUnsecured_depositsToSdxTopic * test: processEncodedTimUnsecured_depositsToSdxTopic -> processEncodedTimUnsecured_depositsToSdxTopicAndTimTmcFiltered * refactor: pass asn1CommandManager into Asn1EncodedDataRouter to improve DI and testability * refactor: introduce ISecurityServicesClient and SecurityServicesClientImpl These can be used to wrap the calls to security services and allow easier mocking of responses. It also allows us to better shield ourselves from API contract changes. users of the client don't need to be aware of changes to the external API as all interactions can be encapsulated within the client * refactor: use ISecurityServicesClient in Asn1EncodedDataRouter This moves the calls to security services from Asn1CommandManager to SecurityServicesClientImpl which allows for more testability, flexibility (mocking client calls), and modularity * test: processSNMPDepositOnly * style: update license header to not be javadoc * test: update tests in Asn1EncodedDataRouterTest to enable running together (make data dynamic) * test: processEncodedTimUnsecured in Asn1EncodedDataRouterTest * test: make consumer groups unique between tests to keep them isolated * test: tweak securityServices settings to get code to run through processSignedMessage * test: disable all Asn1CommandManagerTest tests as they don't work with refactor nor do they confirm behavior * refactor: move asn1CommandManager.depositToSdw to Asn1EncodedDataRouter The responsibility to submit data to topics is already contained within Asn1EncodedDataRouter, and Asn1CommandManager is not responsible for any kafka interactions otherwise, so the responsibility was moved to Asn1EncodedDataRouter * style: added missing Javadocs and reorganized Asn1CommandManager * refactor: don't swallow exception on startup of rsuDepositor in Asn1CommandManager * refactor: ResponseEvent to use generic Address type Updated method signatures and instance usage to specify ResponseEvent with Address as the generic type. This change improves type safety and aligns with best practices for clearer code readability and maintainability. * style: reformat and add missing Javadocs SnmpSession * chore: delete unused AsnCodecRouterServiceController.java * refactor: pull Asn1CommandManager functionality into Asn1EncodedDataRouter * refactor: replace SDXDepositorTopics.java reference with Value reference for topic * refactor: introduce Config beans for Asn1EncodedDataRouter dependencies * refactor: convert Asn1EncodedDataRouter to KafkaListener and update tests - not working well with other tests * Revert "refactor: convert Asn1EncodedDataRouter to KafkaListener and update tests - not working well with other tests" This reverts commit 1c0e260. * Revert "refactor: introduce Config beans for Asn1EncodedDataRouter dependencies" This reverts commit da1b379. * Revert "refactor: replace SDXDepositorTopics.java reference with Value reference for topic" This reverts commit cdb14a2. * Revert "refactor: pull Asn1CommandManager functionality into Asn1EncodedDataRouter" This reverts commit c99bbe8. * Revert "chore: delete unused AsnCodecRouterServiceController.java" This reverts commit 24fbba7. * Revert "style: reformat and add missing Javadocs SnmpSession" This reverts commit 79e0e0c. * Revert "refactor: ResponseEvent to use generic Address type" This reverts commit c1f1178. * Revert "refactor: don't swallow exception on startup of rsuDepositor in Asn1CommandManager" This reverts commit b4473a3. * Revert "style: added missing Javadocs and reorganized Asn1CommandManager" This reverts commit ab2a633. * Revert "refactor: move asn1CommandManager.depositToSdw to Asn1EncodedDataRouter" This reverts commit 7e69562. * test: adjust tests to expect different streamId when message is signed * refactor: move asn1CommandManager.depositToSdw to Asn1EncodedDataRouter The responsibility to submit data to topics is already contained within Asn1EncodedDataRouter, and Asn1CommandManager is not responsible for any kafka interactions otherwise, so the responsibility was moved to Asn1EncodedDataRouter * style: added missing Javadocs and reorganized Asn1CommandManager * refactor: don't swallow exception on startup of rsuDepositor in Asn1CommandManager * refactor: ResponseEvent to use generic Address type Updated method signatures and instance usage to specify ResponseEvent with Address as the generic type. This change improves type safety and aligns with best practices for clearer code readability and maintainability. * style: reformat and add missing Javadocs SnmpSession * chore: delete unused AsnCodecRouterServiceController.java * refactor: pull Asn1CommandManager functionality into Asn1EncodedDataRouter * refactor: replace SDXDepositorTopics.java reference with Value reference for topic * refactor: introduce Config beans for Asn1EncodedDataRouter dependencies * refactor: convert Asn1EncodedDataRouter to KafkaListener and update tests - not working well with other tests * chore: delete now unused Asn1CommandManager.java * refactor: isRunning now only checks for RUNNING in OdeTimJsonTopology This is to allow test code to await valid RUNNING state so that the tests complete as expected * test: Asn1EncodedDataRouterTest await running state of odeTimJsonTopology If the k-stream in odeTimJsonTopology was in any state other than RUNNING, we would not be able to query the k-table, and the tests would fail. We wouldn't be able to produce to the filtered topic because the code would throw an exception and skip the message production * chore: use one mock ISecurityServicesClient for the test suite * chore: properly cleanup OdeTimJsonTopology on shutdown * chore: set destroyMethod for odeTimJsonTopology bean * chore: more gracefully handle/prevent errors when adding topics to EmbeddedKafkaHolder * chore: add debug message when skipping depositing to filtered tim topic * chore: use a test configuration bean to manage OdeTimJsonTopology streams lifecycle during tests * refactor: move TestKafkaStreamsConfig to own file for reusability * chore: reorganize Asn1EncodedDataRouterTest * test: delete OdeTimJsonTopologyTest.java as it is indirectly tested (better) during the Asn1EncodedDataRouterTest suite * refactor: undo unnecessary retries in OdeTimJsonTopology.query * refactor: swap KafkaTemplate for defaultStringMessageProducer in Asn1EncodedDataRouter * style: add Javadocs and remove unused parameters from constructor in Asn1EncodedDataRouter * refactor: remove redundant EventLogger usage * refactor: remove dead code from Asn1EncodedDataRouter.listen * style: rename variables for better clarity in Asn1EncodedDataRouter * chore: replace OdeKafkaProperties loading in Asn1EncodedDataRouterTest to support KafkaProducerConfig loading * refactor: untangle message signing from message production * chore: add explicit imports to KafkaProducerConfig to ensure correct context loading * chore: add explicit load of OdeKafkaProperties in RawEncodedPSMJsonRouterTest * chore: reorder constructor params for easier reading Asn1EncodedDataRouter * refactor: take all internal methods private in Asn1EncodedDataRouter * refactor: move Asn1EncodedDataRouter to kafka.listeners.asn1 package * chore: remove explicit imports. they cause errors with resolution * chore: correct topic in Asn1EncodedDataRouter.listen annotation * chore: remove unused stop method from OdeTimJsonTopology. When using it as the `destroyMethod` in KafkaStreamsConfig it was causing bean lifecycle errors with every other managed bean. Not sure why this happens, but we should be safe to continue without a destroy method as it didn't exist prior to this changeset * chore: include id in KafkaListener annotation on Asn1EncodedDataRouter.listen * refactor: extract streamId from message body to use as key when producing to topic.RawEncodedTIMJson This is needed so that the OdeTimJsonTopology can look up the object by the streamId and subsequently publish the JSON to topic.OdeTimJson * style: reformat files edited in previous commit * chore: add SerializationConfig to test class dependencies SerializationConfig was imported and added to the test configuration for Asn1EncodedDataRouterTest. * test: encapsulate duplicate setup logic in Asn1EncodedDataRouterTest Simplifies resource loading with reusable methods for reduced redundancy. Refactors test setup logic by extracting reusable utilities for creating consumers and listener containers. These changes enhance code readability, maintainability, and test efficiency. * test: streamline resource loading Consolidated test resource lookup into a single method with a standardized path structure. Updated references in tests to use this method and moved resources under a more relevant package structure. This change improves maintainability and aligns resources with the corresponding test classes. * chore: use correct key deserializer in Receiver tests where consuming tim messages with streamIds as keys * style: correct naming of streamId variable * style: add missing Javadocs to RsuDepositorConfig * style: add missing Javadocs and reformat RsuDepositor and test * style: add missing Javadocs to TestKafkaStreamsConfig * refactor: use ObjectMapper bean in Asn1EncodedDataRouter where possible * refactor: renamed and reorganized processUnsignedMessage path in Asn1EncodedDataRouter * refactor: deduplicate extraction of metadata and payload JSONObjects * refactor: error handling and use enum for start flag Updated error handling to clarify null assignments in logs and added "SupportedMessageType" enum to replace the hardcoded start flag "001F". These changes improve log clarity and code maintainability. * refactor: add XmlMapper for XML handling Refactored the ASD packaging logic for clarity and consistency, reducing redundancy and improving maintainability. Introduced `XmlMapper` to streamline XML processing and decoupled responsibilities such as RSU handling and second-encoding publishing. Updated tests to include the new `XmlMapper` dependency, ensuring comprehensive coverage of changes. * revert: undo move of resources to different package GitHub doesn't do well with changed files *also* getting moved. It tracks them as deleted files and new files, so reviewing is difficult. I will follow up in a new PR with the moves of these files to the correct package * Revert "refactor: move Asn1EncodedDataRouter to kafka.listeners.asn1 package" This reverts commit 6578b37. * test: update resource package path after reverting package move * refactor: introduce SignatureResultModel Use a POJO data bag to encapsulate response data from ISecurityServicesClient.signMessage so that the logic is in one place, and it's clearer what to expect from the API call. * refactor: rename SecurityServicesClientImpl.java to SecurityServicesClient * refactor: SecurityServicesClient and add unit test coverage Refactored the SecurityServicesClient to use dependency-injected RestTemplate and improved the request payload structure with a dedicated SignatureRequestModel class. Added unit tests for signMessage functionality and introduced configuration for RestTemplate with WebClientConfig. * chore: remove SecurityServicesConfig and related dependencies. * chore: correct import paths in Asn1EncodedDataRouter and test * test: handle null responses and improve error handling in signing. Added `RestClientException` to `signMessage` for better error handling. Implemented a new test case to verify behavior when the signing service returns a null response. * refactor: use autowired props to drive URL in SecurityServicesClientTest * refactor: use MockSecurityServicesClient in Asn1EncodedDataRouterTest * refactor: service request extraction to use metadata JSON. Simplified `getServiceRequest` method by directly passing the metadata JSON object instead of the full consumed object. Updated relevant calls to align with this change, improving clarity and reducing redundant operations. * docs: simplify comments and remove redundant JavaDoc. Removed verbose and redundant JavaDoc comments, replacing them with concise inline comments to improve code readability. This change ensures that key processing logic remains documented without unnecessary detail duplication. * docs: remove dead comments * chore: simplify RSU message handling logic. Removed redundant null checks before calling `sendToRsus` in the main logic. The null checks are properly handled within the `sendToRsus` method, ensuring clearer and more maintainable code. * chore: simplify error message for SDX deposit failure * refactor: don't use TimTransmorgrifier to buildEncodings Standardizing on the ObjectMapper provided in the app context will reduce variability and decrease risk of unintended differences in serdes behavior * chore: rename buildJsonTimFromPacket to buildTimFromPacket This name better represents the behavior of the method. The Javadocs were updated to be accurate as well * refactor: don't use KafkaListenerConfig in Asn1EncodedDataRouterTest for Kafka container setup Replaced ConcurrentMessageListenerContainer with KafkaMessageListenerContainer and removed KafkaConsumerConfig dependency. This fixes an issue where we would intermittently encounter the following error when running tests: java.lang.IllegalStateException: Expected 1 but got 0 partitions at org.springframework.kafka.test.utils.ContainerTestUtils.waitForAssignment(ContainerTestUtils.java:85) * fix: incorrect expiration date calculation in TIM processing. Updated the expiration date calculation to use `Instant` and ensure accuracy by handling milliseconds properly. Adjusted test cases and mock classes to align with the updated logic. Fixed incorrect test data to reflect the corrected expiration date format. * style: reorganize methods in Asn1EncodedDataRouterTest * fix: replace outdated Date handling to remove system time variability Replaced the outdated `SimpleDateFormat` with the modern `DateTimeFormatter` to handle date formatting and parsing. This improves thread safety and code readability while ensuring alignment with Java's modern date/time APIs. Adjusted test resources to reflect updated timestamp formatting logic. * fix: handle RestClientException during TIM signing process Add exception handling to catch RestClientException when signing TIM messages. Log detailed error messages, including specific handling for HttpClientErrorException.NotFound, to provide better debugging information and highlight potential misconfiguration of jpo-security-svcs. * chore: correct TIM header processing in Asn1EncodedDataRouter Convert TIM start flag to uppercase for consistency and improve logging to dynamically include the expected start flag value. This enhances clarity and ensures robust matching during header stripping. * refactor: TIM signing and certification expiration logic. Consolidated signing logic into `depositToTimCertExpirationTopic` and removed redundant `signTimWithExpiration` method. Added utility for obtaining hex-encoded signed messages in `SignatureResultModel` to streamline encoding operations. * refactor: TIM signing exception handling If we fail to sign the message, we should fail to process the message further. We don't want any unsigned messages in our system (if signing is enabled) * test: add better test failure messaging * test: add better test failure messaging * test: add better test failure messaging (actually stringify iterable) * test: simplify testing assertions for better failure reporting * test: update expected-asn1-encoded-router-snmp-deposit.xml with schemaVersion 8 * fix: correctly handle data signing flow and log clear exception messaging * test: add unit test for handling server error in signMessage * chore: set scope to provided for annotation processors * chore: add mock sec service to docker-compose.yml and sample.env * fix: add error handling for failed decoding * fix: deserialize sec srvcs response from valid json * chore: pull down fix for valid json from security services * chore: correct paths to jpo-security-svcs in docker-compose.yml * docs: update javadoc in RsuDepositorConfig for clarity * chore: Set jpo-security-svcs version to cdot-cv dev branch Updated the version in jpo-security-svcs to align with the cdot-cv development branch. * docs: update accuracy of javadoc for SnmpSession.get * docs: update accuracy of javadoc for SnmpSession.get * docs: update accuracy of javadoc for SnmpSession.get * docs: update accuracy of logs in Asn1EncodedDataRouter * chore: update accuracy of comments in Asn1EncodedDataRouter.stripHeader * refactor: ASN.1 message processing method names. Renamed methods and variables for clarity in message encoding workflows. Updated references in test cases and adjusted test resources to align with naming changes. This improves code readability and better reflects the encoding stages. * docs:correct javadocs in LogFileToAsn1CodecPublisher * refactor: remove unnecessary processSingleEncodedTim code path from Asn1EncodedDataRouter * chore: delete ghost file post merge AsnCodecRouterServiceController.java * refactor: introduce SDXDeposit POJO and use in Asn1EncodedDataRouter * chore: remove inaccurate code comment * chore: correct javadoc comment to indicate TIM may be signed or unsigned * chore: remove unnecessary SNMP flow from Asn1EncodedDataRouter.packageSignedTimIntoAsd * Revert "refactor: remove unnecessary processSingleEncodedTim code path from Asn1EncodedDataRouter" This reverts commit a00ab06. * refactor: support Asn1EncodedDataRouter.processUnsignedTimDeposit flow * refactor: remove addition of ASD element in TimDepositController.depositTim when dataSigningEnabledSDW is false Eliminated processing logic and tests related to unsigned TIM messages, as well as the conditional ASD element addition when dataSigningEnabledSDW is false. This streamlines the codebase by ensuring all TIM messages follow the secured flow, improving maintainability and alignment with production recommendations. * refactor: update Asn1EncodedDataRouter rsus handling Added/replaced logic to handle RSU XML-to-JSON array normalization in `processRsusIfPresent`. Refactored metadata handling in `processUnsignedMessage` and enhanced test cases to include RSU-related input/output scenarios. This ensures proper handling of RSU data while improving test robustness. * test: improve test isolation in Asn1EncodedDataRouterTest * Update sample.env Co-authored-by: Michael7371 <40476797+Michael7371@users.noreply.github.com> --------- Co-authored-by: Michael7371 <40476797+Michael7371@users.noreply.github.com>
* chore: add offset to error message for easier kafka message discovery * chore: swap to offset in error message for easier kafka message discovery * refactor: manage UDP receiver lifecycles with executors Replaced DefaultManagedTaskExecutor with single-thread executors for better lifecycle control of UDP receivers. Added graceful shutdown for executors and receivers to ensure proper cleanup during application shutdown. This enhances resource management and error logging. * chore: remove unnecessary receivers shutdown code
Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com>
Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com>
…nt-for-release Update compatibility table with expected submodule versions for 2025 Q2 release
Update project version to 4.1.0
Update scm tag in pom.xml
Signed-off-by: dmccoystephenson <dmccoystephenson@gmail.com>
Prepare release notes for 2025 Q2 release
dmccoystephenson
requested review from
Michael7371,
SaikrishnaBairamoni,
dan-du-car and
drewjj
Deescalate the error log since it is excessive and well handled already
Document jpo-asn-pojos release process
drewjj
previously approved these changes
May 6, 2025
Collaborator
drewjj
left a comment
drewjj
left a comment
There was a problem hiding this comment.
I have completed local testing of this last night. Looks like everything is functioning from the UDP endpoints to the /tim REST endpoint. Looks like this is good to go.
…icts-on-usdot-pr-574 # Conflicts: # jpo-ode-plugins/src/main/java/us/dot/its/jpo/ode/plugin/j2735/builders/TravelerMessageFromHumanToAsnConverter.java # jpo-ode-svcs/src/test/java/us/dot/its/jpo/ode/traveler/TimDepositControllerTest.java
…-pr-574 Resolve merge conflicts on USDOT PR 574
dan-du-car
approved these changes
May 8, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Details
Description
Problem
Solution
Asn1EncodedDataRouter(CDOT PR 138) andImporterProcessor(CDOT PR 174); update diagrams (CDOT PR 181).message.countcounter (CDOT PR 182).Related Issue
No related GitHub issues.
Motivation and Context
This work builds on previous Spring Kafka migration efforts by completing key component updates, cleaning up unused code, and improving packaging. It also refreshes documentation for better traceability and introduces basic message metrics. These updates aim to reduce maintenance overhead, improve reliability, and support ongoing development.
How Has This Been Tested?
message.countvisible in dashboards.Types of changes
Checklist:
ODE Contributing Guide