Skip to content

Sarah/add notes metadata - #169

Draft
skeating wants to merge 146 commits into
test_branch_zellafrom
sarah/add_notes_metadata
Draft

Sarah/add notes metadata#169
skeating wants to merge 146 commits into
test_branch_zellafrom
sarah/add_notes_metadata

Conversation

@skeating

Copy link
Copy Markdown
Contributor

This request adds a new table to this star database Notes Metadata which will contain times when notes were added to a patient and and what the type of note was.

skeating and others added 30 commits September 3, 2025 15:25
Adds the NotesMetadata entity to represent notes recorded for patients, including fields for hospital visit, note type, and editor role.

Creates a NotesMetadataRepository for interacting with the NotesMetadata table, providing basic CRUD operations.
Adds a repository for interacting with the NotesMetadataAudit table.
Includes a method for testing purposes to retrieve all notes metadata audits
for a given hospital visit ID.

Adds test method to find all notes by encounter.

Renames index for notes metadata table.
Temporarily disables the creation of MRN and hospital visit entities within the NotesMetadataProcessor. This is done to isolate and resolve issues in the note metadata processing pipeline. The related factory class has been added.
Implements processing of notes metadata messages (MDM_T01).
Adds a factory for building notes metadata messages from HL7.
Adds a test helper method for loading notes metadata from files.
Also, configures checkstyle to allow more parameters for method definitions.
Refactors NotesMetadataFactory to use a NotesMetadataHl7 object for parsing note metadata from HL7 messages.

This change simplifies the NotesMetadataFactory class by encapsulating the parsing logic within the new NotesMetadataHl7 class, improving code readability and maintainability. It also addresses an error in the original EDIT_TIME instant in the test class.
If the notes segment (TXA) does not contain the originating time, defaults to using the message timestamp (MSH).

This ensures a timestamp is always available for notes, even when specific note timestamps are absent.
Adds functionality to parse notes metadata from MDM_T08 messages.

This involves extracting relevant information from the message segments
(MSH, PID, PV1, EVN, TXA) and mapping them to the NotesMetadataMessage
object. The start time is derived from the MSH segment if empty.

Includes a new test case to ensure that the T08 message is parsed correctly.
Adds the controller and repository for NotesMetadata.

This controller handles processing of NotesMetadata messages,
creating new NotesMetadata entities if they don't exist or updating
existing ones. It also includes methods for determining if a
message should be updated based on the last edit datetime.

The repository provides methods for finding NotesMetadata by
internal ID and hospital visit.
Corrects a typo in the parameter name of the `findByInternalId` method in the NotesMetadataRepository interface.

The parameter name was incorrectly spelled as `internalId` instead of `notesMetadatacdId`, leading to potential confusion or errors when using the method.
Renames the `notesMetadataId` field to `internalId` in the
`NotesMetadata` entity and related repository method to align naming
conventions across the project.
Implements the processing of NotesMetadata messages, including the creation of minimal entities
and updating existing ones if a newer version is available.

Introduces a new test case to verify the correct processing of
NotesMetadata messages, ensuring that minimal entities (HospitalVisit, Mrn,
NotesMetadata) are created when a new message arrives and no prior data exists.

Renames notesMetadataNumber to notesMetadataId in interchange to correspond to the database column name.
Possibly to do with me using java 23?
Ensures that the last edit datetime for notes metadata is always populated.
If the last edit datetime is not present in the TXA segment, it falls back to the message timestamp.
If the message timestamp is also empty it falls back to the stored from time.
Adds YAML files for testing Notes Metadata messages.

These files represent minimal valid metadata and metadata with a missing date field, providing a basis for comprehensive testing of the NotesMetadataMessage processing logic.
to setup a fanout exchange so waveform messages can go to Emap and Atrium
jeremyestein and others added 27 commits February 26, 2026 14:57
causes permission issues on the GAE anyway.
and remove other glowroot entry
increment of 50 and therefore should hit the server less.
references, thus causing Hibernate to issue unnecessary UPDATEs.
Baseline work to support performance improvements
Speed up emap-star waveform table writes by ~50%
Speed up Emap by increasing DB sequence increment
Speed up Emap by adding a 3-column index
Adds a note on access to the UCLH Rstudio server to the onboarding
document.
Add new unit tests to verify the HL7 parsing of Notes Metadata messages into `NotesMetadataMessage` objects. This includes tests for minimal, T08-specific minimal, and missing metadata scenarios.
- Adjust `notesMetadataId`, `noteType`, and `lastEditDatetime` in Notes Metadata test YAML files for improved test coverage and accuracy.
- Update `TestHL7ParsingMatchesInterchangeFactoryOutput` to correctly reference the "missing_date" test resource.
- Enable automatic null analysis for Java projects by adding a `.vscode/settings.json` file.
- Remove unused imports from `TestNotesMetadata.java`.
@github-actions

Copy link
Copy Markdown

PR checklist

Default guide for a PR (if multiple PRs for the work, only keep one version of it and link to it on the other PRs)

  • From the UCLH data science desktop, a validation run has been set off
  • Check that content that reveals Epic IP (eg. Clarity/Caboodle queries) has not been checked into a public repo.
  • load times
    in UCL teams has been populated with the run information
  • During the run, glowroot has been checked for any queries which are taking a substantial proportion of the
    total processing time. This can be useful to identify indexes that are required.
  • After the run, look for any unexpected errors in the etl_per_message_logging table, the error_search.sql file
    on the shared drive can be used for this \\sharefs6\UCLH6\EMAP\Shared\EmapSqlScripts\devops\error_search.sql.
    Create an issue if you find an unexpected exception and is not related to the changes you've made, otherwise
    fix them!
  • After the run, populate the end time in
    load times
  • Let Aasiyah know about the completed validation and give her information on the changes and where to start
    with the validation
  • Check validation report and give any feedback to Aasiyah if there are any changes needed on her side,
    iterate on getting the validation to match at least 99% (validation and emap code).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants