Skip to content

feature: ATL-51: cache haplotype fetching#1443

Open
SergeySalinAtDataart wants to merge 4 commits intomasterfrom
feature-atl-51-fix-haplotype-fetching
Open

feature: ATL-51: cache haplotype fetching#1443
SergeySalinAtDataart wants to merge 4 commits intomasterfrom
feature-atl-51-fix-haplotype-fetching

Conversation

@SergeySalinAtDataart
Copy link
Copy Markdown
Contributor

@SergeySalinAtDataart SergeySalinAtDataart commented May 4, 2026

This change is Reviewable

@SergeySalinAtDataart SergeySalinAtDataart requested a review from a team as a code owner May 4, 2026 10:08
@SergeySalinAtDataart SergeySalinAtDataart force-pushed the feature-atl-51-fix-haplotype-fetching branch 2 times, most recently from 760c0e7 to a1c7347 Compare May 4, 2026 13:51
Copy link
Copy Markdown
Contributor

@ikogan-da ikogan-da left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds caching for active haplotype-frequency-set lookups in the Match Prediction stack and introduces stronger options binding/validation for several settings classes. It fits into the Match Prediction services by reducing repeated database reads for active set selection and wiring the new cache TTL through app configuration.

Changes:

  • Added HaplotypeFrequencySetCache settings, Terraform wiring, and docs for configuring the active-set cache TTL.
  • Updated HaplotypeFrequencyService to cache all active HF sets and invalidate that cache after imports.
  • Added/expanded settings validation annotations and worker-side validated options registration, plus unit/integration test config updates.

Reviewed changes

Copilot reviewed 30 out of 30 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
terraform/core/variables.tf Adds Terraform variable for active HF-set cache expiry.
terraform/core/modules/match_prediction/variables_release.tf Exposes cache expiry variable to the match prediction module.
terraform/core/modules/match_prediction/function.tf Passes cache expiry into Match Prediction Function app settings.
terraform/core/main.tf Wires root Terraform variable into the module input.
README_Configuration.md Documents the new configuration setting.
Atlas.MultipleAlleleCodeDictionary/Settings/MacDictionarySettings.cs Adds data-annotation validation to MAC settings.
Atlas.MatchPrediction/Services/HaplotypeFrequencies/HaplotypeFrequencyService.cs Implements active-set caching and cache invalidation on import.
Atlas.MatchPrediction/ExternalInterface/Settings/MessagingServiceBusSettings.cs Adds validation attributes to messaging settings.
Atlas.MatchPrediction/ExternalInterface/Settings/MatchPredictionRequestsSettings.cs Adds validation attributes to request topic settings.
Atlas.MatchPrediction/ExternalInterface/Settings/HaplotypeFrequencySetCacheSettings.cs Introduces cache TTL settings type.
Atlas.MatchPrediction/ExternalInterface/Settings/AzureStorageSettings.cs Adds validation attributes to storage settings.
Atlas.MatchPrediction.Worker/Startup.cs Switches worker startup to validated options registration.
Atlas.MatchPrediction.Worker/Settings/SettingsValidationExtensions.cs Adds worker options binding/validation helpers and validators.
Atlas.MatchPrediction.Worker/Settings/MatchPredictionWorkerSettings.cs Adds validation attributes for worker settings.
Atlas.MatchPrediction.Worker/Program.cs Passes configuration into startup.
Atlas.MatchPrediction.Worker/appsettings.json Adds cache settings and validated worker config values.
Atlas.MatchPrediction.Test/Services/HaplotypeFrequencies/HaplotypeFrequencyServiceTests.cs Adds unit tests for caching and invalidation behavior.
Atlas.MatchPrediction.Test.Verification/Startup.cs Registers new cache settings in verification tests.
Atlas.MatchPrediction.Test.Verification/local.settings.template.json Adds cache setting to verification local config.
Atlas.MatchPrediction.Test.Validation/Startup.cs Registers new cache settings in validation tests.
Atlas.MatchPrediction.Test.Validation/local.settings.template.json Adds cache setting to validation local config.
Atlas.MatchPrediction.Test.Integration/DependencyInjection/ServiceConfiguration.cs Registers cache settings for integration tests.
Atlas.MatchPrediction.Test.Integration/appsettings.json Adds cache setting to integration config.
Atlas.MatchPrediction.Functions/Startup.cs Registers cache settings in Match Prediction Functions host.
Atlas.MatchPrediction.Functions/local.settings.template.json Adds cache setting to local function config.
Atlas.MatchPrediction.Data/Repositories/HaplotypeFrequencySetRepository.cs Adds query to fetch all active sets.
Atlas.HlaMetadataDictionary/ExternalInterface/Settings/SearchRelatedMetadataServiceSettings.cs Adds validation attribute to nested metadata cache setting.
Atlas.HlaMetadataDictionary/ExternalInterface/Settings/HlaMetadataDictionarySettings.cs Adds validation attributes to dictionary settings.
Atlas.Functions/Startup.cs Registers cache settings in top-level ATLAS-FUNCTIONS host.
Atlas.Common/Notifications/NotificationsServiceBusSettings.cs Adds validation attributes to notification settings.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@@ -90,6 +90,7 @@ private static void RegisterSettings(IServiceCollection services)

// Match Prediction Algorithm
services.RegisterAsOptions<MatchPrediction.ExternalInterface.Settings.AzureStorageSettings>("MatchPrediction:AzureStorage");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to add the setting to appsettings.json? And the terraform?

try
{
await frequencySetImporter.Import(file, importBehaviour);
cache.Remove(ActiveHaplotypeFrequencySetsCacheKey);
Copy link
Copy Markdown
Contributor

@daria-sorokina-da daria-sorokina-da left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@daria-sorokina-da reviewed 30 files and all commit messages, and made 1 comment.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on daria-sorokina-da and pchenery).

@@ -90,6 +90,7 @@ private static void RegisterSettings(IServiceCollection services)

// Match Prediction Algorithm
services.RegisterAsOptions<MatchPrediction.ExternalInterface.Settings.AzureStorageSettings>("MatchPrediction:AzureStorage");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to add the setting to appsettings.json? And the terraform?

@SergeySalinAtDataart SergeySalinAtDataart force-pushed the feature-atl-51-fix-haplotype-fetching branch from a1c7347 to 52606b5 Compare May 6, 2026 08:44
Copy link
Copy Markdown
Contributor

@daria-sorokina-da daria-sorokina-da left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@daria-sorokina-da reviewed 3 files and all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on pchenery).

@SergeySalinAtDataart SergeySalinAtDataart force-pushed the feature-atl-51-fix-haplotype-fetching branch from 52606b5 to d2dff37 Compare May 6, 2026 11:08
Copy link
Copy Markdown
Contributor

@daria-sorokina-da daria-sorokina-da left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@daria-sorokina-da reviewed 2 files and all commit messages, and resolved 2 discussions.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on pchenery).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants