feat: Entity Resolution Service v1.1.0#14
Merged
Conversation
…is 0 Both ERS_COORDINATOR_SINGLE_REQUEST_TIME_BUDGET and ERS_COORDINATOR_BULK_REQUEST_TIME_BUDGET now accept 0 as a valid value. When single budget is 0, resolve_single skips ERE submission entirely and issues a provisional identifier without any Redis interaction. When bulk budget is 0, resolve_bulk removes the outer asyncio.wait_for timeout and lets the gather run untimed.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Python loggers are process-wide singletons; setting propagate=False in create_app() permanently broke pytest caplog for all ers.* loggers across the test session. Records still reach uvicorn handlers via the explicit addHandler call, so Docker log visibility is unaffected.
feat: add immediate provisional mode (ERS_COORDINATOR_*_TIME_BUDGET=0), fix logging bug
Replace the in-process waiter.notify() callback with a Redis Pub/Sub broadcast so all ERS instances receive ERE outcome notifications regardless of which instance wins the BRPOP race. Enables correct horizontal scaling. - Add ERS_NOTIFICATIONS_CHANNEL config property to RedisConfig - Add publish_notification() to RedisEREClient (reuses existing connection pool) - Add NotificationSubscriberWorker: asyncio background task, SUBSCRIBE loop, exponential backoff reconnect (1s-30s), subscribed asyncio.Event for readiness - Wire subscriber worker and updated on_outcome_stored lambda in app.py lifespan - Unit, integration, and BDD tests for all new behaviour
NotificationSubscriberWorker was creating its aioredis.Redis connection without a socket connect timeout, leaving the subscribe call able to block indefinitely if Redis was slow or hung. - Add REDIS_SOCKET_CONNECT_TIMEOUT to RedisConfig (default 5.0 s) - Add socket_connect_timeout field to RedisConnectionConfig, populated via from_settings() - Forward it to aioredis.Redis() in NotificationSubscriberWorker.run() - Add config and worker unit tests - Update configuration.md (new entry + fix ERS_NOTIFICATIONS_CHANNEL description: Pub/Sub channel, not a list key)
…-instance notifications Merge the two separate interaction diagrams (same-instance and cross-instance BRPOP wins) into a single diagram with an alt/else block, add prose descriptions of both cases above it, and add debug-level log statements in app.py and notification_subscriber_worker.py to trace which path is taken at runtime.
…in entity type discoverability endpoint
Add ServiceUnavailableError subclassing ApplicationError for signalling unreachable backend services (MongoDB, Redis). Also add pythonpath to pytest.ini so the worktree src/ takes precedence over the shared editable install when running tests.
…tcomeIntegrationWorker Backoff starts at 1s, doubles on each consecutive ConnectionError, and is capped at 30s. It resets to 1s after any successful message receive.
Propagate the two config changes from src/config/rdf_mention_config.yaml into all test fixtures, factories and BDD step definitions: - ORGANISATION: add full_address (cccev:registeredAddress/locn:fullAddress) - PROCEDURE: update identifier path to adms:identifier/skos:notation - Add skos namespace to sample fixture (now 7 namespaces) Files updated: sample_rdf_mapping.yaml, factories.py, test_rdf_mapping_config_reader.py, test_rdf_mapping_config.py, test_mention_parser_service.py, parser_configuration.feature, rdf_parsing.feature and their step definitions, ers_rest_api conftest stub.
feat: add black-box ERSys test suite migrated from er-ops
…s-config fix(config): rename ERE_REQUEST/RESPONSE_CHANNEL env vars to ERSYS_ prefix
fix(config): update rdf_mention_config and align test fixtures with real TED data
Pin coordinator config to deterministic values in unit tests via a `default_config` fixture instead of relying on whatever the host .env provides, and remove the env-var coupling from the budget-default tests by unsetting the keys before resolving the descriptor. Fix the integration coordinator fixture and test_it005_concurrent_identical_requests to keep the patch(...) context active for the entire test body — the previous form exited the `with` block before `resolve_single` ran, so the service read the unpatched live config at runtime.
chore: regenerate openapi schema file
test: decouple resolution coordinator tests from .env
…rsions chore: pin dependency versions to exact values
…-index docs: add missing ERS_SUBSCRIBER_READY_TIMEOUT to configuration reference
Chore/docs and demo updates
…nual docs: add ERSys installation guide (ERS1-225)
test: time performance test scripts
chore: pre-release updates for v1.1.0
costezki
approved these changes
May 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Sync
meaningfy-ws/develop→OP-TED/developfor the v1.1.0 release of the Entity Resolution Service.What's new in 1.1.0
Added
ERS_COORDINATOR_*_TIME_BUDGET=0) — returns a provisional identifier without waiting for EREREDIS_TLS)ServiceUnavailableErroron MongoDB and Redis infrastructure outages across all endpointsentity_label_field); returned by/entity-typesOutcomeIntegrationWorkerreconnect on Redis failuresERS_SUBSCRIBER_READY_TIMEOUTstartup readiness gateChanged
ERE_REQUEST_CHANNEL/ERE_RESPONSE_CHANNELrenamed toERSYS_REQUEST_CHANNEL/ERSYS_RESPONSE_CHANNEL— update.envfiles accordinglydisplay_name_fieldrenamed toentity_label_fieldin entity-type configurationerrortomessage;SERVICE_UNAVAILABLEadded to all error enumsPOST /refreshBulkreturns only decisions whose cluster placement changedFixed
stop(),aclose()guaranteed on cancel, malformed Pub/Sub payloads skippedTimeoutErrorwrapping inpublish_notificationfull_addressfield for real TED data