feat: complete ERS implementation (MVP)#3
Merged
Conversation
…or refactor - Use erspec.Decision directly; eliminate ResolutionDecisionRecord - MongoDecisionStoreRepository extends MongoDecisionRepository (parallel sibling to curation repo, same decisions collection) - Add Task 2: lift _build_cursor_condition/_parse_cursor_sort_value to commons base (tier boundary prevents importing from curation) - Renumber tasks 2-6 to 3-7; fix field names, examples, test table, Gherkin path
- domain/errors.py: DecisionStoreError hierarchy with 5 subclasses - DecisionStoreConfig mixin added to ERSConfigResolver in ers/__init__.py; no standalone domain/config.py — reuses MONGO_DATABASE_NAME - Unit tests: test_errors.py and test_config.py (moved to tests/unit/ root, isomorphic to src/ers/__init__.py)
_build_cursor_condition and _parse_cursor_sort_value in MongoDecisionCurationRepository are generic MongoDB cursor-seek utilities with no dependency on curation-specific types (DecisionFilters, DecisionOrdering). The new MongoDecisionStoreRepository (Tier 1) needs the same helpers, but importing from ers.curation (Tier 3) is forbidden by the import-linter contract. Lifting them to the commons base class is the only DRY solution that respects the tier boundary. MongoDecisionCurationRepository inherits them unchanged — no behaviour change. Regression tests added in tests/unit/curation/adapters/.
…_id (EPIC-04 Task 3) - adapters/provisional_id.py: derive_provisional_cluster_id via SHA256ContentHasher - adapters/decision_repository.py: MongoDecisionStoreRepository extending MongoDecisionRepository with upsert_decision, find_by_triad, query_paginated, ensure_indexes; cursor helpers inherited from commons base class - Unit tests for both (15 tests)
…ery (EPIC-04 Task 4)
- Cap CursorParams.limit to MAX_PER_PAGE; remove model_construct bypass - Add unit tests for find_mention_ids_by_cluster, count_distinct_clusters, average_cluster_size - Add span extractor registration smoke tests - Fix missing EOF newline in resolution_decision_store_service.py
…_store_service.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…onfig properties Co-authored-by: gkostkowski <12532923+gkostkowski@users.noreply.github.com> Agent-Logs-Url: https://github.com/meaningfy-ws/entity-resolution-service/sessions/5c8b173d-d6fb-44ef-befa-15dc6df182bd
fix: add lower-bound validation for DECISION_STORE page size config properties
- Correct Redis adapter from "Streams + Pub/Sub" to LPUSH/BRPOP (matches RedisEREClient) - Remove redundant domain models (OutcomeMessage, CorrelationTriad, ClusterAssignment) — erspec types cover all three - Document that timestamp is Optional[datetime] in erspec; service must validate non-null - Document that staleness check is handled atomically by MongoDecisionRepository, not the service - Add missing await on service.integrate_outcome() call in worker entrypoint - Mark ERS-EPIC-04 dependency as Complete
…ordination plan - EPIC-05: fix Redis terminology (Pub/Sub → list queue), add error class signatures, correct Gherkin field names, add on_outcome_stored callback to service constructor and algorithm, add start()/stop() to worker, add deployment constraints section, replace direct repo access with RequestRegistryService (service-layer boundary) - EPIC-06: replace ResolutionDecisionRecord with Decision (erspec), add EPIC-05 to dependencies, clarify JSONRepresentation as parsed_representation field, update triad_key format to direct concatenation, update Assumption 4 for callback pattern - EPIC-07: align models/services/routes with actual implementation (method names, exception handlers, DI pattern, route URLs, ResolutionOutcome enum), add lifespan section for EPIC-05/06 wiring, add architectural constraints 9-10, update stale next actions; add concerns.md (4 open decisions) and coordination-work.md (5 work items for EPIC-05/06 wiring once those EPICs are implemented)
Records the 4 issues found in the review: stub test suite still active, page-size capped at 50 overriding the configured 1000, vacuous truncation BDD scenario, and missing InvalidCursorError in domain/errors.py.
Adds the ere_result_integrator sub-module with all four layers: domain errors, AsyncOutcomeListener ABC, RedisOutcomeListener adapter, OutcomeIntegrationService (6-step algorithm), and OutcomeIntegrationWorker background task. Includes unit, integration, and BDD feature tests. Post-review fixes applied: worker now calls the traced module-level integrate_outcome() function; BDD step defs use asyncio.run() consistently; contract_validation.feature splits schema-layer from service-boundary rejections; span extractor unit tests added; importlinter comment clarified.
… task trackers for EPICs 06-07 Adds task outcome files for tasks 57 (integration tests), 58 (UC-B1.2 e2e wiring), and 59 (resilience gaps identified post-review). Adds new tasks for EPIC-05 covering resilience gaps discovered during the post-implementation review. Adds placeholder task trackers for UC-B1.1 (EPIC-06) and full-stack e2e wiring (EPIC-07). Updates EPIC-05 roadmap accordingly.
…e faults Address 5 resilience gaps identified in post-implementation review (Task 59): - Gap A: RedisOutcomeListener now catches ConnectionError (logs + re-raises to trigger worker restart) and TimeoutError (swallows, resumes polling). Worker wraps the consume loop in a restart loop with 5 s back-off on ConnectionError. - Gap B: OutcomeIntegrationService guards the on_outcome_stored callback in try/except so a failing coordinator notification never unwinds a committed Decision Store write. - Gap C: ValueError from pull_response (malformed/unknown-type payload) is caught in the listener, logged at ERROR, and skipped. - Gap D: Unknown response types (future erspec versions) now produce a WARNING log instead of being silently dropped. - Gap E: ConnectionError from the service layer (registry / decision store) is caught with a distinct infrastructure error message before the generic handler. Also moves the misplaced "Messaging publish failure" scenario from ucb12_integrate_ere_outcomes to ucb11_resolve_entity_mention (Spine A, where publish failures belong) and updates all four affected files. 14 new unit tests added; 869 tests pass.
…n seed script 1. factories.py: add `_organisation_payload()` and `_procedure_payload()` with faker-synthesised fields matching the RDF mapping config field names, plus `_payload_for(entity_type)` dispatcher and `build_for_entity_type()` on `ResolutionRequestRecordFactory` so callers can generate type-correct content and parsed_representation in one call. 2. seed_db.py: use `build_for_entity_type()` when seeding mentions so each record carries content matching its entity type; partition cluster references by entity type so ORGANISATION and PROCEDURE mentions are never mixed in the same cluster, and thread type-scoped cluster IDs through to candidate selection.
- factories.py: content() now delegates to _payload() instead of hardcoding _organisation_payload(), keeping content consistent with parsed_representation for subclasses - factories.py: build_for_entity_type() derives content_hash as the SHA-256 of the generated payload, matching the ResolutionRequestRecord contract - seed_db.py: _build_cluster_references() returns empty maps early when called with an empty mentions list, avoiding ZeroDivisionError
…ow reading rest of env vars
Moves the RDF mention mapping config from er-ops into config/ at the repo root. Updates the default config path, mounts the file in the dev compose setup, and documents the env var in .env.example.
chore: add rdf_mention_config.yaml to repo and wire it up
Co-authored-by: Meaningfy <hi@meaningfy.ws>
…d-data-gen feat: generate distinct entities for procedure type, fix cluster membership issue
Feat/observability
chore: update dependency on entity-resolution-spec repository and change test directory name
twicechild
approved these changes
Apr 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Full implementation of the service across four major areas, passing automatic and manual tests.
Resolution Decision Store (
resolution_decision_store): new service layer withDecisionStoreService, dedicatedDecisionRepositoryadapter, provisional IDgeneration, and span extractors. Replaces the previous stub implementation.
ERE Result Integrator (
ere_result_integrator): new submodule withOutcomeIntegrationService,RedisOutcomeListeneradapter, andOutcomeIntegrationWorkerentrypoint. Wires ERE resolution outcomes into thedecision store (UCB12).
Resolution Coordinator (
resolution_coordinator): major expansion — addedAsyncResolutionWaiter,BulkRefreshCoordinatorService, domain exceptions, andheavily extended
ResolutionCoordinatorServiceto cover the full bulk resolutionand lookup flows (UCB11).
ERS REST API (
ers_rest_api): updated resolve, lookup, and bulk-refreshservices; improved exception handling, health endpoint, and API dependencies.
Curation API (
curation):DecisionRepositorymoved tocommons; new/entity-typesendpoint; updated user-action and decision browsing flows.API documentation: generated AsciiDoc docs for ERS and Curation OpenAPI
schemas; added export scripts and AsciiDoc xref fix script.
Test coverage: comprehensive unit, feature (BDD), integration, and e2e tests
added or updated across all new and changed modules.