Implement DL-5: ProposedChange CRUD operations in MongoDB

[Copilot]

Adds MongoDB-backed staging layer for canonical changes. Any agent can propose changes (facts, entities, relationships, state changes, events); only CanonKeeper can accept/reject them.

Core Operations

• mongodb_create_proposed_change: Creates pending proposals with flexible JSON content, evidence references, and metadata
• mongodb_get_proposed_change: Retrieves full proposal including evidence
• mongodb_list_proposed_changes: Supports filtering by scene_id, story_id, status, change_type, universe_id
• mongodb_update_proposed_change: Atomic status transitions (pending → accepted/rejected) with validation

Key Implementation Details

Thread-safe singleton MongoDB client with double-check locking:

_mongodb_client_lock = threading.Lock()

def get_mongodb_client() -> MongoDBClient:
    global _mongodb_client
    if _mongodb_client is None:
        with _mongodb_client_lock:
            if _mongodb_client is None:
                _mongodb_client = MongoDBClient()
                _mongodb_client.connect()
    return _mongodb_client

Atomic status updates prevent race conditions:

# Only updates if status is still PENDING
updated_doc = collection.find_one_and_update(
    {
        "proposal_id": str(proposal_id),
        "status": ProposalStatus.PENDING.value,
    },
    {"$set": {/* update fields */}},
    return_document=ReturnDocument.AFTER,
)

Status Transition Validation

• Only pending → accepted or pending → rejected allowed
• Accepting requires canonical_ref (UUID of created Neo4j node/edge)
• Attempting to update non-pending proposals returns clear error with current status

Authority Matrix

• Create/read/list: ["*"] (all agents)
• Update: ["CanonKeeper"] (status transitions only)

Files Added

• packages/data-layer/src/monitor_data/db/mongodb.py - Client with connection pooling
• packages/data-layer/src/monitor_data/schemas/proposed_changes.py - Pydantic schemas
• packages/data-layer/src/monitor_data/tools/mongodb_tools.py - CRUD operations
• packages/data-layer/tests/test_tools/test_proposed_change_tools.py - 15 unit tests (95% coverage)

Example Usage

# Agent proposes a new fact
proposal = mongodb_create_proposed_change(ProposedChangeCreate(
    change_type=ProposalType.FACT,
    content={"statement": "The dragon breathes fire", "entity_ids": [dragon_id]},
    scene_id=scene_id,
    universe_id=universe_id,
    confidence=0.9,
    proposed_by="Narrator"
))
# status = PENDING

# CanonKeeper evaluates and accepts
result = mongodb_update_proposed_change(
    proposal.proposal_id,
    ProposedChangeUpdate(
        status=ProposalStatus.ACCEPTED,
        canonical_ref=neo4j_fact_id,  # Required for accepted proposals
        decision_reason="Consistent with scene events"
    )
)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>DL-5: Manage Proposed Changes</issue_title>
<issue_description>Category: data-layer | Epic: 0 | Priority: high

Summary

Implement CRUD operations for ProposedChange documents in MongoDB. These are
staging documents for canonical changes that CanonKeeper evaluates at scene end.
Supports different change types (fact, entity, relationship, state_change, event)
and status transitions (pending → accepted/rejected).

Acceptance Criteria

• mongodb_create_proposed_change creates ProposedChange document
• mongodb_create_proposed_change validates change_type enum
• mongodb_create_proposed_change stores content payload as flexible JSON
• mongodb_create_proposed_change links to scene_id or story_id
• mongodb_get_proposed_change returns full document with evidence
• mongodb_list_proposed_changes supports filtering by status, change_type, scene_id
• mongodb_update_proposed_change allows status transitions
• mongodb_update_proposed_change validates status transitions (pending only → accepted/rejected)
• mongodb_update_proposed_change records decision metadata (decided_by, decided_at, reason)
• Accepted changes include reference to created canonical entity
• All operations track timestamps and proposer identity
• Unit tests achieve >= 80% coverage

Dependencies

This use case depends on:

• DL-4

Blocks

This use case blocks:

• P-3
• CF-3

Implementation

Layer: 1

Files to create:

• packages/data-layer/src/monitor_data/schemas/proposed_changes.py
• packages/data-layer/tests/test_tools/test_proposed_change_tools.py
  Files to modify:
• packages/data-layer/src/monitor_data/tools/mongodb_tools.py
• packages/data-layer/src/monitor_data/middleware/auth.py

MONGODB Operations:

• mongodb_create_proposed_change (authority: *)
• mongodb_get_proposed_change (authority: *)
• mongodb_list_proposed_changes (authority: *)
• mongodb_update_proposed_change (authority: CanonKeeper)

Notes:

• Any agent can propose changes (authority: *)
• Only CanonKeeper can accept/reject (update status)
• Accepted changes should include canonical_ref pointing to created node
• Consider indexing by scene_id for efficient batch retrieval

Testing Requirements

Minimum coverage: 80%

Unit tests:

• test_create_proposed_change_fact: change_type=fact → valid doc
• test_create_proposed_change_entity: change_type=entity → valid doc
• test_create_proposed_change_relationship: change_type=relationship → valid doc
• test_list_by_scene: scene_id filter returns relevant proposals
• test_list_by_status: pending filter works
• ... and 3 more

Integration tests:

• test_proposal_lifecycle: create → list → accept → verify canonical created
• test_batch_canonization: multiple proposals → batch accept/reject

References

Documentation:

• docs/ontology/ONTOLOGY.md#27-proposed-changes
• docs/architecture/AGENT_ORCHESTRATION.md
  Code:
• packages/data-layer/src/monitor_data/db/mongodb.py

---

Generated from /home/sebas/monitor2/docs/use-cases/data-layer/DL-5.yml</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes DL-5: Manage Proposed Changes #40

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[spuentesp]

Closing stale draft PR - conflicts with current master. Will regenerate from current state when ready to implement.