Skip to content

Fix race condition in topic/class description generation on concurrent uploads - #112

Merged
nilskntl merged 2 commits into
masterfrom
108-fix-race-condition-in-topicclass-description-generation-when-multiple-documents-upload-concurrently
Mar 21, 2026
Merged

nilskntl merged 2 commits into
masterfrom
108-fix-race-condition-in-topicclass-description-generation-when-multiple-documents-upload-concurrently

Conversation

@claude

@claude claude Bot commented Mar 21, 2026

Copy link
Copy Markdown

Summary

  • Added areAllDocumentsInTopicTerminal and areAllDocumentsInClassTerminal queries to DocumentRepository using Cypher all() predicate — returns true only when every document in scope has status COMPLETED or FAILED
  • Extracted triggerDescriptionGenerationIfAllTerminal(topicId, classId) in ConvertingResultConsumer: checks the terminal-state condition, then acquires a per-topic/per-class JobLock (atomic Neo4j MERGE) before calling generateTopicDescription / generateClassDescription, ensuring exactly one API instance runs each generation even when multiple document completions race
  • Called the new method asynchronously (fire-and-forget) from both processSuccessfulResult and processFailedResult after the document's terminal status is persisted, so the consumer loop is not blocked
  • Removed the unconditional generateTopicDescription / generateClassDescription calls from getSummarizationFuture (which fired once per completing document)
  • Removed now-unused topicRepository and classRepository fields

Issue

Closes #108

Generated with Claude Code

…t uploads

After each document reaches a terminal state (COMPLETED or FAILED), check whether
all other documents in the same topic are also terminal before triggering
generateTopicDescription and generateClassDescription. A distributed JobLock ensures
exactly one API instance runs generation even when multiple completions race.

Previously getSummarizationFuture triggered both description generation calls on every
completing document, causing N parallel writes to the same Neo4j node. processFailedResult
never triggered description regeneration at all.

Changes:
- DocumentRepository: add areAllDocumentsInTopicTerminal and areAllDocumentsInClassTerminal
  queries using Cypher all() predicate
- ConvertingResultConsumer: inject JobLockRepository; extract triggerDescriptionGenerationIfAllTerminal
  which checks terminal state and acquires a per-topic/per-class JobLock before generating;
  call it asynchronously from both processSuccessfulResult and processFailedResult after
  persisting terminal status; remove now-unused topicRepository and classRepository fields

Implements #108

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@nilskntl
nilskntl merged commit 73b465c into master Mar 21, 2026
2 checks passed
@nilskntl
nilskntl deleted the 108-fix-race-condition-in-topicclass-description-generation-when-multiple-documents-upload-concurrently branch March 21, 2026 00:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix race condition in topic/class description generation when multiple documents upload concurrently

2 participants