feat: Add pre_extraction_item_counts to EventData for sync duration estimation - #224
Merged
Conversation
…stimation
Add ItemInputType enum ('main'/'users') and ItemTypeCount interface, and an
optional pre_extraction_item_counts field on EventData so ADaaS snap-ins can
report per-record-type counts on the metadata-done event. The field is optional
and passes through emit unchanged, so existing snap-ins are unaffected.
ASFND-360
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
i-arnavra3
requested review from
a team,
gasperzgonec and
radovanjorgic
as code owners
July 21, 2026 19:17
radovanjorgic
marked this pull request as draft
July 22, 2026 08:52
Contributor
|
This was tested with the airdrop-template, and it returns this during the No errors or warnings, so platform seems to be accepting this on the QA. This, however, still fails on PROD ( |
Contributor
|
This has been promoted to PROD environment and should be ready to be promoted in the SDK as well. |
gasperzgonec
marked this pull request as ready for review
August 10, 2026 12:37
radovanjorgic
approved these changes
Aug 12, 2026
radovanjorgic
added a commit
that referenced
this pull request
Aug 14, 2026
The behavioral half of v2. Everything here can break a connector.
Adapters:
- WorkerAdapter is split into ExtractionAdapter and LoadingAdapter over
a shared BaseAdapter, so each phase only carries the surface it needs
- processTask is split into processExtractionTask and processLoadingTask
Emit contract:
- a worker's task returns a TaskResult
({ status: 'success' | 'progress' | 'delay' | 'error' }) and the SDK
emits from it in emitFromResult
- adapter.emit is now protected; connectors no longer emit by hand, and
double-emitting is no longer possible
State:
- split into base, extraction, and loading state, mirroring the adapters
Public API:
- the root barrel now names every export explicitly instead of
re-exporting whole modules, so the public surface is one reviewable
file and internals stay internal
- formatAxiosError and serializeAxiosError are replaced by a single
serializeError
Carries #224 (pre_extraction_item_counts) over to the new contract.
Since emit is no longer public, the counts are supplied through the
ExtractionAdapter.preExtractionItemCounts setter, mirroring the existing
artifacts accessor, and the SDK attaches them to the metadata-done
event. Covered by a dummy-connector test asserting the emitted body.
Tests are rewritten against the new adapters and state.
BREAKING CHANGE: connectors must migrate to the split adapters and
process functions, return a TaskResult from their task instead of
emitting, and adopt the split state. Symbols that were previously
reachable through `export *` but are not in the new barrel are no
longer importable from the package root. Connectors that reported
pre_extraction_item_counts via adapter.emit must set
adapter.preExtractionItemCounts instead. See MIGRATION.md.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds an optional
pre_extraction_item_countsfield toEventDataso ADaaS snap-ins can report per-record-type counts on theMetadataExtractionDoneevent. Adds theItemTypeCountinterface andItemInputTypeenum (mainorusers) and exports both. The field is optional, so existing snap-ins are unaffected.Design doc: ADaaS estimated duration extension
Connected Issues
ASFND-360
Checklist
npm run testOR no tests needed.npm run test:backwards-compatibility.npm run lint.