Skip to content

Async Batch Pipeline - Refactor actions-core changes - #3916

Open
sayan-das-in wants to merge 3 commits into
mainfrom
async-batch-pipeline-core-refactor
Open

Async Batch Pipeline - Refactor actions-core changes#3916
sayan-das-in wants to merge 3 commits into
mainfrom
async-batch-pipeline-core-refactor

Conversation

@sayan-das-in

@sayan-das-in sayan-das-in commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

This PR refactors #3768 as follows.

  • Introduces a common type CloudActionDefinition. The sub types ActionDefinition and AsyncActionDefinition now extends this type.
  • Introduces a abstract class CloudAction. The sub classes Action and AsyncAction now extends this abstract class.
  • Addresses the parity gap for audienceMembership and personasContext between Action and AsyncAction.

Testing

Testing completed successfully in local and in staging.
Changes were deployed to stage for more than 24 hours.

  • Added unit tests for new functionality
  • Tested end-to-end using the local server
  • [If destination is already live] Tested for backward compatibility of destination. Note: New required fields are a breaking change.
  • [Segmenters] Tested in the staging environment
  • [Segmenters] [If applicable for this change] Tested for regression with Hadron.

Security Review

Please ensure sensitive data is properly protected in your integration.

  • Reviewed all field definitions for sensitive data (API keys, tokens, passwords, client secrets) and confirmed they use type: 'password'

New Destination Checklist

  • Extracted all action API versions to verioning-info.ts file. example

@sayan-das-in
sayan-das-in requested a review from a team as a code owner August 3, 2026 08:52
Copilot AI lite review requested due to automatic review settings August 3, 2026 08:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Refactors the destination action/core pipeline to share common definition and execution logic between Action and AsyncAction, while closing parity gaps for audienceMembership feature-flag behavior and personasContext forwarding.

Changes:

  • Introduces shared CloudActionDefinition and CloudAction base class to consolidate schema/hook/dynamic-field/request-client logic.
  • Forwards personasContext into async batch execution and updates feature-aware audienceMembership resolution for async batches.
  • Adds unit tests for the new async batching parity behavior.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
packages/core/src/destination-kit/index.ts Forwards personasContext when building async batch execution bundles.
packages/core/src/destination-kit/action.ts Refactors shared logic into CloudAction/CloudActionDefinition; forwards features/personas for async batches.
packages/core/src/tests/batching.test.ts Adds tests covering features forwarding and personasContext propagation to performBatch.

Comment on lines +808 to +809
// All events in a batch share the same personas context because batching is keyed on audience/computation.
personasContext: events[0]?.context?.personas as Personas | undefined,
if (match) {
const [, parent, indexOrChild, child] = match
if (child) {
// It is an array, so we need to extract the index from parent.[index].child and call paret.child handler
Comment on lines +162 to +164
/**
* A subset of the `BaseAction` that's common to both {@link ActionDefinition} and {@link AsyncActionDefinition}.
*/
Copilot AI review requested due to automatic review settings August 3, 2026 10:10
@sayan-das-in
sayan-das-in force-pushed the async-batch-pipeline-core-refactor branch from 5e20569 to cd3194a Compare August 3, 2026 10:10
@sayan-das-in
sayan-das-in force-pushed the async-batch-pipeline-core-refactor branch from cd3194a to f9a1dc2 Compare August 3, 2026 10:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (3)

packages/core/src/destination-kit/action.ts:174

  • CloudActionDefinition is introduced as a common type in the PR description, but it’s not exported here (it’s file-private). If this type is intended to be reused by external consumers or other modules (like ActionDefinition / AsyncActionDefinition are), it should be exported (and potentially re-exported from the package entrypoint) to match the PR’s stated goal.
interface CloudActionDefinition<
  Settings,
  Payload,
  AudienceSettings,
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  GeneratedActionHookInputs = any,
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  GeneratedActionHookOutputs = any
> extends BaseActionDefinition {

packages/core/src/destination-kit/action.ts:514

  • executeDynamicField is already async, so returning Promise.resolve(...) is unnecessary and adds noise. Returning the object directly improves readability and keeps return style consistent across early-return branches.
      return Promise.resolve({
        choices: [],
        nextPage: '',
        error: {
          message: `No dynamic field named ${field} found.`,
          code: '404'
        }
      })

packages/core/src/destination-kit/action.ts:1138

  • CloudAction centralizes request execution and response parsing via performRequest(...), but performPoll bypasses that helper and returns the raw result of performPoll. To avoid inconsistent response handling (especially if some implementations return a Response that needs parsing) and to keep instrumentation behavior uniform, route polling through the shared performRequest path as well.
    return this.definition.performPoll(requestClient, dataBundle)

Copilot AI review requested due to automatic review settings August 3, 2026 10:27
@varadarajan-tw varadarajan-tw changed the title Async Batch Pipeline - Recactor actions-core changes Async Batch Pipeline - Refactor actions-core changes Aug 3, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (3)

packages/core/src/destination-kit/action.ts:226

  • The eslint-disable-next-line @typescript-eslint/no-explicit-any on line 221 applies only to the next line (export interface ActionDefinition<) which does not contain an any, so it’s likely an unused disable directive (and can fail lint if reportUnusedDisableDirectives is enabled). Remove this directive, or move/replace it with a disable that actually targets the any defaults.
// eslint-disable-next-line @typescript-eslint/no-explicit-any
export interface ActionDefinition<
  Settings,
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  Payload = any,
  // eslint-disable-next-line @typescript-eslint/no-explicit-any

packages/core/src/destination-kit/action.ts:515

  • This is inside an async function, so return Promise.resolve({...}) is redundant. Returning the object directly is simpler and avoids an unnecessary promise construction.
    if (typeof fn !== 'function') {
      return Promise.resolve({
        choices: [],
        nextPage: '',
        error: {
          message: `No dynamic field named ${field} found.`,
          code: '404'
        }
      })
    }

packages/core/src/destination-kit/action.ts:169

  • The PR description calls out introducing a common type CloudActionDefinition, but it’s currently file-private (not exported). If this is intended to be a reusable type outside this file/module, it should be exported (and re-exported from the package entrypoint if needed). Otherwise, consider updating the PR description to clarify it’s an internal refactor detail.
/**
 * A subset of {@link BaseActionDefinition} that's common to both {@link ActionDefinition} and {@link AsyncActionDefinition}.
 */

interface CloudActionDefinition<
  Settings,
  Payload,
  AudienceSettings,

Copilot AI review requested due to automatic review settings August 3, 2026 10:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (2)

packages/core/src/destination-kit/action.ts:166

  • CloudActionDefinition is introduced per the PR description, but it is not exported. If the intent is for downstream destinations/consumers to reference this common type directly (beyond it being an internal implementation detail), it should be export interface CloudActionDefinition<...> (or the PR description should be updated to clarify it's internal-only).
/**
 * A subset of {@link BaseActionDefinition} that's common to both {@link ActionDefinition} and {@link AsyncActionDefinition}.
 */

interface CloudActionDefinition<

packages/core/src/destination-kit/action.ts:394

  • The PR description says it introduces an abstract class CloudAction, but the class is currently not exported. If this is intended to be part of the public destination-kit API (so other modules can extend it), it should be export abstract class CloudAction<...>. If it’s intentionally internal, consider adjusting the PR description to avoid implying a new exported surface area.
abstract class CloudAction<
  Settings,
  Payload extends JSONLikeObject,
  AudienceSettings = any,
  Definition extends CloudActionDefinition<
    Settings,
    Payload,
    AudienceSettings,
    unknown,
    unknown
  > = CloudActionDefinition<Settings, Payload, AudienceSettings, unknown, unknown>
> extends EventEmitter {

Copilot AI review requested due to automatic review settings August 5, 2026 06:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (2)

packages/core/src/destination-kit/action.ts:174

  • ActionDefinition / AsyncActionDefinition are exported and extend CloudActionDefinition, but CloudActionDefinition is not exported. In TypeScript this typically fails declaration emit with an error like: "Exported interface 'ActionDefinition' has or is using private name 'CloudActionDefinition'". Export CloudActionDefinition (or change it to an exported type alias) so the public types don’t reference a private symbol.
interface CloudActionDefinition<
  Settings,
  Payload,
  AudienceSettings,
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  GeneratedActionHookInputs = any,
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  GeneratedActionHookOutputs = any
> extends BaseActionDefinition {

packages/core/src/destination-kit/action.ts:222

  • This eslint-disable-next-line @typescript-eslint/no-explicit-any appears unnecessary here (the interface line itself doesn’t contain an explicit any, and the generic defaults already have their own per-line disables). Consider removing it to reduce lint suppression noise.
// eslint-disable-next-line @typescript-eslint/no-explicit-any
export interface ActionDefinition<

@joe-ayoub-segment

Copy link
Copy Markdown
Contributor

I think this is an improvement and it looks safe to me.

Ideally, it would be good to reduce the code duplication in the executeBatch functions for AsyncAction and Action.
It seems to be the handling of the response which is genuinely different between the 2 actions.

If we don't do this I can foresee an issue in the future where a change is unintentionally made to one of the actions but not the other, resulting in issues...

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants