Skip to content

Crawler successfully finish with pending Requests in RequestQueue(V2?) #668

Description

@JJetmar

Issue description

This is a very rare edge case, but I still see it happening every now and then. It seems completely random and is incredibly hard to reproduce, but I've noticed it across several of my standalone scrapers.

The only common thread is that they all use the default RequestQueueV2 (usually just passed in via the default Crawler Options).

When I deploy these as Actors and run them on the Apify Platform, the crawler will sometimes finish successfully even though some requests were completely skipped (according to the logs). These skipped requests just sit in the RequestQueue as "Pending." Sometimes, even the very first (and only) request isn't processed at all—which is exactly what just happened in run C0UfYW3HGFSXREcgA and 2bkyhODHOnyTQwEqv.

I'm not sure if this is specifically a RequestQueueV2 issue or if reverting to RequestQueueV1 would fix it, mostly because it's so hard to catch and reproduce.

Code sample

// any default Apify actor template using default Crawler configuration

Package version

^3.7.2

Node.js version

v24.18.0

Operating system

Linux

Actor or run link

https://console.apify.com/actors/runs/C0UfYW3HGFSXREcgA

I have tested this on the next release

No response

Other context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working.t-toolingIssues with this label are in the ownership of the tooling team.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions