twitter search returns HTTP 404 (SearchTimeline); ClientTransaction init fails with 'NoneType' object has no attribute 'group'

[xiucaicoder]

Summary

Since the x.com homepage redesign, twitter search "query" -n 10 fails with HTTP 404 on every request. twitter feed, tweet, user-posts still work.

Environment

• twitter-cli v0.8.5 (latest on PyPI, 2026-03-17)
• macOS

Reproduce

$ twitter search "agent" -n 3
WARNING twitter_cli.client: Failed to init ClientTransaction: 'NoneType' object has no attribute 'group'
ok: false
error:
  code: not_found
  message: 'Twitter API error (HTTP 404): Twitter API error 404: '

Root cause analysis

1. _ensure_client_transaction() (client.py) fetches https://x.com and parses the homepage with get_ondemand_file_url() in x_client_transaction/utils.py:

   on_demand_file_index = ON_DEMAND_FILE_REGEX.search(str(response)).group(1)  # line 59

   The new x.com homepage HTML no longer matches ON_DEMAND_FILE_REGEX, so .group(1) raises 'NoneType' object has no attribute 'group'. ClientTransaction init is wrapped in try/except (client.py:1092-1093), so it only logs a warning.

2. Because ClientTransaction init fails, the dynamic queryId scan (from JS bundles) is also effectively unavailable, so the client falls back to the hardcoded SearchTimeline queryId in graphql.py (MJpyQGqgklrVl_0X9gNy3A) which is stale — Twitter returns 404 for unknown queryIds.

3. Other endpoints (HomeTimeline, UserTweets, etc.) still work because their queryIds haven't been rotated yet.

Suggested fix

• Update get_ondemand_file_url() regex parsing for the new x.com homepage structure (or make it non-fatal so bundle scanning / queryId refresh still runs)
• Refresh the SearchTimeline queryId from current x.com JS bundles

Happy to provide more debugging info if needed.

[Tommy00748]

Verified on v0.8.5 (2026-08-22, macOS, residential IP): the cookie fix works — this should help pick between the open PRs.

What I tested:

1. Reproduced the bug: anonymous GET https://x.com returns the logged-out shell page (entry-client-logged-out-*.js), which contains no ondemand.s marker → get_ondemand_file_url() fails → search 404. Matches the root cause analysis in fix: send auth cookies when fetching x.com home for ClientTransaction init (fixes #78) #79/fix: send session cookies when fetching x.com for ClientTransaction init #86.
2. Re-ran the exact same init flow with auth_token + ct0 cookies: the homepage response does contain ondemand.s (https://abs.twimg.com/responsive-web/client-web/ondemand.s.da60c70a.js), ClientTransaction init succeeds, and SearchTimeline works — both product=Top and product=Latest return real results.

Regarding #84: the claim that "even with cookies, the redesigned homepage still lacks ondemand.s" did not reproduce for me — with cookies the marker was present. It's possible #84's observation came from an anonymous request or a regional/A-B variant of the homepage, but as of today the cookie approach alone is sufficient here.

Two additional findings not covered in the open PRs:

• Live tab gotcha: the GraphQL product enum value is "Latest", not "Live". Passing "Live" returns GRAPHQL_VALIDATION_FAILED (path: ["variable","product"]). Easy to trip over since the CLI flag is --type latest / the UI tab is called "Live".
• CT cache reuse: the fix should write the successful (cookie-authenticated) homepage + ondemand responses into the existing ~/.twitter-cli/transaction_cache.json cache (same format, 1h TTL) — otherwise every CLI invocation re-fetches x.com twice, which is slow and adds avoidable traffic on a flagged account.

Hope this helps with triage — happy to test any of the open PRs once they're reviewable.