Skip to content

feat: add likes support, Twitter archive import, smart re-import, and quote tweets#41

Open
VirenMohindra wants to merge 1 commit intoviperrcrypto:mainfrom
VirenMohindra:vm/likes-support-and-reimport
Open

feat: add likes support, Twitter archive import, smart re-import, and quote tweets#41
VirenMohindra wants to merge 1 commit intoviperrcrypto:mainfrom
VirenMohindra:vm/likes-support-and-reimport

Conversation

@VirenMohindra
Copy link

@VirenMohindra VirenMohindra commented Mar 10, 2026

Summary

add full likes support, Twitter archive import, smart re-import with update-on-reimport, and quoted tweet extraction for better categorization.

Changes

  • support importing Twitter/X likes alongside bookmarks (bookmarklet, console, file upload, Twitter archive)
  • add Twitter data archive import (.js and .zip files from "Download your data") with auto-detection of source from filename
  • smart re-import: update existing tweets when incoming data is richer (longer text from quoted tweets, real author handles replacing @unknown, new media) — automatically clears enrichment for re-categorization
  • extract quoted tweet content and append to parent tweet for better categorization
  • Live Import API (POST /api/import/twitter) via Twitter GraphQL endpoints for both bookmarks and likes
  • handle Twitter user data structure changes (screen_name moved from legacy to core)
  • move hardcoded Twitter bearer token to X_BEARER_TOKEN env var
  • update UI language from "bookmarks" to "tweets" across all pages where both sources are represented
  • add 34 parser tests (vitest) covering all import formats, edge cases, and backward compatibility
  • update README, CLAUDE.md, .env.example, and .gitignore

Related Issues

Checklist

  • Tested locally
  • npx tsc --noEmit passes
  • No new warnings

@VirenMohindra VirenMohindra force-pushed the vm/likes-support-and-reimport branch 2 times, most recently from 5493279 to 4b28ffc Compare March 10, 2026 21:44
@VirenMohindra
Copy link
Author

how this compares to PR #6 (liked tweets import)

PR #6 added the foundational source field and basic likes import via the Twitter GraphQL API. This PR builds significantly on that foundation:

capability PR #6 this PR
twitter archive import (.js / .zip) no yes — like.js, bookmark.js, partitioned files
smart re-import (update-on-reimport) no — skips all duplicates yes — compares text length, author handle, media count; updates if incoming is richer; clears enrichment for re-categorization
quote tweet extraction no yes — appends [Quote @author]: text for better AI categorization
shared upsert module duplicated logic across routes single lib/upsert-tweet.ts used by both import routes
atomic DB operations individual queries prisma.$transaction() for both update and create paths
error tracking errors silently counted as "skipped" separate errored count in response
parser test coverage none 34 vitest tests covering all import formats and edge cases
UI language "bookmarks" everywhere "tweets" where appropriate, acknowledges likes as a source across all pages
bearer token hardcoded in source moved to X_BEARER_TOKEN env var

the key architectural difference is the shared upsertTweets() module — PR #6's import logic was duplicated between the file import and twitter API routes. this PR extracts it into a single function with proper transaction handling, so any future changes to the dedup/update heuristic only need to happen in one place.

@VirenMohindra
Copy link
Author

comparison to the closed PR #17 (X cleanup for unlike/unbookmark)

PR #17 was closed because its core feature — permanently removing bookmarks/likes from X — was destructive and premature, with hardcoded GraphQL query IDs adding maintenance burden. this PR takes the opposite approach: read-only import enrichment rather than destructive mutations.

concern PR #17 this PR
direction destructive — deletes content from X read-only — imports and enriches locally
GraphQL query IDs hardcoded unbookmark/unlike mutation IDs hardcoded read-only query IDs (bookmarks + likes timeline)
risk if IDs rotate silently fails to delete, user thinks content is gone import returns a clear 400 error, no data loss
schema changes added cleanedFromX timestamp column no new columns — reuses existing source field from #6
reversibility irreversible — once unliked/unbookmarked on X, it's gone fully reversible — local DB only, X untouched
scope single feature (cleanup) multiple features (archive import, smart re-import, quote tweets, shared upsert)

the hardcoded query ID concern from #17's closing comment still applies here, but the blast radius is fundamentally different: a stale read query ID means "import stops working until you update the ID" vs "deletion silently fails." the .env.example documents how to find fresh IDs, and the bearer token is now configurable via X_BEARER_TOKEN env var.

this PR also reuses the source field infrastructure that #6 established, so likes imported via archive files or the live API are properly tracked and displayed throughout the UI.

@VirenMohindra VirenMohindra force-pushed the vm/likes-support-and-reimport branch 3 times, most recently from 6d307ab to 96f7c27 Compare March 10, 2026 22:03
… quote tweets

- Support importing Twitter/X likes alongside bookmarks (bookmarklet, console, file upload, Live Import API)
- Add Twitter data archive import (.js and .zip files from "Download your data")
- Smart re-import: update existing tweets when incoming data is richer (longer text, real author handles, new media), automatically re-queue for AI categorization
- Extract quoted tweet content for better categorization (a tweet quoting a tech thread no longer miscategorizes as health/wellness)
- Live Import API via Twitter's internal GraphQL endpoints for both bookmarks and likes
- Handle Twitter's user data structure changes (screen_name moved from legacy to core)
- Move Twitter bearer token to X_BEARER_TOKEN env var (was hardcoded)
- Update UI language: "bookmarks" → "tweets" where both sources are represented
- Add source filter (bookmarks/likes) to browse page
- Add 34 parser tests (vitest) covering all import formats and edge cases
- Update README and CLAUDE.md with all new features and import methods
- Add .gitignore entries for user data files and DB backups
@VirenMohindra VirenMohindra force-pushed the vm/likes-support-and-reimport branch from 96f7c27 to e697c1d Compare March 11, 2026 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant