Self-hosted Twitter/X bookmark manager with AI-powered organization
Import · Analyze · Categorize · Search · Explore
Siftly turns your Twitter/X bookmarks into a searchable, categorized, visual knowledge base — running entirely on your machine. No cloud, no subscriptions, no browser extensions required. Everything stays local except the AI API calls you configure.
It runs a 4-stage AI pipeline on your bookmarks:
📥 Import (built-in bookmarklet or console script — no extensions needed)
↓
🏷️ Entity Extraction — mines hashtags, URLs, mentions, and 100+ known tools from raw tweet data (free, zero API calls)
↓
👁️ Vision Analysis — reads text, objects, and context from every image/GIF/video thumbnail (30–40 visual tags per image)
↓
🧠 Semantic Tagging — generates 25–35 searchable tags per bookmark for AI-powered search
↓
📂 Categorization — assigns each bookmark to 1–3 categories with confidence scores
After the pipeline runs, you get:
- AI search — find bookmarks by meaning, not just keywords ("funny meme about crypto crashing")
- Interactive mindmap — explore your entire bookmark graph visually
- Filtered browsing — grid or list view, filter by category, media type, and date
- Export tools — download media, export as CSV / JSON / ZIP
- Node.js 18+
- npm (comes with Node.js)
That's it. If you have Claude Code CLI installed and signed in, AI features work automatically — no API key needed.
git clone https://github.com/viperrcrypto/Siftly.git
cd Siftly
./start.shstart.sh installs dependencies, sets up the database, checks for Claude CLI auth, and opens http://localhost:3000 automatically.
If you're using Claude Code to set up the project, it will read CLAUDE.md and know exactly how to get started. Just open the project folder:
git clone https://github.com/viperrcrypto/Siftly.git
claude Siftly/Claude Code will handle setup and start the app using your existing Claude subscription — no extra configuration needed.
git clone https://github.com/viperrcrypto/Siftly.git
cd Siftly
npm install
npx prisma generate
npx prisma migrate dev --name init
npx next devSiftly automatically detects the best available auth method — no configuration needed in the most common case.
| # | Method | How |
|---|---|---|
| 1 | Claude Code CLI (zero config) | Already signed in? Siftly reads your session from the macOS keychain automatically |
| 2 | API key in Settings | Open Settings in the app and paste your key |
| 3 | ANTHROPIC_API_KEY env var |
Set in .env.local or your shell environment |
| 4 | Local proxy | Set ANTHROPIC_BASE_URL to any Anthropic-compatible endpoint |
If you use Claude Code, you're already signed in. Siftly detects your session from the macOS keychain and uses your Claude subscription (Free/Pro/Max) automatically.
The Settings page shows a green "Claude CLI detected — no API key needed" badge with your subscription tier when this is active.
Note: This works on macOS. On Linux/Windows, add an API key in Settings instead.
- Go to console.anthropic.com
- Create a new API key
- Open Siftly → Settings → paste it in
New accounts include $5 free credit — enough for thousands of bookmarks at Haiku pricing (~$0.00025/bookmark).
Siftly has built-in import tools — no browser extensions required. Go to the Import page and choose either method:
- Go to Import in the Siftly sidebar
- Drag the "Export X Bookmarks" link to your browser's bookmark bar (or right-click the bookmark bar → Add Bookmark → paste the URL)
- Go to x.com/i/bookmarks while logged in to X
- Click "Export X Bookmarks" in your bookmark bar — a purple button appears on the page
- Click "▶ Auto-scroll" — the tool scrolls through and captures all your bookmarks automatically
- When complete, click the purple "Export N bookmarks" button —
bookmarks.jsondownloads - Back in Siftly → Import → drop or upload the file
- Go to x.com/i/bookmarks while logged in to X
- Open DevTools: press
F12(Windows/Linux) or⌘⌥J(Mac), then go to the Console tab - Copy the console script from the Siftly Import page, paste it into the console, and press Enter
- Click "▶ Auto-scroll" and wait for all bookmarks to be captured
- Click the export button —
bookmarks.jsondownloads automatically - Back in Siftly → Import → upload the file
Re-import anytime — Siftly automatically skips duplicates and only adds new bookmarks.
Categorization starts automatically as soon as you import. You can also trigger it manually from:
- The Import page (after upload)
- The Mindmap page (when bookmarks are uncategorized)
- The Categorize page in the sidebar
| Stage | What it does |
|---|---|
| Entity Extraction | Mines hashtags, URLs, @mentions, and 100+ known tool/product names from stored tweet JSON — free, zero API calls |
| Vision Analysis | Analyzes every image, GIF, and video thumbnail — OCR text, objects, scene, mood, meme templates, 30–40 visual tags per image |
| Semantic Tagging | Generates 25–35 precise search tags per bookmark by combining tweet text + image context. Also extracts sentiment, people, and company names. |
| Categorization | Assigns 1–3 categories per bookmark with confidence scores using all enriched data |
The pipeline is incremental — if interrupted, it picks up where it left off. Use "Re-run everything (force all)" to re-analyze bookmarks that were already processed.
Natural language queries across all bookmark data:
- "funny meme about crypto crashing"
- "react hooks tutorial"
- "bitcoin price chart"
- "best AI coding tools"
Searches tweet text, image OCR, visual tags, semantic tags, and categories simultaneously using a full-text search index (FTS5) + Claude semantic reranking. Results are ranked by relevance with AI-generated explanations for each match.
Interactive force-directed graph showing all bookmarks organized by category:
- Expand/collapse any category to reveal its bookmarks
- Click a bookmark node to open the original tweet on X
- Color-coded legend by category
- If bookmarks aren't categorized yet, an inline AI Categorize button starts the pipeline without leaving the page
- Grid view (masonry layout) or List view
- Filter by category, media type (photo / video), or search text
- Sort by newest or oldest
- Pagination with 24 items per page
- Active filter chips — removable individually or all at once
- Hover any card to download media or jump to the original tweet
8 default categories pre-seeded with AI-readable descriptions:
| Category | Color |
|---|---|
| Funny Memes | Amber |
| AI Resources | Violet |
| Dev Tools | Cyan |
| Design | Pink |
| Finance & Crypto | Green |
| Productivity | Orange |
| News | Indigo |
| General | Slate |
Create custom categories with a name, color, and optional description. The description is passed directly to the AI during categorization — the more specific, the more accurate the results.
- CSV — spreadsheet-compatible with all fields
- JSON — full structured data export
- ZIP — exports a category's bookmarks + all media files with a
manifest.csv
Press Cmd+K (Mac) or Ctrl+K (Windows/Linux) to search across all bookmarks from anywhere in the app.
All settings are manageable in the Settings page at /settings or via environment variables:
| Setting | Env Var | Description |
|---|---|---|
| Anthropic API Key | ANTHROPIC_API_KEY |
Optional if Claude CLI is signed in — otherwise required for AI features |
| API Base URL | ANTHROPIC_BASE_URL |
Custom endpoint for proxies or local Anthropic-compatible models |
| AI Model | Settings page only | Haiku 4.5 (default, fastest/cheapest), Sonnet 4.6, Opus 4.6 |
| OpenAI Key | Settings page only | Alternative provider if no Anthropic key is set |
| Database | DATABASE_URL |
SQLite file path (default: file:./prisma/dev.db) |
Point Siftly at any Anthropic-compatible server:
ANTHROPIC_BASE_URL=http://localhost:8080siftly/
├── app/
│ ├── api/
│ │ ├── analyze/images/ # Batch image vision analysis (GET progress, POST run)
│ │ ├── bookmarks/ # List, filter, paginate, delete
│ │ │ └── [id]/categories/ # Per-bookmark category management
│ │ ├── categories/ # Category CRUD
│ │ │ └── [slug]/ # Individual category operations
│ │ ├── categorize/ # 4-stage AI pipeline (start, status, stop)
│ │ ├── export/ # CSV, JSON, ZIP export
│ │ ├── import/ # JSON file import with dedup + auto-pipeline trigger
│ │ │ ├── bookmarklet/ # Bookmarklet-specific import endpoint
│ │ │ └── twitter/ # Twitter-specific import endpoint
│ │ ├── link-preview/ # Server-side OG metadata scraper
│ │ ├── media/ # Media proxy/download endpoint
│ │ ├── mindmap/ # Graph nodes + edges for visualization
│ │ ├── search/ai/ # Natural language semantic search (FTS5 + Claude)
│ │ ├── settings/ # API key + model config
│ │ │ ├── cli-status/ # Claude CLI auth detection endpoint
│ │ │ └── test/ # API key validation endpoint
│ │ └── stats/ # Dashboard stats
│ ├── ai-search/ # AI search page
│ ├── bookmarks/ # Browse, filter, paginate
│ ├── categories/ # Category management
│ │ └── [slug]/ # Category detail page
│ ├── categorize/ # Pipeline monitor with live progress
│ ├── import/ # 3-step import flow (instructions → upload → categorize)
│ ├── mindmap/ # Interactive graph
│ ├── settings/ # Configuration
│ └── page.tsx # Dashboard
│
├── components/
│ ├── mindmap/ # Mindmap canvas, nodes, edges
│ │ ├── mindmap-canvas.tsx
│ │ ├── category-node.tsx
│ │ ├── tweet-node.tsx
│ │ ├── root-node.tsx
│ │ ├── chain-edge.tsx
│ │ └── mindmap-context.ts
│ ├── command-palette.tsx # Cmd+K global search
│ ├── nav.tsx # Sidebar navigation
│ └── theme-toggle.tsx # Light/dark mode
│
├── lib/
│ ├── categorizer.ts # AI categorization logic + default categories
│ ├── claude-cli-auth.ts # Claude CLI OAuth session detection (macOS keychain)
│ ├── vision-analyzer.ts # Image analysis + batch semantic tagging
│ ├── image-context.ts # Shared image context builder
│ ├── fts.ts # SQLite FTS5 full-text search index
│ ├── rawjson-extractor.ts # Entity extraction from raw tweet JSON
│ ├── parser.ts # Multi-format JSON parser
│ ├── exporter.ts # CSV, JSON, ZIP export
│ ├── types.ts # Shared TypeScript types
│ └── db.ts # Prisma client singleton
│
├── prisma/
│ └── schema.prisma # SQLite schema
│
├── start.sh # One-command launcher (install + DB setup + open browser)
└── CLAUDE.md # Instructions for Claude Code AI assistant
Bookmark — tweet text, author, date, raw JSON, semantic tags, enrichment metadata
├── MediaItem — images / videos / GIFs with AI-generated image tags
└── BookmarkCategory — category assignments with confidence scores (0–1)
Category — name, slug, hex color, AI-readable description
Setting — key-value store (API keys, model preferences)
ImportJob — tracks import file status and progress
Siftly uses Prisma migrations for relational schema changes.
In development, run npx prisma migrate dev --name <change-name> when schema changes.
For runtime/prod-style startup, apply committed migrations with npx prisma migrate deploy.
FTS5 (bookmark_fts) is managed at runtime in lib/fts.ts, not in schema.prisma.
This is intentional for now because Prisma does not model SQLite virtual table definitions directly.
For Prisma command and workflow details, see:
- https://www.prisma.io/docs/orm/prisma-migrate/workflows/development-and-production
- https://www.prisma.io/docs/orm/prisma-client/setup-and-configuration/generating-prisma-client
| Technology | Version | Role |
|---|---|---|
| Next.js | 16 | Full-stack framework (App Router) |
| TypeScript | 5 | Type safety throughout |
| Prisma | 7 | ORM + migrations |
| SQLite | — | Local database — zero setup, includes FTS5 |
| Tailwind CSS | v4 | Styling |
| Anthropic SDK | — | Vision, semantic tagging, categorization, search |
| @xyflow/react | 12 | Interactive mindmap graph |
| Framer Motion | 12 | Animations |
| Radix UI | — | Accessible UI primitives |
| JSZip | — | Category ZIP export |
| Lucide React | — | Icons |
# One-command start (installs, sets up DB, opens browser)
./start.sh
# Or manually:
npm install
npx prisma generate
npx prisma migrate dev --name init
npx next dev
# Type check
npx tsc --noEmit
# Open database GUI
npx prisma studio
# Build for production
npm run build && npm startEdit DEFAULT_CATEGORIES in lib/categorizer.ts. Each entry needs:
{
name: 'My Category', // Display name
slug: 'my-category', // URL-safe identifier (must be unique)
color: '#6366f1', // Hex color shown in UI
description: '...', // Natural language description — used verbatim in AI prompts
}The description field directly shapes how the AI classifies bookmarks. Be specific.
Add domain strings to KNOWN_TOOL_DOMAINS in lib/rawjson-extractor.ts to have the entity extractor automatically recognize links to those tools in tweet data.
- All data is stored locally in a SQLite file on your machine
- The only external calls are to the AI provider you configure (tweet text + image data)
- No telemetry, no tracking, no accounts required
- Your bookmarks never touch any third-party server except your configured AI endpoint
If Siftly saves you time, consider leaving a tip ☕
MIT — see LICENSE
Built by @viperr · Self-hosted · No extensions · No cloud