hn-daily

Hacker News daily digest fetcher with Jina Reader and crawl4ai. Fetches top stories from yesterday in UTC+8, crawls content and comments, saves to markdown.

Features

• Fetches front-page stories from the Hacker News archive via Jina Reader (yesterday in UTC+8), then sorts locally by points (default 15, configurable)
• Fetches article markdown with Jina Reader first for external URLs, then falls back to local crawling
• Crawls story content and comments using crawl4ai
• Saves story markdown files to drafts/ (configurable via --output)
• Daily digest posts are stored in daily/ as daily/YYYY/MM/YYYY-MM-DD.md for the Hugo site
• Rich CLI output with progress tracking

Installation

pip install -r requirements.txt
python -m playwright install chromium

# Optional: improve Reader throughput and limits
export JINA_API_KEY=your_api_key

Usage

# Run with defaults (yesterday's top 15 stories in UTC+8)
python -m hn_daily

# With options
python -m hn_daily --date 2025-01-19 --limit 15 --output my_drafts

Daily Agent

The scheduled daily digest is generated by Pi, using the project prompt in .pi/prompts/daily.md.

npm install -g --ignore-scripts @earendil-works/pi-coding-agent
export ANTHROPIC_API_KEY=your_api_key
pi --provider anthropic --model claude-sonnet-4-5 --thinking high --no-session \
  --prompt-template .pi/prompts/daily.md \
  --skill .pi/skills/chinese-writing \
  -p "/daily"

To generate a specific date:

pi --provider anthropic --model claude-sonnet-4-5 --thinking high --no-session \
  --prompt-template .pi/prompts/daily.md \
  --skill .pi/skills/chinese-writing \
  -p "/daily 2025-01-19"

GitHub Actions uses ANTHROPIC_API_KEY by default, with ANTHROPIC_AUTH_TOKEN as a compatibility fallback for the previous Claude Code workflow. Set repository variables PI_PROVIDER and PI_MODEL to use another Pi-supported provider/model.

Output

Markdown files are saved to drafts/ with format:

{story_title}_{YYYYMMDD}.md

Each file contains:

• Story metadata (author, points, URL, date)
• Crawled content
• Comments section

Project Structure

hn-daily/
├── hn_daily/
│   ├── cli.py              # CLI entry point
│   ├── models.py           # Story, Comment, CrawlResult
│   └── services/
│       ├── story_service.py    # Fetch stories from HN front archive via Jina Reader
│       ├── comment_service.py  # Fetch comments from Algolia item data
│       ├── crawler_service.py  # crawl4ai integration
│       └── storage_service.py  # Save to markdown
├── tests/
├── drafts/
├── requirements.txt
└── pyproject.toml

Requirements

• Python 3.10+
• Playwright browsers (python -m playwright install chromium)