Skip to content

Rozakos/stock-api

BranchesTags

Repository files navigation

stock-api

Self-hosted yfinance proxy for the ESP8266 stock ticker. Runs on rozakos.eu, replaces the RapidAPI/Yahoo Finance free tier (and its monthly call cap).

ESP8266 ──HTTPS──▶ Cloudflare ──tunnel──▶ stock-api (FastAPI/uvicorn) ──▶ yfinance ──▶ Yahoo
                                                │
                                                └──▶ Postgres (1-min history)

What it does

  • Live quotes — GET /stock/{symbol} returns the last 5 daily closes plus pre-computed change/change_pct. Bearer-token auth.
  • Symbol allowlist — daily refresh of the NASDAQ+NYSE universe (~12.6k tickers) from the NASDAQ Trader files. Unknown symbols are rejected at the API edge before any yfinance call.
  • Live cache — 10-minute in-memory cache. On yfinance failure, falls back to stale cache rather than 502'ing, so the device never blanks out.
  • Quote poller — a background task refreshes the active working set (every symbol requested in the last 15 min) in batches, once a minute through the extended session (pre + regular + post, 04:00–20:00 ET). Device /stock requests serve from the cache the poller fills, so upstream Yahoo load is bounded by the number of distinct symbols, not the number of devices — 500 devices on 500 symbols is ~5 Yahoo requests per minute, the same as 50,000 devices would be. See Scaling.
  • 1-minute history — for the 8 most-recently-requested symbols (LRU), one row per minute bar is written to Postgres during US regular trading hours, retained 30 days. Queryable via GET /history/{symbol}?days=N.
  • Crypto — symbols with a -USD suffix (BTC-USD, ETH-USD, SOL-USD, …) route to CoinGecko instead of yfinance, end-to-end: quotes, history, and logos (Yahoo has no crypto icons). 24/7, with true rolling-24h change. Equities are untouched. Works keyless; set COINGECKO_API_KEY for higher rate limits. See Crypto.
  • Range history — GET /history/{symbol}?range=… returns longer windows (up to max) fetched live from yfinance at fixed period+interval pairs, server-cached with TTLs tuned per range. An optional &limit=N downsamples the series server-side so memory-constrained clients (ESP32) never have to parse a multi-thousand-point max response.

API

Base: https://rozakos.eu/stocks/api/v1. All /stock, /stocks, and /history requests require Authorization: Bearer <token>. Cloudflare bot-fight blocks empty/default User-Agents, so clients must send a non-empty UA.

GET /stock/{symbol}

{
  "symbol": "AMD",
  "closes": [148.32, 151.10, 149.88, 153.44, 155.20],
  "last": 155.20,
  "prev": 153.44,
  "change": 1.76,
  "change_pct": 1.15,
  "market_state": "POST",
  "pre_market": null,
  "pre_market_change_pct": null,
  "post_market": 156.40,
  "post_market_change_pct": 0.77,
  "cached": false,
  "stale": false
}

closes is oldest→newest, length ≤ 5, no nulls. last/change_pct are always the regular-session values. cached/stale are debug flags (stale: true means the upstream failed and we served the last good response).

Extended hours. market_state ∈ {PRE, REGULAR, POST, CLOSED} (coarse, from the US-market clock — doesn't know holidays/half-days). pre_market/post_market carry the latest extended-hours price and its change_pct vs the regular close (last), or null when not applicable: pre_market is set only during PRE, post_market during POST/CLOSED (the last after-hours print of the most recent session). During REGULAR both are null — the live price is already in last. Sourced from a batched prepost intraday pull, so it's additive and never changes the existing fields.

Status codes: 400 unknown symbol, 401 bad/missing bearer, 502 upstream Yahoo failure with no prior cache.

GET /stocks?symbols=AMD,NVDA,AAPL

Batch quotes — one request instead of one per symbol. Built for the embedded ticker: it lets the client fetch its whole watchlist in a single round-trip, keeping the keep-alive connection free for history taps.

{"quotes": [
  {"symbol": "AMD", "last": 488.45, "change_pct": 7.97, "closes": [..]},
  {"symbol": "NVDA", "last": 204.87, "change_pct": 2.22, "closes": [..]}
]}
  • Each entry carries the same quote field names/types as /stock/{symbol}: symbol, last, change_pct, the 5-point closes sparkline (oldest→newest), plus the extended-hours fields market_state, pre_market, pre_market_change_pct, post_market, post_market_change_pct (see /stock above). No cached/stale/prev/ change — kept minimal for embedded buffers (~3 KB for 16 symbols).
  • Served from the same cache + background poller as /stock/{symbol}; every requested symbol joins the poller's active working set. Cache misses are fetched in one batched Yahoo round-trip.
  • Unknown or failed symbols are omitted from quotes — one bad ticker never fails the whole request. Order follows the request; duplicates and case are normalized.
  • Max 16 symbols; more returns 400. A 16-symbol response is ~1.5 KB (well under a 24 KB client buffer). Plain compact JSONResponse with a Content-Length header (no chunked transfer-encoding).

Status codes: 400 empty or >16 symbols, 401 bad/missing bearer.

GET /history/{symbol} — two modes

?range=… (recommended) — served live from yfinance and cached server-side. One of 1d, 1w, 1mo, 6mo, 1y, 5y, max. The server maps each range to a fixed period+interval pair:

range yfinance period yfinance interval interval field cache TTL
1d 1d 5m intraday 60 s
1w 7d 1h intraday 5 min
1mo 1mo 1d daily 1 h
6mo 6mo 1d daily 1 h
1y 1y 1d daily 1 h
5y 5y 1wk daily 1 h
max max 1wk daily 1 h 
{
  "symbol": "AMD",
  "range": "1mo",
  "interval": "daily",
  "count": 22,
  "points": [{"ts": 1776312000, "last": 278.26}, ...]
}

ts is epoch seconds. interval tells the client whether to format the X axis as time-of-day (intraday) or as dates (daily).

Invalid range values return 422 (FastAPI validation).

range=1d session window — the 1d response additionally carries the day's regular trading session as top-level session_open / session_close (epoch seconds, UTC), so a client can render the whole session as a fixed X axis (Revolut-style) and grow the line through it. The bounds come from yfinance chart metadata, so half-days and holidays use the exchange's real session rather than a hardcoded 16:00. points stays ascending and contains only elapsed data up to "now" — the future is not padded. If the bounds can't be determined they're omitted, and the client should fall back to a 6.5h assumption.

{
  "symbol": "AMD",
  "range": "1d",
  "interval": "intraday",
  "session_open": 1779975000,
  "session_close": 1779998400,
  "count": 78,
  "points": [{"ts": 1779975000, "last": 98.12}, ...]
}

&prepost=1 (optional, range=1d only) — extends the 1d chart to the full extended-hours window: pre-market (from 04:00 ET), the regular session, and after-hours (until 20:00 ET), as available at request time. Same {ts, last} shape and same downsampling/limit behavior as the regular 1d. points still contains only elapsed data — during pre-market that's just the pre-market prints so far, and the future is never padded. Ignored for every other range. Omitting the param leaves the 1d response byte-for-byte identical to the regular-session one above.

When set, the response adds four fields (and keeps session_open / session_close as the regular 09:30/16:00 bounds, so a client can style the pre/post segments differently and drop a divider at the open):

  • window_open / window_close — epoch seconds for 04:00 / 20:00 ET of that trading day: the full extended-session x-axis span, so a progressive chart has a stable width even when only a few pre-market prints exist. Fixed wall-clock bounds, so unlike session_* they don't shrink on half-days.
  • market_state — PRE / REGULAR / POST / CLOSED.
  • prev_close — the previous regular session's close, so a client can color the chart by day-change without a second request.

While market_state != CLOSED the payload is cached for a shorter TTL (20 s instead of 60 s) so progressive charts track the live session.

{
  "symbol": "AMD",
  "range": "1d",
  "interval": "intraday",
  "count": 147,
  "points": [{"ts": 1783497600, "last": 512.30}, ...],
  "session_open": 1783517400,
  "session_close": 1783540800,
  "market_state": "POST",
  "window_open": 1783497600,
  "window_close": 1783555200,
  "prev_close": 516.11
}

&limit=N (optional, N ≥ 1, any range) — caps the response at N points by uniform downsampling on the server, always keeping the first and last point so the displayed % change stays correct. Without it, range=max returns the full series (a few thousand points for old listings like AAPL/IBM/KO), which can exhaust RAM on an ESP32 that buffers the whole body before downsampling. With it, e.g. range=max&limit=30 returns ≤ 30 points. The range cache stores the full series, so different limit values for the same symbol/range are all served from one cached fetch. count reflects the returned (downsampled) length; the rest of the shape is unchanged. limit applies only to range= — the legacy days= path ignores it.

?days=N (legacy, 1 ≤ N ≤ 30) — served from the Postgres minute-bar archive. Returned only for symbols currently in the hot LRU. Kept for the in-field ESP8266 firmware until it migrates to range=.

{
  "symbol": "AMD",
  "days": 1,
  "count": 198,
  "points": [{"ts": "2026-05-14T13:30:00+00:00", "last": 442.51}, ...]
}

ts is ISO 8601 here, not epoch seconds — different mode, different shape. 503 if DATABASE_URL is not configured.

GET /logo/{symbol} and GET /logos

Serves cached ticker logos so devices don't need to embed every PNG in firmware flash. Both endpoints require the bearer token.

GET /logo/{symbol} returns a 64×64 PNG. Symbol is case-insensitive. On a miss, the server resolves the logo on the fly through this chain:

  1. Manual override in logo_sources.json — { "IONQ": "https://..." }. Useful for symbols where auto-resolution returns something wrong or ugly (e.g. a halftone/dotted favicon). An override is trusted and short-circuits the yfinance lookup. A value may be a full URL (http(s)://, file://) or a repo-relative path to a committed asset (e.g. "NVDA": "logo_overrides/NVDA.png") — handy for pinning a clean high-res brand mark that won't change out from under you.
  2. Brandfetch (if BRANDFETCH_CLIENT_ID is set) — the transparent high-res symbol from Brandfetch's Logo Link CDN, keyed by the company domain. This is the broad-coverage high-quality source for the whole universe. Brandfetch serves a fixed "B" placeholder for brands with no real symbol (rejected by sha256), and opaque assets are skipped to keep the alpha clean — so a miss here falls through to the favicons below.
  3. The company website from yfinance.Ticker(symbol).info["website"], resolved to a logo via DuckDuckGo's icons.duckduckgo.com/ip3/{domain}.ico and Google's s2/favicons?domain={domain}&sz=256. Both candidates are fetched and the one that decodes to the largest native resolution wins (rather than first-hit), and for multi-size .ico files the largest embedded frame is selected.
  4. If the best logo found is unusably small (native max-dimension below LOGO_MIN_NATIVE, default 32px), a clean monogram is generated instead — a rounded tile in a deterministic per-symbol color with the ticker in white bold.
  5. If nothing resolves at all, the symbol is remembered as a "miss" for 24h to avoid retry storms, and the endpoint returns 404 {"detail": "no logo for X"} (unchanged).

The resolved image is stored as a high-resolution master (the source's native resolution, capped at 256px, alpha-trimmed) in LOGO_CACHE_DIR as {symbol}.png. Keeping the master high-res means a small served size is a single clean downscale from the best available pixels, not a re-downscale of an already-shrunk 64px image.

?size= query parameter — accepts 32, 48, or 64 (default 64); anything else returns 400. Each size is rendered once from the master in a single premultiplied-alpha Lanczos downscale — premultiplication keeps semi-transparent edges true to the logo's color instead of bleeding toward black (no dark halos) — centered on a transparent square at LOGO_CONTENT_RATIO (94%) fill, encoded as an optimized RGBA PNG (typically 1–3 KB). The result is cached per (symbol, size) on disk as {symbol}.{size}.png and served with Cache-Control: public, max-age=2592000, immutable, so the device (and Cloudflare, if you configure it to ignore the auth header on this path) can cache aggressively. The ESP32 CYD firmware uses ?size=48 to keep lodepng's allocation inside its largest free block after WiFi+TLS; sizes never exceed 64px so the device's PNG-decode memory budget is respected.

Changing the pipeline? The master and per-size variants are cached on disk and never auto-invalidated. After changing resolution/resize logic, clear the cache so logos rebuild: rm -f "$LOGO_CACHE_DIR"/*.png (see "pre-warm" below). Existing low-res masters otherwise keep being served.

?test=1 diagnostic mode — bypasses the resolver and cache and returns a synthetic 64×64 RGBA PNG (red background, green diagonal, blue center dot, Cache-Control: no-store). Exists to separate firmware-side PNG render bugs from logo-content/contrast issues. Can also be forced server-wide with env STOCK_API_LOGO_TEST=1.

GET /logos?symbols=AAPL,IONQ,NVDA returns a JSON manifest without fetching anything — handy for the device to check which logos it can download cheaply:

{
  "logos": {
    "AAPL": { "url": "/stocks/api/v1/logo/AAPL", "cached": true },
    "IONQ": { "url": "/stocks/api/v1/logo/IONQ", "cached": false }
  }
}

To pre-warm the cache from the command line:

.venv/bin/python scripts/fetch_logos.py AAPL IONQ NVDA TSLA

Logo normalization is applied only when a logo is first fetched. If the normalization logic changes, remove existing cached PNGs before pre-warming again, otherwise /logo/{symbol} will keep serving the old files:

find "${LOGO_CACHE_DIR:-data/logos}" -maxdepth 1 -type f -name '*.png' -delete
.venv/bin/python scripts/fetch_logos.py AAPL IONQ NVDA TSLA

GET /health

No auth. Useful for uptime checks and seeing service state:

{
  "status": "ok",
  "cached_symbols": ["AMD", "NVDA"],
  "universe_size": 12601,
  "universe_refreshed_at": "2026-05-14T16:07:50.092132",
  "extra_symbols": ["BTC-USD"],
  "history_enabled": true,
  "hot_symbols": ["AMD", "NVDA"],
  "hot_max": 8,
  "tick_seconds": 60,
  "active_symbols": 137,
  "quote_poll_seconds": 60,
  "quote_poll_at": "2026-05-14T16:08:12.114390",
  "quote_poll_ok": true,
  "market_open": true
}

active_symbols is the size of the quote poller's working set; quote_poll_at / quote_poll_ok are the timestamp and result of its last run (handy for alerting if the poller stalls).

Setup

git clone git@github.com:Rozakos/stock-api.git /home/rozakos/stock-api
cd /home/rozakos/stock-api
python3 -m venv .venv
.venv/bin/pip install -r requirements.txt
cp .env.example .env
# edit .env: set API_SECRET (token_hex(24)), DATABASE_URL if you want history
sudo cp stock-api.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now stock-api

Edge routing on rozakos.eu uses Cloudflare Tunnel (cloudflared):

# /etc/cloudflared/config.yml
ingress:
  - hostname: rozakos.eu
    path: ^/stocks/api/v1/(docs|openapi\.json|redoc).*
    service: http_status:404
  - hostname: rozakos.eu
    path: ^/stocks/api/.*
    service: http://127.0.0.1:8001
  - hostname: rozakos.eu
    service: http://127.0.0.1:3000   # the existing Next.js app
  - service: http_status:404

If you're fronting with nginx instead, see nginx.conf.snippet.

Configuration

Variable Default Purpose
API_SECRET — Bearer token. Empty disables auth (LAN-only).
CACHE_TTL_SECONDS 600 TTL for the live-quote cache.
EXTRA_SYMBOLS (empty) Comma-separated tickers to allow alongside the universe. Use this for crypto/indices (e.g. BTC-USD,^GSPC) until proper sources are added.
DATABASE_URL (empty) Postgres DSN. If unset, history features are disabled and /history returns 503.
HISTORY_TICK_SECONDS 60 How often to record minute bars.
HISTORY_RETENTION_DAYS 30 Older rows are pruned on each tick.
HISTORY_MAX_HOT 8 LRU cap on archived symbols. Hitting /stock/{X} bumps X to the front; oldest is evicted when full.
QUOTE_POLL_SECONDS 60 How often the quote poller refreshes the active set during the extended session (04:00–20:00 ET).
QUOTE_POLL_CLOSED_SECONDS 300 Poll interval while the market is closed.
QUOTE_ACTIVE_WINDOW_SECONDS 900 A symbol stays in the poller's working set this long after it was last requested.
QUOTE_MAX_ACTIVE 1000 Cap on the working set. The most recently requested symbols win; the rest fall back to on-demand fetch.
QUOTE_BATCH_SIZE 100 Symbols per yf.download batch (one Yahoo round-trip).
LOGO_CACHE_DIR data/logos Where resolved logos are stored as 64×64 PNGs. Relative paths are resolved from the project root.
LOGO_OVERRIDES_FILE logo_sources.json JSON map of TICKER -> logo URL to override the auto-resolution chain.
LOGO_MIN_NATIVE 32 If the best resolved logo's native max-dimension is below this, serve a generated monogram instead.
BRANDFETCH_CLIENT_ID (empty) Brandfetch Logo Link client ID. If set, the transparent high-res Brandfetch symbol is used as a logo source ahead of the favicon fallbacks.
COINGECKO_API_KEY (empty) CoinGecko Demo API key for crypto (-USD) quotes/history/logos. Optional — crypto works keyless but the free tier is heavily rate-limited; a key raises the limit.

The on-disk file symbols.cache.json stores the last fetched symbol universe so restarts don't depend on the network. It's gitignored and self-heals on the next daily refresh.

Test client

test_client.py mimics what the ESP8266 will do: bearer auth, sets a User-Agent, calls the API, renders the same fields the OLED draws.

.venv/bin/python test_client.py                          # one cycle, defaults
.venv/bin/python test_client.py --loop --interval 30     # mimic device polling
.venv/bin/python test_client.py --symbols TSM AAPL       # custom tickers
.venv/bin/python test_client.py --history AMD --days 7   # unicode sparkline
.venv/bin/python test_client.py --base http://127.0.0.1:8001/stocks/api/v1  # skip the tunnel

Operations

  • Logs: journalctl -u stock-api -f
  • State: curl https://rozakos.eu/stocks/api/v1/health | jq
  • Restart: sudo systemctl restart stock-api
  • DB: rows live in prices(symbol TEXT, ts TIMESTAMPTZ, last DOUBLE PRECISION) with a (symbol, ts DESC) index. Created on first startup.

Crypto

Symbols with a -USD suffix (BTC-USD, ETH-USD, SOL-USD, …) are routed to CoinGecko instead of yfinance, across all three data endpoints. No US equity/ETF ticker uses -USD, so the suffix is the router (_is_crypto), and crypto symbols bypass the NASDAQ allowlist (CoinGecko validates them — an unknown coin just returns no data). The client and response shapes are identical to equities, so the device needs no changes.

  • Quotes (/stock, /stocks) — one batched /coins/markets call yields price, true rolling-24h change (change_pct), a 5-point closes sparkline, and the coin's logo URL. Crypto is always market_state: "REGULAR" with pre_market/post_market null (it trades 24/7). Shared symbols are disambiguated by market-cap rank.
  • History (/history?range=…) — /coins/{id}/market_chart. No session_open/session_close (24/7). The free/Demo tier caps history at 365 days, so 5y/max are clamped to a year. &limit=N downsampling still applies.
  • Logos (/logo) — the CoinGecko coin image (a transparent ~250px mark) feeds the same high-res-master pipeline; a manual override still wins, and a miss falls back to a monogram. This is the gap Yahoo can't fill.

Set COINGECKO_API_KEY (free Demo key) for higher rate limits; it works keyless otherwise, but the public tier is tight and rate-limited crypto requests degrade gracefully (omitted quote / empty history / monogram), never a 5xx.

Scaling

The thing that breaks first under many devices is upstream Yahoo load, not serving. On-demand fetching makes one Yahoo call per cache miss, so N devices on N symbols means N fetches per cache cycle — which both stampedes (no per-symbol lock on the quote path) and risks Yahoo rate-limiting/IP-banning the server. Serving itself is cheap: in-memory cache reads handle thousands/sec.

The quote poller flips this from pull to push: a background task refreshes the active working set on a timer and device requests just read the cache it fills. Consequences:

  • Yahoo load is decoupled from device count. It scales with the number of distinct symbols, not devices. 500 symbols at QUOTE_BATCH_SIZE=100 is ~5 Yahoo requests/minute, constant, whether 500 or 50,000 devices ask.
  • No stampede / no thread-pool blocking on the request path — requests are in-memory reads.
  • A brand-new symbol's first request still falls back to an on-demand fetch; the poller picks it up on the next cycle.

Two deliberate boundaries:

  • /history ranges stay on-demand with their per-range cache TTLs. Long windows (1y, max) don't change minute-to-minute, so they don't belong on the 1-minute poller. Use &limit=N to keep those responses small for constrained clients.
  • The poller is a single point of staleness — if it stalls, quotes freeze. The stale-serve fallback covers brief gaps; watch quote_poll_at / quote_poll_ok in /health for anything longer.

Tests

A smoke test in tests/ exercises every range= value against the live service plus a couple of regression checks. tests/test_quotes.py unit-tests the quote payload builder and the active-set registry offline (no network).

.venv/bin/pip install -r requirements-dev.txt
.venv/bin/python -m pytest tests/ -v

Set STOCK_API_BASE and TEST_SYMBOL to point at a different deployment or ticker. The test reads API_SECRET from .env.

Security notes

  • The bearer token is anti-casual-discovery, not crypto. The device stores it in plaintext in LittleFS, so treat it as rotatable rather than secret.
  • yfinance does HTTPS to Yahoo from the server. The device does HTTPS to rozakos.eu. No keys are compiled into the firmware image.
  • FastAPI's /docs UI is intentionally returned as 404 at the edge so the surface isn't advertised. Reachable on 127.0.0.1 if you tunnel in.
  • No rate limiting at the app layer. The 10-min cache caps real yfinance load to ~1 call per symbol per 10 min, and the symbol allowlist prevents unknown-symbol spam from growing the cache or hitting Yahoo.

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages