Skip to content

Rheosoph/node-package-wayback-machine

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Wayback Machine — Flow-Like Node Package

A Flow-Like WASM node package for the Internet Archive's Wayback Machine. Query the archive, retrieve archived content, trigger new captures, and build or parse replay URLs — all from the visual editor. Every node uses the public Internet Archive HTTP APIs; no authentication or API key is required.

Built with flow-like-wasm-sdk targeting the WASM Component Model (wasm32-wasip2).

Nodes

Node Category Description
Check Availability Query Snapshot closest to a timestamp (Availability API).
List Captures Query All captures of a URL (CDX Server) with date range, match type, collapse & filter. Outputs an array.
Get Latest Snapshot Query The most recent capture of a URL.
Get Oldest Snapshot Query The earliest capture of a URL.
Count Captures Query Number of captures for a URL, optionally within a date range.
Get TimeMap Query Mementos from a URL's Memento TimeMap (link format).
Fetch Snapshot Retrieve Download archived content at a timestamp (or latest); raw or replay mode.
Save Page Now Archive Trigger a fresh capture and resolve the resulting snapshot.
Build Wayback URL URL Construct a replay URL from URL + timestamp + mode (pure).
Parse Wayback URL URL Extract original URL, timestamp & mode from a replay URL (pure).

Shared types

Struct-typed pins keep data structured across the graph:

  • Snapshoturl, timestamp, original, status.
  • Capturetimestamp, original, mime_type, status, digest, length, url.

APIs used

API Endpoint
Availability https://archive.org/wayback/available
CDX Server https://web.archive.org/cdx/search/cdx
Save Page Now https://web.archive.org/save/{url}
Replay https://web.archive.org/web/{timestamp}[mode]/{url}
Memento TimeMap https://web.archive.org/web/timemap/link/{url}

Replay modes map to path flags: replay (toolbar page, no flag), raw (id_, original bytes), image (im_), iframe (if_). An empty/* timestamp builds the all-captures calendar URL.

Prerequisites

  • Rust toolchain (1.82+)
  • WASM target: rustup target add wasm32-wasip2 (or mise run setup)

Build

cargo build --release
# Component at: target/wasm32-wasip2/release/wayback_machine.wasm

# Or, producing node.wasm at the repo root:
mise run build

.cargo/config.toml sets wasm32-wasip2 as the default target, so a plain cargo build produces a WASM component.

Test

Unit tests run on the native host (the SDK stubs host functions during tests):

cargo test --target $(rustc -vV | grep host | awk '{print $2}')
# or: mise run test

Tests cover the pure parsing/URL logic (src/common.rs) and every node's definition. tests/lint.rs enforces package conventions (no Generic pins, struct pins carry schemas, impure nodes have exec pins, etc.).

Project layout

├── .cargo/config.toml         # Default target: wasm32-wasip2
├── Cargo.toml                 # Crate + dependencies
├── flow-like.toml             # Package manifest (metadata, node list, limits)
├── mise.toml                  # Task runner (setup/build/test/clean)
├── src/
│   ├── lib.rs                 # Module wiring + wasm_main!()
│   ├── common.rs              # Shared Snapshot/Capture types + HTTP helpers
│   └── nodes/                 # One node per file
│       ├── check_availability.rs
│       ├── list_captures.rs
│       ├── latest_snapshot.rs
│       ├── oldest_snapshot.rs
│       ├── count_captures.rs
│       ├── get_timemap.rs
│       ├── fetch_snapshot.rs
│       ├── save_page.rs
│       ├── build_wayback_url.rs
│       └── parse_wayback_url.rs
└── tests/lint.rs              # Package-wide lint checks

Add a new node by dropping a src/nodes/<name>.rs file with a #[register_node] struct and declaring it in src/nodes/mod.rs — it is auto-discovered at startup.

Publishing

  1. Build: mise run build
  2. In Flow-Like Desktop: Library → Packages → Publish
  3. Select the .wasm component and flow-like.toml
  4. Submit for review

License

MIT © Rheosoph GmbH

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages