Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
06b1611
Add a local-first Python SDK wrapping atomic_lib via PyO3.
cursoragent Aug 16, 2026
73ff4c0
Fix Python SDK query drive scope, persistence reopen, and changelog.
cursoragent Aug 16, 2026
5921c0a
Plan a Kotlin SDK: same v1 scope as Python, UniFFI over a shared Rust…
cursoragent Aug 16, 2026
9a86206
Clarify why the Python/Kotlin v1 SDKs omit Iroh: scope and SyncSessio…
cursoragent Aug 16, 2026
a9716ff
Add Iroh P2P to the Python SDK: start_peer, sync_with, live save.
cursoragent Aug 16, 2026
c65f76a
Fix wait_for async shape and lock Iroh deps for the Python crate.
cursoragent Aug 16, 2026
e3c78a4
Fix Iroh wait_for runtime and isolate the two-process sync test.
cursoragent Aug 16, 2026
03e2422
Add a Kotlin / UniFFI SDK over atomic_lib with Iroh P2P.
cursoragent Aug 16, 2026
3d18540
Document Windows MSVC for the Python SDK and test it in CI.
cursoragent Aug 19, 2026
c2d08e5
Expose HTTP schema fetch, search, and save_remote on the Python and K…
cursoragent Aug 19, 2026
176884c
Regenerate Kotlin bindings and cache HTTP GET under the resource subj…
cursoragent Aug 19, 2026
b986396
Document property/value query and upload Python abi3 wheels from CI.
cursoragent Aug 23, 2026
9efb14e
fix(python): point PERSONAL_DRIVE export at renamed PRIVATE_DRIVE con…
claude Sep 18, 2026
c827c56
chore(sdk): refresh Python and UniFFI lockfiles for atomic_lib 0.41.0…
cursoragent Sep 18, 2026
d08f54f
ci: honor [hosted-ci] to run Main on GitHub-hosted runners [hosted-ci]
cursoragent Sep 18, 2026
9c2b4a1
ci: match hosted-ci and full Playwright flags as standalone tokens [h…
cursoragent Sep 18, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ concurrency:
# Escape hatch (force hosted, skip Mancave):
# Settings -> Secrets and variables -> Actions -> Variables
# CI_RUNNER = ["ubuntu-latest"]
# Or `[hosted-ci]` standing alone in the commit message (same opt-in
# shape as the full Playwright token). A mention inside a sentence does
# not count — pick greps the whole message. Use it when the self-hosted
# runner is online but the Dagger engine on it cannot start.
#
# Only this workflow uses the self-hosted runner. Releasing/deploying stay
# on ubuntu-latest — a release must not depend on one PC being awake.
Expand All @@ -61,6 +65,7 @@ jobs:
FORCE_RUNNER: ${{ vars.CI_RUNNER }}
EVENT_NAME: ${{ github.event_name }}
REPO: ${{ github.repository }}
COMMIT_MSG: ${{ github.event.head_commit.message }}
run: |
set -euo pipefail

Expand All @@ -76,6 +81,12 @@ jobs:
exit 0
fi

if printf '%s' "${COMMIT_MSG:-}" | grep -qE '(^|[[:space:]])\[hosted-ci\]([[:space:]]|$)'; then
echo "[hosted-ci] in commit message — forcing GitHub-hosted"
echo "host=hosted" >> "$GITHUB_OUTPUT"
exit 0
fi

if [ -z "${GH_TOKEN:-}" ]; then
echo "No CI_RUNNER_STATUS_TOKEN — assuming Mancave online"
echo "host=mancave" >> "$GITHUB_OUTPUT"
Expand Down Expand Up @@ -131,7 +142,7 @@ jobs:
mode=full
elif [[ "$REF" == refs/tags/v* ]]; then
mode=full
elif printf '%s' "${COMMIT_MSG:-}" | grep -q '\[full-e2e\]'; then
elif printf '%s' "${COMMIT_MSG:-}" | grep -qE '(^|[[:space:]])\[full-e2e\]([[:space:]]|$)'; then
mode=full
elif prs=$(gh api "repos/$REPO/commits/$SHA/pulls" -H "Accept: application/vnd.github+json" 2>/dev/null); then
if printf '%s' "$prs" | jq -e 'any(.[]; any(.labels[]; .name == "full-e2e"))' >/dev/null; then
Expand Down
53 changes: 53 additions & 0 deletions .github/workflows/python-sdk.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Python SDK tests (not part of Dagger).
#
# The extension is a PyO3 cdylib. On Windows that needs MSVC's linker;
# ubuntu-latest and windows-latest both have one. A machine without
# Visual Studio Build Tools cannot `uv run pytest` from source.

name: Python SDK

on:
pull_request:
paths:
- "python/**"
- "lib/**"
- ".github/workflows/python-sdk.yml"
push:
paths:
- "python/**"
- "lib/**"
- ".github/workflows/python-sdk.yml"

permissions:
contents: read

jobs:
test:
name: pytest (${{ matrix.os }})
runs-on: ${{ matrix.os }}
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
defaults:
run:
working-directory: python
env:
CARGO_TERM_COLOR: always
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
with:
workspaces: python
- uses: astral-sh/setup-uv@v10.0.1
- name: uv run pytest
run: uv run pytest -q
- name: Build abi3 wheel
run: uv run maturin build --release --out dist
- uses: actions/upload-artifact@v4
with:
name: atomic-data-${{ matrix.os }}
path: python/dist/*.whl
if-no-files-found: error
11 changes: 11 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -47,3 +47,14 @@ scratchpad
node_modules/
# Generated integration certification evidence
/artifacts/integration-certification/

# Python / UniFFI SDKs (excluded from the Cargo workspace, own target/)
python/target
python/.venv
python/*.egg-info
ffi/target
ffi/kotlin/.gradle
ffi/kotlin/build
**/__pycache__
**/.pytest_cache
*.so
4 changes: 4 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,8 @@ Atomic Server is a graph database with real-time sync, built on **Loro CRDT** fo
- **`@tomic/react`** (`browser/react/`) — React hooks.
- **`data-browser`** (`browser/data-browser/`) — The web app (React + TipTap + Loro), feels similar to notion. See the related AGENTS.md
- **`flutter/`** — Cross-platform canvas app (Android/iOS/Web). Uses `flutter_rust_bridge` to call `atomic_lib`. See `flutter/README.md` and `flutter/AGENTS.md`.
- **`python/`** — Python SDK (`atomic_data`). PyO3 bindings over `atomic_lib` (local redb, Iroh, HTTP schema/search/`save_remote`). Excluded from the Cargo workspace; build with `maturin`. See `python/README.md` and `planning/python-sdk.md`.
- **`ffi/`** — UniFFI crate (`atomic-ffi`) and Kotlin SDK (`dev.atomicdata`). Same local redb + Iroh + HTTP surface as Python. Excluded from the Cargo workspace. See `ffi/README.md` and `planning/kotlin-sdk.md`.

### Data model

Expand Down Expand Up @@ -281,6 +283,8 @@ cd browser/lib && pnpm test # JS unit tests
cd browser && pnpm run -r build # Full workspace build
cd browser && pnpm run test-e2e:light # Playwright @smoke (feature-branch CI)
cd browser && pnpm run test-e2e # Full Playwright suite (develop / tags)
cd python && uv run pytest # Python SDK (excluded from workspace)
cd ffi && cargo test && cd kotlin && ./gradlew test # Kotlin / UniFFI SDK (excluded from workspace)
```

`atomic_lib`'s unit tests need the `db` feature — `hierarchy.rs`'s test module
Expand Down
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,11 @@ See [STATUS.md](server/STATUS.md) to learn more about which features will remain
"Publish :develop image" workflow covers the case where the pipeline cannot
go green for reasons unrelated to whether the binary builds; it refuses
`latest` and `v*` tags, which release.yml owns.
- CI: `[hosted-ci]` standing alone in a commit message runs Main on
GitHub-hosted runners (same opt-in shape as the full Playwright token).
A mention inside a sentence does not count. The pick job treats a
self-hosted runner as available whenever it is online, so a box whose
Dagger engine cannot start still never falls back.

- Error-handling hygiene on the commit and read paths. The legacy
`set`/`push`/`remove` rejection in `sync::engine::ingest_commit` now checks
Expand Down Expand Up @@ -363,6 +368,8 @@ See [STATUS.md](server/STATUS.md) to learn more about which features will remain
`ingest_commit_json` serializes; `sync::ws_apply::apply_commit_json` now
returns the `CommitResponse` instead of `()`. See
`planning/runtime-boundary-decision.md`.
- **Python SDK** (`python/`, import `atomic_data`): bindings over `atomic_lib` via PyO3. Local redb plus Iroh P2P (`start_peer`, `sync_with`, live push on save). HTTP GET of `https://` subjects (schema / external resources); optional `server=` for `/search` and `save_remote()`. GitHub Actions uploads Linux/Windows abi3 wheels (no PyPI yet).
- **Kotlin SDK** (`ffi/`, package `dev.atomicdata`): UniFFI bindings over `atomic_lib`. Same local redb + Iroh + HTTP surface as Python (`startPeer`, `syncWith`, `search`, `saveRemote`). JVM tests included; Android AAR is later.

## [v0.41.0-beta.2] - 2026-08-01

Expand Down
4 changes: 3 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ next release reintroduces the bug.
- We try to test at every level, unit tests, integration tests, e2e tests (playwright).
- When tests fail, first make sure the unit tests are green, then do integration tests, then to e2e.
- If e2e tests fail, try walking through the steps 1 by 1 either with the playwright debugger, or by simply reproducing the steps in your browser of choice.
- Feature-branch CI runs Playwright **light** (`@smoke`). `develop` and `v*` tags run the **full** suite. Opt in to full on a branch with a `full-e2e` PR label, `[full-e2e]` in the commit message, or `workflow_dispatch` `e2e_mode=full`. See `planning/e2e-light-heavy.md`.
- Feature-branch CI runs Playwright **light** (`@smoke`). `develop` and `v*` tags run the **full** suite. Opt in to full on a branch with a `full-e2e` PR label, `[full-e2e]` in the commit message, or `workflow_dispatch` `e2e_mode=full`. See `planning/e2e-light-heavy.md`. Put `[hosted-ci]` in a commit message to run Main on GitHub-hosted runners when the self-hosted Dagger engine cannot start.

Feature-specific browser journeys belong in `browser/e2e/tests/` and run through
the shared CI pipeline. For a focused run with CI's service setup, use Dagger
Expand Down Expand Up @@ -393,6 +393,8 @@ Two consequences worth knowing:
with a `full-e2e` PR label, `[full-e2e]` in the commit message, or
`workflow_dispatch` `e2e_mode=full`. (Do not name the Dagger flag
`--e2e-mode`: the CLI camelCases it to `e2EMode` and the call fails.)
`[hosted-ci]` in the commit message is a GitHub Actions pick-job flag
(Main on ubuntu-latest); it is not a Dagger argument.

Every deploy then has to prove itself: the job polls `/server` on the target
until it answers `200` (with enough patience for a store migration). A deploy
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ members = [
"plugin-runtime",
"tools/cargo-bin",
]
exclude = ["flutter/rust", "integrations/localthought/syncables"]
exclude = ["flutter/rust", "integrations/localthought/syncables", "python", "ffi"]

# Debuginfo dominates target/ size: with ~1460 deps (tauri, actix, iroh) the
# default `debug = true` produces a multi-GB tree per build flavor, and this
Expand Down
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ This repo also includes:
- [`atomic_lib`](lib/README.md) Rust library.
- [`atomic-cli`](cli/README.md) terminal client.
- [`flutter`](/flutter) a Dart / Flutter client for Atomic Data, plus AtomicCanvas, a collaborative infinite drawing canvas that syncs peer-to-peer between devices.
- [`python`](/python) local-first Python SDK (`atomic_data`) wrapping `atomic_lib` via PyO3.
- [`ffi`](/ffi) UniFFI crate and Kotlin SDK (`dev.atomicdata`) wrapping `atomic_lib`.
- [`docs`](docs/README.md) documentation / specification for Atomic Data ([docs.atomicdata.dev](https://docs.atomicdata.dev)).

_Status: alpha. [Breaking changes](CHANGELOG.md) are expected until 1.0._
Expand Down Expand Up @@ -48,7 +50,7 @@ _Status: alpha. [Breaking changes](CHANGELOG.md) are expected until 1.0._
- 📲 **Invite and sharing system** with [Atomic Invites](https://docs.atomicdata.dev/invitations.html)
- 🌐 **Embedded server** with support for HTTP / HTTPS / HTTP2.0 (TLS) and Built-in LetsEncrypt handshake.
- 📱 **Runs on mobile**: `atomic_lib` compiles into Flutter apps through [flutter_rust_bridge](https://github.com/fzyzcjy/flutter_rust_bridge), so phones get the same local-first store, signing and peer sync as the browser, not a thin REST wrapper. See [`/flutter`](/flutter).
- 📚 **Libraries**: [Javascript / Typescript](https://www.npmjs.com/package/@tomic/lib), [React](https://www.npmjs.com/package/@tomic/react), [Svelte](https://www.npmjs.com/package/@tomic/svelte), [Rust](https://crates.io/crates/atomic-lib), and a [Dart / Flutter client](/flutter/lib/atomic)
- 📚 **Libraries**: [Javascript / Typescript](https://www.npmjs.com/package/@tomic/lib), [React](https://www.npmjs.com/package/@tomic/react), [Svelte](https://www.npmjs.com/package/@tomic/svelte), [Rust](https://crates.io/crates/atomic-lib), a [Dart / Flutter client](/flutter/lib/atomic), a [Python SDK](/python) (`atomic_data`), and a [Kotlin SDK](/ffi) (`dev.atomicdata`)

https://user-images.githubusercontent.com/2183313/139728539-d69b899f-6f9b-44cb-a1b7-bbab68beac0c.mp4

Expand Down
25 changes: 25 additions & 0 deletions TESTING_COVERAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -449,6 +449,8 @@ templates, and offline variants stay in the full suite. Policy:
| Browser e2e full | `cd browser && pnpm run test-e2e` | `endToEnd` on `develop` and `v*` tags |
| Flutter Dart | `cd flutter && flutter test` | `flutterTest` |
| Flutter Rust bridge | `cargo test --manifest-path flutter/rust/Cargo.toml` | `flutterTest` |
| Python SDK | `cd python && uv run pytest` | `Python SDK` (ubuntu + windows) |
| Kotlin / UniFFI SDK | `cd ffi && cargo test && cd kotlin && ./gradlew test` | **not in CI** |

CI runs `cargo nextest run --workspace --exclude atomic-server-tauri
--no-default-features --features light`. Feature unification pulls in
Expand All @@ -459,6 +461,14 @@ Two things worth knowing about the runners:
- **`flutter/rust` is excluded from the workspace** (root `Cargo.toml`), so
`--workspace` never compiles it. It is covered only by the explicit
`--manifest-path` step in `flutterTest`.
- **`python/` is excluded from the workspace** for the same reason (PyO3).
Tests are `uv run pytest` in `python/`. GitHub Actions workflow
`Python SDK` runs them on ubuntu-latest and windows-latest (MSVC is on
the Windows runner; a host without Visual Studio Build Tools cannot
link the extension). Not in Dagger.
- **`ffi/` is excluded from the workspace** (UniFFI + Iroh). Rust tests are
`cargo test` in `ffi/`; JVM tests are `./gradlew test` in `ffi/kotlin`.
Not in Dagger CI yet.
- **`.config/nextest.toml` sets `retries = 2`.** A flaky test passes CI
silently. Check for `FLAKY` in nextest output, not just the summary line.

Expand Down Expand Up @@ -644,6 +654,21 @@ nothing to test yet. Listed so it is not mistaken for covered.

One 13-line smoke test, never run in CI — the pipeline has no emulator.

### 7b. Python SDK (`python/`)

Glue: `uv run pytest` covers in-memory CRUD, file-backed reopen, HTTP schema
`get()` / `search()`-requires-server / `save_remote()` error path, and a
two-process Iroh sync (`tests/test_iroh.py`). GitHub Actions `Python SDK`
runs that on Linux and Windows. Not in Dagger. No WS session or blobs.

### 7c. Kotlin / UniFFI SDK (`ffi/`)

Glue: `cargo test` in `ffi/` covers the same local CRUD + Iroh-guard + HTTP
schema/`search` cases as Python, in-process. `ffi/kotlin` JUnit covers the
generated bindings plus a two-process Iroh sync (`IrohTest.twoProcessIrohSync`).
Not in Dagger CI. No Android AAR, WS session, or blobs. `startPeer` is
process-global — only one JVM test may start Iroh.

### 8. Known residual races

None outstanding. The concurrent-writer bug that lived here — a local edit
Expand Down
2 changes: 2 additions & 0 deletions docs/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@
- [Rust](rust.md)
- [CLI](rust-cli.md)
- [Lib](rust-lib.md)
- [Python](python.md)
- [Kotlin](kotlin.md)

# Guides

Expand Down
2 changes: 1 addition & 1 deletion docs/src/atomic-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ It's free, open source (MIT license), and has a ton of features:
- 📲 **Invite and sharing system** with [Atomic Invites](https://docs.atomicdata.dev/invitations.html)
- 🌐 **Embedded server** with support for HTTP / HTTPS / HTTP2.0 (TLS) and Built-in LetsEncrypt handshake.
- 📱 **Runs on mobile**: `atomic_lib` compiles into Flutter apps through [flutter_rust_bridge](https://github.com/fzyzcjy/flutter_rust_bridge), so phones get the same local-first store, signing and peer sync as the browser — not a thin REST wrapper.
- 📚 **Libraries**: [Javascript / Typescript](https://www.npmjs.com/package/@tomic/lib), [React](https://www.npmjs.com/package/@tomic/react), [Svelte](https://www.npmjs.com/package/@tomic/svelte), [Rust](https://crates.io/crates/atomic-lib), and a Dart / Flutter client
- 📚 **Libraries**: [Javascript / Typescript](https://www.npmjs.com/package/@tomic/lib), [React](https://www.npmjs.com/package/@tomic/react), [Svelte](https://www.npmjs.com/package/@tomic/svelte), [Rust](https://crates.io/crates/atomic-lib), a Dart / Flutter client, a [Python SDK](python.md), and a [Kotlin SDK](kotlin.md)

## Document undo and redo

Expand Down
111 changes: 111 additions & 0 deletions docs/src/kotlin.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
{{#title Kotlin SDK for Atomic Data}}

# Kotlin SDK

Local-first [Atomic Data](atomic-data-overview.md) for the JVM. The package
wraps [`atomic_lib`](rust-lib.md) through
[UniFFI](https://mozilla.github.io/uniffi-rs/) — the same Rust store the
browser (WASM), Flutter app, and [Python SDK](python.md) use. Reads and
writes go to a local [redb](https://github.com/cberner/redb) file. Edits are
signed Loro commits. A server is optional for local CRUD and Iroh; HTTP GET /
search / `saveRemote()` are still available.

Source: [`ffi/`](https://github.com/atomicdata-dev/atomic-server/tree/develop/ffi)
(crate `atomic-ffi`, Kotlin package `dev.atomicdata`).

Python stays on PyO3. This UniFFI surface is what Swift and the Android
Binder host will share later.

## Install

From a checkout of this repo (needs a Rust toolchain and JDK 21):

```bash
cd ffi
cargo build
./generate-kotlin.sh
cd kotlin && ./gradlew test
```

Point `jna.library.path` at `ffi/target/debug` so the JVM can load
`libatomic_ffi`.

## Quick start

```kotlin
import dev.atomicdata.Store
import dev.atomicdata.Urls

val store = Store.open("./my-atomic-data")
val setup = store.setup("Ada")

val note = store.create(
Urls.PLAIN_TEXT,
"Hello",
null,
mapOf("description" to "A locally stored note"),
)
note.set(Urls.DESCRIPTION, "Edited offline")
note.save()

val got = store.get(note.subject())
println("${got?.get("name")} ${got?.get("description")}")

for (child in store.query(setup.driveSubject, null, null, null, null, 0u)) {
println("${child.subject()} ${child.name()}")
}

store.flush()
```

`setup.agentSecret` is the only way to sign writes after you reopen the
store. Keep it.

```kotlin
val store = Store.open("./my-atomic-data")
store.loadAgent(secret)
```

`Store.inMemory()` is the same API without a directory.

`Resource.destroyResource()` deletes the resource. It is not named
`destroy()` because UniFFI already uses that for FFI handle teardown.

## HTTP (schema, search, remote save)

The core ontology is bundled, so validation works offline. Unknown Class /
Property URLs are loaded with `store.get("https://…")` (HTTP GET + cache).

```kotlin
val store = Store.open("./my-atomic-data", "https://atomicdata.dev")
// or later: store.setServer("https://atomicdata.dev")

val schema = store.get("https://atomicdata.dev/classes/Bookmark")
val hits = store.search("notes", 10u)
note.saveRemote() // POST /commit; did:ad: needs store.server()
```

`has(subject)` is local-only. `search()` errors if no server is set. `save()`
stays local; `saveRemote()` is the HTTP POST.

## P2P sync (Iroh)

```kotlin
val node = store.startPeer() // did:ad:node:…
store.announce(null) // optional pkarr publish
// On the other device, same agent secret (or a grant), then:
other.syncWith(node, null)
```

After the first sync, connected peers get live Loro updates on `.save()`.
`store.waitFor(subject, timeoutSecs)` blocks until a local or peer change
lands.

Two Iroh nodes cannot share one JVM — `startPeer` is process-global, same
as Flutter and Python. Use two processes (or two devices).

## What this is not (yet)

WebSocket-to-server `SyncSession` is not wrapped. Blobs and history are not
wrapped. An Android AAR (`cargo-ndk`) and Binder host are later layers.
Iroh P2P is.
Loading
Loading