Move native startup services into atomic_lib and separate HTTP lifetime

.dagger/src/index.ts

@@ -1505,6 +1505,9 @@ export class AtomicServer {
             `--test-threads ${this.hostKnobs.nextestTestThreads} ` +
             `--retries ${this.hostKnobs.nextestRetries}`,
         ])
+        // Compile the native runtime separately: workspace feature unification
+        // must not hide a dependency on the hosted server/Actix adapter.
+        .withExec(['sh', 'lib/tests/check-native-runtime.sh'])
         .stdout()
     );
   }

CHANGELOG.md

@@ -7,6 +7,8 @@ See [STATUS.md](server/STATUS.md) to learn more about which features will remain
 
 ## UNRELEASED
 
+- Move native storage opening, persisted identity loading and durable flushing into `atomic_lib::runtime`; reject damaged identity configs without replacing the key. Add a standalone no-Actix runtime CI check.
+- Separate embedded node startup from the optional HTTP adapter. Tauri keeps native services alive if HTTP stops; its frontend still requires HTTP/WS. This starts the HTTP-optional runtime migration ([#1196](https://github.com/ontola/atomic-server/issues/1196), [#749](https://github.com/ontola/atomic-server/issues/749)).
 - The outbox drains over a live Iroh link too (`sync::peer::LivePeerCommitTransport`):
   a device with no hub in reach delivers its queued writes to a paired peer as
   signed `COMMIT` frames, which the peer validates and applies like a hub

TESTING_COVERAGE.md

@@ -308,6 +308,29 @@ installed metadata and a dependency sentinel remain untouched. Corepack must
 have this pnpm version cached or be able to download it. A stub Cargo command
 verifies Clippy dispatch, staged input, and failure propagation; this fixture does not compile the Rust workspace.
 
+## HTTP-optional native node lifecycle
+
+`server/tests/it/http_optional.rs` reserves the configured HTTP port, starts
+`serve::run_node`, creates and queries a document through the native store/runtime,
+and explicitly attempts HTTP binding. After that bind fails, native reads,
+edits and change events still work. A second test keeps the hosted embedder
+ready hook before HTTP binding. `runtime::durable_flush::tests` in atomic_lib
+verifies that
+ending the owned flush worker releases redb and preserves its final write.
+These are Rust glue tests, not desktop UI acceptance. Tauri still starts the
+HTTP/WS adapter for its frontend. Missing: native frontend reads/commits/events,
+attachment bytes, restore and peer sync through Tauri with no HTTP listener;
+process-global Iroh teardown/restart is also not covered by this extraction.
+
+`lib/tests/native_runtime.rs` runs on plain Tokio with only `db-redb,config`:
+originless storage + identity + signed creation survive close/reopen, a retained
+config recreates an identity in a replacement database, legacy secrets resolve
+to the same key-derived DID, and malformed existing config is not overwritten.
+`lib/tests/check-native-runtime.sh` rejects atomic-server/Actix in the normal
+core dependency graph and compiles/runs those tests outside workspace feature
+unification. `rustTest` runs this isolation gate after the workspace suite.
+The gate is not a claim that the Tauri dependency graph is already server-free.
+
 ## Browser WebRTC transport (issue #1396)
 
 `browser/lib/src/webrtc-transport.test.ts` covers frame fragmentation/order,

browser/data-browser/src/views/File/convertFileToDocument.test.ts

@@ -111,32 +111,40 @@ describe('plainTextToTiptapJson', () => {
 });
 
 describe('fileContentsToTiptapJson', () => {
-  it('uses the collaborative Markdown schema so Markdown formatting becomes document nodes', async () => {
-    const json = await fileContentsToTiptapJson(
-      '# Heading\n\nThis is **bold**.',
-      'markdown',
-      new Store(),
-    );
-
-    expect(json).toMatchObject({
-      type: 'doc',
-      content: [
-        {
-          type: 'heading',
-          attrs: { level: 1 },
-          content: [{ text: 'Heading' }],
-        },
-        {
-          type: 'paragraph',
-          content: [
-            { text: 'This is ' },
-            { text: 'bold', marks: [{ type: 'bold' }] },
-            { text: '.' },
-          ],
-        },
-      ],
-    });
-  });
+  it(
+    'uses the collaborative Markdown schema so Markdown formatting becomes document nodes',
+    async () => {
+      const json = await fileContentsToTiptapJson(
+        '# Heading\n\nThis is **bold**.',
+        'markdown',
+        new Store(),
+      );
+
+      expect(json).toMatchObject({
+        type: 'doc',
+        content: [
+          {
+            type: 'heading',
+            attrs: { level: 1 },
+            content: [{ text: 'Heading' }],
+          },
+          {
+            type: 'paragraph',
+            content: [
+              { text: 'This is ' },
+              { text: 'bold', marks: [{ type: 'bold' }] },
+              { text: '.' },
+            ],
+          },
+        ],
+      });
+    },
+    // First test in this file to load @tiptap/markdown and the full
+    // collaborative editor schema graph; that cold module transform can
+    // exceed the default 5s under CI load (see vitest.integration.config.ts
+    // for the same reasoning applied elsewhere).
+    15000,
+  );
 
   it('keeps blank Markdown and an unpaired marker as document text', async () => {
     await expect(

desktop/README.md

@@ -12,6 +12,22 @@ cargo tauri dev
 cargo tauri build
 ```
 
+## Node and HTTP lifecycle
+
+The embedded node is initialized by `atomic_server_lib::serve::run_node`.
+Storage opening, identity loading and periodic durable flushing are provided
+by `atomic_lib::runtime`, with no Actix dependency in that core. Tauri binds
+its native commands to the shared `AtomicNode`, then explicitly
+starts `serve_http` for the current webview. A stopped HTTP adapter does not
+stop the native node's flush worker and peer tasks while the app is open.
+
+This is the first step of [HTTP-optional local runtime](../planning/atomic-lib-runtime.md#phase-7-tauri--android-without-loopback).
+The current frontend **still requires HTTP/WebSocket**. There is no no-HTTP
+user setting yet: reads, commits, subscriptions and attachment access must
+move to native commands/events before removing the listener or the Actix
+build dependency. Hosted servers keep the existing `serve` / `serve_with_hook`
+entry points.
+
 ## Running in development
 
 `cargo tauri dev` starts the front-end for you — `beforeDevCommand` in `tauri.conf.json` runs `pnpm -C browser/data-browser dev:tauri`, and the app points at `localhost:6747` (`devUrl`).

desktop/src/lib.rs

@@ -89,7 +89,7 @@ const PAIR_LINK_RETRY_WINDOW: std::time::Duration = std::time::Duration::from_se
 /// A handle on the embedded node, captured once it has booted.
 #[derive(Default)]
 struct EmbeddedNode {
-  store: std::sync::OnceLock<atomic_lib::Db>,
+  runtime: std::sync::OnceLock<atomic_lib::runtime::AtomicNode>,
   config_file: std::sync::OnceLock<std::path::PathBuf>,
   /// Why the node never came up, if it didn't. The server runs on its own
   /// thread, so a boot failure there used to be an unwind into nothing: the
@@ -102,8 +102,8 @@ struct EmbeddedNode {
 impl EmbeddedNode {
   /// The store, or an explanation of why there isn't one.
   fn require_store(&self) -> Result<atomic_lib::Db, String> {
-    if let Some(store) = self.store.get() {
-      return Ok(store.clone());
+    if let Some(node) = self.runtime.get() {
+      return Ok(node.db().clone());
     }
 
     Err(match self.startup_error.get() {
@@ -310,11 +310,7 @@ mod vault_ipc {
   }
 
   pub fn store_of(node: &std::sync::Arc<super::EmbeddedNode>) -> Result<atomic_lib::Db, String> {
-    node
-      .store
-      .get()
-      .ok_or_else(|| "The local node has not finished starting up.".to_string())
-      .cloned()
+    node.require_store()
   }
 
   /// Run vault work on a thread of its own.
@@ -709,12 +705,21 @@ pub fn run() {
           .expect("TLS verifier initialization did not complete");
 
         let rt = actix_rt::Runtime::new().unwrap();
-        // The hook hands us the store once it's up, so `adopt_agent` can point
-        // the node's identity at the signed-in user.
-        let result = rt.block_on(atomic_server_lib::serve::serve_with_hook(
+        // Native commands receive the shared runtime before the HTTP adapter
+        // starts, so `adopt_agent` can set the node's signed-in identity.
+        let result = rt.block_on(atomic_server_lib::serve::run_node(
           config_clone,
-          |appstate| {
-            let _ = node_for_server.store.set(appstate.store.clone());
+          |appstate| async {
+            let _ = node_for_server.runtime.set(appstate.node());
+            // The current webview still needs HTTP/WS. It is an adapter,
+            // explicitly started after native access is ready; replacing the
+            // frontend's transport will let this call become optional.
+            if let Err(error) = atomic_server_lib::serve::serve_http(appstate).await {
+              eprintln!("[node] the HTTP adapter stopped: {error}");
+            }
+            // The window and native commands outlive the HTTP adapter. Keep
+            // peer tasks and durable flushing alive even if its port is busy.
+            std::future::pending().await
           },
         ));
 
@@ -731,7 +736,7 @@ pub fn run() {
       {
         let menu = crate::menu::build(app.handle())?;
         app.handle().set_menu(menu)?;
-        system_tray::setup(app, &config)?;
+        system_tray::setup(app, &config.get_origin(), &config.config_dir)?;
       }
 
       Ok(())

desktop/src/system_tray.rs

@@ -12,7 +12,7 @@ use tauri_plugin_opener::OpenerExt;
 /// a second one. You get two identical icons in the tray, only this one having
 /// a menu (the config can't attach one), and both vanish together when the
 /// process dies — which reads as a rendering glitch rather than two real icons.
-pub fn setup(app: &mut App, config: &atomic_server_lib::config::Config) -> tauri::Result<()> {
+pub fn setup(app: &mut App, origin: &str, config_dir: &std::path::Path) -> tauri::Result<()> {
   let open = MenuItem::with_id(app, "open", "Open", true, None::<&str>)?;
   let browser = MenuItem::with_id(app, "browser", "Open in browser", true, None::<&str>)?;
   let config_item = MenuItem::with_id(app, "config", "Config folder", true, None::<&str>)?;
@@ -22,8 +22,8 @@ pub fn setup(app: &mut App, config: &atomic_server_lib::config::Config) -> tauri
 
   let menu = Menu::with_items(app, &[&open, &browser, &config_item, &docs, &sep, &quit])?;
 
-  let origin = config.get_origin();
-  let config_dir = config.config_dir.to_str().unwrap().to_string();
+  let origin = origin.to_owned();
+  let config_dir = config_dir.to_string_lossy().into_owned();
 
   TrayIconBuilder::new()
     .icon(app.default_window_icon().unwrap().clone())

lib/Cargo.toml

@@ -73,6 +73,7 @@ wasm-bindgen-futures = { version = "0.4.72", optional = true }
 web-sys = { version = "0.3.99", optional = true, features = ["DomException", "FileSystemDirectoryHandle", "FileSystemFileHandle", "FileSystemGetFileOptions", "FileSystemSyncAccessHandle", "FileSystemReadWriteOptions", "StorageManager", "WorkerGlobalScope", "WorkerNavigator"] }
 
 [dev-dependencies]
+tempfile = "3"
 criterion = { version = "0.8.2", features = ["async_tokio"] }
 iai = "0.1"
 lazy_static = "1"

lib/src/runtime/durable_flush.rs

@@ -0,0 +1,78 @@
+//! Owned native durability worker, independent of HTTP and async executors.
+
+/// Fsync runs off the async executor. Dropping the lifecycle wakes the worker
+/// immediately, performs a final flush and joins it; the old detached loop
+/// kept the database open forever after a failed bind or an embedder exit.
+#[must_use = "Keep the flush guard alive for the lifetime of the node"]
+pub struct DurableFlush {
+    stop: Option<std::sync::mpsc::Sender<()>>,
+    thread: Option<std::thread::JoinHandle<()>>,
+}
+
+impl DurableFlush {
+    pub(crate) fn start(store: crate::Db) -> std::io::Result<Self> {
+        let (stop, rx) = std::sync::mpsc::channel();
+        let thread = std::thread::Builder::new()
+            .name("durable-flush".into())
+            .spawn(move || loop {
+                let stopping = !matches!(
+                    rx.recv_timeout(std::time::Duration::from_millis(100)),
+                    Err(std::sync::mpsc::RecvTimeoutError::Timeout)
+                );
+                if let Err(error) = store.flush() {
+                    tracing::warn!("durable flush failed: {error}");
+                }
+                if stopping {
+                    break;
+                }
+            })?;
+        Ok(Self {
+            stop: Some(stop),
+            thread: Some(thread),
+        })
+    }
+}
+
+impl Drop for DurableFlush {
+    fn drop(&mut self) {
+        self.stop.take();
+        if let Some(thread) = self.thread.take() {
+            if thread.join().is_err() {
+                tracing::error!("durable-flush thread panicked");
+            }
+        }
+    }
+}
+
+#[cfg(all(test, feature = "db-redb"))]
+mod tests {
+    use super::DurableFlush;
+    use crate::{urls, Resource, Storelike, Value};
+
+    #[tokio::test]
+    async fn dropping_flush_worker_releases_database_and_persists_final_write() {
+        let dir = tempfile::tempdir().unwrap();
+        let path = dir.path().join("node.redb");
+        let blobs = dir.path().join("blobs");
+        let db = crate::Db::init_redb_file(&path, None, &blobs)
+            .await
+            .unwrap();
+        let worker = DurableFlush::start(db.clone()).unwrap();
+        let mut resource = Resource::new("did:ad:flush-test".into());
+        resource
+            .set_unsafe(urls::NAME.into(), Value::String("Durable".into()))
+            .unwrap();
+        db.add_resource_opts(&resource, false, false, true)
+            .await
+            .unwrap();
+        drop(db);
+        // The worker owns the final Db reference. A detached loop would keep
+        // redb locked and reopening below would fail with DatabaseAlreadyOpen.
+        drop(worker);
+        let reopened = crate::Db::init_redb_file(&path, None, &blobs)
+            .await
+            .unwrap();
+        let resource = reopened.get_resource(resource.get_subject()).await.unwrap();
+        assert_eq!(resource.get(urls::NAME).unwrap().to_string(), "Durable");
+    }
+}