From 49e05c4b65075df7ce665f61adaba2336677f0cb Mon Sep 17 00:00:00 2001 From: Tam Nguyen Duc <1218621+tamnd@users.noreply.github.com> Date: Thu, 20 Aug 2026 09:14:58 +0700 Subject: [PATCH] One connection, and settings that came as text This is the last of the C ABI that was not bound, and none of it is a new thing the engine can do. It is the same calls with less to write around them. A database and a connection are two objects because they are two things, and a program querying from four threads wants one of the first and four of the second. A program that wants exactly one should not have to say so twice, so Connection.open, Connection.create and Connection.memory make the database inside the call and let go of it. Nothing is lost by that: a connection carries its own file handle and a database holds only the path and the configuration. What is given up is the second connection, and duplicate is the way back to one. Settings usually arrive as text, and a program holding a key and a value has no business knowing which of three fields they land in. Config.with takes one by name and Config.of takes a whole map, and both forward to the engine rather than matching the key here, so a key the engine grows later works without a release of this client and a key that never existed is refused with the typo named. Twelve tests. The one worth pointing at is that 512MB is refused rather than read as a number: the two readings of that suffix differ by 4.9%, and the place to decide which one somebody meant is where they typed it. --- README.md | 20 +++ zudb-ffm/src/main/java/dev/zudb/ffm/Abi.java | 12 ++ .../main/java/dev/zudb/ffm/FfmBinding.java | 61 ++++++++ .../test/java/dev/zudb/ffm/ShorthandTest.java | 141 ++++++++++++++++++ zudb/src/main/java/dev/zudb/Config.java | 51 +++++++ zudb/src/main/java/dev/zudb/Connection.java | 74 +++++++++ .../src/main/java/dev/zudb/spi/ZuBinding.java | 50 +++++++ 7 files changed, 409 insertions(+) create mode 100644 zudb-ffm/src/test/java/dev/zudb/ffm/ShorthandTest.java diff --git a/README.md b/README.md index a46e123..13c6a70 100644 --- a/README.md +++ b/README.md @@ -165,6 +165,26 @@ The callback runs on a thread of the library's, one per statement, never two at This is the other of the two places a pointer to Java code goes the other way. An exception crossing an upcall would take the JVM down, so a watcher that throws is logged and answered as though it had asked for the statement to stop, which is the reading that loses least: a callback that threw is a program that has stopped wanting the answer. +## One connection, and settings that came as text + +A `Database` and a `Connection` are two objects because they are two things: the path and the configuration on one side, the caches and the plan cache and the file handle on the other, and a program that queries from four threads wants one of the first and four of the second. A program that wants exactly one connection should not have to say so twice, so it does not have to: + +```java +try (Connection conn = Connection.open("social.zu1")) { + ... +} +``` + +`Connection.create(path)` is the same over a file that is not there yet, and `Connection.memory()` is the same over a graph that is nowhere, which is the shortest thing here that runs a statement. All three make the database inside the call and let go of it, which costs nothing, since a connection carries its own file handle and a database holds only the path. What they give up is the second connection, and `conn.duplicate()` is the way back to one. + +Settings usually arrive as text, out of a properties file or a connection string or a command line, and a program with a key and a value has no business knowing which field of `Config` they land in: + +```java +Config config = Config.of(Map.of("threads", "1", "memory_limit", "1073741824")); +``` + +The keys and the parsing belong to the engine rather than to this client, so a key added to the engine since this client was built works anyway, and a key that never existed is refused with the typo named. A suffix such as `MB` is deliberately not parsed anywhere: its two readings differ by 4.9%, and the place to decide which one a user meant is where the user typed it. + ## How it binds The Foreign Function and Memory API is the primary path. The downcall handles are written by hand against `zu.h` rather than generated with `jextract`, because the C ABI here is around seventy functions with a stable shape, and a hand-written layer is where the interesting decisions live: which calls are `Linker.Option.critical` because they are short pure accessors, where the out-parameter scratch space comes from so that a query does not allocate, and how a `zu_error` becomes a typed Java exception exactly once. There is no native code in this repository beyond `libzu` itself. diff --git a/zudb-ffm/src/main/java/dev/zudb/ffm/Abi.java b/zudb-ffm/src/main/java/dev/zudb/ffm/Abi.java index b1469c9..ae303d9 100644 --- a/zudb-ffm/src/main/java/dev/zudb/ffm/Abi.java +++ b/zudb-ffm/src/main/java/dev/zudb/ffm/Abi.java @@ -56,6 +56,8 @@ final class Abi { final MethodHandle errorExcerpt; final MethodHandle errorFree; + final MethodHandle configSet; + final MethodHandle databaseOpen; final MethodHandle databaseCreate; final MethodHandle databaseMemory; @@ -64,6 +66,9 @@ final class Abi { final MethodHandle databaseClose; final MethodHandle connect; + final MethodHandle openOne; + final MethodHandle createOne; + final MethodHandle memoryOne; final MethodHandle connDuplicate; final MethodHandle connClose; final MethodHandle connInterrupt; @@ -188,6 +193,10 @@ final class Abi { errorExcerpt = h("zu_error_excerpt", FunctionDescriptor.of(ADDRESS, ADDRESS, ADDRESS)); errorFree = h("zu_error_free", FunctionDescriptor.ofVoid(ADDRESS)); + configSet = + h("zu_config_set", FunctionDescriptor.of( + JAVA_INT, ADDRESS, ADDRESS, SIZE_T, ADDRESS, SIZE_T, ADDRESS)); + databaseOpen = h("zu_database_open", FunctionDescriptor.of(JAVA_INT, ADDRESS, SIZE_T, ADDRESS, ADDRESS, ADDRESS)); databaseCreate = @@ -198,6 +207,9 @@ final class Abi { databaseClose = h("zu_database_close", FunctionDescriptor.ofVoid(ADDRESS)); connect = h("zu_connect", FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS, ADDRESS)); + openOne = h("zu_open", FunctionDescriptor.of(JAVA_INT, ADDRESS, SIZE_T, ADDRESS, ADDRESS)); + createOne = h("zu_create", FunctionDescriptor.of(JAVA_INT, ADDRESS, SIZE_T, ADDRESS, ADDRESS)); + memoryOne = h("zu_memory", FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS)); connDuplicate = h("zu_conn_duplicate", FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS, ADDRESS)); connClose = h("zu_conn_close", FunctionDescriptor.ofVoid(ADDRESS)); connInterrupt = h("zu_conn_interrupt", FunctionDescriptor.of(JAVA_INT, ADDRESS)); diff --git a/zudb-ffm/src/main/java/dev/zudb/ffm/FfmBinding.java b/zudb-ffm/src/main/java/dev/zudb/ffm/FfmBinding.java index 8ab8368..d51ccb2 100644 --- a/zudb-ffm/src/main/java/dev/zudb/ffm/FfmBinding.java +++ b/zudb-ffm/src/main/java/dev/zudb/ffm/FfmBinding.java @@ -77,6 +77,29 @@ public String version() { } } + @Override + public long[] configSet( + long memoryLimit, long threads, boolean readOnly, String key, String value) { + Scratch s = Scratch.get(); + MemorySegment sl = s.slots(); + MemorySegment cfg = config(s, memoryLimit, threads, readOnly); + MemorySegment k = s.utf8(key); + MemorySegment v = s.utf8(value); + clear(sl); + try { + int st = + (int) + abi.configSet.invokeExact( + cfg, k, k.byteSize(), v, v.byteSize(), sl.asSlice(ERR, 8)); + check("zu_config_set", st, sl); + return new long[] { + cfg.get(JAVA_LONG, 8), cfg.get(JAVA_LONG, 16), cfg.get(JAVA_INT, 24) + }; + } catch (Throwable t) { + throw fail("zu_config_set", t); + } + } + @Override public long databaseOpen(String path, long memoryLimit, long threads, boolean readOnly) { return openOrCreate(abi.databaseOpen, "zu_database_open", path, memoryLimit, threads, readOnly); @@ -145,6 +168,30 @@ public long connect(long db) { return handle(abi.connect, "zu_connect", db); } + @Override + public long open(String path) { + return one(abi.openOne, "zu_open", path); + } + + @Override + public long create(String path) { + return one(abi.createOne, "zu_create", path); + } + + @Override + public long memory() { + Scratch s = Scratch.get(); + MemorySegment sl = s.slots(); + clear(sl); + try { + int st = (int) abi.memoryOne.invokeExact(sl.asSlice(OUT, 8), sl.asSlice(ERR, 8)); + check("zu_memory", st, sl); + return sl.get(ADDRESS, OUT).address(); + } catch (Throwable t) { + throw fail("zu_memory", t); + } + } + @Override public long connDuplicate(long conn) { return handle(abi.connDuplicate, "zu_conn_duplicate", conn); @@ -1393,6 +1440,20 @@ private long openOrCreate( } } + private long one(java.lang.invoke.MethodHandle mh, String what, String path) { + Scratch s = Scratch.get(); + MemorySegment sl = s.slots(); + MemorySegment p = s.utf8(path); + clear(sl); + try { + int st = (int) mh.invokeExact(p, p.byteSize(), sl.asSlice(OUT, 8), sl.asSlice(ERR, 8)); + check(what, st, sl); + return sl.get(ADDRESS, OUT).address(); + } catch (Throwable t) { + throw fail(what, t); + } + } + private long handle(java.lang.invoke.MethodHandle mh, String what, long in) { Scratch s = Scratch.get(); MemorySegment sl = s.slots(); diff --git a/zudb-ffm/src/test/java/dev/zudb/ffm/ShorthandTest.java b/zudb-ffm/src/test/java/dev/zudb/ffm/ShorthandTest.java new file mode 100644 index 0000000..64b5951 --- /dev/null +++ b/zudb-ffm/src/test/java/dev/zudb/ffm/ShorthandTest.java @@ -0,0 +1,141 @@ +package dev.zudb.ffm; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import dev.zudb.Config; +import dev.zudb.Connection; +import dev.zudb.Result; +import dev.zudb.ZuException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.LinkedHashMap; +import java.util.Map; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +/** + * The one-call openers, and a configuration that arrives as text. + * + *

Neither of these is a new thing the engine can do. They are the same + * calls with less to write around them, which is the whole of what they are + * for: a script that wants one connection should not have to close two + * objects, and a program reading its settings out of a file should not have to + * know which of three fields a key lands in. + */ +class ShorthandTest { + + @BeforeAll + static void engine() { + Libzu.require(); + } + + @Test + void aConnectionInMemoryNeedsNothingAroundIt() { + try (Connection conn = Connection.memory(); + Result r = conn.query("RETURN 1 AS one")) { + assertEquals(1L, r.row(0).getLong(0)); + } + } + + @Test + void aConnectionInMemoryCanBeDuplicatedOntoTheSameGraph() { + try (Connection first = Connection.memory(); + Connection second = first.duplicate(); + Result r = second.query("RETURN 1 AS one")) { + assertEquals(1L, r.row(0).getLong(0)); + assertFalse(first.isClosed()); + } + } + + @Test + void aConnectionOnAFileMakesItAndOpensItAgain(@TempDir Path dir) { + Path file = dir.resolve("graph.zu"); + try (Connection conn = Connection.create(file)) { + assertFalse(conn.isClosed()); + } + assertTrue(Files.isRegularFile(file)); + try (Connection conn = Connection.open(file); + Result r = conn.query("RETURN 2 AS two")) { + assertEquals(2L, r.row(0).getLong(0)); + } + } + + @Test + void creatingOverSomethingAlreadyThereIsRefused(@TempDir Path dir) { + Path file = dir.resolve("graph.zu"); + Connection.create(file).close(); + assertThrows(ZuException.class, () -> Connection.create(file).close()); + } + + @Test + void openingSomethingThatIsNotThereIsRefused(@TempDir Path dir) { + assertThrows(ZuException.class, () -> Connection.open(dir.resolve("nothing.zu")).close()); + } + + @Test + void aStringIsTakenWhereAPathIs(@TempDir Path dir) { + String file = dir.resolve("graph.zu").toString(); + Connection.create(file).close(); + Connection.open(file).close(); + } + + @Test + void anOptionIsSetByName() { + assertEquals(1 << 20, Config.defaults().with("memory_limit", "1048576").memoryLimit()); + assertEquals(4, Config.defaults().with("threads", "4").threads()); + assertTrue(Config.defaults().with("read_only", "true").readOnly()); + assertTrue(Config.defaults().with("read_only", "1").readOnly()); + assertFalse(Config.defaults().withReadOnly(true).with("read_only", "false").readOnly()); + } + + @Test + void settingOneOptionLeavesTheOthersWhereTheyWere() { + Config config = Config.defaults().withThreads(2).withMemoryLimit(1 << 20); + Config after = config.with("read_only", "true"); + assertEquals(2, after.threads()); + assertEquals(1 << 20, after.memoryLimit()); + assertTrue(after.readOnly()); + } + + @Test + void aWholeMapIsTakenAtOnce() { + Map options = new LinkedHashMap<>(); + options.put("threads", "1"); + options.put("memory_limit", "2097152"); + options.put("read_only", "false"); + Config config = Config.of(options); + assertEquals(1, config.threads()); + assertEquals(2 << 20, config.memoryLimit()); + assertFalse(config.readOnly()); + assertEquals(Config.defaults(), Config.of(Map.of())); + } + + @Test + void aKeyTheEngineDoesNotKnowIsRefusedAndNamed() { + ZuException e = + assertThrows(ZuException.class, () -> Config.defaults().with("thread_count", "4")); + assertTrue( + e.getMessage().contains("thread_count"), + "the message did not say which key was the typo: " + e.getMessage()); + } + + @Test + void aValueTheKeyCannotTakeIsRefused() { + assertThrows(ZuException.class, () -> Config.defaults().with("threads", "lots")); + assertThrows(ZuException.class, () -> Config.defaults().with("memory_limit", "512MB")); + assertThrows(ZuException.class, () -> Config.defaults().with("read_only", "yes")); + } + + @Test + void anOptionSetByNameIsAnOptionTheDatabaseIsOpenedWith() { + try (dev.zudb.Database db = dev.zudb.Database.memory(Config.of(Map.of("threads", "1"))); + Connection conn = db.connect(); + Result r = conn.query("RETURN 3 AS three")) { + assertEquals(3L, r.row(0).getLong(0)); + } + } +} diff --git a/zudb/src/main/java/dev/zudb/Config.java b/zudb/src/main/java/dev/zudb/Config.java index e6890a2..0810b4f 100644 --- a/zudb/src/main/java/dev/zudb/Config.java +++ b/zudb/src/main/java/dev/zudb/Config.java @@ -1,5 +1,8 @@ package dev.zudb; +import dev.zudb.spi.ZuBinding; +import java.util.Map; + /** * How a database is opened. Zero means the default in every field, so * {@link #defaults()} opens the same database as passing nothing. @@ -78,4 +81,52 @@ public Config withThreads(long count) { public Config withReadOnly(boolean value) { return new Config(memoryLimit, threads, value); } + + /** + * The same, with one option set by name. + * + *

This is for the configuration that arrives as text, out of a + * properties file or a connection string or a command line, where the + * program has a key and a value and no business knowing which of the three + * fields above they land in. The engine owns the list of keys and the + * parsing of the values, so a key that has been added since this client was + * built works anyway and a key that never existed is refused and named. + * + *

The keys are {@code memory_limit}, {@code threads} and + * {@code read_only}. The first two take a decimal count and no suffix, + * deliberately: the two readings of {@code MB} differ by 4.9%, and the place + * to decide which one a user meant is where the user typed it. The third + * takes true, false, 1 or 0. + * + * @param key the option + * @param value the option's value + * @return a new configuration + * @throws ZuException if the key is not one of the engine's, or the value is + * not something that key can be + */ + public Config with(String key, String value) { + ZuBinding zu = Zu.binding(); + long[] set = zu.configSet(memoryLimit, threads, readOnly, key, value); + return new Config(set[0], set[1], set[2] != 0); + } + + /** + * A configuration out of a map of names to values, over the defaults. + * + *

Order is the map's own, which does not matter: each key lands in a + * field of its own, so the same map is the same configuration however it is + * iterated. + * + * @param options what to set, which may be empty + * @return the configuration + * @throws ZuException at the first entry the engine does not recognise, + * naming it + */ + public static Config of(Map options) { + Config config = DEFAULTS; + for (Map.Entry option : options.entrySet()) { + config = config.with(option.getKey(), option.getValue()); + } + return config; + } } diff --git a/zudb/src/main/java/dev/zudb/Connection.java b/zudb/src/main/java/dev/zudb/Connection.java index 3790914..0f4f14e 100644 --- a/zudb/src/main/java/dev/zudb/Connection.java +++ b/zudb/src/main/java/dev/zudb/Connection.java @@ -1,6 +1,7 @@ package dev.zudb; import dev.zudb.spi.ZuBinding; +import java.nio.file.Path; import java.time.Duration; import java.util.ArrayList; import java.util.List; @@ -35,6 +36,79 @@ public final class Connection implements AutoCloseable { this.handle = new AtomicLong(handle); } + /** + * Opens an existing database with the default configuration and connects + * once, for the program that wants exactly one connection. + * + *

The database handle is made and let go of inside this call, and + * nothing is lost by that: a connection carries its own file handle, and a + * database holds only the path and the configuration. What is given up is + * the second connection, since {@link Database#connect()} is where those + * come from. {@link #duplicate()} is the way back to one. + * + * 

{@code
+   * try (Connection conn = Connection.open(Path.of("social.zu1"))) {
+   *     ...
+   * }
+   * }
+ * + * @param path the file + * @return the connection, which the caller closes + */ + public static Connection open(Path path) { + ZuBinding zu = Zu.binding(); + return new Connection(zu, zu.open(path.toString())); + } + + /** + * The same, named by a string. + * + * @param path the file + * @return the connection, which the caller closes + */ + public static Connection open(String path) { + return open(Path.of(path)); + } + + /** + * Creates a database and connects once. The path must not exist, for the + * reason {@link Database#create(Path)} gives. + * + * @param path the file to make + * @return the connection, which the caller closes + */ + public static Connection create(Path path) { + ZuBinding zu = Zu.binding(); + return new Connection(zu, zu.create(path.toString())); + } + + /** + * The same, named by a string. + * + * @param path the file to make + * @return the connection, which the caller closes + */ + public static Connection create(String path) { + return create(Path.of(path)); + } + + /** + * One scratch graph and one connection on it, which go together when the + * connection closes. + * + *

This is the shortest thing in the client that can run a statement, and + * it is what a test and a scratch script want: nothing on the disk, nothing + * to name, and one thing to close. A second connection on the same graph + * comes from {@link #duplicate()}, which is the only way to one, since a + * graph in memory has no path to reopen. + * + * @return the connection, which the caller closes + */ + public static Connection memory() { + ZuBinding zu = Zu.binding(); + return new Connection(zu, zu.memory()); + } + /** * Runs one statement and hands back everything it answered. * diff --git a/zudb/src/main/java/dev/zudb/spi/ZuBinding.java b/zudb/src/main/java/dev/zudb/spi/ZuBinding.java index 2322a91..bb90b60 100644 --- a/zudb/src/main/java/dev/zudb/spi/ZuBinding.java +++ b/zudb/src/main/java/dev/zudb/spi/ZuBinding.java @@ -58,6 +58,28 @@ public interface ZuBinding { */ String version(); + // ---- configuration ---- + + /** + * Sets one option of a configuration by name, from {@code zu_config_set}. + * + *

The configuration crosses this interface as its three fields rather + * than as a handle, because it is a struct the engine reads by value and + * there is nothing to keep alive between calls. What this call is for is the + * key: the engine owns the list of them and the parsing of the values, so a + * binding forwarding a user's option map does not hard-code a list it would + * have to keep in step. + * + * @param memoryLimit the current bytes the caches may hold + * @param threads the current query workers + * @param readOnly whether writes are currently refused + * @param key the option, which the engine refuses and names if it is not one + * @param value the option's value, which the engine parses + * @return the three fields after the set, in the order they are taken, + * with {@code readOnly} as 1 or 0 + */ + long[] configSet(long memoryLimit, long threads, boolean readOnly, String key, String value); + // ---- databases ---- /** @@ -127,6 +149,34 @@ public interface ZuBinding { */ long connect(long db); + /** + * Opens an existing database with the default configuration and connects + * once, from {@code zu_open}. The database handle is discarded inside the + * call, and nothing is lost by that: the connection carries its own file + * handle. + * + * @param path the file + * @return the connection handle + */ + long open(String path); + + /** + * Creates a database and connects once, from {@code zu_create}. The path + * must not exist. + * + * @param path the file to make + * @return the connection handle + */ + long create(String path); + + /** + * One scratch graph and one connection on it, from {@code zu_memory}, which + * go together when the connection closes. + * + * @return the connection handle + */ + long memory(); + /** * A second connection on the database a connection is already on, made * without a path.