Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,26 @@ The callback runs on a thread of the library's, one per statement, never two at

This is the other of the two places a pointer to Java code goes the other way. An exception crossing an upcall would take the JVM down, so a watcher that throws is logged and answered as though it had asked for the statement to stop, which is the reading that loses least: a callback that threw is a program that has stopped wanting the answer.

## One connection, and settings that came as text

A `Database` and a `Connection` are two objects because they are two things: the path and the configuration on one side, the caches and the plan cache and the file handle on the other, and a program that queries from four threads wants one of the first and four of the second. A program that wants exactly one connection should not have to say so twice, so it does not have to:

```java
try (Connection conn = Connection.open("social.zu1")) {
...
}
```

`Connection.create(path)` is the same over a file that is not there yet, and `Connection.memory()` is the same over a graph that is nowhere, which is the shortest thing here that runs a statement. All three make the database inside the call and let go of it, which costs nothing, since a connection carries its own file handle and a database holds only the path. What they give up is the second connection, and `conn.duplicate()` is the way back to one.

Settings usually arrive as text, out of a properties file or a connection string or a command line, and a program with a key and a value has no business knowing which field of `Config` they land in:

```java
Config config = Config.of(Map.of("threads", "1", "memory_limit", "1073741824"));
```

The keys and the parsing belong to the engine rather than to this client, so a key added to the engine since this client was built works anyway, and a key that never existed is refused with the typo named. A suffix such as `MB` is deliberately not parsed anywhere: its two readings differ by 4.9%, and the place to decide which one a user meant is where the user typed it.

## How it binds

The Foreign Function and Memory API is the primary path. The downcall handles are written by hand against `zu.h` rather than generated with `jextract`, because the C ABI here is around seventy functions with a stable shape, and a hand-written layer is where the interesting decisions live: which calls are `Linker.Option.critical` because they are short pure accessors, where the out-parameter scratch space comes from so that a query does not allocate, and how a `zu_error` becomes a typed Java exception exactly once. There is no native code in this repository beyond `libzu` itself.
Expand Down
12 changes: 12 additions & 0 deletions zudb-ffm/src/main/java/dev/zudb/ffm/Abi.java
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ final class Abi {
final MethodHandle errorExcerpt;
final MethodHandle errorFree;

final MethodHandle configSet;

final MethodHandle databaseOpen;
final MethodHandle databaseCreate;
final MethodHandle databaseMemory;
Expand All @@ -64,6 +66,9 @@ final class Abi {
final MethodHandle databaseClose;

final MethodHandle connect;
final MethodHandle openOne;
final MethodHandle createOne;
final MethodHandle memoryOne;
final MethodHandle connDuplicate;
final MethodHandle connClose;
final MethodHandle connInterrupt;
Expand Down Expand Up @@ -188,6 +193,10 @@ final class Abi {
errorExcerpt = h("zu_error_excerpt", FunctionDescriptor.of(ADDRESS, ADDRESS, ADDRESS));
errorFree = h("zu_error_free", FunctionDescriptor.ofVoid(ADDRESS));

configSet =
h("zu_config_set", FunctionDescriptor.of(
JAVA_INT, ADDRESS, ADDRESS, SIZE_T, ADDRESS, SIZE_T, ADDRESS));

databaseOpen =
h("zu_database_open", FunctionDescriptor.of(JAVA_INT, ADDRESS, SIZE_T, ADDRESS, ADDRESS, ADDRESS));
databaseCreate =
Expand All @@ -198,6 +207,9 @@ final class Abi {
databaseClose = h("zu_database_close", FunctionDescriptor.ofVoid(ADDRESS));

connect = h("zu_connect", FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS, ADDRESS));
openOne = h("zu_open", FunctionDescriptor.of(JAVA_INT, ADDRESS, SIZE_T, ADDRESS, ADDRESS));
createOne = h("zu_create", FunctionDescriptor.of(JAVA_INT, ADDRESS, SIZE_T, ADDRESS, ADDRESS));
memoryOne = h("zu_memory", FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS));
connDuplicate = h("zu_conn_duplicate", FunctionDescriptor.of(JAVA_INT, ADDRESS, ADDRESS, ADDRESS));
connClose = h("zu_conn_close", FunctionDescriptor.ofVoid(ADDRESS));
connInterrupt = h("zu_conn_interrupt", FunctionDescriptor.of(JAVA_INT, ADDRESS));
Expand Down
61 changes: 61 additions & 0 deletions zudb-ffm/src/main/java/dev/zudb/ffm/FfmBinding.java
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,29 @@ public String version() {
}
}

@Override
public long[] configSet(
long memoryLimit, long threads, boolean readOnly, String key, String value) {
Scratch s = Scratch.get();
MemorySegment sl = s.slots();
MemorySegment cfg = config(s, memoryLimit, threads, readOnly);
MemorySegment k = s.utf8(key);
MemorySegment v = s.utf8(value);
clear(sl);
try {
int st =
(int)
abi.configSet.invokeExact(
cfg, k, k.byteSize(), v, v.byteSize(), sl.asSlice(ERR, 8));
check("zu_config_set", st, sl);
return new long[] {
cfg.get(JAVA_LONG, 8), cfg.get(JAVA_LONG, 16), cfg.get(JAVA_INT, 24)
};
} catch (Throwable t) {
throw fail("zu_config_set", t);
}
}

@Override
public long databaseOpen(String path, long memoryLimit, long threads, boolean readOnly) {
return openOrCreate(abi.databaseOpen, "zu_database_open", path, memoryLimit, threads, readOnly);
Expand Down Expand Up @@ -145,6 +168,30 @@ public long connect(long db) {
return handle(abi.connect, "zu_connect", db);
}

@Override
public long open(String path) {
return one(abi.openOne, "zu_open", path);
}

@Override
public long create(String path) {
return one(abi.createOne, "zu_create", path);
}

@Override
public long memory() {
Scratch s = Scratch.get();
MemorySegment sl = s.slots();
clear(sl);
try {
int st = (int) abi.memoryOne.invokeExact(sl.asSlice(OUT, 8), sl.asSlice(ERR, 8));
check("zu_memory", st, sl);
return sl.get(ADDRESS, OUT).address();
} catch (Throwable t) {
throw fail("zu_memory", t);
}
}

@Override
public long connDuplicate(long conn) {
return handle(abi.connDuplicate, "zu_conn_duplicate", conn);
Expand Down Expand Up @@ -1393,6 +1440,20 @@ private long openOrCreate(
}
}

private long one(java.lang.invoke.MethodHandle mh, String what, String path) {
Scratch s = Scratch.get();
MemorySegment sl = s.slots();
MemorySegment p = s.utf8(path);
clear(sl);
try {
int st = (int) mh.invokeExact(p, p.byteSize(), sl.asSlice(OUT, 8), sl.asSlice(ERR, 8));
check(what, st, sl);
return sl.get(ADDRESS, OUT).address();
} catch (Throwable t) {
throw fail(what, t);
}
}

private long handle(java.lang.invoke.MethodHandle mh, String what, long in) {
Scratch s = Scratch.get();
MemorySegment sl = s.slots();
Expand Down
141 changes: 141 additions & 0 deletions zudb-ffm/src/test/java/dev/zudb/ffm/ShorthandTest.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
package dev.zudb.ffm;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;

import dev.zudb.Config;
import dev.zudb.Connection;
import dev.zudb.Result;
import dev.zudb.ZuException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.LinkedHashMap;
import java.util.Map;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;

/**
* The one-call openers, and a configuration that arrives as text.
*
* <p>Neither of these is a new thing the engine can do. They are the same
* calls with less to write around them, which is the whole of what they are
* for: a script that wants one connection should not have to close two
* objects, and a program reading its settings out of a file should not have to
* know which of three fields a key lands in.
*/
class ShorthandTest {

@BeforeAll
static void engine() {
Libzu.require();
}

@Test
void aConnectionInMemoryNeedsNothingAroundIt() {
try (Connection conn = Connection.memory();
Result r = conn.query("RETURN 1 AS one")) {
assertEquals(1L, r.row(0).getLong(0));
}
}

@Test
void aConnectionInMemoryCanBeDuplicatedOntoTheSameGraph() {
try (Connection first = Connection.memory();
Connection second = first.duplicate();
Result r = second.query("RETURN 1 AS one")) {
assertEquals(1L, r.row(0).getLong(0));
assertFalse(first.isClosed());
}
}

@Test
void aConnectionOnAFileMakesItAndOpensItAgain(@TempDir Path dir) {
Path file = dir.resolve("graph.zu");
try (Connection conn = Connection.create(file)) {
assertFalse(conn.isClosed());
}
assertTrue(Files.isRegularFile(file));
try (Connection conn = Connection.open(file);
Result r = conn.query("RETURN 2 AS two")) {
assertEquals(2L, r.row(0).getLong(0));
}
}

@Test
void creatingOverSomethingAlreadyThereIsRefused(@TempDir Path dir) {
Path file = dir.resolve("graph.zu");
Connection.create(file).close();
assertThrows(ZuException.class, () -> Connection.create(file).close());
}

@Test
void openingSomethingThatIsNotThereIsRefused(@TempDir Path dir) {
assertThrows(ZuException.class, () -> Connection.open(dir.resolve("nothing.zu")).close());
}

@Test
void aStringIsTakenWhereAPathIs(@TempDir Path dir) {
String file = dir.resolve("graph.zu").toString();
Connection.create(file).close();
Connection.open(file).close();
}

@Test
void anOptionIsSetByName() {
assertEquals(1 << 20, Config.defaults().with("memory_limit", "1048576").memoryLimit());
assertEquals(4, Config.defaults().with("threads", "4").threads());
assertTrue(Config.defaults().with("read_only", "true").readOnly());
assertTrue(Config.defaults().with("read_only", "1").readOnly());
assertFalse(Config.defaults().withReadOnly(true).with("read_only", "false").readOnly());
}

@Test
void settingOneOptionLeavesTheOthersWhereTheyWere() {
Config config = Config.defaults().withThreads(2).withMemoryLimit(1 << 20);
Config after = config.with("read_only", "true");
assertEquals(2, after.threads());
assertEquals(1 << 20, after.memoryLimit());
assertTrue(after.readOnly());
}

@Test
void aWholeMapIsTakenAtOnce() {
Map<String, String> options = new LinkedHashMap<>();
options.put("threads", "1");
options.put("memory_limit", "2097152");
options.put("read_only", "false");
Config config = Config.of(options);
assertEquals(1, config.threads());
assertEquals(2 << 20, config.memoryLimit());
assertFalse(config.readOnly());
assertEquals(Config.defaults(), Config.of(Map.of()));
}

@Test
void aKeyTheEngineDoesNotKnowIsRefusedAndNamed() {
ZuException e =
assertThrows(ZuException.class, () -> Config.defaults().with("thread_count", "4"));
assertTrue(
e.getMessage().contains("thread_count"),
"the message did not say which key was the typo: " + e.getMessage());
}

@Test
void aValueTheKeyCannotTakeIsRefused() {
assertThrows(ZuException.class, () -> Config.defaults().with("threads", "lots"));
assertThrows(ZuException.class, () -> Config.defaults().with("memory_limit", "512MB"));
assertThrows(ZuException.class, () -> Config.defaults().with("read_only", "yes"));
}

@Test
void anOptionSetByNameIsAnOptionTheDatabaseIsOpenedWith() {
try (dev.zudb.Database db = dev.zudb.Database.memory(Config.of(Map.of("threads", "1")));
Connection conn = db.connect();
Result r = conn.query("RETURN 3 AS three")) {
assertEquals(3L, r.row(0).getLong(0));
}
}
}
51 changes: 51 additions & 0 deletions zudb/src/main/java/dev/zudb/Config.java
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
package dev.zudb;

import dev.zudb.spi.ZuBinding;
import java.util.Map;

/**
* How a database is opened. Zero means the default in every field, so
* {@link #defaults()} opens the same database as passing nothing.
Expand Down Expand Up @@ -78,4 +81,52 @@ public Config withThreads(long count) {
public Config withReadOnly(boolean value) {
return new Config(memoryLimit, threads, value);
}

/**
* The same, with one option set by name.
*
* <p>This is for the configuration that arrives as text, out of a
* properties file or a connection string or a command line, where the
* program has a key and a value and no business knowing which of the three
* fields above they land in. The engine owns the list of keys and the
* parsing of the values, so a key that has been added since this client was
* built works anyway and a key that never existed is refused and named.
*
* <p>The keys are {@code memory_limit}, {@code threads} and
* {@code read_only}. The first two take a decimal count and no suffix,
* deliberately: the two readings of {@code MB} differ by 4.9%, and the place
* to decide which one a user meant is where the user typed it. The third
* takes true, false, 1 or 0.
*
* @param key the option
* @param value the option's value
* @return a new configuration
* @throws ZuException if the key is not one of the engine's, or the value is
* not something that key can be
*/
public Config with(String key, String value) {
ZuBinding zu = Zu.binding();
long[] set = zu.configSet(memoryLimit, threads, readOnly, key, value);
return new Config(set[0], set[1], set[2] != 0);
}

/**
* A configuration out of a map of names to values, over the defaults.
*
* <p>Order is the map's own, which does not matter: each key lands in a
* field of its own, so the same map is the same configuration however it is
* iterated.
*
* @param options what to set, which may be empty
* @return the configuration
* @throws ZuException at the first entry the engine does not recognise,
* naming it
*/
public static Config of(Map<String, String> options) {
Config config = DEFAULTS;
for (Map.Entry<String, String> option : options.entrySet()) {
config = config.with(option.getKey(), option.getValue());
}
return config;
}
}
Loading
Loading