Conversation
## Abstract
This document proposes adding **Path Hash**, a general-purpose,
binary-safe data type backed by a radix tree, to Valkey. Each Valkey key
stores one Path Hash, and each logical path maps to a `field -> value`
map. The type provides exact lookup, longest-prefix matching, matching
of all stored ancestor prefixes, atomic updates, and prefix traversal.
It gains persistence and high availability through Valkey's existing
RDB, AOF, primary-replica replication, and Cluster capabilities.
The first target use case is a KV Cache placement index for AI
inference: Worker KV Cache events continuously update which Workers
contain which cached prefixes. A Router queries all reusable prefixes
along a request's cumulative block-hash path and then combines the
results with real-time load to select a Worker. The type itself does not
interpret tokens, models, Workers, or cache tiers. Applications encode
those semantics into binary-safe paths, fields, and values, so the type
can also serve hierarchical routing, longest-prefix matching,
autocomplete indexes, and other prefix-oriented workloads.
## 1. Background and Inference Architecture
### 1.1 Problem Statement
Large-scale model inference services commonly reuse previously computed
KV Cache through a prefix cache. When a request reaches a Router, the
Router needs to know more than whether the prompt hits the cache. It
also needs to know:
1. How deep the request's contiguous prefix matches on each Worker;
2. Whether the same prefix exists simultaneously on multiple Workers or
cache tiers;
3. How placement changes continuously as blocks are created, migrated,
evicted, and as Workers restart;
4. That multiple Routers must observe the same persistent and
recoverable placement index;
A regular Hash can store `block_hash -> workers`, but it cannot express
"find all stored ancestor prefixes of this query" in a single query.
Storing every prefix as a separate top-level key also inflates the key
count, makes atomic updates across keys difficult, and introduces many
network round trips. The ordering of a Sorted Set is likewise not
equivalent to a byte-prefix relationship.
A radix tree compresses paths by their shared byte prefixes, making it
naturally suitable for longest-prefix matching and ancestor-prefix
enumeration. Valkey already contains a mature internal `rax`
implementation that supports path compression, binary-safe keys, values
on internal nodes, insertion, deletion, and ordered iteration.
Therefore, no new tree implementation dependency is required.
### 1.2 Goals
- Provide a general-purpose, binary-safe prefix-index data type.
- Query the longest prefix or all stored ancestor prefixes with one
command.
- Allow multiple fields on the same path to be updated independently.
- Make writes idempotent and support atomic single-key batch application
for event streams.
- Use existing Valkey persistence, replication, failover, and Cluster
semantics.
- Reuse the path-compression and iteration capabilities of the current
internal `rax`.
- Provide stable and explicit response structures for RESP2 and RESP3
clients.
### 1.3 Non-Goals
- The type does not provide vector similarity or semantic-similarity
matching; all matching is exact byte-prefix matching.
- The type does not provide a global Path Hash spanning multiple Valkey
keys.
- The type does not replace the inference system's Worker snapshot,
live-event, or event-reconciliation protocols.
- The type does not guarantee synchronous replication or zero RPO; data
reliability still follows the selected Valkey deployment model.
- v1 does not provide a separate TTL for an individual path; TTL still
applies to the top-level Valkey key.
### 1.4 Inference Architecture
```mermaid
flowchart LR
Client[Client] --> Router[Router]
Router -->|prefix query| Valkey[Valkey Path Hash]
Router -->|inference request| Worker[Inference Worker]
Worker -->|KV events| Bridge[KV Event Bridge]
Bridge -->|atomic batch| Valkey
Worker -->|load report| Router
```
The data plane has two paths:
- Update path: A Worker produces KV events such as block created,
stored, and evicted. The Bridge converts each event into a complete path
and writes it idempotently to Valkey through `PHSET` or batches updates
to multiple paths through `PHMSET`.
- Query path: The Router converts a prompt into a cumulative block-hash
path, retrieves all matching prefixes and placements in one `PHPREFIXES`
call, and then combines them with Worker load for routing.
The Valkey Path Hash is a placement index shared by multiple Routers.
The Bridge is responsible for protocol adaptation, event ordering, batch
coalescing, retries, generation management, and short-term in-memory
buffering rather than serving queries.
### 1.5 Failure and Recovery Boundaries
Path Hash data is persisted through RDB, AOF, and primary-replica
replication. The Bridge uses bounded in-memory buffering and retries for
unacknowledged events. If it detects a sequence gap or buffer overflow,
it should reconcile against a Worker snapshot before continuing to
consume live events.
A Path Hash placement index is generally rebuildable soft state. Losing
an add usually produces a false miss and therefore reuses less cache.
Losing a delete can produce a more dangerous stale positive.
Applications should therefore include the Worker generation in the value
and have the Router accept only the currently live generation. After a
Worker restarts, an old generation will not participate in routing even
if an old delete was lost; stale records can be reclaimed later by a
background scan.
This strategy reduces application-level risk during a failure window but
does not change the RPO of Valkey's asynchronous replication itself.
## 2. Data Model
### 2.1 Logical Structure
Each top-level Valkey key stores one tree:
```text
Valkey key
└── Path Hash
├── path P1
│ ├── field F1 -> value V1
│ └── field F2 -> value V2
├── path P1 || P2
│ └── field F1 -> value V3
└── path P1 || P3
└── field F3 -> value V4
```
Formally:
```text
Tree: BinaryString -> Map<BinaryString, BinaryString>
```
- Top-level key: A regular Valkey key responsible for the namespace,
Cluster slot, and tree-wide TTL.
- path: A binary-safe byte string and the logical key in the Path Hash.
- field: A binary-safe byte string identifying an owner or property that
can be updated independently under the same path.
- value: A binary-safe opaque value encoded by the application.
An empty path is valid and represents the root payload. An internal
navigation node is a "stored logical path" only when it has a non-empty
field/value map. After its last field is deleted, the logical path
disappears automatically, and `rax` can subsequently compress internal
nodes that are no longer needed.
The top-level Path Hash key may remain present with zero logical paths.
Deleting the final path, or clearing the tree with `PHDELPREFIX key ""`,
leaves an empty Path Hash object whose `PHCARD` is zero. This follows
Stream semantics, where `XTRIM` may leave an empty stream; only `DEL` or
`UNLINK` removes the top-level key. Empty Path Hash objects are
preserved by RDB, AOF rewrite, replication, and `COPY`.
### 2.2 Why Each Path Contains a Field/Value Map
If each path stored only one opaque value, updating placement for
multiple Workers would require reading the entire value, modifying the
list, and writing it back. Two Bridges or two event batches could then
cause a lost update. A field/value map reduces the conflict granularity
to an individual field:
```text
path = cumulative-hash-path
field = worker-a | hbm
value = generation 7 | component mask | metadata
```
`worker-a` and `worker-b` can independently execute `PHSET` or `PHDEL`
operations, and the commands are naturally idempotent.
### 2.3 Definition of Prefix
Let `p` and `q` be byte strings. `p` is a prefix of `q`, written `p ⪯
q`, if and only if there is a byte string `s` such that `q = p || s`.
Matching does not interpret UTF-8, integers, tokens, or segments.
For the set `S` of stored paths in the tree:
```text
Exact(q) = q, provided q belongs to S
Longest(q) = arg max |p|, where p belongs to S and p ⪯ q
Prefixes(q) = all p in S satisfying p ⪯ q, ordered from shortest to longest
```
If an empty path is stored, it is the first match for every query.
`PREFIXES` returns only ancestors of the query, not descendants for
which the query is a prefix.
## 3. Inference Matching Flow and Example
### 3.1 From Token Blocks to a Stored Path
The application first divides the prompt into blocks using a fixed page
size. Suppose there are three blocks, `B1`, `B2`, and `B3`, and a
chained cumulative hash is used:
```text
C1 = Hash(B1)
C2 = Hash(C1 || B2)
C3 = Hash(C2 || B3)
```
Each cumulative hash is encoded as a fixed-width big-endian byte string.
For example, a 64-bit hash uses 8 bytes:
```text
P1 = BE64(C1)
P2 = BE64(C1) || BE64(C2)
P3 = BE64(C1) || BE64(C2) || BE64(C3)
```
The request query path is:
```text
Q = BE64(C1) || BE64(C2) || BE64(C3)
```
The stored representation is therefore a "cumulative hash sequence." If
`Hash(B1)`, `Hash(B2)`, and `Hash(B3)` were treated as three unrelated
keys, they would have no ancestor relationship at the byte level, and
the Path Hash could not derive the block order.
The server does not enforce an 8-byte segment size. Fixed-width encoding
is simply how the application ensures that no logical path ends in the
middle of a hash.
### 3.2 Complete Example
Suppose `page_size = 4` and the request contains 12 tokens:
```text
B1 = token 1..4
B2 = token 5..8
B3 = token 9..12
```
The placement tree has the following logical content:
```text
P1
├── worker-a|hbm -> generation=7, components=1111
└── worker-b|hbm -> generation=3, components=1111
P2
├── worker-a|hbm -> generation=7, components=1111
└── worker-b|hbm -> generation=3, components=1111
P3
└── worker-b|hbm -> generation=3, components=1111
```
The shared-prefix relationship in the tree is:
```mermaid
flowchart TD
Root[Root]
P1[P1 cached by A and B]
P2[P1 plus P2 cached by A and B]
P3[P1 plus P2 plus P3 cached by B]
Root --> P1
P1 --> P2
P2 --> P3
```
The Router executes the following operation for `Q`. `LENGTHS` avoids
repeatedly returning the shared bytes of `P1`, `P2`, and `P3`; the
Router can divide the matched length by 8 to obtain the page depth
directly:
```text
PHPREFIXES kv:{model-id}:placement Q
LENGTHS
WITHVALUES
COUNT 3
MAXLEN 24
```
The response is conceptually equivalent to:
```text
[
[8, [worker-a|hbm, value-a7, worker-b|hbm, value-b3]],
[16, [worker-a|hbm, value-a7, worker-b|hbm, value-b3]],
[24, [worker-b|hbm, value-b3]]
]
```
The Router aggregates by path length and obtains:
```text
worker-a has a contiguous match of 2 pages = 8 tokens
worker-b has a contiguous match of 3 pages = 12 tokens
```
It then combines those results with signals such as load, queue length,
and cross-machine communication cost for routing. The largest cache hit
does not necessarily determine the final Worker. This is also why the
Router needs all ancestor prefixes rather than only a global `LONGEST`
result.
If the Router is considering only a set of healthy Workers, it can use
`FIELDS` filtering to reduce the response size:
```text
PHPREFIXES kv:{model-id}:placement Q
LENGTHS
FIELDS 2 worker-a|hbm worker-b|hbm
COUNT 3
MAXLEN 24
```
### 3.3 How the Event Bridge Constructs a Path
The Path Hash API accepts a complete path. It does not accept a
`parent_hash` and then query the parent chain on the server. If a Worker
event contains only the current block hash and parent hash, the Bridge
has two options:
1. Maintain a short-lived `hash -> full path` cache and use it to
construct the complete path;
2. Modify the event protocol so that the Worker carries either the
cumulative hash-segment sequence or the fully encoded path directly.
The second approach makes a stateless Bridge easier to implement, while
the first reduces event size. This trade-off belongs to the inference
event protocol and is not part of the general-purpose Path Hash data
type.
## 4. API Design
### 4.1 Command Overview
The `PH` command prefix stands for Path Hash. The public type name is
Path Hash; the command group, ACL category, and `TYPE`/`SCAN TYPE` name
use `path-hash`. The keyspace-notification configuration character
remains `r`.
In every command, `key` is the top-level Valkey key, while `path`,
`query`, `field`, and `value` are binary-safe bulk strings. A command
returns the standard `WRONGTYPE` error when an existing key has the
wrong type.
| Command | Syntax | Purpose | Time Complexity |
|---|---|---|---|
| `PHSET` | `PHSET key path [FNX \| FXX] FIELDS numfields field value
[field value ...]` | Atomically set one or more fields under one path |
`O(L + F·U)` |
| `PHMSET` | `PHMSET key path FIELDS numfields field value [field value
...] [path FIELDS numfields field value ...]` | Atomically set fields
grouped by path | `O(ΣL + M·U)` |
| `PHGET` | `PHGET key path field [field ...]` | Read one or more fields
by exact path | `O(L + F·U)` |
| `PHMGET` | `PHMGET key path FIELDS numfields field [field ...] [path
FIELDS numfields field ...]` | Read fields grouped by exact path | `O(ΣL
+ M·U)` |
| `PHGETALL` | `PHGETALL key path` | Read all field/value pairs under a
path | `O(L + O)` |
| `PHEXISTS` | `PHEXISTS key path` | Test whether an exact logical path
exists | `O(L)` |
| `PHDEL` | `PHDEL key path [field [field ...]]` | Delete fields or an
entire logical path | `O(L + F·U)` for specified fields; `O(L + S)` for
the entire payload |
| `PHLONGEST` | `PHLONGEST key query [LENGTH] [WITHVALUES \| FIELDS
numfields field [field ...]]` | Return the longest stored prefix of a
query | `O(L + O)` |
| `PHPREFIXES` | `PHPREFIXES key query [LENGTHS] [WITHVALUES \| FIELDS
numfields field [field ...]] [COUNT count] [MAXLEN max-path-bytes]` |
Return all stored ancestor prefixes of a query | `O(L + O)` |
| `PHDELPREFIX` | `PHDELPREFIX key prefix` | Delete all logical paths
under a prefix | `O(L + Σ(Pᵢ + Fᵢ))` |
| `PHSCAN` | `PHSCAN key cursor [PREFIX prefix] [COUNT count]
[WITHVALUES]` | Incrementally traverse logical paths | `O(L + C + O)` |
| `PHCARD` | `PHCARD key` | Return the number of logical paths | `O(1)`
|
Where:
- `L` is the byte length of the input path or query;
- `F` is the number of requested fields;
- `S` is the number of stored fields in a payload removed in full;
- `M` is the total number of fields across all path groups in a
multi-path command;
- `U` is the lookup cost within a node payload: linear for a small
listpack and amortized constant time for a dict;
- `O` is the number of bytes or fields actually returned;
- `Pᵢ` is the byte length of the i-th path matched by a subtree
deletion;
- `Fᵢ` is the number of fields in that path's payload;
- `C` is the number of paths examined by the current scan.
### 4.2 `PHSET`
```text
PHSET key path [FNX | FXX]
FIELDS numfields field value [field value ...]
```
Atomically set one or more field/value pairs in the field map for
`path`. `numfields` must be greater than zero and must equal the number
of field/value pairs that follow. Without a condition, a missing tree or
path is created automatically and existing fields are overwritten.
- `FNX`: Apply the entire field/value group only if none of the
specified fields exists under the path.
- `FXX`: Apply the entire field/value group only if every specified
field exists under the path.
- `FNX` and `FXX` are mutually exclusive.
- Conditions are evaluated against all specified fields before any write
is performed. If any field fails the condition, none of the supplied
field/value pairs is written.
- If the key or path does not exist, `FNX` succeeds and creates it,
while `FXX` fails without creating it.
- If the same field is specified more than once, assignments are applied
from left to right and the last value wins.
Return: `OK` if the write succeeds; null if the condition is not
satisfied.
For example, the following command writes both fields only if neither
`field-1` nor `field-2` exists under `path-a`. The path may already
exist with other fields. If either specified field exists, neither field
is modified:
```text
PHSET tree path-a FNX FIELDS 2 field-1 value-1 field-2 value-2
```
### 4.3 `PHMSET`
```text
PHMSET key
path FIELDS numfields field value [field value ...]
[path FIELDS numfields field value ...]
```
Atomically apply one or more path groups within one Path Hash. Each path
is followed by `FIELDS`, a positive `numfields`, and exactly that many
field/value pairs. Grouping reflects the underlying `path -> field/value
map`, avoids repeating path bytes, and allows one path lookup per path
group. Missing paths are created automatically and existing fields are
overwritten.
`PHMSET` does not support `FNX` or `FXX`; every assignment is
unconditional. The complete grouped argument structure is validated
before any mutation is applied. If the same `(path, field)` target
appears more than once within or across groups, assignments are applied
from left to right and the last value wins.
Return: `OK` after all assignments have been applied.
For example:
```text
PHMSET tree
path-a FIELDS 2 field-1 value-1 field-2 value-2
path-b FIELDS 1 field-3 value-3
```
### 4.4 `PHGET`
```text
PHGET key path field [field ...]
```
Perform one exact path lookup and read one or more fields from its
payload.
- When one field is requested, return its value as a bulk string, or
null if the key, path, or field does not exist.
- When multiple fields are requested, return an array of values in
request order. A missing key or path produces null for every requested
field, and each missing field produces null at its corresponding
position.
### 4.5 `PHMGET`
```text
PHMGET key
path FIELDS numfields field [field ...]
[path FIELDS numfields field ...]
```
Read one or more field groups from one Path Hash. Each path is followed
by `FIELDS`, a positive `numfields`, and exactly that many fields. Each
path lookup is exact and performed once per group; the command does not
perform ancestor-prefix or descendant-prefix matching. The complete
grouped argument structure is validated before reading results.
Return one flat array whose elements correspond to all requested fields
in path-group and field order. A missing key, path, or field produces
null at the corresponding position. The command always returns an array,
including when only one field is requested. Paths and fields do not need
to be distinct, and duplicate requests are preserved.
For example, table-cache rows stored under a `row` field can be fetched
by primary key in one command:
```text
PHMGET student:pk
student-10001 FIELDS 2 name grade
student-10002 FIELDS 1 name
student-10003 FIELDS 1 name
```
### 4.6 `PHGETALL`
```text
PHGETALL key path
```
Return all field/value pairs for the specified path. RESP2 returns a
flat array:
```text
[field-1, value-1, field-2, value-2, ...]
```
RESP3 returns a map:
```text
{field-1: value-1, field-2: value-2, ...}
```
Field order is undefined in both protocols. If the path or key does not
exist, return an empty array in RESP2 or an empty map in RESP3.
### 4.7 `PHEXISTS`
```text
PHEXISTS key path
```
Perform an exact path lookup and return 1 if `path` is a stored logical
path, or 0 if the key or path does not exist. Prefix ancestors and
descendants do not count as an exact match.
### 4.8 `PHDEL`
```text
PHDEL key path [field [field ...]]
```
- When fields are provided, delete only those fields and return the
number of fields actually deleted.
- When no field is provided, delete the entire payload for the path and
return 1 or 0.
- Deleting a path payload does not delete descendant paths.
- After the last field is deleted, the path is automatically removed
from the logical tree.
- Deleting the final logical path leaves an empty Path Hash key. Use
`DEL` or `UNLINK` to remove the top-level key.
The command is an idempotent no-op for a target that does not exist.
When fields are provided, the command takes `O(L + F·U)` time. When no
field is provided, removing the complete payload synchronously releases
all of its stored fields and takes `O(L + S)` time in the worst case.
`PHDEL` is therefore an `@slow` command.
### 4.9 `PHLONGEST`
```text
PHLONGEST key query [LENGTH] [WITHVALUES | FIELDS numfields field [field ...]]
```
Find the longest stored path that is a byte prefix of `query`.
- By default, return the matching path; return null if there is no
match.
- `LENGTH`: Replace the matching path with its byte length, for cases
where the caller already holds the query.
- `WITHVALUES`: Return `[path, [field, value, ...]]` in RESP2 or `[path,
{field: value, ...}]` in RESP3. Field order is undefined.
- `FIELDS`: Read only the specified fields. The inner value array
preserves request order and places null for a missing field. Return
`[path, [value-or-null, ...]]` in both RESP2 and RESP3.
- `WITHVALUES` and `FIELDS` are mutually exclusive.
An empty path can be returned as a match if it is stored.
### 4.10 `PHPREFIXES`
```text
PHPREFIXES key query
[LENGTHS]
[WITHVALUES | FIELDS numfields field [field ...]]
[COUNT count]
[MAXLEN max-path-bytes]
```
Return all stored paths satisfying `path ⪯ query`, ordered from shortest
to longest.
- By default, return `[path-1, path-2, ...]`.
- `LENGTHS`: Replace each matching path with its byte length to avoid
repeating the shared bytes of a long query in the response.
- `WITHVALUES`: Return `[[path-1, [field, value, ...]], ...]` in RESP2
or `[[path-1, {field: value, ...}], ...]` in RESP3. Field order within
each payload is undefined.
- `FIELDS`: Return `[[path-1, [value-or-null, ...]], ...]` in both RESP2
and RESP3, with each value array aligned to the requested field order
and null for each missing field.
- `COUNT`: Return at most `count` matches. This is a hard limit, not a
hint. If more matches exist, retain the deepest `count` prefixes; return
the selected results in ascending path-length order.
- `MAXLEN`: Match only paths whose byte length does not exceed
`max-path-bytes`. Apply this limit before `COUNT`.
- Return an empty array when there is no match.
`COUNT` must be an integer from `1` to `LONG_MAX`, and `MAXLEN` must be
an integer from `0` to `LONG_MAX`, using the server build's `long` range
(maximum `2147483647` on 32-bit builds or `9223372036854775807` on
64-bit builds). Invalid or out-of-range values return an error, even
when the key does not exist. Omitting either option imposes no
corresponding limit; `MAXLEN 0` permits only a stored empty root path.
The command visits only ancestor nodes along the query path and does not
scan the entire tree.
For example, suppose the matching path lengths are `[8, 16, 24, 32]`:
```text
PHPREFIXES key query LENGTHS COUNT 2 MAXLEN 24
```
`MAXLEN 24` first produces `[8, 16, 24]`; `COUNT 2` then retains the two
deepest matches, so the final result is `[16, 24]`.
### 4.11 `PHDELPREFIX`
```text
PHDELPREFIX key prefix
```
Delete every logical path satisfying `prefix ⪯ path` and return the
number of paths deleted. An empty prefix clears the entire tree but
preserves the empty top-level Path Hash key. Use `DEL` or `UNLINK` to
remove the key itself.
This is an `@slow` command with time complexity `O(L + Σ(Pᵢ + Fᵢ))`:
deleting each match copies and removes its complete path and
synchronously releases every field in its payload.
The implementation may collect matching paths in bounded-size chunks to
avoid iterator invalidation and limit the number of temporary path
copies. This chunking does not make the operation incremental: the
complete matching subtree is deleted synchronously and atomically in one
command invocation, without yielding between chunks. A large subtree can
therefore cause high latency.
### 4.12 `PHSCAN`
```text
PHSCAN key cursor [PREFIX prefix] [COUNT count] [WITHVALUES]
```
Incrementally traverse paths in lexicographic order. The cursor is an
opaque bulk string: pass `0` on the first call; the server returns
`[next-cursor, entries]`, where `next-cursor = 0` means the traversal is
complete. A nonzero cursor encodes the previous position and must not be
interpreted by the client.
- `PREFIX` restricts traversal to the specified subtree.
- `COUNT` is a hint for the number of paths to examine in each call and
does not guarantee an exact result count.
- `WITHVALUES` returns each entry as `[path, [field, value, ...]]` in
RESP2 or `[path, {field: value, ...}]` in RESP3. Field order within each
payload is undefined. The outer response remains `[next-cursor,
entries]` in both protocols.
- As with Valkey `SCAN`, concurrent modifications can cause duplicates
or omissions, and clients must process results idempotently.
The implementation can encode a version number and the last returned
path in the cursor, then use `raxSeek` to resume at the lexicographic
position without retaining an iterator session on the server.
### 4.13 `PHCARD`
```text
PHCARD key
```
Return the number of logical paths in the tree that carry a non-empty
payload. Return 0 for either an empty Path Hash or a missing key; use
`EXISTS` or `TYPE` when the caller needs to distinguish those states.
## 5. Possible Implementation Using the Current Internal `rax`
### 5.1 Overall Memory Layout
```c
typedef struct RadixPayload {
uint8_t encoding; /* listpack or dict */
uint32_t num_fields;
void *data;
} RadixPayload;
typedef struct RadixObject {
rax *index; /* path -> RadixPayload* */
uint64_t num_paths;
uint64_t num_fields;
} RadixObject;
```
```text
Valkey key
└── RadixObject
├── rax index
│ └── data pointer ─────────┐
└── counters │
▼
RadixPayload
└── listpack or dict
└── field -> value
```
The `rax` data pointer points to a `RadixPayload`. `rax` already allows
a key to terminate at an internal node that still has descendants, so
`P1` and `P1 || P2` can both carry payloads. A small payload uses a
listpack to reduce memory overhead when each block belongs to only a few
Workers. After the entry or value threshold is exceeded, the payload is
promoted one-way to a dict to provide near-`O(1)` field updates. It is
not automatically demoted after deletion, avoiding encoding oscillation.
### 5.2 `rax` Capabilities That Can Be Reused Directly
- `raxNew` creates an empty tree;
- `raxInsert` and `raxTryInsert` insert a path and payload pointer;
- `raxFind` supports exact-path operations such as `PHGET` and
`PHEXISTS`;
- `raxRemove` removes a logical path after its payload becomes empty and
recompresses compressible nodes;
- `raxStart`, `raxSeek`, `raxNext`, and `raxPrev` support lexicographic
scan, RDB save, and AOF rewrite;
- `raxSize` provides the number of logical keys;
- `raxAllocSize` can be included in `MEMORY USAGE`;
- Path compression stores the shared bytes of many cumulative hash
prefixes only once.
### 5.3 Prefix-Walk Capability to Add
Currently, `raxFind` returns data only when the query exactly matches a
stored key. `raxSeek` provides lexicographic positioning but does not
efficiently enumerate ancestors of the query. If a client calls
`raxFind` separately for every byte prefix of the query, the worst-case
complexity degrades from `O(L)` to `O(L²)`.
Two helpers should be added at the internal `rax` layer, or a
callback-based walk should be added and shared by both:
```c
void *raxFindLongestPrefix(
rax *rt,
const unsigned char *query,
size_t len,
size_t *matched_len);
int raxForEachPrefix(
rax *rt,
const unsigned char *query,
size_t len,
raxPrefixCallback callback,
void *context);
```
The algorithm walks the query only once:
```text
1. Start at the head; if the root is a logical key, record length 0.
2. At a regular node, select the child for the next query byte.
3. At a compressed node, compare the entire compressed edge at once; stop immediately if any byte differs.
4. Whenever an iskey node is reached, record the number of bytes consumed and the data pointer.
5. LONGEST retains only the final record; PREFIXES invokes the callback for every record in order.
6. Stop when the query is exhausted or the required branch does not exist.
```
The time complexity is `O(L + K)`, where `K` is the number of logical
paths actually matched; siblings and descendants are not scanned. The
implementation must correctly handle a root key, a mismatch in the
middle of a compressed edge, a key that is also an ancestor of another
key, an empty query, and binary zero.
The current `raxLowWalk` already contains most of the path-descent
logic, but it is `static inline` and returns only the stopping node and
split position. The implementation can extract a shared internal walker
without changing the existing insertion semantics, or add a separate
read-only prefix walker. The latter has a smaller change surface and is
easier to validate with differential tests.
### 5.4 `PHDELPREFIX` and `PHSCAN`
`PHSCAN PREFIX p` can use `raxSeek(">=", p)` to locate the first
candidate, then call `raxNext` until a key no longer starts with `p`.
`PHDELPREFIX` cannot iterate and delete without accounting for iterator
invalidation. The v1 implementation collects a bounded number of
complete path copies, closes the iterator, removes those paths, and
repeats until no match remains. The bound applies only to the number of
temporary path copies. The outer loop still deletes the complete subtree
synchronously and atomically in one event-loop turn, so chunking does
not bound command latency. Its work includes copying and removing every
matched path and releasing every field in the associated payloads.
If truly nonblocking deletion of a large subtree is needed in the
future, `rax` can gain a subtree-detach capability and hand detached
nodes to the lazy-free thread. This is not required for v1.
---------
Signed-off-by: bodong.ybd <bodong.ybd@alibaba-inc.com>
Co-authored-by: Jacob Murphy <jkmurphy@google.com>
`process_file` runs inside a `multiprocessing.Pool` worker, and on a
reply schema mismatch it re-raised the `jsonschema` exception. That
exception holds a reference to a `TypeChecker` lambda in
`jsonschema._types`, which cannot be pickled, so the parent process
reports `MaybeEncodingError`/`PicklingError` and the actual mismatch is
only visible in the worker's own output, hundreds of lines earlier in
the CI log. This raises a `RuntimeError` carrying the message instead,
so the failure the validator exits on is the failure it found.
<details>
<summary>Details</summary>
## Problem
`utils/req-res-log-validator.py:239` ends the schema-mismatch handler
with a bare `raise`. The pool is created at `:334` and `process_file` is
the worker body from `:186`.
Reproduced standalone with the pinned `jsonschema==4.17.3`:
```
pickle of jsonschema.ValidationError: PicklingError: Can't pickle <function <lambda> at 0x7f6d8c90add0>: attribute lookup <lambda> on jsonschema._types failed
real message the worker saw: None is not of type 'string'
bare `raise` (upstream today): parent saw MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7f6d8c73b450>'. Reason: 'PicklingError("Can't pickle <function <lambda> at 0x7f6d8c90add0>: attribute lookup <lambda> on jsonschema._types failed")'
RuntimeError (this patch) : parent saw RuntimeError: JSON schema validation error on fake.log: None is not of type 'string'
```
This is not specific to any one command. Every reply-schema mismatch
presents the same way, which is why the `reply-schemas-validator` job's
failure output does not name the schema that failed. Contrast #2676,
where the worker raised a `ValueError` and the parent reported it
verbatim.
It surfaced while triaging a Daily failure where `XPENDING` on an empty
PEL was rejected by a stale schema (fixed separately in #4653). The
schema was the bug; the pickling error was what made it expensive to
find.
## Scope
Only the `jsonschema` handler at `:239` is changed. The two handlers at
`:210` and `:214` also re-raise from the worker, but they carry
`json.decoder.JSONDecodeError` and whatever `Request`/`Response`
construction throws, which pickle fine, so they are left alone rather
than widened speculatively.
## Alternative rejected
Catching `MaybeEncodingError` in the parent and reformatting it would
keep the worker code unchanged, but the parent has no access to the
original exception's fields by then, only the pickling failure's text.
The information has to be flattened before it crosses the process
boundary.
</details>
This was generated by AI but verified, with love, by a human.
---------
Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
PHLONGEST, PHPREFIXES and PHSCAN declare NONDETERMINISTIC_OUTPUT_ORDER in their json, but the commands.def committed in a84b2f5 (#4506) has their tips as NULL, so those three commands report no tips in COMMAND INFO and COMMAND DOCS. This is the output of `make commands.def` with nothing else changed. The `validate commands.def up to date` step is currently failing on unstable because of it. This was generated by AI but verified, with love, by a human. Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
Harden release script. Tie release to commit SHA, not git tag. --------- Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
The external test jobs launch valkey-server with --daemonize yes and then immediately connect from the next workflow step. daemonize() exits the parent as soon as it forks (src/server.c:7332), while the child does not bind the listening socket until initListeners() (src/server.c:8168), so the next step can be refused. test-external-cluster hits this first because valkey-cli cluster addslots is the very next step, and its 'sleep 5' runs after addslots rather than before it. The standalone and nodebug jobs are equally exposed: in external mode the tcl harness connects once with no retry (tests/support/server.tcl:424). Poll valkey-cli ping until it answers PONG before the first client connect, bounded at 60s and dumping the server log on timeout. PING is keyless so it answers before any slots are assigned (src/server.c:4766), which is what the cluster job needs. Signed-off-by: Madelyn Olson <madelyneolson@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The three external test jobs launch
valkey-server --daemonize yesin one workflow step and connect to it from the next one, but daemonizing exits the parent process the instant it forks, well before the child binds its listening socket.test-external-clusterfails first becausevalkey-cli cluster addslotsis the very next step and gets a connection refused, and thesleep 5on that line runs afteraddslotsrather than before it, so it never helps. The standalone and nodebug jobs are exposed to the same race and only survive on timing, since the tcl harness connects once with no retry when running against an external server. This adds a boundedvalkey-cli pingpoll before the first client connect in each job.AI generated details
Problem
Run: https://github.com/valkey-io/valkey/actions/runs/35123001126
Job: https://github.com/valkey-io/valkey/actions/runs/35123001126/job/104885142811
The
external-server.logartifact from that job shows the server started fine, and shows the fork as two PIDs:The parent exits at
src/server.c:7332:daemonize()is called frommain()atsrc/server.c:8138. The bind happens 30 lines later, atsrc/server.c:8168:Mapping the log to
main(): the PID-file warning iscreatePidFile()(8156) at .652, "Running mode=cluster" isserverAsciiArt()(8158) at .652, "No cluster configuration found" isclusterInit()(8161) at .653, and the static Lua load (8177) is at .895.initListeners()(8168) therefore ran inside the (.653, .895] window. The refusal at .667 lands inside that window, pinning the bind after the connect attempt empirically as well as by code order..github/workflows/external.yml:79-80before this change:The
sleep 5is a;-separated statement afteraddslots, so it only delays the transition into./runtest.Why all three jobs, not just the cluster one
External mode connects once and never retries:
wait_server_startedis not on this path; the helpers bail out early for external servers (tests/support/server.tcl:55-56). On this run the standalone job connected ~78ms after launching the server:It wins on two margins, neither guaranteed: standalone startup skips
clusterInit()(8161) andclusterInitLast()(8170) so it reachesinitListeners()sooner, and./runtestburns a few tens of ms on cleanup, spawning tclsh, and loading test files before connecting.Decisions
Why not move the existing
sleep 5beforeaddslots. The observed fork-to-bind gap was 240ms on a healthy runner, but nothing inmain()bounds it.moduleLoadFromQueue()(8164),ACLLoadUsersAtStartup()(8167), andloadDataFromDisk()(8236) all sit on that path, so a fixed sleep converts a fast flake into a slow one while costing 5s on every green run.Why
PING. In the cluster job the probe has to run beforeaddslots, so it cannot depend on slot coverage.PINGis keyless and is not redirected, atsrc/server.c:4766-4767:Shape. Follows the existing polling idiom at
.github/workflows/daily.yml:806and:824(while [ $(./src/valkey-cli ...) ]; do sleep 1; done), with a bound and a log dump added, since those two loops are unbounded.Not fixed here. "Failed to write PID file: Permission denied" is unrelated.
--daemonize yeswithout--pidfiledefaults to/var/run/valkey.pid(src/server.h:170), unwritable by the runner user. The write is best-effort and does not retry or block (src/server.c:7319-7326), so it costs one failedfopen.Not caused by the PR under test.
gh pr diff 4711 --name-onlyreturnssrc/t_stream.candtests/unit/type/stream.tcl, +42/-2. Nothing in startup, cluster init, or workflows, and the failure is a connect that precedes any test code.Testing
The local machine starts in 41ms, faster than the shell needs to expand
{0..16383}, so the race does not fire on its own. Widening the window with--aclfileholding 200k users, which loads atsrc/server.c:8167immediately before the bind, reproduces the CI failure exactly:Same config with the new step in front of
addslots:Timeout path, bound shortened to 3 for the run, against a port with nothing on it:
That failure branch is worth checking because GitHub runs these steps as
shell: /usr/bin/bash -e {0}. A[ ... ] && exit 0whose test fails is part of an&&list and is not the command following the final&&, so-edoes not abort the loop while the probe is still unready:This was generated by AI but verified, with love, by a human.