Rust SCSI Target

A high-performance, multi-protocol storage target implementation in Rust supporting iSCSI and NVMe over Fabrics (NVMe-oF) via TCP transport.

Features

Dual Protocol Support: Run iSCSI and NVMe-oF from the same codebase
iSCSI Protocol Compliance: Full SCSI command router with MODE SENSE/SELECT, WRITE AND VERIFY, LOG SENSE, FORMAT UNIT, PERSISTENT RESERVE, UNMAP, WRITE SAME, COMPARE AND WRITE
NVMe-oF/TCP Target: Discovery controller, Fabrics Connect/Disconnect, Identify, logs, features, reservations, dataset management, and read/write I/O paths
Unified Management: Single admin API and metrics endpoint for both protocols
Dynamic Configuration: Add and remove iSCSI IQNs and NVMe NQNs through the CLI or JSON-RPC Admin API
Cluster Coordination: Built-in support for distributed deployments via etcd, Consul, or Redis
ALUA/MPIO: Asymmetric Logical Unit Access for multi-path I/O
Performance Optimized: TCP_NODELAY, SO_REUSEPORT, TCP Fast Open (Linux)
Pluggable Backends: Memory, mmap, file-backed, and block-device backends

Architecture

┌─────────────────────────────────────────┐
│         Unified Daemon                  │
│  (targets/unified)                      │
├─────────────────────────────────────────┤
│  Admin API (:5000)  │  Metrics (:5001)  │
├─────────────────────────────────────────┤
│      MultiProtocolRuntime               │
│  ┌─────────────┐  ┌─────────────────┐   │
│  │ iSCSI       │  │ NVMe-oF         │   │
│  │ TargetManager│  │ NvmeRuntime     │   │
│  └─────────────┘  └─────────────────┘   │
├─────────────────────────────────────────┤
│  iSCSI (:3260)      NVMe-oF (:4420)     │
└─────────────────────────────────────────┘

Workspace Crates

Crate	Description
`target-core`	Protocol-agnostic abstractions: `Backend`, `StorageRuntime`, metrics, coordinator
`iscsi-target`	iSCSI implementation: session, connection, login, command routing
`nvme-target`	NVMe-oF implementation: controller, queue pair, capsule protocol
`network-tokio`	Tokio-based TCP frontend for iSCSI
`network-common`	Shared TCP listener optimizations
`admin-api`	JSON-RPC admin API for target management
`metrics`	Prometheus metrics export
`iscsi-protocol`	iSCSI protocol types and PDU codec
`nvme-protocol`	NVMe command and fabrics protocol types
`backend-block`	Block device backend
`backend-memory`	Memory and mmap backends

Building

# Development build, including libraries, binaries, examples, and tests
cargo build --all-targets

# Release binaries
cargo build --release

# Standard quality gates
cargo fmt --all -- --check
cargo test
cargo clippy --workspace --all-targets -- -D warnings

Running

Default endpoints:

Binary	Protocol bind	Admin API	Metrics	Health
`unified`	iSCSI `0.0.0.0:3260`, NVMe-oF `0.0.0.0:4420`	`127.0.0.1:5000`	`127.0.0.1:5001`	`127.0.0.1:8080`
`iscsid`	iSCSI `0.0.0.0:3260`	`127.0.0.1:5000`	`127.0.0.1:5001`	`127.0.0.1:8080`
`nvmeofd`	NVMe-oF `0.0.0.0:4420`	`127.0.0.1:5002`	`127.0.0.1:5003`	`127.0.0.1:8082`

Binding to well-known ports such as 3260 may require elevated privileges on some systems. Use explicit --*-bind or config-file bind addresses for unprivileged local testing.

Unified Daemon (Recommended)

cargo run --bin unified run
cargo run --bin unified run --config unified.toml

Configuration file (unified.toml):

[server]
iscsi_bind = "0.0.0.0:3260"
nvme_bind = "0.0.0.0:4420"
admin_bind = "127.0.0.1:5000"
metrics_bind = "127.0.0.1:5001"

[[iscsi_targets]]
iqn = "iqn.2024-06.com.rust-iscsi:disk1"
alias = "Rust iSCSI Target"

[[iscsi_targets.luns]]
lun = 0
backend_type = "memory"
size = "1GiB"
block_size = 512
readonly = false
# Supported backend_type values: "memory", "mmap", "file", "block".
# For persistent iSCSI storage, use backend_type = "file" or "block".
# path = "/var/lib/rust-scsi/iscsi-disk1.img"

# Optional: per-target CHAP authentication
[iscsi_targets.auth]
username = "initiator1"
password = "secret123"
mutual = false

[nvme_target]
nqn = "nqn.2014-08.org.nvmexpress:uuid:rust-target-0001"

[nvme_target.backend]
type = "memory"
size = "1G"
block_size = 512
readonly = false
# Supported type values: "memory", "mmap", "file", "block".
# For persistent NVMe-oF storage, use type = "file" or "block".
# path = "/var/lib/rust-scsi/nvme-target.img"

[coordinator]
backend = "local"
node_id = "node-1"
# backend may also be "etcd", "consul", or "redis".
# endpoints = ["127.0.0.1:2379"]
# datacenter = "dc1"  # Consul only

iSCSI Only

cargo run --bin iscsid run
cargo run --bin iscsid run --config config.toml

With CHAP authentication:

cargo run --bin iscsid run --auth-user initiator1 --auth-pass secret123
cargo run --bin iscsid run --auth-user initiator1 --auth-pass secret123 --auth-mutual

Configuration file (config.toml):

[server]
bind_address = "0.0.0.0:3260"
threads = 4

[iscsi]
max_connections = 8
max_sessions = 256
max_queue_depth = 128
error_recovery_level = 0
initial_r2t = false
immediate_data = true
header_digest = "None"
data_digest = "None"

[[targets]]
iqn = "iqn.2024-06.com.rust-iscsi:disk1"
alias = "Rust iSCSI Target"

[[targets.luns]]
lun = 0
backend = "memory"
size = "1GiB"
readonly = false
# Supported backend values: "memory", "mmap", "file", "block".
# For persistent iSCSI storage, use backend = "file" or "block".
# path = "/var/lib/rust-scsi/iscsi-disk1.img"

[auth.chap]
username = "initiator1"
password = "secret123"

NVMe-oF Only

cargo run --bin nvmeofd run
cargo run --bin nvmeofd run --config nvmeofd.toml

Configuration file: /etc/nvmeofd/config.toml or ./nvmeofd.toml

Example file-backed NVMe-oF target:

nqn = "nqn.2014-08.org.nvmexpress:uuid:rust-target-0001"

[server]
bind_address = "0.0.0.0:4420"

[backend]
type = "file"
size = "1G"
block_size = 512
readonly = false
path = "/var/lib/rust-scsi/nvme-target.img"

[coordinator]
backend = "local"
node_id = "nvme-node-1"

Coordinator Backends

All daemons use the same coordinator options. local is the default single-node backend. Distributed deployments may use etcd, consul, or redis; unknown backend names fail daemon startup instead of silently falling back to local.

[coordinator]
backend = "etcd"        # local | etcd | consul | redis
node_id = "node-1"
endpoints = ["127.0.0.1:2379"]
datacenter = "dc1"      # used by Consul; defaults to dc1

Default endpoints are 127.0.0.1:2379 for etcd, 127.0.0.1:8500 for Consul, and 127.0.0.1:6379 for Redis when endpoints is omitted.

Storage Backends

All daemons accept the same backend names in CLI and dynamic Admin API paths:

Backend	Use case	Notes
`memory`	Fast disposable test LUNs/namespaces	Data is lost when the daemon exits
`mmap`	Anonymous memory-mapped storage	Uses Linux mmap optimizations and falls back to in-memory storage on other platforms
`file`	Persistent file-backed storage	Prefer an explicit `path`; a deterministic temp path is used when omitted
`block`	Existing block device or block-device-like path	Prefer an explicit `path`; requires appropriate OS permissions and isolated test devices

size accepts raw bytes or human-readable units such as 512MiB, 1GiB, 1G, and 2GB. block_size defaults to 512 bytes where the field exists. readonly = true wraps the selected backend in a read-only adapter and rejects write-affecting commands while preserving read paths.

Security

CHAP Authentication

iSCSI targets support CHAP one-way and mutual authentication:

One-way: Target verifies the initiator's identity
Mutual: Both target and initiator verify each other

Supported hash algorithms: MD5, SHA1, SHA256, SHA3-256.

ACL (Access Control Lists)

Per-target ACL controls which initiators can connect:

allowed_initiators: IQN allow list (supports wildcards like iqn.2024-01.com.test:*)
denied_initiators: IQN deny list (takes precedence)
allowed_ips: IP address allow list

If no ACL is configured, all initiators are allowed.

Graceful Shutdown & Health Checks

All daemons support graceful shutdown on SIGINT/SIGTERM/SIGHUP (Unix) or Ctrl+C:

# Stop accepting new connections, wait for active I/O to complete
cargo run --bin unified run
# Press Ctrl+C or send SIGTERM

HTTP health check endpoints (default 127.0.0.1:8080):

curl http://127.0.0.1:8080/health  # {"status":"healthy"}
curl http://127.0.0.1:8080/ready   # {"status":"ready"} or 503

Change health check bind address:

cargo run --bin unified run --health-bind 0.0.0.0:8080
cargo run --bin iscsid run --health-bind 127.0.0.1:8081
cargo run --bin nvmeofd run --health-bind 127.0.0.1:8082

CLI Dynamic Configuration

All binaries expose a CLI for dynamic target management without restarting the daemon. Use --admin <addr> on any management command when the daemon was started with a non-default Admin API bind address.

# iSCSI
cargo run --bin iscsid -- target list
cargo run --bin iscsid -- target add --iqn iqn.2024-06.com.rust-iscsi:disk2 --backend memory --size 2G
cargo run --bin iscsid -- target add --iqn iqn.2024-06.com.rust-iscsi:disk3 --backend file --size 2G --block-size 512 --path /var/lib/rust-scsi/iscsi-disk3.img
cargo run --bin iscsid -- target remove --iqn iqn.2024-06.com.rust-iscsi:disk2
cargo run --bin iscsid -- session list
cargo run --bin iscsid -- session disconnect --id <session_id>
cargo run --bin iscsid -- health
cargo run --bin iscsid -- metrics
cargo run --bin iscsid -- config validate config.toml

# Unified
cargo run --bin unified -- target list
cargo run --bin unified -- target add --iqn iqn.2024-06.com.rust-iscsi:disk2 --backend memory --size 2G
cargo run --bin unified -- target add --iqn iqn.2024-06.com.rust-iscsi:disk3 --backend file --size 2G --path /var/lib/rust-scsi/iscsi-disk3.img
cargo run --bin unified -- target add --protocol nvme --nqn nqn.2014-08.org.nvmexpress:uuid:disk2 --backend file --size 2G --path /var/lib/rust-scsi/nvme-disk2.img
cargo run --bin unified -- target remove --iqn iqn.2024-06.com.rust-iscsi:disk2
cargo run --bin unified -- target remove --nqn nqn.2014-08.org.nvmexpress:uuid:disk2
cargo run --bin unified -- session list
cargo run --bin unified -- session disconnect --id <session_id>
cargo run --bin unified -- health
cargo run --bin unified -- metrics
cargo run --bin unified -- config validate unified.toml

# NVMe-oF
cargo run --bin nvmeofd -- target list
cargo run --bin nvmeofd -- target add --nqn nqn.2014-08.org.nvmexpress:uuid:disk2 --backend file --size 2G --path /var/lib/rust-scsi/nvme-disk2.img
cargo run --bin nvmeofd -- target remove --nqn nqn.2014-08.org.nvmexpress:uuid:disk2
cargo run --bin nvmeofd -- health
cargo run --bin nvmeofd -- metrics
cargo run --bin nvmeofd -- config validate nvmeofd.toml

Admin API

JSON-RPC 2.0 over newline-delimited TCP (default port 5000 for unified/iSCSI, 5002 for NVMe-oF). CLI commands use this API through the built-in AdminClient.

Example methods:

target_list — List all targets
target_get — Get one target by target_id, id, iqn, or nqn
target_add — Dynamically add a target (rejects duplicate target IDs)
target_remove — Remove a target by target_id, id, iqn, or nqn
session_list — List active sessions
session_disconnect — Disconnect a session
stats — Get runtime statistics
health_check — Cluster health status
get_metrics — Return runtime counters as JSON
get_backend_status — Return backend size/block status
get_logs — Return recent in-memory log entries, with optional limit
get_alua_state — Return ALUA state, optionally filtered by target
cluster_members — Return coordinator cluster membership
set_tpg_state — Change ALUA TPG state
failover — Trigger LUN failover

For iSCSI target_add, the JSON config accepts iqn, optional alias, luns, CHAP fields (auth_username/auth_password, chap_username/chap_password, or nested auth.chap.username/auth.chap.password plus auth_mutual or auth.chap.mutual), and ACL arrays (allowed_initiators, denied_initiators, allowed_ips). Each LUN accepts backend_type or backend, optional path for file/block backends, and size may be either bytes as a number or a human-readable string such as "32MiB". If a file/block LUN omits path, the daemon creates a deterministic file under the system temp directory using both target IQN and LUN number.

For NVMe-oF target_add, the JSON config accepts nqn, backend_type, backend, or type, optional readonly, block_size, optional path for file/block backends, and size as either bytes as a number or a human-readable string such as "32MiB". If a file/block target omits path, the daemon creates a deterministic file under the system temp directory for that NQN.

Direct TCP examples:

# List targets on a unified or iSCSI daemon.
printf '%s\n' '{"jsonrpc":"2.0","method":"target_list","id":1}' | nc 127.0.0.1 5000

# Add an iSCSI target with CHAP and ACL through Admin API.
printf '%s\n' '{
  "jsonrpc":"2.0",
  "method":"target_add",
  "params":{
    "iqn":"iqn.2024-06.com.rust-iscsi:api-disk",
    "alias":"API iSCSI Target",
    "auth":{"chap":{"username":"initiator1","password":"secret123","mutual":true}},
    "allowed_initiators":["iqn.2024-01.com.test:*"],
    "denied_initiators":[],
    "allowed_ips":["127.0.0.1"],
    "luns":[{"lun":0,"backend_type":"file","size":"32MiB","block_size":512,"path":"/tmp/rust-scsi-api-iscsi.img"}]
  },
  "id":2
}' | nc 127.0.0.1 5000

# Add an NVMe-oF target on nvmeofd.
printf '%s\n' '{
  "jsonrpc":"2.0",
  "method":"target_add",
  "params":{
    "nqn":"nqn.2014-08.org.nvmexpress:uuid:api-disk",
    "backend_type":"file",
    "size":"32MiB",
    "block_size":512,
    "path":"/tmp/rust-scsi-api-nvme.img"
  },
  "id":3
}' | nc 127.0.0.1 5002

# Remove by protocol-specific identifier.
printf '%s\n' '{"jsonrpc":"2.0","method":"target_remove","params":{"iqn":"iqn.2024-06.com.rust-iscsi:api-disk"},"id":4}' | nc 127.0.0.1 5000
printf '%s\n' '{"jsonrpc":"2.0","method":"target_remove","params":{"nqn":"nqn.2014-08.org.nvmexpress:uuid:api-disk"},"id":5}' | nc 127.0.0.1 5002

Testing

# Workspace unit, integration, and doc tests
cargo test

# Build every target
cargo build --all-targets

# Warning gate
cargo clippy --workspace --all-targets -- -D warnings

# End-to-end mock test (in-process iSCSI server + client)
cargo test -p iscsi-target --test e2e_mock

# Quick mock startup for manual testing
cargo run --bin iscsid run --config scripts/mock-config.toml

# Portable userspace iSCSI daemon smoke test
scripts/iscsi_userspace_smoke.py

# Optional Linux nvme-cli smoke test for NVMe-oF/TCP
scripts/nvmeof_smoke.sh

# Portable userspace NVMe-oF/TCP daemon smoke test
scripts/nvmeof_userspace_smoke.py

# Optional libiscsi initiator/conformance run for iSCSI
tests/integration/run_tests.sh

# Local unified daemon smoke test (no external initiators required)
scripts/unified_smoke.sh

# Aggregated local verification
scripts/e2e_verify.sh

Recommended verification matrix:

Command	Coverage
`cargo fmt --all -- --check`	Workspace formatting
`cargo test`	Rust unit, integration, and doc tests across protocol, backend, admin, and daemon crates
`cargo build --all-targets`	All binaries and test targets compile
`cargo clippy --workspace --all-targets -- -D warnings`	Warning-free workspace
`scripts/unified_smoke.sh`	Unified daemon startup, readiness, admin API, metrics, and dynamic iSCSI/NVMe target add/remove
`scripts/iscsi_userspace_smoke.py`	Portable iSCSI login, CHAP, discovery, SCSI commands, R2T/Data-Out, reservations, UNMAP, NOP, dynamic target API
`scripts/nvmeof_userspace_smoke.py`	Portable NVMe/TCP discovery, connect, admin/I/O queues, reads/writes, logs, reservations, dataset management, dynamic target API
`scripts/nvmeof_smoke.sh`	Optional Linux `nvme-cli` initiator smoke; falls back to the userspace NVMe-oF smoke unless strict mode is requested
`tests/integration/run_tests.sh`	Optional libiscsi discovery, initiator smoke, SCSI/iSCSI conformance, digest, CHAP, and mutual CHAP coverage

The E2E mock test (crates/iscsi-target/tests/e2e_mock.rs) verifies the full iSCSI protocol stack:

iSCSI Login (Security Negotiation → Operational Negotiation → Full Feature Phase)
SCSI command responses: TEST UNIT READY, INQUIRY, MODE SENSE(6/10), READ CAPACITY(10), LOG SENSE
Mode page validation: Caching (0x08), Control (0x0A), Error Recovery (0x01), Informational Exceptions (0x1C)

For automated CI verification:

./scripts/e2e_verify.sh   # tests + e2e_mock + build + smoke check

The optional scripts/nvmeof_smoke.sh check starts a memory-backed nvmeofd instance and uses Linux nvme-cli to discover, connect, identify, and disconnect. It skips when Linux, nvme-cli, or required privileges are missing; set NVME_CMD_PREFIX=sudo instead of running the whole script as root. By default it performs no namespace writes; set RUN_READ=1 or RUN_WRITE_ZEROES=1 only for an isolated test host.

When nvme-cli prerequisites are missing, scripts/nvmeof_smoke.sh automatically falls back to scripts/nvmeof_userspace_smoke.py. Set NVME_SMOKE_REQUIRE_NVMECLI=1 to preserve the old strict behavior and exit SKIP instead of running the fallback.

scripts/nvmeof_userspace_smoke.py starts a memory-backed nvmeofd instance and verifies NVMe/TCP ICReq/ICResp, Discovery Controller connect plus Discovery Log Page, Fabrics Connect, Get/Set Features, Keep Alive, Identify Controller, and Identify Namespace, checks the Supported Log Pages and Command Effects logs, then performs a one-block write/read round trip, Flush, Write Zeroes, and zero read-back using only the Python standard library. It also verifies the non-inline write path with R2T and H2CData, Verify, Compare, Copy, Write Uncorrectable, Format NVM via a separate admin queue, Reservation Register/Report/Acquire/Release, Dataset Management deallocate with zero read-back, explicit Fabrics Disconnect, and dynamic Admin API add/remove of a second NVMe-oF target with duplicate-add rejection, backend aliases, human-readable sizes, discovery, and connect validation. It is portable across macOS and Linux, requires no root privileges or kernel NVMe initiator, and cleans up the temporary daemon.

scripts/iscsi_userspace_smoke.py starts a memory-backed iscsid instance and verifies mutual CHAP Login, SendTargets, TEST UNIT READY, INQUIRY, READ CAPACITY(10), MODE SENSE(6/10), MODE SELECT(6/10), LOG SENSE, REPORT SUPPORTED OPERATION CODES, immediate and R2T/Data-Out WRITE(10)/READ(10) round trips, PERSISTENT RESERVE IN/OUT, Task Management LUN RESET, and UNMAP/GET LBA STATUS, plus NOP-Out/NOP-In and dynamic Admin API add/remove of a second CHAP/ACL-configured iSCSI target with SendTargets discovery updates. The dynamic target path also exercises duplicate-add rejection, backend aliases, and human-readable size strings. File/block LUN path handling is covered by unit tests and daemon smoke checks. It finishes with Logout using only the Python standard library. It requires no root privileges, kernel iSCSI initiator, libiscsi tools, or jq.

scripts/unified_smoke.sh starts a temporary unified daemon with one memory-backed iSCSI target and one memory-backed NVMe-oF target, verifies /ready, admin health, target list, metrics, and dynamic admin add/remove for both protocols, then cleans up. It does not require root, nvme-cli, libiscsi, or jq.

The optional tests/integration/run_tests.sh check builds iscsid, starts a disposable memory-backed iSCSI LUN, then uses the vendored libiscsi tools for discovery, INQUIRY, READ CAPACITY, SCSI conformance, iSCSI protocol checks, and thin-provisioning coverage such as UNMAP, GET LBA STATUS, ORWRITE, and WRITE SAME. It expects normal Rust build dependencies plus libiscsi build tooling such as autoconf, automake, libtool, pkg-config, and CUnit. Conformance failures are strict by default; set ALLOW_CONFORMANCE_FAILURES=1 only for exploratory runs. Use SMOKE_ONLY=1 to run only discovery, INQUIRY, and READ CAPACITY when CUnit is unavailable, or CHECK_ONLY=1 to preflight local build dependencies without starting a target. On macOS the script disables libiscsi's -Werror while building the vendored user-space tools, because current SDK headers emit warnings for APIs used by libiscsi. Common overrides include TEST_PORT=13260, ISCSID_BIN=/path/to/iscsid, and LIBISCSI_CONFIGURE_FLAGS="...".

Platform Notes

Linux is required for the kernel nvme-cli initiator path in scripts/nvmeof_smoke.sh; macOS and Linux can both run the userspace NVMe-oF smoke.
Kernel iSCSI initiators are not required for the default tests; the iSCSI userspace smoke and libiscsi integration tests use user-space clients.
Binding to privileged ports, opening raw block devices, or using kernel initiators may require elevated permissions.
File and block backends mutate their backing storage. Use temporary files or isolated test devices unless the data is disposable.
The repository vendors libiscsi under tests/libiscsi; the integration script can build those tools locally when the host has the C build dependencies.

License

MIT OR Apache-2.0

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
crates		crates
scripts		scripts
targets		targets
tests		tests
.gitignore		.gitignore
AGENTS.md		AGENTS.md
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
README.md		README.md

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Repository files navigation

Rust SCSI Target

Features

Architecture

Workspace Crates

Building

Running

Unified Daemon (Recommended)

iSCSI Only

NVMe-oF Only

Coordinator Backends

Storage Backends

Security

CHAP Authentication

ACL (Access Control Lists)

Graceful Shutdown & Health Checks

CLI Dynamic Configuration

Admin API

Testing

Platform Notes

License

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

Rust SCSI Target

Features

Architecture

Workspace Crates

Building

Running

Unified Daemon (Recommended)

iSCSI Only

NVMe-oF Only

Coordinator Backends

Storage Backends

Security

CHAP Authentication

ACL (Access Control Lists)

Graceful Shutdown & Health Checks

CLI Dynamic Configuration

Admin API

Testing

Platform Notes

License

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages