StubLoader.unpackLibrary uses InputStream.available() as copy-loop guard, silently truncating the extracted stub (→ SIGBUS BUS_ADRERR / "failed to map segment")

[Hexcles]

Summary

StubLoader.unpackLibrary extracts the bundled libjffi stub with a copy loop whose termination condition and transfer size are both InputStream.available():

jffi/src/main/java/com/kenai/jffi/internal/StubLoader.java

Lines 465 to 474 in 9c5dc82

	private static void unpackLibrary(File dstFile, InputStream sourceIS) throws IOException {
	// Write the library to the tempfile
	try (FileOutputStream os = new FileOutputStream(dstFile)) {
	ReadableByteChannel srcChannel = Channels.newChannel(sourceIS);
	for (long pos = 0; sourceIS.available() > 0; ) {
	pos += os.getChannel().transferFrom(srcChannel, pos, Math.max(4096, sourceIS.available()));
	}
	}
	}

private static void unpackLibrary(File dstFile, InputStream sourceIS) throws IOException {
    try (FileOutputStream os = new FileOutputStream(dstFile)) {
        ReadableByteChannel srcChannel = Channels.newChannel(sourceIS);
        for (long pos = 0; sourceIS.available() > 0; ) {
            pos += os.getChannel().transferFrom(srcChannel, pos, Math.max(4096, sourceIS.available()));
        }
    }
}

InputStream.available() is explicitly not an end-of-stream indicator — its contract says it returns only what can be read without blocking, and that the value must not be used to size a buffer or detect EOF. getStubLibraryStream() returns a resource stream out of the jar (a ZipFile/inflater stream), whose available() can return 0 before the true end. When that happens at a chunk boundary the loop exits early and the extracted .so is silently truncated — unpackLibrary reports no error and loadFromJar proceeds to System.load() a short file.

Note that jffi already has a size + SHA-256 check in verifyExistingLibrary (L476–L501), but it only runs for a pre-existing cached file. The fresh-extraction path does no length/digest verification, so a truncated unpack is loaded blindly.

Symptom

A truncated stub crashes the JVM at load. We hit it as an intermittent fatal SIGBUS:

# SIGBUS (0x7) ...
# Problematic frame: # C  [ld-linux-x86-64.so.2+0x26b4a]      (dlopen)
siginfo: si_signo: 7 (SIGBUS), si_code: 2 (BUS_ADRERR)
Current thread: JavaThread "dd-task-scheduler"
  C   [ld-linux-x86-64.so.2 ...]                              (dlopen)
  V   [libjvm.so ...] JVM_LoadLibrary
  j   com.kenai.jffi.internal.StubLoader.loadFromJar / <clinit>
  j   jnr.ffi.Runtime.getSystemRuntime
  j   jnr.unixsocket.UnixSocketAddress.<init>
  j   com.timgroup.statsd.NonBlockingStatsDClientBuilder.build   (DogStatsD over a UDS)

/proc/self/maps showed the extracted …/jffi<rand>.so with its final rw-p segment page (file offset 0x1b000–0x1c000, a 4 KiB boundary) beyond EOF, so the dynamic linker's relocation write into that page faulted past end-of-file → BUS_ADRERR. The page-aligned cut is consistent with the transferFrom(…, max(4096, available())) loop terminating one chunk early. Depending on where the file is cut, the same truncation can instead surface at dlopen as failed to map segment from shared object — which I believe is what #46 hit (debugged there to loadFromJar "somehow corrupting the library", but closed as a permission issue).

Why it's intermittent

available() only returns 0 prematurely some of the time, and more often under CPU contention. We observe it across a high-volume CI fleet (many thousands of short-lived JVMs on saturated hosts) that reach jffi via the Datadog java-dogstatsd-client UDS path (jnr-unixsocket → jffi). The same agent in long-lived production processes — one extraction at controlled startup — effectively never hits it.

Suggested fix

Don't use available() for control flow; copy to true EOF and (ideally) verify the result:

private static void unpackLibrary(File dstFile, InputStream sourceIS) throws IOException {
    try (OutputStream os = new FileOutputStream(dstFile)) {
        byte[] buf = new byte[16384];
        int n;
        while ((n = sourceIS.read(buf)) != -1) {
            os.write(buf, 0, n);
        }
    }
    // ideally also verify written length / SHA-256 against the packaged resource
    // (the logic already exists in verifyExistingLibrary) and fail loudly on mismatch.
}

The same available()-as-length pattern also appears in verifyExistingLibrary (L477/L481/L490) and is similarly unreliable for jar streams.

Environment

• jffi as bundled in dd-java-agent 1.62.0 (unshaded com.kenai.jffi); the loop is unchanged on current master (9c5dc82).
• JVM: Azul Zulu OpenJDK 21.0.6+7
• OS: Ubuntu 24.04, Linux 5.15 (OCI VM), containerized (podman)
• Reached via com.datadoghq:java-dogstatsd-client opening a DogStatsD Unix domain socket → jnr-unixsocket → jffi.

Related: #46 (same corruption symptom, closed as a permission issue), #158.