Merge pull request #442 from maxmind/greg/stf-1488

Bound decoder work to prevent a pointer fan-out DoS (STF-1571)

Author: horgh

CHANGELOG.md

@@ -1,15 +1,22 @@
 CHANGELOG
 =========
 
-4.1.1
+4.2.0
 ------------------
 
-* Fixed decoding of data pointers with offsets of 2 GiB or greater. The
-  pointer payload was decoded into an `int`, so such offsets were
-  sign-extended to a negative value and rejected by `Buffer.position()`
-  with an `IllegalArgumentException`. Every record past the 2 GiB
-  boundary was unreachable in databases larger than 2 GiB, which have
-  been supported since 4.0.0.
+* Fixed decoding of data pointers with offsets of 2 GiB or greater. Records
+  beyond that boundary could previously fail with `IllegalArgumentException`.
+* Fixed skipping unknown four-byte pointers during typed decoding. Skipped
+  values that extend past the database are now rejected.
+* Fixed UTF-8 decoding across buffer chunks. Malformed decoded strings and
+  truncated values now throw `InvalidDatabaseException`.
+* Added decoder limits to prevent excessive CPU and memory use from crafted
+  databases: 65,536 decoded or skipped values, 128 nested containers, and 2 MiB
+  of encoded string and bytes payload per operation. Exceeding a limit throws
+  `InvalidDatabaseException`. See [UPGRADING.md](UPGRADING.md) for accounting,
+  decoded-value validation, and collection capacity-hint changes.
+* Improved decoder performance and reduced per-lookup allocation, including
+  UTF-8 string decoding.
 
 4.1.0 (2026-05-12)
 ------------------

UPGRADING.md

@@ -1,3 +1,32 @@
+# Upgrading to 4.2.0
+
+## Decoder Resource Limits
+
+Version 4.2.0 limits the work and memory used by one record or metadata decode.
+The decoder rejects an operation that exceeds any of these limits:
+
+- 65,536 decoded or skipped values under the Java reader's work accounting
+- 128 nested maps or arrays
+- 2 MiB of encoded string and bytes payload materialized by the decoder
+
+A decoded pointer costs one value in addition to its target's logical costs.
+Cached targets retain their value count, container depth, and payload bytes, so
+cache state does not change whether a decode exceeds a limit. Skipped pointers
+count as one value and their targets remain unvisited. Skipped fields receive structural bounds and
+resource checks, but their contents are not fully validated.
+
+These limits leave a wide margin above MaxMind-produced records. A custom
+database containing an unusually large record that decoded in an earlier
+release may now throw `InvalidDatabaseException`. The limits are not
+configurable in this release.
+
+Initial collection capacity hints are capped at 128. Built-in collections grow
+as needed. Custom `List` and `Map` types constructed through an `int` constructor
+receive this capped hint instead of the full declared size.
+
+When following a pointer, the decoder rejects targets that are themselves
+pointers. Decoded integers wider than their format type permits are also rejected.
+
 # Upgrading to 4.0.0
 
 This guide covers the breaking changes introduced in version 4.0.0 and how to

src/main/java/com/maxmind/db/Buffer.java

@@ -1,8 +1,5 @@
 package com.maxmind.db;
 
-import java.nio.charset.CharacterCodingException;
-import java.nio.charset.CharsetDecoder;
-
 /**
  * A generic buffer abstraction that supports sequential and random access
  * to binary data. Implementations may be backed by a single {@link
@@ -96,13 +93,4 @@ sealed interface Buffer permits SingleBuffer, MultiBuffer {
      * @return a duplicate buffer
      */
     Buffer duplicate();
-
-    /**
-     * Decodes the buffer's content into a string using the given decoder.
-     *
-     * @param decoder the charset decoder
-     * @return the decoded string
-     * @throws CharacterCodingException if decoding fails
-     */
-    String decode(CharsetDecoder decoder) throws CharacterCodingException;
 }

src/main/java/com/maxmind/db/DecodedValue.java

@@ -1,16 +1,42 @@
 package com.maxmind.db;
 
 /**
- * {@code DecodedValue} is a wrapper for the decoded value.
+ * An opaque decoded value and its resource costs, produced by {@link NodeCache.Loader}.
+ * Caches retain this instance unchanged for its original key. See {@link NodeCache}.
  */
 public final class DecodedValue {
+    private static final int PAYLOAD_SHIFT = 8;
+    private static final int VALUES_SHIFT = 30;
+    private static final long PAYLOAD_MASK = (1L << 22) - 1;
+
+    // Final fields preserve their initialized values when a cache publishes this object.
     final Object value;
+    private final long costs;
 
-    DecodedValue(Object value) {
+    DecodedValue(Object value, int values, long payloadBytes, int depth) {
         this.value = value;
+        this.costs = ((long) values << VALUES_SHIFT)
+            | (payloadBytes << PAYLOAD_SHIFT)
+            | depth;
     }
 
     Object value() {
         return value;
     }
+
+    static int values(long costs) {
+        return (int) (costs >>> VALUES_SHIFT);
+    }
+
+    static long payloadBytes(long costs) {
+        return (costs >>> PAYLOAD_SHIFT) & PAYLOAD_MASK;
+    }
+
+    static int depth(long costs) {
+        return (int) (costs & 0xFF);
+    }
+
+    long costs() {
+        return this.costs;
+    }
 }