UPSTREAM PR #27102: Fix OOM in packed field decoders due to integer overflow in length validation

[loci-dev]

Note

Source pull request: protocolbuffers/protobuf#27102

Problem

Parsing a crafted protobuf message with a packed repeated field whose declared
byte length is close to Integer.MAX_VALUE causes OutOfMemoryError and
crashes the JVM.

The root cause is an integer overflow in ArrayDecoders: the bounds check
fieldLimit > data.length runs after computing
fieldLimit = position + packedDataByteSize, which wraps to a negative value
when packedDataByteSize is large. A negative fieldLimit always passes the
> data.length comparison, allowing ensureCapacity() to receive the
unchecked byte size. For decodePackedFixed32List, this allocates a
536-million-element int[] (~2 GB) from a 7-byte input.

Impact

• Availability: A single unauthenticated 7-byte message triggers OOM on
  any JVM heap size. The allocation happens inside the protobuf parser before
  application code runs.
• Affected methods: All 9 packed field decoders in ArrayDecoders:
  fixed32, fixed64, float, double, varint32, varint64, bool, sint32, sint64.
• Affected versions: protobuf-java ≥ 4.28.0 (where ensureCapacity was
  added to fixed-width decoders). The overflow in fieldLimit computation
  exists in earlier versions as well but was less exploitable without the
  pre-allocation.

Fix

Replace the post-addition bounds check with an overflow-safe pre-addition
validation in all 9 packed decoders:

// Before (vulnerable):
final int fieldLimit = position + packedDataByteSize;
if (fieldLimit > data.length) { ... }

// After (safe):
if (packedDataByteSize < 0 || packedDataByteSize > data.length - position) {
    throw InvalidProtocolBufferException.truncatedMessage();
}
final int fieldLimit = position + packedDataByteSize;

The subtraction data.length - position cannot underflow because position
is always a valid index into data at this point. This rejects both negative
varint lengths and lengths that exceed the remaining buffer, before any
addition or allocation occurs.

Testing

Added PackedFixed32OverflowTest with 9 test methods (one per packed decoder
type). Each crafts a minimal payload with length = Integer.MAX_VALUE and
asserts InvalidProtocolBufferException is thrown. Without the fix, these
tests crash with OutOfMemoryError.

[loci-review]

No meaningful performance changes were detected across 10164 analyzed functions in the following binaries: build.protoc-stable.

💬 Questions? Tag @loci-dev