GH-939: Remove reflection for gRPC buffers

[xborder]

What's Changed

• The branch removes the previous reflection-based access to internal gRPC buffers and replaces it with supported gRPC interfaces (Detachable, HasByteBuffer) to safely transfer ownership of direct buffers to Arrow without copying.
• gRPC direct ByteBuffer streams are detached and wrapped as ForeignAllocation for zero‑copy reads
• Fall back to safe buffer copies when gRPC does not expose a direct ByteBuffer or data is fragmented
• Added test coverage

Closes #939.

[lidavidm]

Since we detached the stream, if we give up here, aren't we erroneously going to discard the actual data? My understanding of Detachable is that the original stream is now invalid, right?

[xborder]

good catch. Actually this is redundant, the isDirect is already checked at the beginning of the method so I just removed it

[lidavidm]

All builds fail:

Error:  Non-test scoped test only dependencies found:
Error:     io.grpc:grpc-core:jar:1.78.0:compile

[lidavidm]

Approved CI

[lidavidm]

CC @ejona86 in case you do want to look over this

[lidavidm]

@xborder it seems this doesn't actually work...

[xborder]

@lidavidm Seems like a chicken and the egg problem. grpc-core can't be a compile dependency because it is detected as unused by CI although it is still required in runtime (AddWritableBuffer still needs it for reflection).
Marking it with a runtime scope causes the tests I created to fail because they require the dependency in compile time.

I've changed the tests to use a mock for the dependency it had on grpc-core. This seemed to have solved the issues from the two previous runs but probably diminishes the relevance of these tests

[jbonofre]

@xborder thanks for the update. Let me take a look.

[xborder]

@lidavidm I think there's something I overlooked. frame() is iterating over the stream, if I detach it when reading APP_METADATA_TAG it invalidates the same stream it is iterating. Let me rework on this and I'll keep you posted