GH-47534: [C++] Detect conda-installed packages in Meson CI

[WillAyd]

Rationale for this change

This detects conda-installed packages in Meson CI, so that dependencies which are installed via conda do not need to be built from scratch

What changes are included in this PR?

An update to the meson setup invocation

Are these changes tested?

Yes

Are there any user-facing changes?

No

• GitHub Issue: [C++] Meson CI jobs do not detect conda-installed packages #47534

[github-actions]

⚠️ GitHub issue #47534 has been automatically assigned in GitHub to PR creator.

[WillAyd]

Looks like this is working now and cuts the Meson build time down by about ~35 minutes! @kou

[kou]

+1

There are still some packages that can't be discovered. Let's work on it as a separated task.

https://github.com/apache/arrow/actions/runs/17565284388/job/49890903257?pr=47535#step:6:459

Run-time dependency libbrotlidec found: YES 1.1.0
Run-time dependency libbrotlienc found: YES 1.1.0
...
Run-time dependency bzip2 found: NO (tried pkgconfig)
...
Run-time dependency liblz4 found: YES 1.9.4
Run-time dependency snappy found: NO (tried pkgconfig)
...
Run-time dependency Boost (found: filesystem) found: YES 1.85.0 (/opt/conda/envs/arrow)
Run-time dependency GTest found: YES 1.17.0
Run-time dependency gtest_main found: YES 1.17.0
Dependency gtest found: YES 1.17.0 (cached)
Run-time dependency GMock found: YES 1.17.0
Run-time dependency benchmark-main found: NO (tried pkgconfig and cmake)
...
Run-time dependency gflags found: YES 2.2.2
Run-time dependency grpc++ found: YES 1.49.1
Run-time dependency protobuf found: YES 3.21.12
Run-time dependency absl_synchronization found: YES 20220623
Program protoc found: YES (/opt/conda/envs/arrow/bin/protoc)
Program grpc_cpp_plugin found: YES (/opt/conda/envs/arrow/bin/grpc_cpp_plugin)

[WillAyd]

Nice observation @kou - there appear to be various inconsistencies with the packages that are not found:

bzip2 does not provide any information about its dependency (neither pkg-config nor CMake files), so I think that would need an upstream patch

snappy does not provide a pkg-config file. It does provide a CMake file, but unfortunately only for MODULE search mode and not CONFIG search mode. It provides a file located in "${CONDA_PREFIX}/lib/cmake" called Snappy/SnappyConfig.cmake. The case-sensitive nature of the search overlooks that file.

If we cared enough, we could patch our meson configuration to try something like:

snappy_dep = dependency('Snappy', required: false)
if not snappy_dep.found()
    snappy_dep = dependency('snappy')
endif

Although that does special-case snappy. We could alternatively just see if they can distribute pkg-config information, or a CMake file using the MODULE search mode

Regarding benchmark-main, it appears that there is an inconsistency in the wrapdb. Google benchmark distributes a file by the name of benchmark_main (underscore not hyphen). We can therefore change our code to:

benchmark_main_dep = dependency(
    'benchmark_main',
    fallback: ['benchmark', 'google_benchmark_main_dep'],
)

I've opened an upstream issue about that in mesonbuild/wrapdb#2388, and an issue for a workaround here in #47537

[kou]

bzip2 does not provide any information about its dependency (neither pkg-config nor CMake files), so I think that would need an upstream patch

I agree with it.

Can we use find_library() as a workaround like the following?

diff --git a/cpp/src/arrow/meson.build b/cpp/src/arrow/meson.build
index fbd9cc8551..fd80539c80 100644
--- a/cpp/src/arrow/meson.build
+++ b/cpp/src/arrow/meson.build
@@ -223,7 +223,14 @@ endif
 
 if needs_bz2
     arrow_util_srcs += ['util/compression_bz2.cc']
-    arrow_util_deps += dependency('bzip2')
+    bzip2 = cpp_compiler.find_library('bz2',
+                                      has_headers: ['bzlib.h'],
+                                      required: false)
+    if bzip2.found()
+        arrow_util_deps += bzip2
+    else
+        arrow_util_deps += dependency('bzip2')
+    endif
 endif
 
 if needs_lz4

Can we use dependency('snappy', 'Snappy') for Snappy?

[WillAyd]

Can we use dependency('snappy', 'Snappy') for Snappy?

Ah cool! I wasn't aware of this syntax but it seems to work locally. Let's try it

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 2b203ff.

There weren't enough matching historic benchmark results to make a call on whether there were regressions.

The full Conbench report has more details.