Repository navigation
[C++] 32 bit MSVC doesn't build #32111
Description
When specifying Win32 as a platform, and building with MSVC, the build fails with the following compile errors :
C:\Users\avertleyb\git\arrow\cpp\src\arrow/util/bit_util.h(70,59): error C3861: '__popcnt64': identifier not found [C:\Users\avertleyb\git\arrow\cpp\build32\src\arrow\arrow_shared.vcxproj]
C:\Users\avertleyb\git\arrow\cpp\src\arrow/util/bit_util.h(204,7): error C3861: '_BitScanReverse64': identifier not found [C:\Users\avertleyb\git\arrow\cpp\build32\src\arrow\arrow_shared.vcxproj]
C:\Users\avertleyb\git\arrow\cpp\src\arrow/util/bit_util.h(250,7): error C3861: '_BitScanForward64': identifier not found [C:\Users\avertleyb\git\arrow\cpp\build32\src\arrow\arrow_shared.vcxproj]
Environment: Win32, MSVC
Reporter: Arkadiy Vertleyb
PRs and other links:
Note: This issue was originally created as ARROW-16778. Please see the migration documentation for further details.
Activity
asfimport commentedon Jun 9, 2022
Antoine Pitrou / @pitrou:
[~avertleyb]Would you like to submit a Github PR to fix those issues?asfimport commentedon Jun 9, 2022
Arkadiy Vertleyb:
I was going to, but after I got the build fixed the arrow-utility-test hangs. Not sure if it is related to my fix, haven't figured out yet how debug it properly.
This was my patch:
index 8583e10b2..5122cb3d5 100644
— a/cpp/src/arrow/util/bit_util.h
+++ b/cpp/src/arrow/util/bit_util.h
@@ -67,7 +67,14 @@ static constexpr uint8_t kBytePopcount[] = {
5, 4, 5, 5, 6, 3, 4, 4, 5, 4, 5, 5, 6, 4, 5, 5, 6, 5, 6, 6, 7, 3, 4, 4, 5, 4, 5, 5, 6,
4, 5, 5, 6, 5, 6, 6, 7, 4, 5, 5, 6, 5, 6, 6, 7, 5, 6, 6, 7, 6, 7, 7, 8};
-static inline uint64_t PopCount(uint64_t bitmap) { return ARROW_POPCOUNT64(bitmap); }
+static inline uint64_t PopCount(uint64_t bitmap) {
+#if defined(_MSC_VER) &&  && )defined(_M_X64)
+#else
+#endif
+}
static inline uint32_t PopCount(uint32_t bitmap) { return ARROW_POPCOUNT32(bitmap); }
//
@@ -199,7 +206,7 @@ static inline int CountLeadingZeros(uint64_t value) {
#if defined(clang) || defined(GNUC)
if (value == 0) return 64;
return static_cast(__builtin_clzll(value));
-#elif defined(_MSC_VER)
+#elif defined(_MSC_VER) && (defined(_M_AMD64) || defined(_M_X64))
unsigned long index; // NOLINT
if (_BitScanReverse64(&index, value)) { // NOLINT
return 63 - static_cast(index);
@@ -245,7 +252,7 @@ static inline int CountTrailingZeros(uint64_t value) {
#if defined(clang) || defined(GNUC)
if (value == 0) return 64;
return static_cast(__builtin_ctzll(value));
-#elif defined(_MSC_VER)
+#elif defined(_MSC_VER) && (defined(_M_AMD64) || defined(_M_X64))
unsigned long index; // NOLINT
if (_BitScanForward64(&index, value)) {
return static_cast(index);
asfimport commentedon Jun 9, 2022
Antoine Pitrou / @pitrou:
Under Linux you would debug using
gdb <binary-dir>/arrow-utility-test. Under Windows I have no idea.asfimport commentedon Jun 9, 2022
Arkadiy Vertleyb:
Can't build Debug version. Is:
cmake --build . --config Debug
The right way to do it?
Getting the following:
Performing copy step for 'googletest_ep'
Error copying file "C:/.../arrow/cpp/build32_debug/googletest_ep-prefix/bin/gtest.dll" to "C:/.../arrow/cpp/build32_debug/release//RELEASE".
C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(241,5): error MSB8066: Custom build for 'C:...\arrow\cpp\build32_debug\CMakeFiles\d9834a26c074484d22
c6b56466b8c71e\googletest_ep-copy.rule;C:...\arrow\cpp\build32_debug\CMakeFiles\0e7b907092777ef17a2918749093be30\googletest_ep-complete.rule;C:...\arrow\cpp\build32_debug\CMakeFiles\4d50031ad3e0595d6aa1
aebb9c9d11b8\googletest_ep.rule' exited with code 1. [C:...\arrow\cpp\build32_debug\googletest_ep.vcxproj]
asfimport commentedon Jun 9, 2022
Antoine Pitrou / @pitrou:
You must first configure the CMake build type.
For this you can pass the
-DCMAKE_BUILD_TYPE=Debugflag in the CMake configuration command.asfimport commentedon Jun 9, 2022
Will Jones / @wjones127:
Also there are some other useful debug settings. See the
base-debugpreset in CMakePresets.json.asfimport commentedon Jun 9, 2022
Will Jones / @wjones127:
FYI I'm working on creating a win32 build right now to see if I can reproduce what you are seeing.
asfimport commentedon Jun 9, 2022
Arkadiy Vertleyb:
Thanks Will.
Saw you latest comment at Bloomberg.
Please also keep in mind that using -DCMAKE_BUILD_TYPE=Debug somehow ends up with treating warnings as errors, which doesn't work well in a 32 bit system because of conflicts between int64_t and size_t. As I understand this comes from Google guidelines conflicting with C++ standard. Not sure how to avoid this.
As you probably guessed by now, I am no cmake expert :-(
asfimport commentedon Jun 9, 2022
Will Jones / @wjones127:
Yes I got that error as warning too. For now I was able to bypass with
-DARROW_CXXFLAGS='/wd4244 /wd4554'.asfimport commentedon Jun 9, 2022
Will Jones / @wjones127:
Also note, it seems that our bundled version of Boost Build doesn't compile on VS2022. We may wish to upgrade that.
I'll go forward in VS2019 for now.
asfimport commentedon Jun 9, 2022
Arkadiy Vertleyb:
2019 is good for now.
asfimport commentedon Jun 10, 2022
Will Jones / @wjones127:
[~avertleyb]I've been able to get both of the following configurations to compile. Do you want to try them out and lmk if they work for you?asfimport commentedon Jun 10, 2022
Antoine Pitrou / @pitrou:
Instead of disabling warnings one-by-one, you can change the
BUILD_WARNING_LEVELCMake variable. I don't know why we are not documenting it ( @kou do you know?), but it is described thusly incmake_modules/SetupCxxFlags.cmake:asfimport commentedon Jun 10, 2022
Will Jones / @wjones127:
Wow that will save me a lot of time! :)
asfimport commentedon Jun 10, 2022
Arkadiy Vertleyb:
@wjones127 This builds with three path parameters removed to use defaults and with my patch applied.
But then ctest crashes running arrow-io-file-test.
11 remaining items
asfimport commentedon Jun 13, 2022
Kouhei Sutou / @kou:
Assigned.
asfimport commentedon Jun 30, 2022
James Duong / @jduo:
[~avertleyb], I saw that your PR is closed and that you were going to resubmit it. Do you need help finishing this off? I'm highly interested in this patch.asfimport commentedon Jul 7, 2022
Arkadiy Vertleyb:
@jduo sorry, I am still waiting on my employer to approve my open source contribution. If you can still see my patch you can apply it manually - it's a few lines of code here and there. Fixed compile and unit tests, but not warnings.
asfimport commentedon Nov 6, 2022
Apache Arrow JIRA Bot:
This issue was last updated over 90 days ago, which may be an indication it is no longer being actively worked. To better reflect the current state, the issue is being unassigned per project policy. Please feel free to re-take assignment of the issue if it is being actively worked, or if you plan to start that work soon.
jmckenna commentedon Oct 29, 2023
Still unable to build with today's
mainon MSVC, exact same errors as the initial report here.jmckenna commentedon Jun 4, 2025
@WillAyd does the Meson port allow the Meson
PLATFORM=x86setting at the project level? (assuming your mention of Meson, means that all settings are supported). (as this is important for MSVC, as noted here in this ticket - trying to avoid hijacking your email thread)WillAyd commentedon Jun 4, 2025
I doubt in its current state that the Meson configuration supports Windows - we don't test for it in CI and I have not done anything to appease the Windows requirements for symbol visibility
Assuming the symbol visibility gets implemented though, if you wanted to set an argument at the project level you could do something like:
And that will be added to compilation command for every library built within the project's config.
Hope that answers your question but let me know if I misunderstood. I have very limited Windows knowledge, but can point you to better resources as needed
jmckenna commentedon Jun 4, 2025
@WillAyd Sorry to not be specific: I do know how to configure Meson for Windows builds at the project level (using the setting I mentioned), however my question is what you hinted at in your first sentence: Would the Arrow library compile with these Meson settings, for MSVC builds (ie. the subject of this ticket). This is critical, for whatever build system that you choose (CMake, Meson, etc), really, the build system you choose is irrelevant, what is important is that (the elephant in the room) MSVC builds are supported.
Hope this time I am more clear.
WillAyd commentedon Jun 4, 2025
Meson as a tool definitely supports MSVC. Our current configuration is likely missing things to make that work, so I think patches are welcome to fix any issues you encounter there.
jmckenna commentedon Jun 5, 2025
Sorry to not be specific: I should have written "I do know how to configure Meson for MSVC Windows builds at the project level".
It is great to read here that the Arrow project is interested in patches, to get the underlying Arrow code to work for MSVC. (as reported here in this issue, it currently fails with CMake, which I think I commented above in 2023).
kou commentedon Jun 5, 2025
If we support 32bit Windows, I think that we need at least one active maintainer for 32bit Windows.
kou commentedon Jun 5, 2025
(Unfortunately, I don't have enough bandwidth for it.)
amoeba commentedon Jun 5, 2025
I think once we tackle #46002, whether we support 32-bit MSVC will be much more clear and so will how to bring it into support. It looks like getting a build working may not be far off but we'd still want to have an active maintainer as @kou points out.
WillAyd commentedon Aug 7, 2025
@jmckenna I have been working recently on getting MSVC supported via Meson. You may want to keep tabs on #47282 and perhaps even try it out for 32 bit builds locally
jmckenna commentedon Aug 7, 2025
@WillAyd wow great, thanks very much, I'll follow along and provide testing feedback there.