_cext.pyd depends on MSVCP140.dll, which is not distributed

[LordAro]

Version: 1.4.7

_cext.pyd still depends MSVCP140.dll, even though this was supposed to have been fixed by #179 (?)

$ python3.12 -m pip download kiwisolver
Collecting kiwisolver
  Using cached kiwisolver-1.4.7-cp312-cp312-win_amd64.whl.metadata (6.4 kB)
Using cached kiwisolver-1.4.7-cp312-cp312-win_amd64.whl (55 kB)
Saved c:\foo\kiwisolver-1.4.7-cp312-cp312-win_amd64.whl
Successfully downloaded kiwisolver

$ file kiwisolver-1.4.7-cp312-cp312-win_amd64.whl
kiwisolver-1.4.7-cp312-cp312-win_amd64.whl: Zip archive data, at least v2.0 to extract, compression method=deflate

$ unzip kiwisolver-1.4.7-cp312-cp312-win_amd64.whl
Archive:  kiwisolver-1.4.7-cp312-cp312-win_amd64.whl
  inflating: kiwisolver/__init__.py
  inflating: kiwisolver/_cext.cp312-win_amd64.pyd
  inflating: kiwisolver/_cext.pyi
  inflating: kiwisolver/exceptions.py
  inflating: kiwisolver/py.typed
  inflating: kiwisolver-1.4.7.dist-info/LICENSE
  inflating: kiwisolver-1.4.7.dist-info/METADATA
  inflating: kiwisolver-1.4.7.dist-info/WHEEL
  inflating: kiwisolver-1.4.7.dist-info/top_level.txt
  inflating: kiwisolver-1.4.7.dist-info/RECORD

$ objdump -p kiwisolver/_cext.cp312-win_amd64.pyd | grep DLL
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: MSVCP140.dll
        DLL Name: python312.dll
        DLL Name: KERNEL32.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll

Found by installing Python in Windows Sandbox and trying to import matplotlib. Matplotlib (3.9.2) does not depend on these external dependencies

Matplotlib deps

$ find matplotlib -name '*.pyd' -print -exec sh -c 'objdump -p {} | grep DLL' \;
matplotlib/backends/_backend_agg.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: python312.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: KERNEL32.dll
matplotlib/backends/_tkagg.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: COMCTL32.dll
        DLL Name: PSAPI.DLL
        DLL Name: python312.dll
        DLL Name: KERNEL32.dll
        DLL Name: VCRUNTIME140_1.dll
matplotlib/ft2font.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-locale-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: python312.dll
        DLL Name: api-ms-win-crt-utility-l1-1-0.dll
        DLL Name: api-ms-win-crt-convert-l1-1-0.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: KERNEL32.dll
matplotlib/_c_internal_utils.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: ole32.dll
        DLL Name: SHELL32.dll
        DLL Name: USER32.dll
        DLL Name: python312.dll
        DLL Name: KERNEL32.dll
        DLL Name: VCRUNTIME140_1.dll
matplotlib/_image.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: python312.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: KERNEL32.dll
matplotlib/_path.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: python312.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: KERNEL32.dll
matplotlib/_qhull.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: python312.dll
        DLL Name: api-ms-win-crt-utility-l1-1-0.dll
        DLL Name: api-ms-win-crt-time-l1-1-0.dll
        DLL Name: api-ms-win-crt-convert-l1-1-0.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: KERNEL32.dll
matplotlib/_tri.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: api-ms-win-crt-locale-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: python312.dll
        DLL Name: api-ms-win-crt-filesystem-l1-1-0.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: KERNEL32.dll
matplotlib/_ttconv.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: python312.dll
        DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: KERNEL32.dll

Apologies if this counts as a duplicate of #168 or #181 (or the others), but the messaging in those issues seemed... muddy.

[MatthieuDartiailh]

Thanks for the separate ticket which makes things clearer on my end. I will do my best to investigate ASAP.

[Qix-]

Another alternative is statically linking the runtimes. Then none of the redist DLLs would be necessary to begin with, instead of needing to package them alongside the plugin.

[MatthieuDartiailh]

Python ships VCRUNTIME so there is no problem there but MSVCP is problematic.

@QuLogic I would welcome your insight since I mostly followed your suggestions here. One difference I noticed between kiwi setup and Matplotlib is that in kiwi I re-enabled FH4, do you think it may be related ?

[QuLogic]

I see you copied the setup from Matplotlib here:

kiwi/.github/workflows/release.yml

Lines 106 to 114 in 16735f9

	# On Windows, we explicitly request MSVC compilers (as GitHub Action runners have
	# MinGW on PATH that would be picked otherwise), switch to a static build for
	# runtimes, but use dynamic linking for `VCRUNTIME140.dll`, `VCRUNTIME140_1.dll`,
	# and the UCRT. This avoids requiring specific versions of `MSVCP140.dll`, while
	# keeping shared state with the rest of the Python process/extensions.
	CIBW_CONFIG_SETTINGS_WINDOWS: >-
	setup-args="--vsenv"
	setup-args="-Db_vscrt=mt"
	setup-args="-Dcpp_link_args=['ucrt.lib','vcruntime.lib','/nodefaultlib:libucrt.lib','/nodefaultlib:libvcruntime.lib']"

These are settings for Meson(-python) but I just noticed you are using setuptools to build. Those flags should make MSVCP statically linked with Meson, but they actually do nothing with setuptools (or perhaps something else entirely, as it didn't error?) Maybe there was a warning that could be seen if the build was in verbose mode?

From setuptools docs, it looks like you could set some environment variables to override build flags. The equivalent should be something like:

CFLAGS="/MT"
CPPFLAGS="/MT"
LDFLAGS="ucrt.lib vcruntime.lib /nodefaultlib:libucrt.lib /nodefaultlib:libvcruntime.lib"

[MatthieuDartiailh]

Thanks @QuLogic I missed this point. I considered switching to meson but never made the jump.

[MatthieuDartiailh]

I will have to try harder since my first attempt did not produce the expected result (https://github.com/nucleic/kiwi/actions/runs/11949548832).

[QuLogic]

Maybe try CIBW_ENVIRONMENT_WINDOWS

[MatthieuDartiailh]

Thanks for the suggestion but it does not seem to do the trick... (https://github.com/nucleic/kiwi/actions/runs/11999734728/job/33447988060)
The logs looks like the env vars are set but still something is not behaving properly. I will investigate directly patching setuptools in cppy. I will have the added benefit of applying to all the projects that uses it.

[MatthieuDartiailh]

Building locally after patching cppy, I obtain the following dependencies, are those fine;

        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
        DLL Name: api-ms-win-crt-locale-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: python313.dll
        DLL Name: KERNEL32.dll

[MatthieuDartiailh]

Based on the first post of this issue I think so but I would be happy if somebody can confirm.

[LordAro]

Yup, vcruntime*.dll is distributed with cpython, kernel32.dll & api-*.dll are internal windows libraries, and python313 is... python :)

[QuLogic]

This is consistent with what we get: matplotlib/matplotlib#28687 (comment)

[MatthieuDartiailh]

Thanks

[MatthieuDartiailh]

Can somebody more knowledgeable than myself in cibuildwheel config help me figure out the reason this pipeline fails ?

https://github.com/nucleic/kiwi/actions/runs/12036490541/job/33557857394

The override of MD with MT while not ideal does not cause issues locally nor in the CI test build.

[QuLogic]

Ah, that's a 32-bit build; we don't have those any more in Matplotlib. Perhaps slightly different library names are needed there and @zooba might know.