CMake: Replace the Visual Studio project#14475
Conversation
JoshuaVandaele
force-pushed
the
cmake-replaces-vs
branch
3 times, most recently
from
1692381 to
8744a79
Compare
Can't you just do the fallback if the symlink fails? Or test if the setting is on/symlinks are ok to make this more pleasant to use? It might just be me though, if everyone else is ok with that, feel free to ignore me. |
JoshuaVandaele
force-pushed
the
cmake-replaces-vs
branch
2 times, most recently
from
1692381 to
22d7c01
Compare
There is now a copy fallback if the symlink cannot be created, and a warning will appear asking the user to enable Developer Mode alongside a link to official Microsoft docs on how to do so on different versions of Windows |
JoshuaVandaele
force-pushed
the
cmake-replaces-vs
branch
from
22d7c01 to
2e3941f
Compare
iwubcode
left a comment
iwubcode
left a comment
There was a problem hiding this comment.
Wonderful work! I will approve after I test.
JoshuaVandaele
force-pushed
the
cmake-replaces-vs
branch
from
2e3941f to
c7b0f2a
Compare
There was a problem hiding this comment.
I tested this with Visual Studio 2022.
CMake experience
To test, I used Visual Studio's "Open Folder" for CMake. CMake worked fine (I changed the preset). I did see a lot of warnings like:
1> [CMake] -- Could NOT find MBEDTLS (missing: MBEDTLS_INCLUDE_DIR MBEDTLS_LIBRARY MBEDX509_LIBRARY MBEDCRYPTO_LIBRARY) (Required is at least version "2.28")
1> [CMake] -- No system MBEDTLS was found. Using static MBEDTLS from Externals.
Which is harmless but on Windows, maybe was doing extra processing? We will never have 'system' packages there.
Build experience
I tried the "Visual Studio" flavor build. The build was quite a bit slower than I remember the Visual Studio default builds being. I did not do an actual time comparison, so I might be remembering wrong (I rarely do rebuilds). FWIW, I do believe the "Ninja" builds are expected to be faster. I will give those a try too!
Once it built, it took a moment to find the dolphin executable. It might be worthwhile to set the startup project but it wasn't a challenge to manually define that. (looks like that option is only for generating the VS sln files, which I didn't try)
I didn't do anything too exotic but I tried to place a breakpoint, check the debugger, and launched a title. Everything worked, exactly as expected!
Conclusion
There will be some growing pains here (maybe just for me, am I the only Windows dev? 😆 ) but not having to manage two build platforms is what we've needed for years. Thank you @JoshuaVandaele for putting in the effort.
This gets a 👍 from me!
|
Thanks a lot for your testing! A few comments I'd like to address:
In my testing they're about the same, but that was entirely done in virtual machines so you may be correct.
They are a lot faster actually! I'd recommend to use these unless you have cross-compiling needs
It is possible to install some libraries in such a way on Windows by adding their DLLs to the path, and although it is currently unlikely someone is to manually install libraries like this, it will be needed for the mingw support @cscd98 is currently working on, or if we more generally want gcc/clang support in the future on Windows.
I'm not sure about this since it doesn't match the current behavior on other Desktop platforms, my understandement is that you need to select your "startup project" in the dropdown at the top of the Visual Studio window, lest it uses the default of building all Dolphin.exe, Updater.exe, and DolphinTool.exe? I'd like more feedback on this.
|
Dentomologist
left a comment
Dentomologist
left a comment
There was a problem hiding this comment.
I successfully built this with various combinations of VS 2022 and 2026, debug and release, ninja and Visual Studio configurations.
The Install targets fail with various warnings, and I'm not sure they make sense to use from VS. If not, can they be either removed or hidden-by-default in VS?
I'm not sure how many people this would actually affect besides me, but anyone using a portable build in the old output directory on Windows will have their config effectively reset since the user folder will now be in %AppData%.
How are the symlink changes going to interact with the buildbot? Could they result in dev versions where symlinks are copied shallowly and then don't refer to anything on the user's computer, or are such copies always deep?
| * [Windows](https://github.com/dolphin-emu/dolphin/wiki/Building-for-Windows) | ||
| * [Linux](https://github.com/dolphin-emu/dolphin/wiki/Building-for-Linux) | ||
| * [macOS](https://github.com/dolphin-emu/dolphin/wiki/Building-for-macOS) |
There was a problem hiding this comment.
[Reminder for later] We'll have to make sure to update these once this is merged.
There was a problem hiding this comment.
Updated versions exist at https://github.com/JoshuaVandaele/dolphin/tree/wiki-cmake
In my testing, any compression tool (such as 7z which is what we use on the buildbot for Windows) copies the files over instead of the symlinks |
JoshuaVandaele
force-pushed
the
cmake-replaces-vs
branch
from
c7b0f2a to
b97856b
Compare
Oh, this would impact me too, I just tested with a fresh build. This is good to know, I'll have to remember to update my output path layout once this is merged so I don't hit this issue... |
|
This PR fails to build on my end with both Visual Studio and CLion: I suspect that's because I'm still on Visual Studio 2022. Downgrading the requirements produces a pch.h file not found error on CLion. FWIW, if I checkout the commit right before this PR, Visual Studio's CMake is working properly. |
The If it's desirable to support older versions I can downgrade that requirement, but it might be a bit of a pain since the Visual Studio Installer doesn't let me install VS2022 by itself alone since VS2026 released. |
Generic builds have never been supported on Windows.
JoshuaVandaele
force-pushed
the
cmake-replaces-vs
branch
from
b97856b to
b791058
Compare
|
I managed to get it to work on both CLion and VS2026 but it's painfully slow to build. Here are the pain points I noticed. DefaultsOn both CLion or VS, it doesn't work by default. On both of them, I need to enable/select the preset (release, debug, etc.) AND select the project (the default one doesn't work). Visual Studio 2026On VS it seems to default to the first detected project (in alphabetical order, i.e. CLionOn CLion it targets "dolphin-emu" as follows but fails with a PCH error (file not found, related to WinUpdater?): Selecting the dolphin-emu app configuration fixes the issue: However, we can't build both the release and debug builds on CLion as they are sharing the same Build timeI can't provide accurate measurements but it seams 5x slower with my current configuration (intel core ultra 7, 16G RAM). I'm not sure if it's something I need to tweak on VS (I tried both 2022 and 2026) or CLion but both are slow when using CMake:
I can see multiple CL processes in the task manager but I suspect there might be less processes being spawned when using CMake. VS2026 UI glitchesI had several UI glitches when selecting projects/presets, e.g. sometimes the dropdown menu is quickly appearing and disappearing. I had to hold the mouse left click to select a preset/project which isn't ideal. DolphinThe project is properly built and seems to work as expected. I haven't compared the previous binary size/speed, though. |
|
Build times in seconds on my computer after running
The PR debug Visual Studio build had surprisingly variable build times over a number of builds (ranging from 101-165), but the others were consistent across clean builds within a second or two. As you can see the Visual Studio configuration builds significantly slower than master, though more like 2x than the 5x sepalani saw. The release Ninja config builds as fast as release master, and the debug Ninja config builds modestly slower than debug master. That's unfortunate for me since I run in debug by default, but it's not a huge difference.
These happened for me too. I think the menu thing happens particularly when the list is tall enough to fill the screen. (From my first comment):
I've hidden these, but it'd still be good to do that by default. |
No difference that I've noticed. |
JoshuaVandaele
force-pushed
the
cmake-replaces-vs
branch
from
b791058 to
df96c56
Compare
There shouldn't be any noticeable difference, the Visual Studio project, The Visual Studio Generator, and the Ninja generator all use the MSVC toolchain to build Dolphin on Windows.
This should now be a lot better: the install presets are now hidden on Windows, and by default the Visual Studio Generator will now only show 5 targets.
I have decided to separate the generation directories for release and debug builds. I had initially decided against it to keep the workflow as similar as the current one as possible and to simplify launch scripts, however it comes with its own advantages of faster iteration when switching between different configurations, and KitWare recommends against reusing the same directory for release and debug builds. |
This introduces a CMakePresets file for both Unix-likes and Windows that replace the old CMakeSettings. It adds presets for **Debug** and **Release** profiles for both **x64** and **ARM64** architectures, as well as **Generic builds**. Presets can be used using[ Visual Studio's built-in CMakePresets support](https://learn.microsoft.com/en-us/cpp/build/cmake-presets-vs?view=msvc-170), or [Visual Studio Code's CMake Tools](https://marketplace.visualstudio.com/items?itemName=ms-vscode.cmake-tools) extension. They can also be used from the command line, like so: - x64/Unix-like/Ninja: - Configure: `cmake --preset ninja-release-x64` - Build: `cmake --build --preset ninja-build-release-x64` - Configure + Build: `cmake --workflow --preset ninja-release-x64` - ARM64/Windows/Visual Studio: - Configure: `cmake --preset visualstudio-release-arm64` - Build: `cmake --build --preset visualstudio-build-release-arm64` - Configure + Build: `cmake --workflow --preset visualstudio-release-arm64` The Ninja generator is available to both Windows and Unix-likes, while the Visual Studio Generator is only available on Windows. **Cross-compiling** On **Windows**, the Visual Studio generator automatically takes care of everything, you just need to select an ARM64 preset. On **Unix-likes**, to cross-compile you need to install a cross-compiler and (optionally) a sysroot of the target system. Here is an example to compile from x64 to ARM64 with a sysroot: - `cmake --preset ninja-release-arm64 -DCMAKE_C_COMPILER=aarch64-linux-gnu-gcc -DCMAKE_CXX_COMPILER=aarch64-linux-gnu-g++ -DCMAKE_SYSROOT=/opt/sysroots/aarch64-linux-gnu` - `cmake --build --preset ninja-build-release-arm64` You will need a sysroot to link against Qt, since we do not vendor it in on platforms other than Windows. **User presets** A `CMakeUserPresets.json` file may be created locally at the root of the project to further customize your presets. For example, here are the user presets I used to test this PR on Arch Linux with a generic Arch Linux ARM sysroot: ```json { "version": 10, "configurePresets": [ { "name": "gcc-debug-arm64", "inherits": "ninja-debug-arm64", "cacheVariables": { "CMAKE_C_COMPILER": "aarch64-linux-gnu-gcc", "CMAKE_CXX_COMPILER": "aarch64-linux-gnu-g++", "CMAKE_EXE_LINKER_FLAGS": "-L/opt/sysroots/ArchLinuxARM/lib", "CMAKE_SYSROOT": "/opt/sysroots/ArchLinuxARM" } }, { "name": "clang-debug-arm64", "inherits": "ninja-debug-arm64", "cacheVariables": { "CMAKE_C_COMPILER": "clang", "CMAKE_CXX_COMPILER": "clang++", "CMAKE_C_FLAGS": "-target aarch64-linux-gnu", "CMAKE_CXX_FLAGS": "-target aarch64-linux-gnu", "CMAKE_SYSROOT": "/opt/sysroots/ArchLinuxARM" } }, { "name": "clang-debug-x64", "inherits": "ninja-debug-x64", "cacheVariables": { "CMAKE_C_COMPILER": "clang", "CMAKE_CXX_COMPILER": "clang++" } } ], "buildPresets": [ { "name": "gcc-build-debug-arm64", "configurePreset": "gcc-debug-arm64" }, { "name": "clang-build-debug-arm64", "configurePreset": "clang-debug-arm64" }, { "name": "clang-build-debug-x64", "configurePreset": "clang-debug-x64" } ], "workflowPresets": [ { "name": "gcc-debug-arm64", "steps": [ { "type": "configure", "name": "gcc-debug-arm64" }, { "type": "build", "name": "gcc-build-debug-arm64" } ] }, { "name": "clang-debug-arm64", "steps": [ { "type": "configure", "name": "clang-debug-arm64" }, { "type": "build", "name": "clang-build-debug-arm64" } ] }, { "name": "clang-debug-x64", "steps": [ { "type": "configure", "name": "clang-debug-x64" }, { "type": "build", "name": "clang-build-debug-x64" } ] } ] } ``` They are then used like so: Configure + Build with GCC: `cmake --workflow --preset gcc-debug-arm64` Configure + Build with Clang: `cmake --workflow --preset clang-debug-arm64` Configure + Build with Clang (x64): `cmake --workflow --preset clang-debug-x64` *Addendum: It should also now be possible to cross-compile from Windows to Unix-likes, and Unix-like to other Unix-like (e.g. Linux -> FreeBSD), however this is untested.*
JoshuaVandaele
force-pushed
the
cmake-replaces-vs
branch
from
df96c56 to
c1ded34
Compare
|
I gave my own go at benchmarking the build times of master against the different presets.
By default the presets build multiple targets, so I also timed the build times of just the
I would recommend using the |
7 participants
The Visual Studio project is currently used to compile Dolphin on Windows, while CMake is used for all other platforms.
This difference has become a maintenance burden when introducing new dependencies, as it effectively doubles the amount of work and requires the developer to be familiar with both CMake and Visual Studio projects. Additionally, most dependencies do not provide Visual Studio projects, which means they need to be created and maintained by us, something that is very time-consuming and error-prone.
The situation has become cumbersome enough that the Visual Studio project now invokes CMake to configure and build larger dependencies, such as glslang.
Changes
Unification of the Output, Sys, and Translation Directories
Previously, on Windows the output directory was located in the
/Binary/<architecture>/directory, whereas the output directory is located underbuild/Binariesfor other platforms. Both platforms now output to/build/<release|debug>/<architecture>/Binaries.On Non-Windows and non-Apple platforms, the Sys directory had a set location unless the
LINUX_LOCAL_DEVflag was set, in which case Dolphin used a location relative to the executable. This is no longer the case, Dolphin will now first check for the existence of a Sys folder next to the Dolphin executable, before falling back to the defined SYSDATA directory. During build time, the Sys directory is now unconditionally Symlinked next to the executable, similar to the behavior on Windows, which copied the required files.With this change, the
LINUX_LOCAL_DEVflag is no longer required, and therefore has been removed.Warning
Because creating symbolic links on Windows normally require a special permission, we fall back to copying with a warning unless Developer Mode is enabled.
The translation directories have also been merged on both platforms, in a similar fashion to the Sys directory.
Introducing CMakePresets, Replacing the Old CMakeSettings
It adds presets for Debug and Release profiles for both x64 and ARM64 architectures, as well as Generic builds.
Presets can be used using Visual Studio's built-in CMakePresets support, or Visual Studio Code's CMake Tools extension.
They can also be used from the command line, like so:
cmake --preset ninja-release-x64cmake --build --preset ninja-build-release-x64cmake --workflow --preset ninja-release-x64cmake --preset visualstudio-release-arm64cmake --build --preset visualstudio-build-release-arm64cmake --workflow --preset visualstudio-release-arm64The Ninja generator is available to both Windows and Unix-likes, while the Visual Studio Generator is only available on Windows.
Cross-Compiling
On Windows, the Visual Studio generator automatically takes care of everything, you just need to select an ARM64 preset.
On Unix-likes, to cross-compile you need to install a cross-compiler and (optionally) a sysroot of the target system.
Here is an example to compile from x64 to ARM64 with a sysroot:
cmake --preset ninja-release-arm64 -DCMAKE_C_COMPILER=aarch64-linux-gnu-gcc -DCMAKE_CXX_COMPILER=aarch64-linux-gnu-g++ -DCMAKE_SYSROOT=/opt/sysroots/aarch64-linux-gnucmake --build --preset ninja-build-release-arm64You will need a sysroot to link against Qt, since we do not vendor it in on platforms other than Windows.
User Presets
A
CMakeUserPresets.jsonfile may be created locally at the root of the project to further customize your presets.Example: User Presets used to test this PR on Arch Linux with a generic Arch Linux ARM sysroot
{ "version": 10, "configurePresets": [ { "name": "gcc-debug-arm64", "inherits": "ninja-debug-arm64", "cacheVariables": { "CMAKE_C_COMPILER": "aarch64-linux-gnu-gcc", "CMAKE_CXX_COMPILER": "aarch64-linux-gnu-g++", "CMAKE_EXE_LINKER_FLAGS": "-L/opt/sysroots/ArchLinuxARM/lib", "CMAKE_SYSROOT": "/opt/sysroots/ArchLinuxARM" } }, { "name": "clang-debug-arm64", "inherits": "ninja-debug-arm64", "cacheVariables": { "CMAKE_C_COMPILER": "clang", "CMAKE_CXX_COMPILER": "clang++", "CMAKE_C_FLAGS": "-target aarch64-linux-gnu", "CMAKE_CXX_FLAGS": "-target aarch64-linux-gnu", "CMAKE_SYSROOT": "/opt/sysroots/ArchLinuxARM" } }, { "name": "clang-debug-x64", "inherits": "ninja-debug-x64", "cacheVariables": { "CMAKE_C_COMPILER": "clang", "CMAKE_CXX_COMPILER": "clang++" } } ], "buildPresets": [ { "name": "gcc-build-debug-arm64", "configurePreset": "gcc-debug-arm64" }, { "name": "clang-build-debug-arm64", "configurePreset": "clang-debug-arm64" }, { "name": "clang-build-debug-x64", "configurePreset": "clang-debug-x64" } ], "workflowPresets": [ { "name": "gcc-debug-arm64", "steps": [ { "type": "configure", "name": "gcc-debug-arm64" }, { "type": "build", "name": "gcc-build-debug-arm64" } ] }, { "name": "clang-debug-arm64", "steps": [ { "type": "configure", "name": "clang-debug-arm64" }, { "type": "build", "name": "clang-build-debug-arm64" } ] }, { "name": "clang-debug-x64", "steps": [ { "type": "configure", "name": "clang-debug-x64" }, { "type": "build", "name": "clang-build-debug-x64" } ] } ] }They are then used like so:
Configure + Build with GCC:
cmake --workflow --preset gcc-debug-arm64Configure + Build with Clang:
cmake --workflow --preset clang-debug-arm64Configure + Build with Clang (x64):
cmake --workflow --preset clang-debug-x64Addendum: It should also now be possible to cross-compile from Windows to Unix-likes, and Unix-like to other Unix-like (e.g. Linux -> FreeBSD), however this is untested.
Pre-Merge Requirements