NGMP implementation

.github/instructions/ngmp.instructions.md

@@ -0,0 +1,49 @@
+---
+applyTo: '**/GeneralsOnline/**,**/NextGenMP/**'
+---
+
+# NGMP Subsystem Implementation Instructions
+
+These instructions govern the Next-Gen Multiplayer (NGMP) client protocol integration into **GeneralsX**. All changes in `GeneralsOnline` and `NextGenMP` components must adhere strictly to these guidelines.
+
+---
+
+## Golden Constraints
+
+1. **Strict Prohibition of `<windows.h>` & Win32 APIs**:
+   - Never include `<windows.h>`, `<winsock2.h>`, `<ws2tcpip.h>`, `<wincred.h>`, or `<shellapi.h>` in NGMP sources.
+   - Use standard C++ (`<chrono>`, `<thread>`, `<mutex>`), POSIX networking (`<sys/socket.h>`, `<netinet/in.h>`, `<arpa/inet.h>`), or SDL3 primitives (`SDL_GetTicks()`, `SDL_Delay()`, `SDL_GetPrefPath()`).
+
+2. **Platform Layer Isolation**:
+   - Low-level network socket and OS calls must reside exclusively in `Core/GameEngineDevice/`.
+   - Higher-level network logic under `GeneralsMD/Code/GameEngine/Source/GameNetwork/GeneralsOnline/` must use pure abstract interfaces and cross-platform abstractions.
+
+3. **Thread Safety & UI Main Thread Separation**:
+   - Network callbacks from HTTP/WebSocket/P2P threads **MUST NOT** directly access or mutate UI controls (`GameWindow`, `WOLLobbyMenu`, etc.).
+   - Network events must be posted to a thread-safe thread/FIFO queue, consumed strictly on the main render thread during `Shell::update()`.
+
+4. **Credential & Token Storage**:
+   - Do NOT use Windows Credential Manager.
+   - Save session JWT tokens in an obfuscated local configuration file under the directory provided by `SDL_GetPrefPath("GeneralsX", "GeneralsOnline")`.
+
+5. **Server Endpoint Environment**:
+   - Development backend server target: `ws://192.168.1.120:9001/ws` (WebSocket) and `http://192.168.1.120:9001/api` (REST).
+   - Server URLs must be configurable via INI/JSON config rather than hardcoded string literals.
+
+6. **Cross-Platform Math & Endianness**:
+   - Ensure network packet data serialization handles network byte order (`htons`/`ntohs`, `htonl`/`ntohl`) explicitly to support cross-play between x86_64 Linux and ARM64 macOS.
+
+7. **Reference repositories**:
+   - There are two reference repositories for NGMP, take a look on these to understand the protocol and implementation details:
+     1. `references/GameClient`: The client-side implementation.
+     2. `references/GameServer`: The server-side implementation.
+
+8. **REST API & Routing Conventions (Lessons Learned)**:
+   - **Environment/Contract Prefix**: The C# Kestrel backend strictly requires environment and contract version routing. Do **NOT** use `GetServerRESTEndpoint()` directly to build URLs. Always use `NGMP::GetAPIEndpoint("EndpointName")` (e.g. `NGMP::GetAPIEndpoint("Lobbies")`), which appends the required `/env/dev/contract/1/` prefix automatically.
+   - **Authorization**: Almost all endpoints require authorization. You must inject the `Authorization: Bearer <token>` HTTP header in all requests (GET, POST, PUT, DELETE). Missing this will result in a `401 Unauthorized` response.
+   - **HTTP Methods & Payloads**: Backend endpoints have strict method bindings (`[HttpGet]`, `[HttpPut]`, `[HttpPost]`). For example, creating a lobby requires `PUT /Lobbies`, while joining a lobby requires `PUT /Lobby/{id}`. The JSON payloads must also contain all expected fields (even if default or empty), or the server will reject the request with `400 Bad Request`.
+
+9. **WebSocket Protocol Conventions (Lessons Learned)**:
+   - **Message ID Routing**: The backend `WebSocketController` strictly uses integer `msg_id` values to route actions, NOT string action types (e.g., `msg_id: 1` for sending chat, `msg_id: 2` for receiving chat, `msg_id: 4` for member list updates). Always include the correct `msg_id` in sent JSON payloads and use it to parse incoming messages.
+   - **Network Rooms (Lobby Registration)**: The backend requires the client to explicitly register into a network room to receive lobby and player updates. When initializing the global lobby chat, you must send `{"msg_id": 3, "room": 0}` (NETWORK_ROOM_CHANGE_ROOM). If you fail to do this, the server considers you in room `-1`, which causes the HTTP `GET /Lobbies` endpoint to return `0 lobbies` (as it filters by your current room) and prevents you from receiving player list updates (`msg_id: 4`).
+   - **Heartbeats**: The WebSocket server automatically drops connections if a PING is not received within a timeout window (often 20s). The client must periodically send `{"msg_id": 8}` (ping) at least every 10 seconds to keep the socket alive.

.github/workflows/build-linux-flatpak.yml

@@ -85,10 +85,15 @@ jobs:
             flatpak-builder-${{ runner.os }}-${{ inputs.preset }}-
 
       - name: Build Flatpak Bundle
+        env:
+          NGMP_DEFAULT_HOST: ${{ secrets.NGMP_DEFAULT_HOST }}
+          NGMP_SERVER_HOST: ${{ secrets.NGMP_DEFAULT_HOST }}
+          NGMP_SERVER_PORT: ${{ secrets.NGMP_SERVER_PORT }}
+          NGMP_USE_SSL: ${{ secrets.NGMP_USE_SSL }}
         run: |
           set -o pipefail
           mkdir -p logs
-          ./scripts/build/linux/build-linux-flatpak.sh "${{ inputs.preset }}" "${{ matrix.game.value }}" 2>&1 | tee "logs/build_flatpak_${{ matrix.game.log_suffix }}.log"
+          bash ./scripts/build/linux/build-linux-flatpak.sh "${{ inputs.preset }}" "${{ matrix.game.value }}" 2>&1 | tee "logs/build_flatpak_${{ matrix.game.log_suffix }}.log"
 
       - name: Locate generated Flatpak bundle
         id: bundle

.github/workflows/build-macos.yml

@@ -340,6 +340,10 @@ jobs:
           CXX: clang++
           CMAKE_C_COMPILER_LAUNCHER: ccache
           CMAKE_CXX_COMPILER_LAUNCHER: ccache
+          NGMP_DEFAULT_HOST: ${{ secrets.NGMP_DEFAULT_HOST }}
+          NGMP_SERVER_HOST: ${{ secrets.NGMP_DEFAULT_HOST }}
+          NGMP_SERVER_PORT: ${{ secrets.NGMP_SERVER_PORT }}
+          NGMP_USE_SSL: ${{ secrets.NGMP_USE_SSL }}
         run: |
           mkdir -p logs
           echo "VULKAN_SDK=$VULKAN_SDK"

.github/workflows/build-windows.yml

@@ -82,9 +82,13 @@ jobs:
           New-Item -ItemType Directory -Force ci-artifacts | Out-Null
           Copy-Item build/win32-vcpkg/${{ matrix.game.folder }}/Release/${{ matrix.game.exe }} artifacts/
           Copy-Item build/win32-vcpkg/${{ matrix.game.folder }}/Release/${{ matrix.game.pdb }} artifacts/
-          Copy-Item build/win32-vcpkg/vcpkg_installed/x86-windows/bin/zlib1.dll artifacts/
           Copy-Item build/win32-vcpkg/${{ matrix.game.folder }}/Release/${{ matrix.game.exe }} ci-artifacts/
-          Copy-Item build/win32-vcpkg/vcpkg_installed/x86-windows/bin/zlib1.dll ci-artifacts/
+
+          Get-ChildItem -Path @("vcpkg_installed/x86-windows/bin/*.dll", "build/win32-vcpkg/vcpkg_installed/x86-windows/bin/*.dll") -ErrorAction SilentlyContinue | ForEach-Object {
+            Write-Host "Collecting vcpkg DLL: $($_.FullName)"
+            Copy-Item $_.FullName artifacts/
+            Copy-Item $_.FullName ci-artifacts/
+          }
 
           foreach ($name in @("binkw32.dll", "mss32.dll")) {
             $stub = Get-ChildItem build/win32-vcpkg -Filter $name -Recurse |

.gitignore

@@ -91,5 +91,6 @@ flatpak/*.flatpak
 #tmp and log files on root folder
 logs_*.zip
 *.log
+/cmake/ngmp_env.cmake
 references/GameClient/
 references/GameServer/

.gitmodules

@@ -2,3 +2,9 @@
 	path = references/fbraz3-dxvk
 	url = https://github.com/fbraz3/dxvk.git
 	branch = generalsx-macos-v2.6
+[submodule "references/GameClient"]
+	path = references/GameClient
+	url = https://github.com/GeneralsOnlineDevelopmentTeam/GameClient.git
+[submodule "references/GameServer"]
+	path = references/GameServer
+	url = https://github.com/GeneralsOnlineDevelopmentTeam/Services.git

AGENTS.md

@@ -289,5 +289,6 @@ The `**` at applyTo means all files, you MUST load it everytime.
 | [.github/instructions/platform-macos.instructions.md](.github/instructions/platform-macos.instructions.md) | `scripts/build/macos/**,references/fbraz3-dxvk/**` | macOS/DXVK build notes |
 | [.github/instructions/docs.instructions.md](.github/instructions/docs.instructions.md) | `**/*.md` | Documentation structure and workflow |
 | [.github/instructions/scripts.instructions.md](.github/instructions/scripts.instructions.md) | `scripts/**` | Script organization and naming |
+| [.github/instructions/ngmp.instructions.md](.github/instructions/ngmp.instructions.md) | `**/GeneralsOnline/**,**/NextGenMP/**` | NGMP cross-platform multiplayer integration guidelines |
 
 Update this table when instruction files are added, removed, or renamed.

CMakeLists.txt

@@ -102,6 +102,9 @@ include(cmake/miniaudio.cmake)
 # curl.cmake is self-guarded with if(SAGE_UPDATE_CHECK), so always safe to include.
 include(cmake/curl.cmake)
 
+# GeneralsX @feature GeneralsOnline NGMP protocol dependency setup
+include(cmake/ngmp.cmake)
+
 # GeneralsX @feature fbraz 03/05/2026 Phase 4: Deterministic math library integration
 # gamemath.cmake integrates fdlibm-based GameMath for cross-platform replay validation.
 # Upstream reference: Okladnoj, PR #2670

Core/GameEngine/Include/Common/GameDefines.h

@@ -17,7 +17,6 @@
 */
 
 #pragma once
-
 #include "WWLib/WWDefines.h"
 
 // Note: Retail compatibility must not be broken before this project officially does.

Core/GameEngine/Include/GameNetwork/GameSpy/LobbyUtils.h

@@ -53,3 +53,15 @@ enum GameSortType CPP_11(: Int)
 
 Bool HandleSortButton( NameKeyType sortButton );
 void PopulateLobbyPlayerListbox();
+
+enum LobbyGameModeFilter CPP_11(: Int)
+{
+    LOBBY_FILTER_ALL = 0,
+    LOBBY_FILTER_1V1,
+    LOBBY_FILTER_TEAM,
+    LOBBY_FILTER_FFA,
+    LOBBY_FILTER_AOD,
+};
+
+extern LobbyGameModeFilter theLobbyFilter;
+LobbyGameModeFilter detectGameMode(const std::string& name);

Core/GameEngine/Include/GameNetwork/GameSpy/PersistentStorageThread.h

@@ -30,6 +30,13 @@
 
 #include "gamespy/gstats/gpersist.h"
 
+#if defined(min)
+#undef min
+#endif
+#if defined(max)
+#undef max
+#endif
+
 #define MAX_BUDDY_CHAT_LEN 128
 
 typedef std::map<Int, UnsignedInt> PerGeneralMap;

Core/GameEngine/Include/GameNetwork/GameSpyOverlay.h

@@ -39,6 +39,7 @@ void ClearGSMessageBoxes();	///< Tear down any GS message boxes (e.g. in case we
 void GSMessageBoxOk(UnicodeString titleString,UnicodeString bodyString, GameWinMsgBoxFunc okFunc = nullptr);	///< Display a Message box with Ok button and track it
 void GSMessageBoxOkCancel(UnicodeString title, UnicodeString message, GameWinMsgBoxFunc okFunc, GameWinMsgBoxFunc cancelFunc);	///< Display a Message box with Ok/Cancel buttons and track it
 void GSMessageBoxYesNo(UnicodeString title, UnicodeString message, GameWinMsgBoxFunc yesFunc, GameWinMsgBoxFunc noFunc);	///< Display a Message box with Yes/No buttons and track it
+void GSMessageBoxCancel(UnicodeString title, UnicodeString message, GameWinMsgBoxFunc cancelFunc = nullptr);	///< Display a Message box with Cancel button and track it
 void RaiseGSMessageBox();		///< Bring GS message box to the foreground (if we transition screens while a message box is up)
 
 enum GSOverlayType CPP_11(: Int)

Core/GameEngine/Source/Common/UserPreferences.cpp

@@ -51,6 +51,13 @@
 #include "GameClient/ChallengeGenerals.h"
 #include "GameNetwork/GameSpy/PeerDefs.h"
 
+#if defined(SAGE_USE_NGMP) && __has_include("GameNetwork/GeneralsOnline/OnlineServices_Manager.h")
+#include "GameNetwork/GeneralsOnline/OnlineServices_Manager.h"
+#include "GameNetwork/GeneralsOnline/OnlineServices_Auth.h"
+#include "GameNetwork/GeneralsOnline/NGMP_interfaces.h"
+#define HAVE_NGMP_PREFS 1
+#endif
+
 
 //-----------------------------------------------------------------------------
 // DEFINES ////////////////////////////////////////////////////////////////////
@@ -242,7 +249,12 @@ void UserPreferences::setAsciiString(AsciiString key, AsciiString val)
 QuickMatchPreferences::QuickMatchPreferences()
 {
 	AsciiString userPrefFilename;
-	Int localProfile = TheGameSpyInfo->getLocalProfileID();
+#if defined(HAVE_NGMP_PREFS)
+	NGMP_OnlineServices_AuthInterface* pAuthInterface = NGMP_OnlineServicesManager::GetInterface<NGMP_OnlineServices_AuthInterface>();
+	Int localProfile = pAuthInterface != nullptr ? (Int)pAuthInterface->GetUserID() : 0;
+#else
+	Int localProfile = TheGameSpyInfo ? TheGameSpyInfo->getLocalProfileID() : 0;
+#endif
 	userPrefFilename.format("GeneralsOnline\\QMPref%d.ini", localProfile);
 	load(userPrefFilename);
 }
@@ -427,7 +439,12 @@ Int QuickMatchPreferences::getSide()
 CustomMatchPreferences::CustomMatchPreferences()
 {
 	AsciiString userPrefFilename;
-	Int localProfile = TheGameSpyInfo->getLocalProfileID();
+#if defined(HAVE_NGMP_PREFS)
+	NGMP_OnlineServices_AuthInterface* pAuthInterface = NGMP_OnlineServicesManager::GetInterface<NGMP_OnlineServices_AuthInterface>();
+	Int localProfile = pAuthInterface != nullptr ? (Int)pAuthInterface->GetUserID() : 0;
+#else
+	Int localProfile = TheGameSpyInfo ? TheGameSpyInfo->getLocalProfileID() : 0;
+#endif
 	userPrefFilename.format("GeneralsOnline\\CustomPref%d.ini", localProfile);
 	load(userPrefFilename);
 }
@@ -776,7 +793,12 @@ void CustomMatchPreferences::setUseStats( Bool useStats )
 GameSpyMiscPreferences::GameSpyMiscPreferences()
 {
 	AsciiString userPrefFilename;
-	Int localProfile = TheGameSpyInfo->getLocalProfileID();
+#if defined(HAVE_NGMP_PREFS)
+	NGMP_OnlineServices_AuthInterface* pAuthInterface = NGMP_OnlineServicesManager::GetInterface<NGMP_OnlineServices_AuthInterface>();
+	Int localProfile = pAuthInterface != nullptr ? (Int)pAuthInterface->GetUserID() : 0;
+#else
+	Int localProfile = TheGameSpyInfo ? TheGameSpyInfo->getLocalProfileID() : 0;
+#endif
 	userPrefFilename.format("GeneralsOnline\\GSMiscPref%d.ini", localProfile);
 	load(userPrefFilename);
 }

Core/GameEngine/Source/GameClient/GUI/GUICallbacks/InGameChat.cpp

@@ -199,7 +199,7 @@ void ToggleInGameChat( Bool immediate )
 	if (TheGameLogic->isInReplayGame())
 		return;
 
-	if (!TheGameInfo->isMultiPlayer() && TheGlobalData->m_netMinPlayers)
+	if (TheGameInfo == nullptr || (!TheGameInfo->isMultiPlayer() && TheGlobalData->m_netMinPlayers))
 		return;
 
 	if (chatWindow)

Core/GameEngine/Source/GameClient/GUI/Gadget/GadgetListBox.cpp

@@ -2478,6 +2478,9 @@ void GadgetListBoxRemoveMultiSelect( GameWindow *listbox )
 //=============================================================================
 void GadgetListBoxSetListLength( GameWindow *listbox, Int newLength )
 {
+	if (!listbox)
+		return;
+
 	ListboxData *listboxData = (ListboxData *)listbox->winGetUserData();
 
 

Core/GameEngine/Source/GameNetwork/GameSpy/LobbyUtils.cpp

@@ -139,6 +139,41 @@ static void showSortIcons()
 		}
 	}
 }
+LobbyGameModeFilter theLobbyFilter = LOBBY_FILTER_ALL;
+
+LobbyGameModeFilter detectGameMode(const std::string& name)
+{
+	std::string modeName = name;
+	std::transform(modeName.begin(), modeName.end(), modeName.begin(), ::tolower);
+
+	// remove spaces
+	modeName.erase(std::remove(modeName.begin(), modeName.end(), ' '), modeName.end());
+
+	// handle common variations
+	for (size_t i = 0; i < modeName.size(); ++i)
+	{
+		if (modeName.compare(i, 2, "vs") == 0)
+		{
+			modeName.erase(i + 1, 1);
+			continue;
+		}
+
+		if (modeName[i] == 'x')
+			modeName[i] = 'v';
+	}
+
+	if (modeName.find("aod") != std::string::npos)
+		return LOBBY_FILTER_AOD;
+	if (modeName.find("ffa") != std::string::npos || modeName.find("1v1v1") != std::string::npos)
+		return LOBBY_FILTER_FFA;
+	if (modeName.find("1v1") != std::string::npos)
+		return LOBBY_FILTER_1V1;
+	if (modeName.find("2v2") != std::string::npos || modeName.find("3v3") != std::string::npos || modeName.find("4v4") != std::string::npos)
+		return LOBBY_FILTER_TEAM;
+
+	return LOBBY_FILTER_ALL;
+}
+
 void setSortMode( GameSortType sortType ) { theGameSortType = sortType; showSortIcons(); RefreshGameListBoxes(); }
 void sortByBuddies( Bool doSort ) { sortBuddies = doSort; showSortIcons(); RefreshGameListBoxes(); }
 

Core/GameEngine/Source/GameNetwork/GameSpyOverlay.cpp

@@ -137,6 +137,14 @@ void GSMessageBoxYesNo(UnicodeString title, UnicodeString message, GameWinMsgBox
 	cancelFunc = newNoFunc;
 }
 
+// GeneralsX @feature fbraz3 17/08/2026 Add GSMessageBoxCancel support for NGMP asynchronous lobby creation
+void GSMessageBoxCancel(UnicodeString title, UnicodeString message, GameWinMsgBoxFunc newCancelFunc)
+{
+	ClearGSMessageBoxes();
+	messageBoxWindow = MessageBoxCancel(title, message, messageBoxCancel);
+	cancelFunc = newCancelFunc;
+}
+
 /**
 	* If the screen transitions underneath the dialog box, we
 	* need to raise it to keep it visible.
@@ -194,6 +202,7 @@ void GameSpyOpenOverlay( GSOverlayType overlay )
 {
 	if (overlay == GSOVERLAY_BUDDY)
 	{
+#if !defined(SAGE_USE_NGMP)
 		if (!TheGameSpyBuddyMessageQueue->isConnected())
 		{
 			// not connected - is it because we were disconnected?
@@ -209,6 +218,7 @@ void GameSpyOpenOverlay( GSOverlayType overlay )
 			}
 			return;
 		}
+#endif
 		AudioEventRTS buttonClick("GUICommunicatorOpen");
 
 		if( TheAudio )

Generals/Code/GameEngine/Source/GameClient/GUI/GUICallbacks/Menus/MainMenu.cpp

@@ -1464,7 +1464,11 @@ WindowMsgHandledType MainMenuSystem( GameWindow *window, UnsignedInt msg,
 				dropDownWindows[DROPDOWN_MULTIPLAYER]->winHide(FALSE);
 				TheTransitionHandler->reverse("MainMenuMultiPlayerMenuTransitionToNext");
 
+#if defined(SAGE_USE_NGMP)
+				TheShell->push( "Menus/GameSpyLoginProfile.wnd" );
+#else
 				StartPatchCheck();
+#endif
 //				localAnimateWindowManager->reverseAnimateWindow();
 				dropDown = DROPDOWN_NONE;
 

Generals/Code/GameEngine/Source/GameClient/GUI/GUICallbacks/Menus/PopupPlayerInfo.cpp

@@ -762,9 +762,12 @@ void PopulatePlayerInfoWindows( AsciiString parentWindowName )
 	Int currentRank = 0;
 	Int rankPoints = CalculateRank(stats);
 	Int i = 0;
-	while( rankPoints >= TheRankPointValues->m_ranks[i + 1])
-		++i;
-	currentRank = i;
+	if (TheRankPointValues)
+	{
+		while (i + 1 < MAX_RANKS && rankPoints >= TheRankPointValues->m_ranks[i + 1])
+			++i;
+		currentRank = i;
+	}
 
 	PerGeneralMap::iterator it;
 	Int numWins = 0;
@@ -971,7 +974,15 @@ void PopulatePlayerInfoWindows( AsciiString parentWindowName )
 		}
 		else
 		{
-			GadgetProgressBarSetProgress(win, 100 * INT_TO_REAL(rankPoints - TheRankPointValues->m_ranks[currentRank])/( TheRankPointValues->m_ranks[currentRank + 1] - TheRankPointValues->m_ranks[currentRank]));
+			Int diff = TheRankPointValues->m_ranks[currentRank + 1] - TheRankPointValues->m_ranks[currentRank];
+			if (diff > 0)
+			{
+				GadgetProgressBarSetProgress(win, 100 * INT_TO_REAL(rankPoints - TheRankPointValues->m_ranks[currentRank]) / INT_TO_REAL(diff));
+			}
+			else
+			{
+				GadgetProgressBarSetProgress(win, 100);
+			}
 		}
 	}
 	win = findWindow(nullptr, parentWindowName, "WinRank");

Generals/Code/GameEngine/Source/GameClient/GUI/GUICallbacks/Menus/WOLLobbyMenu.cpp

@@ -308,9 +308,12 @@ static void playerTooltip(GameWindow *window,
 
 	Int rank = 0;
 	Int i = 0;
-	while( info->m_rankPoints >= TheRankPointValues->m_ranks[i + 1])
-		++i;
-	rank = i;
+	if (TheRankPointValues)
+	{
+		while (i + 1 < MAX_RANKS && info->m_rankPoints >= TheRankPointValues->m_ranks[i + 1])
+			++i;
+		rank = i;
+	}
 	AsciiString sideName = "GUI:RandomSide";
 	if (info->m_side > 0)
 	{
@@ -388,9 +391,12 @@ const Image* LookupSmallRankImage(Int side, Int rankPoints)
 
 	Int rank = 0;
 	Int i = 0;
-	while( rankPoints >= TheRankPointValues->m_ranks[i + 1])
-		++i;
-	rank = i;
+	if (TheRankPointValues)
+	{
+		while (i + 1 < MAX_RANKS && rankPoints >= TheRankPointValues->m_ranks[i + 1])
+			++i;
+		rank = i;
+	}
 
 	if (rank < 0 || rank >= 10)
 		return nullptr;