gui/quickfort stopped loading blueprints

[badmancrab-byte]

I cannot use quickfort anymore after reinstalling the game on a new laptop. I have added some blueprints of my own and sorted them in folders, and they were transferred in the new re-install from what I can see.

This is the meassage that pops up when I try loading any blueprints:

[ab9rf]

This is due to a multibyte encoding error; you have a file somewhere with a character in its file name that cannot be represented in your system code page. While this is a DFHack bug and we'll have to fix it, you may be able to mitigate this bug by finding the file in question and renaming it to something that doesn't have such a character in it.

This can also happen if your installation folder or user profile folder contains an accented character that isn't in your system code page. If your primary language is not English, you may need to change your system's primary code page away from the Windows default of 1252 to one that supports the language that you're using. In retrospect, this likely will not help

[ab9rf]

Most likely, this is caused by std::filesystem::path's constructor throwing an exception while trying to construct a path from a UTF-8 string, at LuaTypes.cpp:199-208

void df::path_identity::lua_write(lua_State* state, int fname_idx, void* ptr, int val_index) const
{
    size_t size;
    const char* bytes = lua_tolstring(state, val_index, &size);
    if (!bytes)
        field_error(state, fname_idx, "path expected", "write");

    std::u8string str((char8_t*)bytes, size);
    *(std::filesystem::path*)ptr = std::filesystem::path(str);
}

On Windows, std::filesystem::path's constructor can throw an exception if the transcoding from the input encoding to the output encoding fails. In this case, the issue is likely that the input string (which is forced to be interpreted as UTF-8) actually was not (it was likely encoded in the platform code page instead) and so transcoding from UTF-8 to UCS-2 (which is how std::filesystem::paths are represented internally on Windows) failed because of an invalid input encoding.

[ab9rf]

Additional: the core problem is in LuaType.cpp:192-197:

void df::path_identity::lua_read(lua_State* state, int fname_idx, void* ptr) const
{
    auto ppath = (std::filesystem::path*)ptr;
    auto str = DFHack::Filesystem::as_string(*ppath);
    lua_pushlstring(state, (char*)str.data(), str.size());
}

Filesystem::as_string uses filesystem::path::string which returns the path as a "regular" string, which on Windows means it transcodes the pathname from its native UCS-2 representation to the system code page. But we later use this as if it were UTF-8, which it won't be (except on the rare system where the system code page is UTF-8, which is not yet fully supported in Windows.) We either need to change as_string to return a u8string (rather than a string), produced using filesystem::path::u8string, or add an as_u8string API to DFHack::Filesystem and use it in this context. I'm inclined to lean toward the former solution.

Note that changing Filesystem::as_string to return a std::u8string may also necessitate adding identity support for u8strings.