Timezone implementation

[DalisonCPU]

This implementation allows developers to set different timezones at runtime.
Functions:
set_timezone(string timezone_id) - Sets the active timezone using standard timezone identifiers (e.g., "America/New_York", "Asia/Tokyo")
reset_timezone() - Resets to system default timezone
get_current_timezone_name() - Returns the name of the currently active timezone
get_current_timezone_offset() - Gets the UTC offset in seconds for the current timezone
list_available_timezones() - Returns an array of all supported timezone identifiers

The system supports 200+ official timezone identifiers following PHP timezone standards
Calendar objects automatically respect the active timezone when created
A test of how to use the new implementation was added in test/quick/timezone.nvgt

Note: I've tested this and it seems to work well, but I'm still quite new to C++. Please test it more thoroughly before approving.

[samtupy]

Hi,

Thanks for the conrtibution!

So I wonder if this might be best as a plugin rather than a core feature so as to reduce confusion from users about what API's work with this system and so that updates to timezone data are a bit less instantly manditory? For example, this won't apply if I call TIME_HOUR, nore if I use the datetime object, and any time someone adds a new time based API to NVGT's core, they'd potentially need to be aware to work with this system.

The plugin could have all the functions it has now, and can even register functions on top of the calendar object, such as a constructor that takes a timezone string.

Also, I think it might be good to write some sort of autogen script that creates the timezone data, as right now it doesn't seem easy to apply updates to timezone data, which unfortunately take place at any random time.

Alternatively, have we considered any cross platform libraries that exist which can call into native platform APIs to handle this for us so that we don't need to be the ones responsible for updating the IANA data?

These are just my thoughts at present, thanks again this is a good idea and a good start!

[DalisonCPU]

Hi.
I hadn't really considered that someone might develop an alternative time API. Presently, this feature modifies the behavior of TIME_HOUR, TIME_MINUTE, and TIME_SECOND. It also affects calendar functionality upon reset, reverting to the designated timezone's time.
Regarding the creation of a plugin, I must admit that would be a more advantageous approach. While a plugin option would be superior, I'm uncertain about my ability to accomplish it entirely on my own. Would it be possible to use a library for automatic timezone data updates? I'm not familiar with any libraries that offer this capability...
My current implementation was conceived with the primary goal of simplifying the experience for users with servers in different timezones, especially for individuals like myself in Brazil, where the constant effort of updating server times is quite burdensome.

[ultrasound1372]

Most if not all Unix systems (check macos?) actually ship with zoneinfo, and so likely provide APIs to interact with it. Windows, unfortunately, uses an entirely separate timezone implementation with its own naming and not conforming to IANA at all. It also has APIs to interact with this, but the two are not interchangeable. I would also add that if this were to be put in core, which it perhaps should, datetime objects should be made timezone aware, as in Python. That is they should have an actual timezone attribute that stores the reference timezone. For examples of how this kind of functionality can be standardized, including the IANA and Windows interfacing, see Python's implementations of time and datetime, and the pytz package.

[DalisonCPU]

@samtupy Please then give us some direction on how to proceed with the pull request. The timezone implementation is very important for the project and we should not leave this implementation aside.

[ethindp]

I do believe this should probably be a plug-in. Given that we may need to ship a TZ database.

[erray007]

Dostum basit bir dll oluştura bilirsin bu nvgt ye bir özellik eklemenin en basit yoludur

[DalisonCPU]

I do not know which is the best idea, but I really want this functionality to be implemented. I cannot find any sense in it being a plugin, but if it is decided that this is the best option, I will be happy to help.

[samtupy]

So to be clear, I think it should either be a plugin, or we need to use an external timezone database E. reading zoneinfo files on unix, or interacting with the windows API to list system timezones and offsets, or at the very least we need a script to fetch and generate the latest timezone info. The basic problem here is that the timezone data is baked into a map in the code, but in reality time zones shift in the world at any random time, for example because of border changes or other legal updates. If this feature is included in core, I think we should have a robust way of handling that.

[ultrasound1372]

Oh, the zoneinfo maps are actually embedded in the code? No, definitely not, they need to be fetched externally. Whether core or not.

[DalisonCPU]

I was doing some research and hadn't come across this before, but I found this library:
https://github.com/unicode-org/cldr-json/
Projects like Globalize.js, locale-util, cldr-language-helpers, and many others utilize it, and it struck me as quite interesting. What are your thoughts?
It's all provided in JSON, which would be a huge help for developing the Python script to generate the updated list during the NVGT compilation.

I discovered these links that might be useful:
https://raw.githubusercontent.com/unicode-org/cldr-json/main/cldr-json/cldr-core/supplemental/metaZones.json
https://raw.githubusercontent.com/unicode-org/cldr-json/main/cldr-json/cldr-core/supplemental/windowsZones.json

I'm hesitant about relying on the operating system's built-in timezone database. While I could get AMERICA/SAO_PAULO configured correctly on Linux, it might not work on Windows due to discrepancies in timezone naming conventions.

[ultrasound1372]

If you're going to generate it yourself, shouldn't you use IANA's data directly? Or does CLDR bind to that.

[DalisonCPU]

Yes, the CLDR organizes this data into JSON formats, which makes everything much easier.

[ethindp]

Unless I missed something (which is of course possible) the CLDR database doesn't contain actual timezone information in the same way the TZ database does. It does contain some calendar stuff but I haven't looked at the contents of those (although from scanning the descriptions of what they are they appear to be for specific regions/cultures/etc.).

[DalisonCPU]

Basically, I'm not totally certain, but it looks like a lot of projects, even Microsoft, are using this system. I'm not sure how accurate it is, but that's what it seems like. We could try the official site, but it'll be way more work because it's not in JSON format.

[ethindp]

I just looked at, e.g., the Chinese calendar example for English (https://github.com/unicode-org/cldr-json/blob/main/cldr-json/cldr-cal-chinese-full/main/en/ca-chinese.json) to pick one at random, and it doesn't look like it contains any actual timezone information, just patterns for months/dates/times, or translations/names.

[DalisonCPU]

My idea would be to use CLDR to get the names and zoneinfo from Python to get the offsets. I don't know what you think of this. It seems like the easiest way to me.

[ethindp]

I mean, that would work but if we're going to do that we might as well just pull in ICU and be done with it given that ICU already does the CLDR localization for us and has the CLDR data built-in. There's pretty much no point in manually handling it given that the vast super-majority of users compile a binary representation of it instead of bundling the raw JSON.

As for tzdata, we would neeed to build a tzfile(5) parser.

[DalisonCPU]

Are you referring to downloading this data in real-time, every time NVGT script's is opened? If so, I don't think it's a good idea, as there would always be a significant delay when the program is launched. If that's not the case, please send me a link or something I can look at to understand better.

[ethindp]

No. We bundle it with each release of NVGT. All the timezone stuff is in tzfile(5) format, or we embed it into the program as rodata. For the CLDR, we'd need to figure out how to represent it.

[DalisonCPU]

Could you create something initial for us to begin with the implementation?

[DalisonCPU]

And then, will you help or did you just comment so we lose time?

[ethindp]

Could you create something initial for us to begin with the implementation?

Are you asking for a tzfile(5) parser?

[DalisonCPU]

That's it. Something to download these files, and then we'll see how we'll implement these files.

[ultrasound1372]

Surely there are existing libraries, C/C++, that can parse tzfile(5)? Or is that just part of something like glibc? CLDR, as I understand given what CLDR actually is, is just to display those timezones to users in a more user friendly format, and apparently to handle alternative calendar systems.

[samtupy]

There is this, but it looks slightly incomplete E. no communication with the windows API would mean we need to ship zoneinfo files on windows: https://github.com/google/cctz

[ethindp]

@samtupy I think that shipping the files on windows might actually be smart: it means our system is unified. Although (maybe) Windows integration might be a good idea? I'm not sure honestly.

[samtupy]

Thing to remember is that it's not NVGT that would need to ship zoneinfo, but every game created with NVGT that queries timezones, so the bundler would have to stuff it into the lib folder or something like that. That's not the end of the world? It's just not desired, it's just more files we'd be making go viral as they spread with everyone's games. We can do it if we absolutely must. I'd really rather not.

[DalisonCPU]

I am much more in favor of extracting the data at the time of NVGT compilation and adding it to the code similar to what is currently being done, of course with adjustments.
I also don't like too many files in folders.

[DalisonCPU]

I have added an initial script for your feedback. Currently, it generates the map in the same folder as the script.

[DalisonCPU]

@samtupy @ultrasound1372 @ethindp

[ethindp]

Hi there,

This script looks pretty good. Some thoughts:

1. You could probably shorten the branching in organize_by_region by splitting on / and adding the first part to the regions dict.
2. Make the maps you generate constexpr.

Other than that, this looks good as a plug-in.

[DalisonCPU]

@ethindp How does it look now?

I need to know your opinions, should we implement it as a plugin or directly in the code? I personally prefer it to be in the code and not as a plugin.

[ultrasound1372]

Shipping the zoneinfo files on Windows is exactly what Python does, via the tzdata package, because they do not synchronize, and thus a CLDR mapping of Windows timezones is prone to errors, plus datetime arithmetic is likely to break down if it overlaps timezone changes, as I'm unsure how many rules Windows maintains. We might be best served doing something similar to what Python did. Zoneinfo as a plug-in, but add timezone support to the datetime objects, a la Python's aware datetime, and some default objects that zoneinfo can derive from. Using lazy evaluation if at all possible! Zone name at datetime, UTC offset at datetime, dst at datatime, or however Python implements that. Other language's implementation could also be followed, or if there's some well-known C++ datetime library that does this, I'm just familiar with Python's conventions.

[ethindp]

I honestly think that TZ stuff should purely be a plug-in. Timezones aren't just offsets from UTC, and you have to factor in whether DST is active or not and all that. Depending on how frequent the tzdata package is updated, we would need to release NVGT updates quite frequently to keep up, potentially.

[ultrasound1372]

Yes, the actual timezone data, that is the stuff from IANA, should be a plug-in. I'm saying that there should be support for the construct in core, a base object that doesn't do a whole lot but at least defines a UTC time zone, and support for attaching it and subclasses to datetimes.

[DalisonCPU]

Will it be possible to have DATE_HOUR, DATE_MINUTE etc. updated with a plugin? If it's not possible, I don't see much point in using it as a plugin. Would the same be possible in a calendar?

[ethindp]

@ultrasound1372 Right, but I don't think that kind of split is appropriate. Without the zoneinfo data, the code in core doesn't really offer much other than hard-coded offsets, which do not exactly contribute much other than adding a maintenance burden, and also adding a risk that the zoneinfo files may get updated while the offsets in core remain unchanged.

@DalisonCPU Yes, plug-ins can extend classes with new methods, properties and the like. You cannot extend functions themselves for obvious reasons, but nothing stops you from adding new functions/methods and such.

[DalisonCPU]

Can I remodel the default TIME_HOUR property if I deem it necessary, or not?

[DalisonCPU]

I also think it's very difficult for one person to do all of this, does anyone volunteer to help?

[ethindp]

As I said: you cannot modify the code of functions from a plug-in. You can de-register the built-in one and replace it with your own, though.