`test_make_compat.py`'s `test_make_compat` method fails when round-tripping an ICS with `TZID=Pacific Standard Time`

[ChrisCarini]

Describe the bug
A clear and concise description of what the bug is, and the version of the ics library used.

It appears that an ICS file with something like DTSTART;TZID=Pacific Standard Time:20241018T000000 (noting the Pacific Standard Time tz) will cause this lib to fail the 'round-trip' of converting the ICS file into a Calendar object, and then back to an ICS (str) via IcsCalendarStream.calendar_to_ics(calendar) will change the line to DTSTART;TZID=PST:20241018T000000

To Reproduce

See #535

Expected behavior

I would expect at the very least that the ICS str coming from IcsCalendarStream.calendar_to_ics(calendar) would be re-parsable by this lib.

I understand that Pacific Standard Time is not a 'standard' TZ, and is part of the recent compatibility that was added for Office 365, but "most-ideally", it'd be great if IcsCalendarStream.calendar_to_ics(calendar) could return a value that is near-identical to the original ICS file that was passed in.

Additional context

I found this when trying to do some testing/debugging for home-assistant/core#148315 - likely tangential and not directly related, just but 'linking' for self-posterity.

[allenporter]

Does it also output a timezone named VTIMEZONE with the name PST or only Pacific Standard Time? I'm trying to understand if the output calendar is "changed" (but still valid RFC5545) or if its "invalid"

[ChrisCarini]

Hey @allenporter - sorry for the delay!

I see Pacific Standard Time:

BEGIN:VTIMEZONE
TZID:Pacific Standard Time
BEGIN:STANDARD
DTSTART:16010101T020000
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=1SU;BYMONTH=11
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T020000
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=2SU;BYMONTH=3
END:DAYLIGHT
END:VTIMEZONE

Full excerpt (lines 6-134 of my remote ics; the entirety of VTIMEZONE) below:

BEGIN:VTIMEZONE
TZID:Eastern Standard Time
BEGIN:STANDARD
DTSTART:16010101T020000
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=1SU;BYMONTH=11
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T020000
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=2SU;BYMONTH=3
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VTIMEZONE
TZID:Pacific Standard Time
BEGIN:STANDARD
DTSTART:16010101T020000
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=1SU;BYMONTH=11
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T020000
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=2SU;BYMONTH=3
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VTIMEZONE
TZID:Central Standard Time
BEGIN:STANDARD
DTSTART:16010101T020000
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=1SU;BYMONTH=11
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T020000
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=2SU;BYMONTH=3
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VTIMEZONE
TZID:Mountain Standard Time
BEGIN:STANDARD
DTSTART:16010101T020000
TZOFFSETFROM:-0600
TZOFFSETTO:-0700
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=1SU;BYMONTH=11
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T020000
TZOFFSETFROM:-0700
TZOFFSETTO:-0600
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=2SU;BYMONTH=3
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VTIMEZONE
TZID:Romance Standard Time
BEGIN:STANDARD
DTSTART:16010101T030000
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=-1SU;BYMONTH=10
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T020000
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=-1SU;BYMONTH=3
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VTIMEZONE
TZID:India Standard Time
BEGIN:STANDARD
DTSTART:16010101T000000
TZOFFSETFROM:+0530
TZOFFSETTO:+0530
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T000000
TZOFFSETFROM:+0530
TZOFFSETTO:+0530
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VTIMEZONE
TZID:UTC
BEGIN:STANDARD
DTSTART:16010101T000000
TZOFFSETFROM:+0000
TZOFFSETTO:+0000
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T000000
TZOFFSETFROM:+0000
TZOFFSETTO:+0000
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VTIMEZONE
TZID:US Mountain Standard Time
BEGIN:STANDARD
DTSTART:16010101T000000
TZOFFSETFROM:-0700
TZOFFSETTO:-0700
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T000000
TZOFFSETFROM:-0700
TZOFFSETTO:-0700
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VTIMEZONE
TZID:GMT Standard Time
BEGIN:STANDARD
DTSTART:16010101T020000
TZOFFSETFROM:+0100
TZOFFSETTO:+0000
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=-1SU;BYMONTH=10
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T010000
TZOFFSETFROM:+0000
TZOFFSETTO:+0100
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=-1SU;BYMONTH=3
END:DAYLIGHT
END:VTIMEZONE

If it helps any, here's a quick analysis of the TZID in the file:

$> grep -o 'TZID=[^:;]*' "/Users/ccarini/Library/Application Support/JetBrains/IntelliJIdea2026.1/scratches/calendar.ics" | sed 's/TZID=//' | sort | uniq -c | sort -rn
     6162 Pacific Standard Time
       42 Eastern Standard Time
       36 UTC
       22 Central Standard Time
       12 India Standard Time
       10 Mountain Standard Time
        4 US Mountain Standard Time
        4 Romance Standard Time
        4 GMT Standard Time
        2 W. Europe Standard Time

[allenporter]

OK so what happens is we map the value here

ical/ical/tzif/extended_timezones.py

Line 87 in 8f86756

"Pacific Standard Time": "America/Los_Angeles",

to a native timestamp when in compat mode.

Does it round trip successfully when NOT in compat mode? I assume it preserves it since its not pulling in that logic. Basically remote calendars in home assistant enable special hacks, and this seems like an outcome of that.

[ChrisCarini]

Does it round trip successfully when NOT in compat mode?

What's a good way/existing-test to use to test this? Any recommendations?

[allenporter]

The APIs are described here: https://allenporter.github.io/ical/ical.html

Adding examples to here, there are some tests of input / output similar to the compat tests:
https://github.com/allenporter/ical/tree/main/tests/examples

[ChrisCarini]

I might be doing something wrong (and thus might need a bit more guidance, if you're open to it), because I am not getting the existing cals to round-trip and be identical - though perhaps that's not expected?

This is the test I have added into test_ics_examples.ics:

@pytest.mark.parametrize("filename", TEST_FILES, ids=TEST_IDS)
def test_roundtrip(filename: Path) -> None:
    """Test to read golden files and verify they are parsed."""
    ics = None
    with filename.open(encoding="utf-8") as ics_file:
        ics = ics_file.read()

    input = IcsCalendarStream.calendar_from_ics(ics)
    output = IcsCalendarStream.calendar_to_ics(input)

    output_file = Path(str(filename) + "-out.ics")
    output_file.write_text(output, encoding='utf-8')

    assert output == ics

Comparing the output file, the ordering of the 'fields' (nomenclature is probably wrong) for each event are not the same, but even after re-ordering them by hand, there are still some missing bits. Here is a screenshot after I manually re-ordered the apple ical example that's already in the repo:

Without manually reordering anything, it looks like below:

Because of this, I'm a bit hesitant to feed it my ICS to get a signal if it's successful or not doing a round-trip, since the existing calendars don't see to be successful at round-tripping with the above test (which, again, I very well may be doing something wrong or have some misunderstanding). It's worth noting that my ICS does seem to be read in fine, granted it takes in excess of 10minutes each time though, so I'd rather get a clean signal w/ smaller cals (like the current ones in testdata) before trying my own (granted, I tried my own, too, but yeah, same issue - stuff is all over).

[allenporter]

Ah I misunderstood the original report. I was thinking this was a TZID attribute on a start/end event, but i see now you're saying its dropping unknown attributes on an entire component. This is #65

I was originally thinking that any unknown property should be collected in extras (ref) and so maybe we can put them there. Then when writing, it would append them to the output.

I think we may not preserve ordering within a component however.

Would that do what you expect?

[allenporter]

OK I added propper extras support so now we should be seeing the extended properties round tripped properly. Perhaps now we can revisit the previous item you were looking at?