Skip to content

utils: parse microsecond-precise ISO 8601 and Apache offsets, keep --start-time/--end-time comparable - #27

Open
HrachShah wants to merge 1 commit into
mainfrom
fix/parse-timestamp-microseconds-with-tz
Open

utils: parse microsecond-precise ISO 8601 and Apache offsets, keep --start-time/--end-time comparable#27
HrachShah wants to merge 1 commit into
mainfrom
fix/parse-timestamp-microseconds-with-tz

Conversation

@HrachShah

@HrachShah HrachShah commented Jul 4, 2026

Copy link
Copy Markdown
Owner

Summary

parse_timestamp in log_analyzer_cli/utils.py could not parse several common timestamp shapes that the regex it tries already captures:

  • ISO 8601 with microseconds and a timezone offset (2025-10-10T13:55:36.123456+00:00)
  • ISO 8601 with microseconds and Z (2025-10-10T13:55:36.123456Z)
  • ISO 8601 with a space separator, microseconds, and an offset (2025-10-10 13:55:36.123456+00:00)

It also lost the offset on Apache common log timestamps: the regex captured 10/Oct/2025:13:55:36 only, dropping the trailing ±HHMM entirely.

Changes

  • src/log_analyzer_cli/utils.py:
    • _try_parse_datetime now orders formats from most specific to least specific, so a T%H:%M:%S.%f%z shape is tried before a plain T%H:%M:%S%z (and before T%H:%M:%S.%f without a timezone). Added the previously missing T%H:%M:%S.%f%z, H:%M:%S.%f%z, T%H:%M:%S %z, and Apache's %d/%b/%Y:%H:%M:%S %z formats.
    • The Apache timestamp regex now optionally captures the ±HHMM offset, and the new %d/%b/%Y:%H:%M:%S %z format preserves it on the returned datetime.
  • src/log_analyzer_cli/cli.py: _parse_file now strips a tz-aware tzinfo from the parsed timestamp before comparing it against a naive --start-time/--end-time bound, so the bounds can filter a log whose entries carry a UTC offset without raising TypeError: can't compare offset-naive and offset-aware datetimes.
  • tests/test_utils.py: 26 new tests covering microsecond-precise ISO shapes (with and without timezone, with and without Z, with positive and negative offsets), the space-separator form, Apache common log with and without offset, RFC 3164 syslog, naive inputs, and a regression that asserts fractional seconds and the timezone offset round-trip.

Test results

python3 -m pytest tests/73 passed (47 pre-existing + 26 new).

Summary by Sourcery

Extend timestamp parsing and comparison to handle timezone-aware and microsecond-precise log entries while keeping CLI time range filters robust.

New Features:

  • Support parsing ISO 8601 timestamps with microseconds and timezone offsets, including Z and space-separated variants.
  • Support parsing Apache common log timestamps with optional numeric timezone offsets.

Enhancements:

  • Order datetime parsing formats from most specific to least specific to avoid losing fractional seconds or timezone information.
  • Normalize timezone-aware timestamps to naive when applying --start-time/--end-time filters to prevent comparison errors.

Tests:

  • Add comprehensive tests covering microsecond-precise ISO 8601 timestamps, Apache common log formats with and without offsets, syslog-style timestamps, and regression cases for preserving fractional seconds and timezone offsets.

@sourcery-ai

sourcery-ai Bot commented Jul 4, 2026

Copy link
Copy Markdown

Reviewer's Guide

Enhances timestamp parsing to support microsecond-precise ISO 8601 and Apache log offsets, and makes CLI time-bound filtering robust when comparing offset-aware log entries against naive start/end times, backed by targeted tests.

Sequence diagram for CLI timestamp parsing and naive bounds comparison

sequenceDiagram
    actor User
    participant CLI as _parse_file
    participant Utils as parse_timestamp
    participant Dt as datetime

    User->>CLI: _parse_file(file_path, start_time, end_time)
    loop for each line
        CLI->>Utils: parse_timestamp(line)
        Utils-->>CLI: timestamp
        alt start_time or end_time is set
            alt timestamp.tzinfo is not None
                CLI->>Dt: timestamp.replace(tzinfo=None)
                Dt-->>CLI: compare_ts (naive)
            else timestamp.tzinfo is None
                CLI->>CLI: compare_ts = timestamp
            end
            alt compare_ts < start_time
                CLI->>CLI: continue (skip line)
            else compare_ts > end_time
                CLI->>CLI: continue (skip line)
            end
        end
        CLI->>CLI: parser.parse(line)
    end
    CLI-->>User: parsed results
Loading

File-Level Changes

Change Details Files
Broaden and refine timestamp detection and parsing to support microseconds and timezone offsets without losing precision or tzinfo.
  • Extend ISO 8601 regex to cover optional fractional seconds and timezone offsets including Z and ±HHMM with optional colon.
  • Update Apache common log regex to optionally capture a trailing numeric offset while still matching entries without an offset.
  • Reorder datetime format attempts from most specific to least specific so microsecond+timezone formats are tried before coarser ones that would drop tzinfo or microseconds.
  • Add explicit datetime formats for ISO 8601 with microseconds and timezone (T and space separator), ISO 8601 with timezone but no fractional seconds, Apache common log with optional timezone, and keep existing formats for syslog and naive timestamps.
src/log_analyzer_cli/utils.py
Normalize parsed timestamps before applying CLI start/end-time filters to avoid naive vs aware datetime comparison errors.
  • Introduce a compare_ts variable that strips tzinfo from offset-aware timestamps while leaving naive timestamps unchanged.
  • Use compare_ts for comparisons against start_time and end_time bounds to prevent TypeError and keep filtering semantics consistent.
src/log_analyzer_cli/cli.py
Add regression tests to cover new timestamp parsing capabilities and ensure fractional seconds and timezone offsets round-trip correctly.
  • Create a new test module dedicated to utils, especially parse_timestamp behavior.
  • Add tests for ISO 8601 timestamps with microseconds and various timezone representations (Z, +00:00, ±HHMM, negative offsets) and with/without space separator.
  • Add tests for Apache common log timestamps with and without offsets, RFC 3164-style syslog entries, lines without timestamps, and preservation of microseconds and timezone offsets.
  • Include sanity tests for existing helpers detect_log_level and normalize_error_pattern to guard against regressions.
tests/test_utils.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai

coderabbitai Bot commented Jul 4, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@HrachShah, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 56 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 85f94330-df0a-45e3-85d9-90426bd50a6f

📥 Commits

Reviewing files that changed from the base of the PR and between e93757f and 79893e6.

📒 Files selected for processing (3)
  • src/log_analyzer_cli/cli.py
  • src/log_analyzer_cli/utils.py
  • tests/test_utils.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/parse-timestamp-microseconds-with-tz

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant