Skip to content

use FROM_UNIXTIME() for earliest and latest condition date conversion - #359

Open
elliVM wants to merge 3 commits into
teragrep:mainfrom
elliVM:use-from-unixtime
Open

use FROM_UNIXTIME() for earliest and latest condition date conversion#359
elliVM wants to merge 3 commits into
teragrep:mainfrom
elliVM:use-from-unixtime

Conversation

@elliVM

@elliVM elliVM commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Description

Resolves #355

Use MariaDB's FROM_UNIXTIME() function to calculate dates in EarliestCondition and LatestCondition.

Previously, epoch values were converted to java.sql.Date in the JVM timezone before being used in the SQL condition. This caused the generated logdate condition to depend on the JVM timezone, while the logtime condition depended on the MariaDB session timezone.

By moving the epoch-to-date conversion to MariaDB, both conditions now use the MariaDB session timezone consistently.

This affects existing tests containing earliest or latest conditions because the generated SQL now uses DATE(FROM_UNIXTIME(...)) instead of a date literal.

Note: MariaDB documentation recommends using a named timezone (for example UTC or Europe/Helsinki) instead of offsets to avoid timezone-related performance issues.

FROM_UNIXTIME() documentation

Testing

General

  • I have checked that my test files and functions have meaningful names.
  • I have checked that each test tests only a single behavior.
  • I have done happy tests.
  • I have tested only my own code.
  • I have tested at least all public methods.

Assertions

  • I have checked that my tests use assertions and not runtime overhead.
  • I have checked that my tests end in assertions.
  • I have checked that there is no comparison statements in assertions.
  • I have checked that assertions are in tests and not in helper functions.
  • I have checked that assertions for iterables are outside of for loops and both sides of the iteration blocks.
  • I have checked that assertions are not tested inside consumers.

Testing Data

  • I have tested algorithms and anything else with the possibility of unbound growth.
  • I have checked that all testing data is local and fully replaceable or reproducible or both.
  • I have checked that all test files are standalone.
  • I have checked that all test-specific fake objects and classes are in the test directory.
  • I have checked that my tests do not contain anything related to customers, infrastructure or users.
  • I have checked that my tests do not contain non-generic information.
  • I have checked that my tests do not do external requests and are not privately or publicly routable.

Statements

  • I have checked that my tests do not use throws for exceptions.
  • I have checked that my tests do not use try-catch statements.
  • I have checked that my tests do not use if-else statements.

Java

  • I have checked that my tests for Java uses JUnit library.
  • I have checked that my tests for Java uses JUnit utilities for parameters.

Other

  • I have only tested public behavior and not private implementation details.
  • I have checked that my tests are not (partially) commented out.
  • I have checked that hand-crafted variables in assertions are used accordingly.
  • I have tested Object Equality.
  • I have checked that I do not have any manual tests or I have a valid reason for them and I have explained it in the PR description.

Code Quality

  • I have checked that my code follows metrics set in Procedure: Class Metrics.
  • I have checked that my code follows metrics set in Procedure: Method Metrics.
  • I have checked that my code follows metrics set in Procedure: Object Quality.
  • I have checked that my code does not have any NULL values.
  • I have checked my code does not contain FIXME or TODO comments.

@elliVM elliVM self-assigned this Jul 16, 2026
@elliVM elliVM added the bug Something isn't working label Jul 16, 2026
+ " and \"journaldb\".\"logfile\".\"logdate\" <= date '2024-10-20'\n"
+ " and (UNIX_TIMESTAMP(STR_TO_DATE(SUBSTRING(REGEXP_SUBSTR(path,'[0-9]+(\\.rfc5424)?(\\.log)?\\.gz(\\.[0-9]*)?$'), 1, 10), '%Y%m%d%H')) <= 1729435021)\n"
+ ")";
// spotless:off

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test is easier to read without spotless formatting.
Change:
date '2022-01-26' -> DATE(FROM_UNIXTIME(1643205600))
date '2024-10-20' -> DATE(FROM_UNIXTIME(1729435021))

+ " ),\n" + " \"bloomdb\".\"pattern_test_ip\".\"filter\"\n" + " ) = true\n"
+ " and \"bloomdb\".\"pattern_test_ip\".\"filter\" is not null\n" + " )\n"
+ " or \"bloomdb\".\"pattern_test_ip\".\"filter\" is null\n" + " )\n" + ")";
// spotless:off

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change:
date '2022-01-26' -> DATE(FROM_UNIXTIME(1643205600))
date '2024-10-20' -> DATE(FROM_UNIXTIME(1729435021))

@elliVM
elliVM requested a review from Tiihott July 16, 2026 07:10

@Tiihott Tiihott left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All tests pass and changes look ok.
The tests that were previously disabled because of issue #355 should be enabled again.

Comment thread src/test/java/com/teragrep/pth_06/planner/StreamDBClientTest.java Outdated
Comment thread src/test/java/com/teragrep/pth_06/planner/StreamDBClientTest.java Outdated
@elliVM
elliVM requested a review from Tiihott July 30, 2026 12:26

@Tiihott Tiihott left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@elliVM
elliVM requested a review from kortemik July 30, 2026 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

EarliestCondition and LatestCondition fail to generate proper conditions when timezone used by JVM and MariaDB session don't match.

2 participants