Skip to content

Refactor: use of i64 and db swap#35

Merged
purusang merged 11 commits into
mainfrom
refactor/use-u64
Apr 3, 2026
Merged

Refactor: use of i64 and db swap#35
purusang merged 11 commits into
mainfrom
refactor/use-u64

Conversation

@purusang

@purusang purusang commented Mar 30, 2026

Copy link
Copy Markdown
Collaborator

In this pr I have attempted to fix an ugly semantic flaw we had in our codebase due to a limitation of Postgres. Pg does not support U64, so we translated u64 (block height, checkpoint idx, etc) to i64 and vice versa which added unnecessary application level complication throughout the code.
So the major changes are:

  • removal of PGU64 abstraction which used to do i64<->u64 translations.
  • use of mariadb instead of pg
  • removal of separate migration service to avoid deployment complication. Now the migration will be handled by backend at the startup.
  • have added func-tests by following our alpen repo, this test sets up a mock strata fullnode and ckpt-exp backend and mostly tests if it is serving through our major routes (/search, /checkpoint?p=<x> and /). This was mostly written by Claude so I do not want it to be critically reviewed. Yet its good to have an eye.
  • hooked func-tests to ci.yml workflow

I have tested locally by running docker compose up --build -d and also by running cd functional-tests && ./run-tests.sh

PS: We have done some additional works which were not planned but felt worthwhile after a few rounds of reviews.

  • we have added ci workflows for lints and just recipes.
  • added functional tests
  • updated readme so that anybody can run this project easily.
  • removed sentinel values for status and txid with proper enums and option.

purusang and others added 5 commits March 30, 2026 17:04
- Replace sqlx-postgres with sqlx-mysql; use BIGINT UNSIGNED for native u64 storage
- Eliminate separate migration container by calling Migrator::up at startup
- Mock Strata fullnode (Python HTTP server) serving deterministic checkpoints and blocks over JSON-RPC
- Full stack isolation: mock fullnode + ephemeral MariaDB container + backend binary per test run

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@purusang purusang requested a review from krsnapaudel March 30, 2026 15:28
@purusang purusang self-assigned this Mar 30, 2026
@purusang purusang changed the title Refactor the use of i64 instead of u64 Refactor the use of i64 Mar 30, 2026
@purusang purusang changed the title Refactor the use of i64 Refactor: use of i64 and db swap Mar 30, 2026
@purusang

purusang commented Mar 30, 2026

Copy link
Copy Markdown
Collaborator Author

@alexhui01 JFYI, after this pr goes in, checkpoint explorer helm template will need a refactor. Major changes for deployment being:

  • mariadb instead of postgres
  • NO separate migration job, it will be handled by backend itself

Please refer to the the updated docker-compose as a reference.

NOTE: I do not see our Testnet I (we can keep using images from releases/0.2.0 branch )being impacted by this, but for future deployments this must be followed.

@krsnapaudel krsnapaudel left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for cleaning of the u64, i64 mess.

Other than the comment about why mariadb, you have pycache in the commit and some other issues.

Comment thread backend/model/src/checkpoint.rs Outdated
Comment thread backend/model/src/checkpoint.rs Outdated
Comment thread functional-tests/envs/services/backend.py Outdated
Comment thread docker-compose.yml
Comment thread backend/bin/checkpoint-explorer/src/services/checkpoint_service.rs Outdated
Comment thread .github/workflows/ci.yaml Outdated
@purusang purusang requested a review from krsnapaudel March 31, 2026 07:57
- replace dummy l1_reference fields with ExplorerL1Ref containing only
  txid; set to None when txid is absent. simplify frontend type to match
- add MariaDB healthcheck and condition: service_healthy to both
  docker-compose files so backend waits for DB to be ready
- write backend subprocess output to service.log instead of DEVNULL,
  matching alpen functional-test pattern
- remove invalid working-directory input from setup-uv ci step
- fix typo in checkpoint_service doc comment
- restore comment on confirmation_status parse

@krsnapaudel krsnapaudel left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A lot of nits.

Comment thread backend/bin/checkpoint-explorer/src/services/checkpoint_service.rs Outdated
Comment thread backend/database/src/services/block_service.rs Outdated
Comment thread backend/database/src/services/block_service.rs Outdated
Comment thread backend/database/src/services/block_service.rs Outdated
Comment thread backend/database/src/services/block_service.rs Outdated
Comment thread backend/database/src/services/checkpoint_service.rs Outdated
Comment thread backend/database/src/services/checkpoint_service.rs Outdated
Comment thread backend/database/src/services/checkpoint_service.rs Outdated
Comment thread backend/database/src/services/checkpoint_service.rs Outdated
Comment thread backend/model/src/checkpoint.rs Outdated
@purusang

purusang commented Apr 1, 2026

Copy link
Copy Markdown
Collaborator Author

A lot of nits.

@krsnapaudel it will be a good refactor. I am going with our alpen's tracing style (eg. info!(?task_id, "Task proving started"); )

@purusang purusang requested a review from krsnapaudel April 1, 2026 06:18

@krsnapaudel krsnapaudel left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still some issues.

Comment thread backend/bin/checkpoint-explorer/src/services/api_service.rs
Comment thread backend/bin/checkpoint-explorer/src/services/block_service.rs
Comment thread backend/model/src/checkpoint.rs Outdated
Comment thread backend/model/src/checkpoint.rs Outdated
Comment thread backend/model/src/checkpoint.rs Outdated
Comment thread backend/model/src/checkpoint.rs Outdated
Comment thread backend/model/src/checkpoint.rs
Comment thread backend/database/src/services/checkpoint_service.rs Outdated
Comment thread backend/database/src/services/checkpoint_service.rs Outdated
Replace String fields with typed equivalents in the checkpoint DB model:
- status: String → RpcCheckpointConfStatus via DeriveActiveEnum
- checkpoint_txid: String with "-" sentinel → Option<Txid>

Eliminates all string-based status comparisons in queries and removes
the "-" sentinel pattern from the From/Into impls.
@purusang purusang requested a review from krsnapaudel April 2, 2026 06:45

@krsnapaudel krsnapaudel left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise looks good.

Comment thread backend/bin/checkpoint-explorer/src/utils/config.rs
@purusang purusang requested a review from krsnapaudel April 3, 2026 10:35
@purusang purusang added this pull request to the merge queue Apr 3, 2026
Merged via the queue into main with commit 43f9b93 Apr 3, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants