Index Credit Transfers

[S4mmyb]

Summary

Adds indexing support for ecocredit credit transfers. The indexer now tracks EventTransfer events from MsgSend transactions, storing transfer details in a new transfers table. This enables tracking all credit movements between accounts, which is essential for:

• Understanding credit flow and ownership changes
• Tracking both tradable and retired credit transfers
• Supporting queries about sender/recipient transfer history
• Analyzing credit movement patterns across the network

What changed

• New transfers table: Stores individual credit transfer records with:
  • Event type and transaction metadata (chain_num, block_height, tx_idx, msg_idx, tx_hash, timestamp)
  • Transfer details (sender, recipient, batch_denom)
  • Amounts (tradable_amount, retired_amount as TEXT to preserve precision)
• New index_transfers.py: Parses EventTransfer events and indexes them
  • Filters to MsgSend only (EventTransfer can emit from other message types)
  • Aggregates amounts when multiple credits of the same batch_denom are transferred
• Updated utils.py: Added transfers to TABLE_EVENT_NAMES_MAP with both v1 event types; v1alpha was skipped because mainnet has no MsgSend during v1alpha
• Updated main.py: Added index_transfers() call to start the transfers indexer process
• Migration 000008.sql: Creates the transfers table with proper indexes and foreign key constraints
• Test script: Added test_index_transfers.sh for basic smoke testing

What was tested

• Connected to regen-upgrade testnet
• Created test transactions with MsgSend containing credit transfers individually, and bundled
• Verified EventTransfer events were correctly parsed and inserted into transfers table
• Confirmed both tradable-only and retired transfers were indexed

[gemini-code-assist]

Summary of Changes

Hello @S4mmyb, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new indexing capability to track ecocredit transfers within the system. By capturing and storing EventTransfer events originating from MsgSend transactions, the system can now provide a comprehensive history of credit movements between accounts. This enhancement is crucial for analyzing credit flow, understanding ownership changes, and supporting detailed queries on transfer patterns, thereby improving the transparency and auditability of ecocredit transactions.

Highlights

• New transfers table: A dedicated database table has been introduced to store detailed records of ecocredit transfers, capturing essential information about each movement.
• Ecocredit transfer indexing: A new Python script (index_transfers.py) is added to process EventTransfer events, specifically filtering for MsgSend transactions and aggregating amounts for both tradable and retired credits.
• Integration into main indexer: The new transfer indexing process is integrated into the main application flow, ensuring continuous data capture and processing of ecocredit transfers.
• Database schema migration: A new SQL migration (000008.sql) creates the transfers table with appropriate columns, unique constraints, and indexes for efficient data storage and retrieval.

Changelog

• index_transfers.py
  • Added a new Python script responsible for polling and processing EventTransfer events.
  • Includes logic to filter events to only those originating from MsgSend messages.
  • Aggregates tradable_amount and retired_amount for transfers, handling potential multiple entries for the same batch.
  • Inserts normalized transfer data into the new transfers database table.
• main.py
  • Added an import statement for the new index_transfers module.
  • Included a call to index_transfers() to initiate the transfer indexing process alongside other existing indexers.
• migrations/committed/000008.sql
  • Introduced a new SQL migration file to create the public.transfers table.
  • The table schema includes fields such as type, tradable_amount, retired_amount, batch_denom, sender, recipient, timestamp, block_height, chain_num, tx_idx, msg_idx, and tx_hash.
  • Defined a unique constraint on (chain_num, block_height, tx_idx, msg_idx, batch_denom, sender, recipient) to prevent duplicate entries.
  • Created B-tree indexes on sender, recipient, batch_denom, and tx_hash for optimized query performance.
• migrations/schema_snapshot.sql
  • Updated the database version and pg_dump version comments.
  • Included the schema definition for the newly added public.transfers table, its unique constraint, and indexes.
  • Added \restrict and \unrestrict commands.
• utils.py
  • Modified the TABLE_EVENT_NAMES_MAP dictionary to include 'transfers' as a key, mapping it to the regen.ecocredit.v1.EventTransfer event type. This ensures the events_to_process utility function correctly identifies and retrieves transfer events for the new indexer.

Activity

• The new indexing functionality was tested on the regen-upgrade testnet.
• Test transactions involving MsgSend with both individual and bundled credit transfers were created.
• Verification confirmed that EventTransfer events were correctly parsed and inserted into the transfers table.
• Both tradable-only and retired credit transfers were successfully indexed.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6b5ed2f0a9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[gemini-code-assist]

Code Review

This pull request introduces indexing for ecocredit credit transfers, a valuable addition for tracking credit movements. However, the implementation contains two high-severity data integrity issues: a logic error that causes data corruption when a single message emits multiple transfer events, and precision loss due to the use of floating-point arithmetic for high-precision credit amounts, making the indexed transfer data unreliable for financial auditing or ownership tracking. Additionally, there are opportunities for performance and robustness improvements in index_transfers.py.

[blushi]

Maybe we could use a view backed by indexes instead of an additional table to save on db storage costs without too much impact on performance?

[S4mmyb]

Maybe we could use a view backed by indexes instead of an additional table to save on db storage costs without too much impact on performance?

Good idea - honestly ended up being a lot simpler too. Lmk what you think now. I tested a few transactions, but this was the one for the edge case:

[S4mmyb]

hey @blushi , do you think I could get a review on this?

[blushi]

utACK

[blushi]

@S4mmyb You'll have to submit a PR from dev to main for the changes to reflect on prod.