Skip to content

Feat/refactor markets tokens initialization #355

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
aarmoa merged 2 commits into from
Nov 13, 2024
Merged

Feat/refactor markets tokens initialization #355

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2b202ee
(feat) Refactored markets and tokens initialization logic in AsyncCli…
aarmoa Nov 12, 2024
845e813
Merge branch 'dev' of https://github.com/InjectiveLabs/sdk-python int…
aarmoa Nov 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

All notable changes to this project will be documented in this file.

## [1.8.0] - 9999-99-99
### Changed
- The markets initialization in AsyncClient has been modified to get markets information from the chain endpoints instead of the Indexer endpoints

### Removed
- Removed the legacy deprecated markets and tokens initialization using the denoms INI files in the SDK. Removed also the INI files from the SDK

## [1.7.2] - 2024-11-13
### Fixed
- Fixed link to official `ofac.json` file
Expand Down
146 changes: 72 additions & 74 deletions pyinjective/async_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,8 +157,8 @@ def __init__(
self._initialize_timeout_height_sync_task()

self._tokens_and_markets_initialization_lock = asyncio.Lock()
self._tokens_by_denom: Optional[Dict[str, Token]] = None
self._tokens_by_symbol: Optional[Dict[str, Token]] = None
self._tokens_by_denom = dict()
self._tokens_by_symbol = dict()
self._spot_markets: Optional[Dict[str, SpotMarket]] = None
self._derivative_markets: Optional[Dict[str, DerivativeMarket]] = None
self._binary_option_markets: Optional[Dict[str, BinaryOptionMarket]] = None
Expand Down Expand Up @@ -3291,87 +3291,81 @@ async def _initialize_tokens_and_markets(self):
derivative_markets = dict()
binary_option_markets = dict()
tokens_by_symbol, tokens_by_denom = await self._tokens_from_official_lists(network=self.network)
markets_info = (await self.fetch_spot_markets(market_statuses=["active"]))["markets"]
valid_markets = (
market_info
for market_info in markets_info
if len(market_info.get("baseTokenMeta", {}).get("symbol", "")) > 0
and len(market_info.get("quoteTokenMeta", {}).get("symbol", "")) > 0
)

for market_info in valid_markets:
base_token = self._token_representation(
token_meta=market_info["baseTokenMeta"],
denom=market_info["baseDenom"],
tokens_by_denom=tokens_by_denom,
tokens_by_symbol=tokens_by_symbol,
)
quote_token = self._token_representation(
token_meta=market_info["quoteTokenMeta"],
denom=market_info["quoteDenom"],
tokens_by_denom=tokens_by_denom,
tokens_by_symbol=tokens_by_symbol,
)
self._tokens_by_denom.update(tokens_by_denom)
self._tokens_by_symbol.update(tokens_by_symbol)

markets_info = (await self.fetch_chain_spot_markets(status="Active"))["markets"]
for market_info in markets_info:
base_token = self._tokens_by_denom.get(market_info["baseDenom"])
quote_token = self._tokens_by_denom.get(market_info["quoteDenom"])

market = SpotMarket(
id=market_info["marketId"],
status=market_info["marketStatus"],
status=market_info["status"],
ticker=market_info["ticker"],
base_token=base_token,
quote_token=quote_token,
maker_fee_rate=Decimal(market_info["makerFeeRate"]),
taker_fee_rate=Decimal(market_info["takerFeeRate"]),
service_provider_fee=Decimal(market_info["serviceProviderFee"]),
min_price_tick_size=Decimal(market_info["minPriceTickSize"]),
min_quantity_tick_size=Decimal(market_info["minQuantityTickSize"]),
min_notional=Decimal(market_info["minNotional"]),
maker_fee_rate=Token.convert_value_from_extended_decimal_format(Decimal(market_info["makerFeeRate"])),
taker_fee_rate=Token.convert_value_from_extended_decimal_format(Decimal(market_info["takerFeeRate"])),
service_provider_fee=Token.convert_value_from_extended_decimal_format(
Decimal(market_info["relayerFeeShareRate"])
),
min_price_tick_size=Token.convert_value_from_extended_decimal_format(
Decimal(market_info["minPriceTickSize"])
),
min_quantity_tick_size=Token.convert_value_from_extended_decimal_format(
Decimal(market_info["minQuantityTickSize"])
),
min_notional=Token.convert_value_from_extended_decimal_format(Decimal(market_info["minNotional"])),
)

spot_markets[market.id] = market

markets_info = (await self.fetch_derivative_markets(market_statuses=["active"]))["markets"]
valid_markets = (
market_info
for market_info in markets_info
if len(market_info.get("quoteTokenMeta", {}).get("symbol", "")) > 0
)

for market_info in valid_markets:
quote_token = self._token_representation(
token_meta=market_info["quoteTokenMeta"],
denom=market_info["quoteDenom"],
tokens_by_denom=tokens_by_denom,
tokens_by_symbol=tokens_by_symbol,
)

market = DerivativeMarket(
id=market_info["marketId"],
status=market_info["marketStatus"],
ticker=market_info["ticker"],
oracle_base=market_info["oracleBase"],
oracle_quote=market_info["oracleQuote"],
oracle_type=market_info["oracleType"],
oracle_scale_factor=market_info["oracleScaleFactor"],
initial_margin_ratio=Decimal(market_info["initialMarginRatio"]),
maintenance_margin_ratio=Decimal(market_info["maintenanceMarginRatio"]),
markets_info = (await self.fetch_chain_derivative_markets(status="Active", with_mid_price_and_tob=False))[
"markets"
]
for market_info in markets_info:
market = market_info["market"]
quote_token = self._tokens_by_denom.get(market["quoteDenom"])

derivative_market = DerivativeMarket(
id=market["marketId"],
status=market["status"],
ticker=market["ticker"],
oracle_base=market["oracleBase"],
oracle_quote=market["oracleQuote"],
oracle_type=market["oracleType"],
oracle_scale_factor=market["oracleScaleFactor"],
initial_margin_ratio=Token.convert_value_from_extended_decimal_format(
Decimal(market["initialMarginRatio"])
),
maintenance_margin_ratio=Token.convert_value_from_extended_decimal_format(
Decimal(market["maintenanceMarginRatio"])
),
quote_token=quote_token,
maker_fee_rate=Decimal(market_info["makerFeeRate"]),
taker_fee_rate=Decimal(market_info["takerFeeRate"]),
service_provider_fee=Decimal(market_info["serviceProviderFee"]),
min_price_tick_size=Decimal(market_info["minPriceTickSize"]),
min_quantity_tick_size=Decimal(market_info["minQuantityTickSize"]),
min_notional=Decimal(market_info["minNotional"]),
maker_fee_rate=Token.convert_value_from_extended_decimal_format(Decimal(market["makerFeeRate"])),
taker_fee_rate=Token.convert_value_from_extended_decimal_format(Decimal(market["takerFeeRate"])),
service_provider_fee=Token.convert_value_from_extended_decimal_format(
Decimal(market["relayerFeeShareRate"])
),
min_price_tick_size=Token.convert_value_from_extended_decimal_format(
Decimal(market["minPriceTickSize"])
),
min_quantity_tick_size=Token.convert_value_from_extended_decimal_format(
Decimal(market["minQuantityTickSize"])
),
min_notional=Token.convert_value_from_extended_decimal_format(Decimal(market["minNotional"])),
)

derivative_markets[market.id] = market
derivative_markets[derivative_market.id] = derivative_market

markets_info = (await self.fetch_binary_options_markets(market_status="active"))["markets"]
markets_info = (await self.fetch_chain_binary_options_markets(status="Active"))["markets"]
for market_info in markets_info:
quote_token = tokens_by_denom.get(market_info["quoteDenom"], None)
quote_token = self._tokens_by_denom.get(market_info["quoteDenom"])

market = BinaryOptionMarket(
id=market_info["marketId"],
status=market_info["marketStatus"],
status=market_info["status"],
ticker=market_info["ticker"],
oracle_symbol=market_info["oracleSymbol"],
oracle_provider=market_info["oracleProvider"],
Expand All @@ -3380,21 +3374,25 @@ async def _initialize_tokens_and_markets(self):
expiration_timestamp=market_info["expirationTimestamp"],
settlement_timestamp=market_info["settlementTimestamp"],
quote_token=quote_token,
maker_fee_rate=Decimal(market_info["makerFeeRate"]),
taker_fee_rate=Decimal(market_info["takerFeeRate"]),
service_provider_fee=Decimal(market_info["serviceProviderFee"]),
min_price_tick_size=Decimal(market_info["minPriceTickSize"]),
min_quantity_tick_size=Decimal(market_info["minQuantityTickSize"]),
min_notional=Decimal(market_info["minNotional"]),
maker_fee_rate=Token.convert_value_from_extended_decimal_format(Decimal(market_info["makerFeeRate"])),
taker_fee_rate=Token.convert_value_from_extended_decimal_format(Decimal(market_info["takerFeeRate"])),
service_provider_fee=Token.convert_value_from_extended_decimal_format(
Decimal(market_info["relayerFeeShareRate"])
),
min_price_tick_size=Token.convert_value_from_extended_decimal_format(
Decimal(market_info["minPriceTickSize"])
),
min_quantity_tick_size=Token.convert_value_from_extended_decimal_format(
Decimal(market_info["minQuantityTickSize"])
),
min_notional=Token.convert_value_from_extended_decimal_format(Decimal(market_info["minNotional"])),
settlement_price=None
if market_info["settlementPrice"] == ""
else Decimal(market_info["settlementPrice"]),
else Token.convert_value_from_extended_decimal_format(Decimal(market_info["settlementPrice"])),
)

binary_option_markets[market.id] = market

self._tokens_by_denom = tokens_by_denom
self._tokens_by_symbol = tokens_by_symbol
self._spot_markets = spot_markets
self._derivative_markets = derivative_markets
self._binary_option_markets = binary_option_markets
Expand Down
103 changes: 5 additions & 98 deletions pyinjective/composer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
import json
from configparser import ConfigParser
from decimal import Decimal
from time import time
from typing import Any, Dict, List, Optional
from warnings import warn

from google.protobuf import any_pb2, json_format, timestamp_pb2

from pyinjective import constant
from pyinjective.constant import ADDITIONAL_CHAIN_FORMAT_DECIMALS, INJ_DECIMALS, INJ_DENOM
from pyinjective.core.market import BinaryOptionMarket, DerivativeMarket, SpotMarket
from pyinjective.core.token import Token
Expand Down Expand Up @@ -137,17 +135,11 @@ def __init__(

"""
self.network = network
self.spot_markets = dict()
self.derivative_markets = dict()
self.binary_option_markets = dict()
self.tokens = dict()
if spot_markets is None or derivative_markets is None or binary_option_markets is None or tokens is None:
self._initialize_markets_and_tokens_from_files()
else:
self.spot_markets = spot_markets
self.derivative_markets = derivative_markets
self.binary_option_markets = binary_option_markets
self.tokens = tokens
self.spot_markets = spot_markets or dict()
self.derivative_markets = derivative_markets or dict()
self.binary_option_markets = binary_option_markets or dict()
self.tokens = tokens or dict()

self._ofac_checker = OfacChecker()

def Coin(self, amount: int, denom: str):
Expand Down Expand Up @@ -2556,91 +2548,6 @@ def unpack_transaction_messages(transaction_data: Dict[str, Any]) -> List[Dict[s

return msgs

def _initialize_markets_and_tokens_from_files(self):
config: ConfigParser = constant.CONFIGS[self.network]
spot_markets = dict()
derivative_markets = dict()
tokens = dict()

for section_name, configuration_section in config.items():
if section_name.startswith("0x"):
description = configuration_section["description"]

quote_token = Token(
name="",
symbol="",
denom="",
address="",
decimals=int(configuration_section["quote"]),
logo="",
updated=-1,
)

if "Spot" in description:
base_token = Token(
name="",
symbol="",
denom="",
address="",
decimals=int(configuration_section["base"]),
logo="",
updated=-1,
)

market = SpotMarket(
id=section_name,
status="",
ticker=description,
base_token=base_token,
quote_token=quote_token,
maker_fee_rate=None,
taker_fee_rate=None,
service_provider_fee=None,
min_price_tick_size=Decimal(str(configuration_section["min_price_tick_size"])),
min_quantity_tick_size=Decimal(str(configuration_section["min_quantity_tick_size"])),
min_notional=Decimal(str(configuration_section.get("min_notional", "0"))),
)
spot_markets[market.id] = market
else:
market = DerivativeMarket(
id=section_name,
status="",
ticker=description,
oracle_base="",
oracle_quote="",
oracle_type="",
oracle_scale_factor=1,
initial_margin_ratio=None,
maintenance_margin_ratio=None,
quote_token=quote_token,
maker_fee_rate=None,
taker_fee_rate=None,
service_provider_fee=None,
min_price_tick_size=Decimal(str(configuration_section["min_price_tick_size"])),
min_quantity_tick_size=Decimal(str(configuration_section["min_quantity_tick_size"])),
min_notional=Decimal(str(configuration_section.get("min_notional", "0"))),
)

derivative_markets[market.id] = market

elif section_name != "DEFAULT":
token = Token(
name=section_name,
symbol=section_name,
denom=configuration_section["peggy_denom"],
address="",
decimals=int(configuration_section["decimals"]),
logo="",
updated=-1,
)

tokens[token.symbol] = token

self.tokens = tokens
self.spot_markets = spot_markets
self.derivative_markets = derivative_markets
self.binary_option_markets = dict()

def _order_mask(self, is_conditional: bool, is_buy: bool, is_market_order: bool) -> int:
order_mask = 0

Expand Down
18 changes: 0 additions & 18 deletions pyinjective/constant.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,25 +1,7 @@
import os
from configparser import ConfigParser

GAS_PRICE = 160_000_000
GAS_FEE_BUFFER_AMOUNT = 25_000
MAX_MEMO_CHARACTERS = 256
ADDITIONAL_CHAIN_FORMAT_DECIMALS = 18
TICKER_TOKENS_SEPARATOR = "/"
INJ_DENOM = "inj"
INJ_DECIMALS = 18

devnet_config = ConfigParser()
devnet_config.read(os.path.join(os.path.dirname(__file__), "denoms_devnet.ini"))

testnet_config = ConfigParser()
testnet_config.read(os.path.join(os.path.dirname(__file__), "denoms_testnet.ini"))

mainnet_config = ConfigParser()
mainnet_config.read(os.path.join(os.path.dirname(__file__), "denoms_mainnet.ini"))

CONFIGS = {
"devnet": devnet_config,
"testnet": testnet_config,
"mainnet": mainnet_config,
}
10 changes: 10 additions & 0 deletions pyinjective/core/token.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
from dataclasses import dataclass
from decimal import Decimal

from pyinjective.constant import ADDITIONAL_CHAIN_FORMAT_DECIMALS


@dataclass(eq=True, frozen=True)
class Token:
Expand All @@ -12,5 +14,13 @@ class Token:
logo: str
updated: int

@staticmethod
def convert_value_to_extended_decimal_format(value: Decimal) -> Decimal:
Copy link
Contributor

@PavelInjective PavelInjective Nov 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for me these methods names seem too long since they're commonly used across the sdk

return value * Decimal(f"1e{ADDITIONAL_CHAIN_FORMAT_DECIMALS}")

@staticmethod
def convert_value_from_extended_decimal_format(value: Decimal) -> Decimal:
return value / Decimal(f"1e{ADDITIONAL_CHAIN_FORMAT_DECIMALS}")
Comment on lines +21 to +23
Copy link
Contributor

@coderabbitai coderabbitai bot Nov 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add documentation and optimize the implementation

  1. The method could benefit from documentation explaining how it differs from chain_formatted_value.
  2. Similar to the previous method, using Decimal.shift() would be more efficient.
    @staticmethod
    def convert_value_from_extended_decimal_format(value: Decimal) -> Decimal:
+        """
+        Converts a value from extended decimal format by removing additional chain format decimals.
+        This differs from chain_formatted_value which handles token-specific decimal places.
+        """
-        return value / Decimal(f"1e{ADDITIONAL_CHAIN_FORMAT_DECIMALS}")
+        return value.shift(-ADDITIONAL_CHAIN_FORMAT_DECIMALS)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
@staticmethod
def convert_value_from_extended_decimal_format(value: Decimal) -> Decimal:
return value / Decimal(f"1e{ADDITIONAL_CHAIN_FORMAT_DECIMALS}")
@staticmethod
def convert_value_from_extended_decimal_format(value: Decimal) -> Decimal:
"""
Converts a value from extended decimal format by removing additional chain format decimals.
This differs from chain_formatted_value which handles token-specific decimal places.
"""
return value.shift(-ADDITIONAL_CHAIN_FORMAT_DECIMALS)


def chain_formatted_value(self, human_readable_value: Decimal) -> Decimal:
return human_readable_value * Decimal(f"1e{self.decimals}")
Loading