Backtesting - Getting KeyError procesed data

[rapcmia]

Describe the bug

When checking github issues, found this behavior where the response from hummingbot-api's backtesting resulted to 200 OK but error on dashboard

KeyError: 'processed_data'

Traceback:
File "/home/dashboard/main.py", line 17, in <module>
    main()
File "/home/dashboard/main.py", line 13, in main
    pg.run()
File "/opt/conda/envs/dashboard/lib/python3.12/site-packages/streamlit/navigation/page.py", line 300, in run
    exec(code, module.__dict__)  # noqa: S102
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dashboard/frontend/pages/config/pmm_simple/app.py", line 30, in <module>
    bt_results = backtesting_section(inputs, backend_api_client)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/dashboard/frontend/components/backtesting.py", line 38, in backtesting_section
    if len(backtesting_results["processed_data"]) == 0:
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^

Steps to reproduce bug

1. Setup humminbot/deploy using ./setup.sh
2. Select PMM_simple and run backtest on hyperliquid HYPE-USDT or mexc MX-USDT
3. Observed hummingbot-api logs and dashboard page

[YanFengWang]

[Oscar00003]

Having exactly the same issue.

[masnail]

Same issue

[SiPust]

Errors seems to be in

/opt/conda/envs/hummingbot-api/lib/python3.12/site-packages/hummingbot/data_feed/candles_feed/mexc_spot_candles

now = self._round_timestamp_to_interval_multiple(self._time())
        max_duration = 500
        if (now - start_time) / self.interval_in_seconds >= max_duration:
            raise ValueError(
                f"{self.interval} candles are only available for the last {max_duration} bars from now.")

If I understand it correctly,

if (now - start_time) / self.interval_in_seconds >= max_duration:

always splits by seconds, and does not recognise used candle interval.

[mail6562]

I increased the memory and this problem no longer exists.

[Oscar00003]

@mail6562 please can you let me know how can I also do that considering I am a beginner? Any help would be appreciated. Thank you.

[rpraes]

Has anyone managed to resolve this issue?

[SiPust]

I don´t have this issue anymore. I am not sure if pulling the latest docker image, resolved this.
But I also recognized network issues. I am using pihole to block ads in my private network. This may also be something to check, if your firewall or ad blocker is blocking network traffic. I needed to allow access to mexc domain.
I hope this helps..

[fj11]

i have this problem when using kraken and coinbase_advanced_trade

[megusta-01]

something is wrong with api. I've no problems with kucoin
On hyperliquid you trade with USDC, but Portfolio is in USD

[lgrawet]

@rapcmia I am coding my own Ichimoku strategy I'm willing to share with the community and stumbled into this.

[RyanisAgile]

Seems that some of the connectors are excluded from the backtesting data provider in v2

class BacktestingDataProvider(MarketDataProvider):
    CONNECTOR_TYPES = [ConnectorType.CLOB_SPOT, ConnectorType.CLOB_PERP, ConnectorType.Exchange,
                       ConnectorType.Derivative]
    EXCLUDED_CONNECTORS = ["hyperliquid_perpetual", "dydx_perpetual", "cube", "vertex",
                           "coinbase_advanced_trade", "kraken", "dydx_v4_perpetual", "hitbtc",
                           "hyperliquid", "injective_v2_perpetual", "injective_v2"]

[himanshu24x77]

Hi, is this issue still open ? How to resolve this ?

[himanshu24x77]

@rapcmia hi, did you manage to resolve this issue?