[Bug]: `MPRester` fails due to 403 on `/heartbeat` endpoint (Cloudflare bot detection)

[lan496]

Description

When using mp_api.client.MPRester, I encounter an error during initialization related to accessing the /heartbeat endpoint.

/usr/local/lib/python3.13/site-packages/mp_api/client/mprester.py:229: in __init__
    emmet_version = MPRester.get_emmet_version(self.endpoint)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/local/lib/python3.13/site-packages/mp_api/client/mprester.py:494: in get_emmet_version
    response = get(url=endpoint + "heartbeat").json()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Observed behavior

From reading the code, MPRester appears to access https://api.materialsproject.org/heartbeat without an API key in order to retrieve version information (database and emmet versions). However, this request consistently returns a 403 Forbidden, likely due to Cloudflare’s bot detection. For example: curl -i https://api.materialsproject.org/heartbeat also returns a 403 status.

Suggestion

I understand the operational challenges of running a large API service like the Materials Project. That said, it would be very helpful if the client could support a fallback behavior, for example:

• Gracefully skipping get_database_version / get_emmet_version when /heartbeat is unreachable
• Catching and ignoring 403 (or network) errors for version checks
• Optionally allowing users to disable version checks via a flag or environment variable

Related issue

This may be related to the following issue: #817

Version

mp-api 0.45.15

Which OS?

• MacOS
• Windows
• Linux

[tsmathis]

hi @lan496, over the last 24 h MP has been experiencing a botnet like attack (mp status) that has caused us to manually tighten things up to maintain performance for the majority of the user base.

When did you start seeing this issue?

[lan496]

Thank you for the quick update and the context.

I’m not entirely sure when the issue first started, but I can confirm that my CI using MPRester was still working successfully on Jan 26, 3:28 PM (GMT+9). After that point, it began failing with the 403 error on the /heartbeat endpoint.

Also, just as a side note, the MP status dashboard is very helpful. Thanks for providing and maintaining that 👍

[tsmathis]

Of course, and thanks for the time stamp.

If we have some useful post-mortem findings in the next few days we'll update, otherwise please bear with us while we try to maintain performance. Things should return to expected performance levels soon.

[tsmathis]

Hi @lan496, just wanted to let you know to try your CI workflows again, everything should be back to normal.

Let us know if there are any issues!

[lan496]

I’ve now confirmed that the /heartbeat endpoint has returned to normal. Thank you very much for the quick response and all the work on this.

Feel free to close this issue. Alternatively, if you think it would still make sense to improve client-side handling (e.g., emitting warnings instead of failing hard), I’m also happy for this issue to remain open for discussion.

[esoteric-ephemera]

Thanks @lan496, this should get fixed in #1053 - if the heartbeat page returns a 403, a warning linking to the status page will be emitted and the client should work as normal