Rate Limits for the JSON API

[Lachele]

We are thrilled that folks are using our [JSON API](https://github.com/GLYCAM-Web/website/tree/master/Examples/JsonApi)! 🎉

In fact, y'all are so good at it that you occasionally cause an [unintentional DOS](https://en.wikipedia.org/wiki/Denial-of-service_attack#Unintentional_denial-of-service)!

Current rate limits:

The previous rate limits were not sufficient. They are now:

• POST method: One per 20 seconds; burst of 5.
• GET method: One per 5 seconds, burst of 5.

Information below here is explanatory

The rate-limit numbers might not be up to date, but the other information should be good. Let us know if we need to make an update!

To help keep our website up and usable by all, we are in the process of implementing these mitigations:

• Rate limits on JSON API usage. Specifically:
  • Starting a new action, anything that sends a POST method to the /json/ endpoint (building a new sequence, for example), will be limited to one action every 20 seconds.
  • Checking on the status of an existing action, generally involving a GET method, will be limited to one per 5 seconds.
  • These limits will apply to the IP address of the user.
    • If multiple people share an IP address, their accesses will be limited collectively.
• Updated scripts that will help you to automatically stay within the limits.
• Associated repositories containing instructions for building and running the jobs on your local machines.
  • The first of these will support the generation of Docker images.
  • Let us know what other methods you want!

Please reply to this post with questions and comments.

Timing for implementation

Rate limits

I am actively working on the rate limits. I'm definitely more chemist than sysadmin, but I'm getting there.

Expect these within days.

Updated scripts

Expect these by the first of the year.

Docker support

The first release should happen by December 1.

[Lachele]

To stay within the limits, just add a sleep command to your code.

For bash, a 10-second sleep is:

sleep 10

Just put a 10 (or more) second sleep after any "POST" method and a 1 (or more) second sleep after all others.

The rate limits should be in place by Friday and certainly by Monday.