"0 percent complete" forever

[nuzillard]

Dear James,

I am facing a new problem now.
After a successful trial of client.sdf_query() with a batch of 1000 molecules in about 1 minute,
running a second batch of 1000 molecules resulted after about 15 minutes (still running...)
in many "0 percent complete" lines (about 1 per minute).
Is this a problem from the server side?
Is there some usage quota?

Best regards,

Jean-Marc

[nuzillard]

About issue #1, the resulting file was empty.

Trying now with a series of 1 molecule batches works, even if the reply from the server
includes an "alpha". This is not a practical situation, considering the number
of molecules to be classified.

Best,

Jean-Marc

[JamesJeffryes]

Well the file does get written before the results are returned so an interrupted query will result in an empty file.

There is a usage quota but I’m not familiar with any of its details as I was not involved with the creation of ClassyFire. I suspect it has something to do with the number of requests in time period not the size of the requests. If you hit the cap you should see an exception caused by the 429 HTTP response.

Additionally it is possible to change the chunk size for SDF queries. The default chunk size is 1000 which means that sdf files with more than 1000 molecules get split up into simultaneous requests. You can set this like so client.chunk_size = 500 before executing your sdf query to split the sdf file into two requests then recombine the results. However, reducing this size very much will likely result in you hitting the request quota.

[nuzillard]

Many thanks for this detailed answer.
I will ask for how request quota is managed.

Best,

Jean-Marc

[JamesJeffryes]

One more thing, it would not be too difficult to make the sdf request function just a touch more sophisticated to limit the parallel requests to whatever the quota. When I wrote this, I don't think one existed.

[nuzillard]

Do you mean that setting chunk_size = 10000 would be OK for a file with a bit less than 10000 molecules, or does this limit of 1000 was set by ClassyFire?

[JamesJeffryes]

Looks like there's a 3MB limit on uploaded file size per the website

[nuzillard]

Finally, I submitted batches of 500 structures, leaving 10 seconds of sleeping time in between and everything was OK. I only had a problem with the structure in file
buffer.zip
which resulted in Error 500, Internal Server, but I used directly the ClassyFire website to obtain the classification for that one.

Many thanks again for the help.

Jean-Marc

[nuzillard]

From David Arndt, who answered my questions about ClassyFire and quotas:
"We have limits of 12 requests per minute when submitting, and 10 requests per second for other operations (like GET). We may relax those eventually if we think the server can handle it."