How is the forum caching supposed to work?

[baccccccc]

I've scraped a certain forum thread, say, two days ago. Ended up having post with ID 14186154 in Last_Scraped_Forum_Posts.

yesterday another post was added to the thread, say ID 16241361.

I run CDL against this forum thread today. Did not download any new content. Last_Scraped_Forum_Posts still shows 14186154. Effectively, CDL ignores the existence of post 16241361.

I double-checked that forum scraping is deemed successful. I.e., the thread URL has not ended up in Scrape_Error_URLs for whatever reason. (It sometimes happens due to DDoS-guard.)

there are no errors in downloader.log (other than some 404s and 451s on some filehosts. None on the forum itself.)

I tried to reduce forum_cache_expire_after to PT1H. Reran CDL. Still nothing, and 14186154 is the last scraped post.

is this expected, perhaps because some forums have hardcoded rate limiting settings? Or I'm holding it wrong again?

[NTFSvolume]

@jbsparrow maybe the last page check is broken

[jbsparrow]

Sounds like it's broken. I'll take a look at it this evening.

[NTFSvolume]

Caching was completely removed for v8.8.0.

All the cache related options are still there but they do nothing. I new implementation of cache will be added in some future version