Write a scraper for each problem hosting site (codeforces, codechef, ...) which retrives problem count.

[trintlermint]

Write a scraper to retrieve problem count from each problem hosting site

Description

Create a scraper for each problem hosting site (codeforces, codechef, atCoder, leetcode, CSES, etc.) which retrieves the total number of available problems from each platform, at some point it needs to be able to retrieve problem count from each site, this way we can avoid it from being done manually . . .

We need to find out how to retrieve problem count from each website at such as a mass scale since we have an extremely high number of websites to solve problems from (wishing this to be extended one day to find all problems into one giant compilation of problem sets bla bla bla . . . (so original))

Requirements

• le scraper should support one platform and retrieve the current problem count via web scraping or official api if available.
• The script should handle common error cases (e.g., connectivity, site changes).
• Output should be a simple summary, e.g.:

  Codeforces: 8500
  CodeChef: 3650
  Leetcode: 2800
  ...
  

• Python/Golang for implementation. (or wasm if youre crazy like that)

Criteria

• At least one working scraper for a major site. (codeforces ive already found)
• solution can be executed from the shell and prints a summary.
• Each supported site is implemented as a separate module/class/function.
• If a site offers an official API, prefer using it over scraping.

---

Example

• Codeforces would work something like this (but this needs help to be able to make it compile acmsguru problems too):

#scraper.py
def get_codeforces_problem_count() -> int:
    try:
        url = "https://codeforces.com/api/problemset.problems"
        response = make_request(url)
        if not response:
            logger.error("failed to fetch api")
            return -1
        data = response.json()
        if data.get('status') == 'OK' and 'result' in data:
            problems = data['result'].get('problems', [])
            return len(problems)     
        logger.warning("unexpected")
        return -1    
    except Exception as e:
        logger.error(f"error scraping codefores: {str(e)}")
        return -1

---

ask to be assigned if want =) write a forked branch with your changes and show me or work with me (trintler or Trintler#4377) on discord