feat:implement geeksforgeeks webscraper

[ashish-066]

contributors have to implement tgeeksforgeeks webscraper that does all the things like the leetcode api or codeforces api.as geeks for geeks doesnt provide a public api.Your task is to create a scraper that scrapes

-user stats
-skills

[Louser21]

@ashish-066 assign me this

[ashish-066]

@Louser21 assigned

[ashish-066]

make sure to give a brief on your understanding or plan for this task.so we can make sure you dont go into wrong path

[Louser21]

Since there is no official GeeksforGeeks API, the solution is to directly scrape data from the GeeksforGeeks website.
The scraper will open a user’s public profile page and extract the required information from the HTML itself.

We can use:

axios/fetch → to request the webpage

cheerio → to parse the HTML and extract data

The scraper will:

1. Take the username as input.

2. Visit the corresponding GFG profile URL.

3. Read the HTML content of the page.

4. Extract:

user stats

solved problems

coding score

streaks

skills/topics

5. Convert the extracted data into structured JSON.

6. Return the data through backend routes.

If some data is dynamically rendered and not present in raw HTML, we can use puppeteer to launch a headless browser and scrape the rendered page.

The overall goal is to simulate an API service for GeeksforGeeks using web scraping.

[ashish-066]

it is good to go but puppeteer may take more ram right.if there are many geeksforgeeks user.we can face a server crash also.do you have any better solutions for that?

[Louser21]

i think mostly we dont need puppeteer its only if the server renders the data step by step in pages instead of statically

i will go through other powerful and faster alternatives as well

[ashish-066]

@Louser21 any progress?

[Louser21]

The scrapper i coded is fetching only some users
It occured due toh structure of Gfg I need to adjust for that

[Louser21]

Can I submit it by Monday as I have a 48 hr hackathon starting tonight

[ashish-066]

you can take time.just checking in if you are facing any problems s