This would allow sites that require JS to load properly to bootstrap the DOM before scraping. Edit: Started working on this. Here are some todos - [ ] figure out why last ~20 sites error out - [ ] move chrome user data directory to config directory - [ ] troubleshoot timeouts and viewport settings - [ ] figure out abstraction for supporting both `request` and `puppeteer`
This would allow sites that require JS to load properly to bootstrap the DOM before scraping.
Edit: Started working on this. Here are some todos
requestandpuppeteer