scrapit is a simple Python script designed to scrape content from a provided URL and save it in an output/ directory. After scraping, it commits and pushes the content to a Git repository.
Clone this repository:
git clone https://github.com/your_username/scrapit.git
Navigate to the cloned directory:
cd scrapit
Install the required Python packages:
pip install -r requirements.txt
To scrape a webpage:
python scrapit.py <URL>
Replace with the target webpage you want to scrape.
If you wish to use this script for your own purposes and perhaps target a specific webpage regularly:
- Fork this repository.
- Rename the forked repository to match the name of your target webpage or any other name you prefer.
- Clone your forked and renamed repository to your local machine.
- Follow the installation steps above.
- Don't forget to take out the
output/from the.gitignoreso that your scrapped web page is being commited.
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
MIT