This Python script is a simple web scraper designed to crawl a given website, extract valid page links, and download images from those pages. It starts with a "mother" URL, finds additional pages using a regular expression that matches specific relative link patterns, and then downloads all images found on those pages into a designated folder.
- Web Request Handling: Uses the
requestslibrary to fetch page content. - HTML Parsing: Leverages
BeautifulSoupfrom thebs4library to parse HTML and extract links and image tags. - Link Extraction: Filters and collects valid page links from anchor tags using a regex pattern.
- Image Downloading: Downloads images from each page, saving them to a local directory (
downloaded_imagesby default). - Error Handling: Includes basic error handling to manage failed requests and download errors.
-
Clone the Repository:
Clone this repository to your local machine. -
Install Dependencies:
Ensure you have Python installed, then install the required libraries with:pip install requests beautifulsoup4
-
Configure the Script (Optional):
- The starting URL is set in the script with:
url = "https://www.ferreteriafurriols.cat/"
- Modify this URL if you want to target a different website.
- The starting URL is set in the script with:
-
Run the Script:
Execute the script from your terminal:python script_name.py
Replace
script_name.pywith the actual name of your Python file. -
Output:
The script will create a folder calleddownloaded_images(if it doesn't already exist) and save all downloaded images there. It also prints progress and any error messages to the console.
Feel free to modify and distribute this script under your preferred license.