birdrent_scraper

Tools for scraping rental listings from bird.rent and visualizing them on an interactive 3-D map of Kyiv, with metro lines and building footprints pulled from OpenStreetMap.

⚠️ Educational project. Scrapes a third-party site through its undocumented API. Be respectful of the source: keep request volume reasonable and don't redistribute the collected data.

What's here

File	Purpose
scraper.py	Fetches every building and its listings from the bird.rent API (multi-threaded) and writes bird_listings.json / bird_listings.csv.
fetch_map_data.py	Enriches the listings with Kyiv metro lines/stations and OSM building footprints (via the Overpass API) and writes map_data.js.
bird_rent_3d_map_with_metro_and_csv.html	Self-contained MapLibre GL map that renders the listings, footprints, and metro.
birdtest.py	Scratch script for probing a single listing endpoint.
bird_listings.{json,csv,xlsx}	Generated listing datasets.
map_data.js	Generated GeoJSON constants consumed by the HTML map.

Requirements

• Python 3.8+
• requests

pip install requests

Usage

1. Scrape the listings

python scraper.py

Walks the full building list from https://bird.rent/api/map/buildings, fetches each building's properties in parallel (WORKERS = 20, tune in scraper.py), and saves:

• bird_listings.json
• bird_listings.csv

Each listing record includes price, currency, rooms, area, floor, coordinates, contact, source URL, and timestamps. Region and auth cookie are configured at the top of the file (REGION, COOKIE).

2. Build the map data

python fetch_map_data.py

Reads bird_listings.json and queries the Overpass API (rotating across mirrors) for:

• Metro lines — subway route relations, stitched into continuous lines
• Metro stations — station nodes assigned to their nearest line
• Building footprints — OSM polygons matched to each building's coordinates
• Listing points — one point per building for low-zoom markers

Output is written to map_data.js. First run takes ~5–15 minutes because of batched Overpass queries over ~3700 buildings. Re-run whenever the listings or OSM data change. The Kyiv bounding box lives in BBOX near the top of the file.

3. View the map

Open bird_rent_3d_map_with_metro_and_csv.html in a browser. It loads bird_listings.json and map_data.js from the same directory, so serve the folder over HTTP if your browser blocks local fetch:

python -m http.server 8000
# then open http://localhost:8000/bird_rent_3d_map_with_metro_and_csv.html

The map shows extruded 3-D building footprints (height from OSM levels or floor count), metro lines/stations, and clickable popups with per-listing details.

Regenerating from scratch

python scraper.py          # → bird_listings.json / .csv
python fetch_map_data.py   # → map_data.js
# open the HTML map