Skip to content

PRO100CHOK/apartments-com-listings-scraper-python

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Apartments.com Listings Scraper — Python Example

Scrape Apartments.com rental listings, full property details (90+ fields), and property-management contact emails in Python — no Akamai bypass code to write yourself.

Apify Actor Python 3.10+ License: MIT

Production-ready Python project that turns Apartments.com — the largest US multifamily rental marketplace — into a programmable feed via the Apartments.com Scraper Apify actor. Three chained actions: discover listings from any search-results URL, pull every detail field for each listing (rent ranges, beds, baths, sqft, walk score, transit score, amenities, pet policy, reviews, similar properties), then extract contact emails and phone numbers from the property-management company's own website.

What this does

Apartments.com sits behind Akamai Bot Manager — the same anti-scraping system used by major airlines and banks. Generic Python scrapers (requests + BeautifulSoup) get blocked within minutes, and headless-browser approaches need session warming and fingerprint randomization to last. This project uses an Apify actor that handles the Akamai bypass via residential US IPs, manages session rotation, paginates the search results, and parses Apartments.com's own internal JSON endpoints into a clean schema with 90+ fields per property.

Use cases

  • Renter search agents — track new listings under your filter criteria daily and email yourself fresh matches.
  • Real-estate market research — compare rent-per-square-foot across cities or submarkets for investment models.
  • Multifamily competitive intelligence — monitor competitors' published rents, concessions, and amenity lists.
  • Property-management lead gen — pull contact emails and phones from property-management company websites for B2B outreach.
  • Rental fraud detection — cross-reference suspicious listings against the Apartments.com canonical record.
  • Academic / journalism research — build a longitudinal dataset of US rental market prices for housing-policy studies.

Requirements

  • Python 3.10+
  • A free Apify account
  • No Apartments.com login required

Quick start

git clone https://github.com/pro100chok/apartments-com-listings-scraper-python.git
cd apartments-com-listings-scraper-python
pip install -r requirements.txt
cp .env.example .env
# paste your APIFY_API_TOKEN
python main.py

main.py runs the full two-step pipeline: searches Brooklyn, NY listings, then pulls full detail records for the first 15 hits — names, rent ranges, walk scores. Edit SEARCH_URL to point at any apartments.com search-results page (the URL can include all the site's built-in filters: price range, beds, pets, etc.).

Three actions

action Inputs Output
search startUrls (any apartments.com search URL), maxPages One record per listing on the search results — basic info + listing URL.
details listingUrls or datasetId from a previous search run Full 90+ field record per listing: rent, beds/baths/sqft, scores, amenities, pet policy, reviews, photos.
emails websiteUrls or datasetId Visits each property-website URL and extracts emails, phones, social links.

The three actions chain naturally — pass the previous run's datasetId instead of re-listing URLs:

search = client.actor("...").call(run_input={"action": "search", "startUrls": [...], "maxPages": 3})
details = client.actor("...").call(run_input={"action": "details", "datasetId": search["defaultDatasetId"]})
emails = client.actor("...").call(run_input={"action": "emails", "datasetId": details["defaultDatasetId"]})

How it works

  1. Your Python script calls the actor with action: "search" and an apartments.com search URL.
  2. The actor sends the request through Apify's residential US proxy pool, handles the Akamai challenge, and walks through the paginated result list. Each listing is pushed to the run's dataset.
  3. For each listing you care about, call the actor again with action: "details" to get the full 90+ field property record from apartments.com's internal listing endpoint.
  4. Optionally, run action: "emails" on each listing's websiteUrl to extract contact info from the property-management company's own website.

The actor's session warming and IP rotation are tuned for apartments.com specifically — block rates stay below 1% in normal operation. Higher concurrency works (up to 18 parallel sessions) but pushes the block rate up.

Example: detailed listing extraction

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_API_TOKEN"])

run = client.actor("pro100chok/apartments-scraper-usage").call(run_input={
    "action": "details",
    "listingUrls": [
        {"url": "https://www.apartments.com/the-max-new-york-ny/abc123/"}
    ],
})

for it in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(it["name"])
    print(f"  Rent: ${it['rent']['min']}–${it['rent']['max']}")
    print(f"  Walk score: {it['scores']['walk']}")
    print(f"  Pets allowed: {it['petsAllowed']}")

Example output (details action, abbreviated)

{
  "name": "The Max",
  "url": "https://www.apartments.com/the-max-new-york-ny/abc123/",
  "address": {
    "fullAddress": "123 Example St, Brooklyn, NY 11201",
    "city": "Brooklyn", "state": "NY", "zip": "11201",
    "lat": 40.6975, "lng": -73.9961
  },
  "rent": { "min": 2850, "max": 4200 },
  "beds": { "min": 1, "max": 3 },
  "baths": { "min": 1, "max": 2 },
  "sqft": { "min": 580, "max": 1240 },
  "scores": { "walk": 92, "transit": 87, "bike": 88 },
  "petsAllowed": ["dogs", "cats"],
  "amenities": ["gym", "rooftop", "concierge", "in-unit laundry"],
  "websiteUrl": "https://themaxnyc.com",
  "phone": "+1-718-555-0123",
  "photos": ["https://images-...jpg", "..."]
}

Input parameters

Parameter Type Used by Description
action string all search, details, or emails.
startUrls object[] search apartments.com search URLs. Pre-build filters in the URL.
listingUrls object[] details Specific listing URLs to scrape full details for.
websiteUrls object[] emails Property-website URLs to extract contacts from.
datasetId string details / emails Re-use the previous run's dataset ID instead of supplying URLs.
maxPages integer search Max search-result pages per URL (~40 listings/page, max 18 pages). 0 = all.
maxItems integer details / emails Cap on listings to scrape / websites to visit. 0 = all.
concurrency integer all Parallel requests. Default 10.
proxyConfiguration object all Residential US recommended for search/details. Auto proxies work for emails.

More examples

File Demonstrates
examples/01_basic_usage.py Single-city search.
examples/02_filtered_search.py Apply apartments.com URL filters (beds, price, pets).
examples/03_property_emails.py Extract emails + phones from property-website URLs.
examples/04_export_to_csv.py Multi-city scrape + rent-per-sqft normalization.
examples/05_export_to_google_sheets.py Daily snapshot to a shared Sheet.

FAQ

How much does it cost? The actor is metered per item scraped — typically a few cents per 100 listings. The free Apify tier ($5/month) covers thousands of listings per month.

Is there a monthly flat-rate option? Yes — Apartments.com Scraper monthly at $20/month gives unlimited usage with identical functionality.

Do I need to handle Apartments.com's bot protection myself? No. The actor includes Akamai-bypass logic, session rotation, and IP-warming heuristics tuned specifically for apartments.com. You just pass URLs and receive parsed JSON.

What filters can I apply? Anything Apartments.com's own UI supports — just navigate to the filtered search page on apartments.com and copy its URL into startUrls. The URL encodes all filters (price range, beds, baths, pet-friendly, amenities, etc.).

Can I scrape detailed records for thousands of listings? Yes. details action paginates internally and the actor's residential proxy rotation keeps block rates near zero. For very large batches, set concurrency to 4–6 to be gentle on the IPs.

Does it return historical listings? No — apartments.com only exposes currently-listed properties. For longitudinal data, schedule daily / weekly snapshots and store them yourself.

How accurate is the rent data? Rents come directly from apartments.com's internal API — the same data their site shows. Note that some properties show "starting at" pricing instead of full ranges; the actor reports whatever apartments.com publishes.

Can I scrape outside the US? Apartments.com is US-only. For international rental data, use a country-specific source (Zillow, Rightmove, Idealista, etc.).

Related actors

See all my actors at apify.com/pro100chok.

License

MIT — see LICENSE.


Built on top of the Apartments.com Scraper Apify actor.