BlackPrint Technologies · Sep 2026
Official Store-Locator Registry
Monthly Retail Footprint Scraping Framework
Overview
Built a monthly branch registry for Mexican retail chains that uses only official sources: it scrapes each brand's own store locator, normalizes locations to a single schema and diffs runs to detect openings and closings.
Highlights
- 01
Engineered a declarative adapter framework with six extraction strategies (JSON APIs, embedded JSON, sitemap/JSON-LD, HTML listings, grid sweeps and real-browser XHR capture) and shipped 90 brand adapters that yielded 20,805 accepted branch locations in the September 2026 run.
- 02
Enforced a single-source rule: no aggregators or map APIs as location sources, and geocoding only for addresses the brand itself published. The rule is backed by QA gates, frozen fixtures and 206 pytest tests.
- 03
Handled anti-bot store locators conservatively, using a real, headed Chrome profile that stops at any CAPTCHA challenge and never solves or retries it. Exported results to Parquet and GeoJSON with a per-batch progress board across 406 registered brands.
Stack
- Python
- httpx
- Playwright
- Pydantic
- Typer
- BeautifulSoup
- JSON-LD
- H3
- Shapely
- Parquet
- GeoJSON
- pytest
Working on something similar?
Tell me about the problem and the data behind it. I reply within 24–48 hours.
Discuss a project