All projects · AI Agents & RAG
E-Commerce Product Scraper Studio (Python & Playwright)
An autonomous e-commerce product catalog web scraper built with Python and Playwright, featuring dynamic DOM hydration waiting, OneTrust cookie…
An autonomous e-commerce product catalog web scraper built with Python and Playwright, featuring dynamic DOM hydration waiting, OneTrust cookie consent bypass, cascading CSS/XPath selectors, terminal execution replaying, and Excel-safe UTF-8 BOM CSV export.
- Autonomous Playwright Chromium scraper extracting Product Title, Price, Review Count, and canonical Product URL
- Handles dynamic cookie consent dialogs (#onetrust-accept-btn-handler) before executing element queries
- Resilient multi-tier selector cascades for Titles, Prices, and Ratings preventing scraper failures
- Exports clean dataset with UTF-8 BOM encoding (utf-8-sig) to eliminate Microsoft Excel currency symbol mojibake
- Interactive dual-studio matching reference challenge screenshot with live MyProtein catalog and terminal log replayer
- Includes production Dockerfile, local setup guides, and Linux VM dependency automation (playwright install-deps)
- Deconstructs the challenge clue 'Use AI to write it' with naive vs master prompt engineering blueprints
- Built-in 8-point automated in-browser diagnostic test suite verifying DOM structure, field parsing, and CSV compliance
Built with: Python, Playwright, Headless Chromium, Web Scraping, E-Commerce, CSV Export, Anti-Bot Heuristics, Docker