Wine-Searcher Pricing Scraper
Overview
A price-comparison bot built so the client's company could always sell wine at the lowest price on the market. It collects wine names, providers and prices from Wine-Searcher and works out the cheapest provider for every wine, refreshing every four hours so the client's own pricing can be set against what the market is actually charging that day. The scraper mimics human browsing behaviour to stay under bot detection.
Outcome
A standing view of the lowest price on the market for each wine, refreshed six times a day, so the company could keep undercutting it without anyone checking prices by hand.
Architecture & Pipeline
-
01
Scheduler (4 h)
Recurring runs
-
02
Wine-Searcher
Source site
-
03
Anti-Detection Browse
Human-like behavior
-
04
Per-Wine Cheapest Provider
Cross-provider compare
-
05
Spreadsheet Builder
Pandas
- Pandas
-
06
Email / Drive Delivery
Client report
End-to-end flow, in order. Scroll sideways on a narrow screen.
Key Features
- Cheapest provider identified for every wine, on every run
- Four-hourly refresh so pricing decisions are made on current market rates
- Comparison built specifically to keep the company's own price the lowest
- Techniques to avoid bot detection
- Automated delivery by email or file-sharing
- Structured spreadsheet output ready for analysis