AI Scraping Pipeline & Lead CRM Platform
Overview
An end-to-end pipeline that turns Germany's official company register into a working book of investor leads. A containerised, VPN-rotated Selenium fleet searches the register for each firm, opens that firm's document directory and downloads every filing it holds; each document is then stored in AWS S3 with a reference back to the firm it belongs to. LLMs then read those filings and extract who the investors in that firm are.
The extracted leads flow into a purpose-built lead management platform where the sales team works them. Each lead carries its own record of the conversations held with it, and anyone who accepts the offer to sell their holding moves through the platform's pipeline stages toward a completed sale. The database underneath — schema design, development and ongoing management — was built as part of the project.
Outcome
Firm filings become a live book of investor leads without anyone opening the register by hand — each lead carrying its own conversation history and moving through pipeline stages to an agreed share sale.
Screens & Demos
Architecture & Pipeline
-
01
Scheduler
Regular runs
-
02
German Company Register
Official filings
-
03
Selenium Fleet
Docker · VPN-rotated · per-firm directories
- Docker
- Selenium
-
04
AWS S3
PDFs stored with firm references
- AWS
-
05
LLM Extraction
Qwen 3.6 Flash · GPT 5.6 Luna
- LLM
-
06
Deduplicate & Validate
Investor records
-
07
Lead Platform
Conversations · pipeline stages
End-to-end flow, in order. Scroll sideways on a narrow screen.
Key Features
- Containerised Selenium fleet with VPN rotation, searching the register firm by firm and downloading every filing in each company's document directory
- Every document stored in AWS S3 with a reference back to its firm
- LLM extraction (Qwen 3.6 Flash, GPT 5.6 Luna) pulling investor details out of the filings
- Deduplication and validation before a lead reaches the platform
- Lead management dashboard holding each lead's details and full conversation history
- Pipeline stages tracking a lead from first contact through to an agreed share sale
- Database schema designed, developed and managed end to end
- Scheduled runs so newly filed documents are picked up automatically