AutoUncle CoDriver Crawler

Purpose

AutoUncle CoDriver is an AI assistant for automotive dealerships. The crawler fetches public pages from dealership websites when a dealer configures AutoUncle CoDriver or when AutoUncle CoDriver needs to understand a dealer-provided vehicle URL.

• Discovering dealership locations, departments, contact information, and opening hours.
• Finding stable business documents such as warranty, financing, trade-in, or test drive information.
• Reading dealer-provided vehicle detail pages to understand stock or reference identifiers.

Scope and Limits

AutoUncle CoDriver is not a public search engine crawler and does not crawl pages for AI model training, content resale, competitive price scraping, or public search indexing.

• Crawls are initiated by dealership configuration or by URLs supplied to AutoUncle CoDriver.
• Organization setup crawls are limited to a small number of public internal pages from the dealer website.
• The crawler does not log in, submit forms, access private customer areas, or attempt credentialed browsing.
• Pages identified as sensitive, irrelevant, or authentication-only are skipped by design.

Request Behavior

• Requests are made with a headless Chromium browser so JavaScript-rendered dealer pages can be read accurately.
• Requests are limited and event-driven, not a continuous broad crawl.
• Fetched HTML and rendered text may be cached in AutoUncle CoDriver to avoid repeated requests for the same URL.
• AutoUncle CoDriver stores crawled content only to provide dealer account setup, lead processing, and enrichment features.

Crawler IPs

AutoUncle CoDriver publishes a machine-readable crawler IP list at /.well-known/codriver-crawler-ips.json. This endpoint is intended for bot verification services such as Cloudflare.

The list contains the fixed production app and worker server egress IPs used by standard non-proxy crawler fetches. Shared fallback proxy pools are intentionally excluded because they are not a fixed AutoUncle CoDriver-owned allowlist for verified bot validation.