Discovers NEW products from a Shopify store using hash-based change detection.
Batch 1b: This endpoint runs in parallel with Batch 1a (update-ai-site).
It’s completely decoupled - no scraped content needed.
What This Endpoint Does
- ✅ Fetch products from Shopify
/products.json
- ✅ Compare hash to detect changes
- ✅ Save NEW products to database
- ✅ Generate prompts for NEW products (calls
generate_prompts_for_product internally)
- ✅ Generate
/llms/{slug}.txt for NEW products (calls generate_product_llms_txt internally)
- ✅ Return files for combined deploy (when
skip_deploy=True)
How It Works
Hash-Based Change Detection
Instead of re-processing all products every time, we use efficient change detection:
- products_hash - MD5 hash of sorted product handles
- products_snapshot - Full product list from last sync
Hash comparison:
- If hash unchanged → skip entirely (no work needed)
- If hash changed → compare snapshots to find NEW products only
Request Body
| Field | Type | Required | Description |
|---|
business_id | string | Yes | Clerk org ID |
skip_deploy | boolean | No | If true, return files instead of deploying (default: false) |
Response
{
"status": "success",
"products_checked": 25,
"new_products": [
{
"name": "Product A",
"entity_id": "uuid-123"
}
],
"prompts_generated": 10,
"llms_files_deployed": 1,
"project_name": "business-ai-site",
"ai_site_url": "https://business.searchcompany.dev",
"source_url": "https://mystore.com",
"llms_files": [...] // Only if skip_deploy=true
}
Database Updates
| Table | Action |
|---|
entities | Insert new products with type: "product" |
prompts | Insert 10 prompts per new product |
ai_sites | Update products_hash and products_snapshot |
Database Columns Used (ai_sites)
| Column | Type | Purpose |
|---|
products_hash | TEXT | MD5 hash of sorted product handles for quick comparison |
products_snapshot | JSONB | Full product list from last sync [{handle, name, ...}] |
Code Location
src/app/apis/cron/discover_products/routes.py
src/app/shared/products/discover.py # Core Shopify product fetching logic
src/app/shared/products/generate_llms_txt.py # LLMs generation
Internal Services Called
This endpoint internally calls these shared services:
| Service | Purpose |
|---|
fetch_shopify_products | Fetch all products from Shopify API |
save_product | Save new product to entities table |
generate_prompts_for_product | Generate 10 visibility prompts |
generate_product_llms_txt | Generate /llms/{slug}.txt content |
Why all-in-one? For the cron job, we want a single endpoint that handles
the entire product discovery flow. The separate endpoints (generate-product-prompts,
generate-product-llms-txt) exist for manual triggering and debugging.
