Skip to main content
Internal Service β€” This is not an HTTP endpoint. It’s called directly by the generate-all orchestrator.

Purpose

Uses the Firecrawl Agent API to extract what a business does. This information is used to generate the AI website content (llms.txt, Q&A pages, data.json). Runs in GROUP 1a (parallel with all other GROUP 1 tasks).

Function Signature

Parameters

Returns

How It Works

Firecrawl Agent Prompt

The service sends this prompt to Firecrawl Agent:

Schema

Used By

This is separate from the Scrape Website service (GROUP 1b). Scraped pages are only used for markdown replica generation, not for LLM content.

Why Separate from Scraping?

Benefits:
  • Focused prompts: LLM content generation gets clean, structured business info
  • No token waste: Don’t send full page markdown to Gemini for llms.txt generation
  • Parallel execution: Both can run simultaneously since they’re independent

Code Location

Error Handling

Common errors:
  • Timeout - Firecrawl agent takes too long (max 3 minutes)
  • API error - Firecrawl API returns non-200 status
  • Missing API key - FIRECRAWL_API_KEY not configured
If this service fails, the orchestrator will not proceed with GROUP 2a (Create AI Website) since business_info is required.