Skip to main content
Streaming chat endpoint for multi-business research. Uses OpenRouter (Cerebras) for fast inference with competitor context in the system prompt. The response is streamed via Server-Sent Events (SSE), allowing real-time token delivery to the frontend.

Request Body

Conversation History Format

System Prompt Caching

For better performance, the frontend should:
  1. On the first message, omit system_prompt and include timezone
  2. The backend returns a system_prompt event with the generated prompt
  3. On subsequent messages, pass back the cached system_prompt
This avoids rebuilding the system prompt (which queries the database) on every message.

Example Request

Response Format (SSE)

The response is a stream of Server-Sent Events. Each event contains a JSON payload:

System Prompt Event (first message only)

Sent on the first message to allow frontend caching:

Token Event

Sent for each generated token:

Search Event

Sent when the LLM requests a web search:

Search Results Event

Sent after search completes:

Done Event

Sent when generation completes:

Error Event

Sent on error:

Example Response Stream

Features

Competitor Context

Selected competitors are included in the system prompt, giving the LLM access to:
  • Company name and URL
  • Description
  • Scraped content (key information about the business)
When the LLM needs current information, it can request a web search. The search is performed via Exa API, and results are injected into the conversation.

Conversation History

Include previous messages to maintain context across multiple turns.

Frontend Integration (JavaScript)