Skip to main content

​
Internal Services: IndexNow + GSC Submission

After Step 2 of domain connection completes, a background orchestrator handles Google verification and search engine submission:
  1. Google Verification - Polls TXT record verification (10s × 12 = 2 min)
  2. IndexNow - Notifies Bing, Yandex, and other IndexNow-compatible engines
  3. Google Search Console - Submits sitemap (only if domain is verified)

​
Purpose

These tasks ensure search engines are immediately notified about the AI site after the proxy is connected. This is one of only TWO places where search engine submission happens:
  1. Here - One-time after domain connection (Step 2)
  2. BATCH 3 in daily cron - After new content is deployed

​
Why Poll Google Verification in Step 2?

The Google TXT record is added during Step 1 (alongside the SSL TXT record), but we verify it in Step 2 because:
  1. DNS Propagation: TXT records need time to propagate globally
  2. More Time: By Step 2, the record has had time to propagate
  3. Resilience: Polling handles slow DNS gracefully (up to 2 minutes)
  4. Non-blocking: User doesn’t wait - it runs in background

​
Function Signatures

async def _background_google_verify_and_submit(
    ai_site_id: str,
    entity_id: str,
    org_id: str,
    source_url: str,
    custom_domain: str,
    google_verification_token: str
)

async def _background_indexnow_submission(
    entity_id: str,
    org_id: str,
    source_url: str
)

async def _background_gsc_submission(
    ai_site_id: str,
    org_id: str,
    source_url: str
)

​
Parameters

​
Google Verify + Submit Orchestrator

ParameterTypeDescription
ai_site_idstrAI site database ID
entity_idstrDatabase entity ID
org_idstrClerk organization ID
source_urlstrCustomer’s source URL (e.g., “https://example.com”)
custom_domainstrThe domain to verify (e.g., “www.example.com”)
google_verification_tokenstrToken from Step 1’s start-google-verification

​
IndexNow

ParameterTypeDescription
entity_idstrDatabase entity ID
org_idstrClerk organization ID
source_urlstrCustomer’s source URL (e.g., “https://example.com”)

​
GSC

ParameterTypeDescription
ai_site_idstrAI site database ID
org_idstrClerk organization ID
source_urlstrCustomer’s source URL

​
URLs Submitted (IndexNow)

CategorySource
Sitemap URLsFetched from deployed AI site’s sitemap.xml
/sitemap.xmlAdded explicitly (not in sitemap itself)
Boosted pagesAll paths from boosted_pages.deployment_url
The function fetches sitemap.xml from the deployed AI site, parses all <loc> URLs, replaces the AI site domain with the customer’s source domain, and submits to IndexNow.

​
Execution Flow

​
Polling Details

SettingValue
Poll interval10 seconds
Max attempts12
Total timeout2 minutes
On successIndexNow + GSC
On failureIndexNow only

​
Dependencies

  • src/app/shared/google_search_console.py - verify_domain(), add_to_search_console(), submit_sitemap()
  • src/app/shared/indexnow.py - submit_urls_to_indexnow()

​
Code Location

src/app/apis/domain/mark_step_complete/routes.py