Google Search Console (GSC) integration allows us to verify customer domains with Google and submit sitemaps for better indexing. This is critical for visibility in Google AI Overviews, as Google does not support IndexNow.
Why Google Search Console?
| Search Engine | Indexing Method | AI Product |
|---|
| Google | Search Console API | AI Overviews |
| Bing | IndexNow | Copilot |
| Yandex | IndexNow | N/A |
Architecture
We use a master account approach:
┌─────────────────────────────────────────────────────────────┐
│ admin_1@searchcompany.co │
│ (Master GSC Account) │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ customer1 │ │ customer2 │ │ customer3 │ ... │
│ │ .com │ │ .io │ │ .co │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │
│ Up to 1,000 sites per account │
└─────────────────────────────────────────────────────────────┘
- One Google account owns all verified customer domains
- Customers don’t need their own Google accounts
- We use OAuth2 with a refresh token for API access
- The refresh token never expires (as long as used every 6 months)
Limits
Each Google account can manage a maximum of 1,000 sites in Search Console.
When approaching this limit, you must add a new admin account (see below).
Current Setup
| Environment Variable | Description |
|---|
GOOGLE_CLIENT_ID_GSC | OAuth client ID from Google Cloud Console |
GOOGLE_CLIENT_SECRET_GSC | OAuth client secret |
GOOGLE_REFRESH_TOKEN_GSC_ADMIN_1 | Refresh token for admin_1@searchcompany.co |
Adding a New Admin Account
When you reach ~900 sites on admin_1, it’s time to add admin_2@searchcompany.co.
Step 1: Create the Google Account
- Create a new Google Workspace account:
admin_2@searchcompany.co
- Ensure it has access to Google Search Console
Step 2: Get OAuth Credentials
The same OAuth client (Client ID + Secret) can be used for multiple accounts. You only need a new refresh token.
- Go to OAuth Playground
- Click the gear icon (⚙️) → Check “Use your own OAuth credentials”
- Enter the existing
GOOGLE_CLIENT_ID_GSC and GOOGLE_CLIENT_SECRET_GSC
- Select scopes:
https://www.googleapis.com/auth/siteverificationhttps://www.googleapis.com/auth/webmasters
- Click “Authorize APIs”
- Sign in as
admin_2@searchcompany.co (not admin_1)
- Grant permissions
- Click “Exchange authorization code for tokens”
- Copy the Refresh Token
Step 3: Add Environment Variable
Add the new refresh token to your .env:
# Google Search Console OAuth
GOOGLE_CLIENT_ID_GSC=your-client-id.apps.googleusercontent.com
GOOGLE_CLIENT_SECRET_GSC=your-client-secret
GOOGLE_REFRESH_TOKEN_GSC_ADMIN_1=1//token-for-admin-1
GOOGLE_REFRESH_TOKEN_GSC_ADMIN_2=1//token-for-admin-2 # NEW
Step 4: Update the Code
Modify Backend/src/app/shared/google_search_console/client.py to support multiple accounts:
def _get_credentials(account: str = "ADMIN_1") -> tuple[str, str, str]:
"""Get OAuth credentials from environment."""
client_id = os.getenv("GOOGLE_CLIENT_ID_GSC")
client_secret = os.getenv("GOOGLE_CLIENT_SECRET_GSC")
refresh_token = os.getenv(f"GOOGLE_REFRESH_TOKEN_GSC_{account}")
if not all([client_id, client_secret, refresh_token]):
raise ValueError(f"Missing Google OAuth credentials for {account}")
return client_id, client_secret, refresh_token
Step 5: Implement Account Selection Logic
Add logic to select which account to use based on current site counts:
async def get_account_for_new_site() -> str:
"""Determine which admin account to use for a new site."""
# Check site counts for each account
admin_1_count = await count_sites_for_account("ADMIN_1")
if admin_1_count < 950:
return "ADMIN_1"
admin_2_count = await count_sites_for_account("ADMIN_2")
if admin_2_count < 950:
return "ADMIN_2"
# Add more accounts as needed
raise Exception("All GSC accounts at capacity")
Step 6: Store Account Assignment
Add a column to ai_sites table to track which account owns each site:
ALTER TABLE ai_sites ADD COLUMN google_admin_account TEXT DEFAULT 'ADMIN_1';
Token Refresh Behavior
The refresh token stays valid as long as:
- Used at least once every 6 months - Our daily cron (sitemap resubmission) handles this automatically
- User doesn’t revoke access - Only if someone logs into the Google account and revokes app access
- Under 50 refresh tokens per account - We only have 1 per account, so no issue
If a refresh token is revoked or expires, you’ll need to repeat the OAuth
Playground flow to get a new one.
Endpoints
| Endpoint | Purpose |
|---|
| Start Google Verification | Get TXT verification token from Google |
| Complete Google Verification | Verify domain + add to Search Console + submit sitemap |
| Resubmit Sitemap | Resubmit sitemap after new boosted pages (cron) |
Flow During Domain Connection
Google verification runs in a two-step process:
Step 1: Get TXT Records (Frontend)
User clicks "Start Secure Connection"
│
├──────────────────────────────────────┐
│ │
▼ ▼
/start-certificate /start-google-verification
(Let's Encrypt TXT) (Google TXT)
│ │
└──────────────────────────────────────┘
│
▼
Entri shows BOTH
TXT records to add
│
▼
User adds via Entri
│
▼
/complete-certificate
(Issue + attach cert)
Step 2: Verify Google + Submit (Backend)
User clicks "Point Domain" (switches CNAME)
│
▼
/mark-step-complete
(status = DEPLOYED)
│
▼
Background: Poll Google Verification
(10 seconds × 12 = 2 minutes max)
│
┌──────────────────┴──────────────────┐
│ │
Verified Not Verified
│ │
▼ ▼
Add to Search Console Skip GSC
Submit sitemap (IndexNow still runs)
Why poll in Step 2? The Google TXT record is added during Step 1, but DNS
propagation takes time. By polling in Step 2 (after the user completes the
CNAME switch), we give DNS more time to propagate. The backend polls every 10
seconds for up to 2 minutes.
Google verification failure is non-blocking. If it fails, the domain is
still connected and IndexNow submission still runs. Only GSC sitemap
submission is skipped.
Database Columns
The ai_sites table stores Google verification state:
| Column | Type | Purpose |
|---|
google_verification_token | TEXT | TXT record value from Google Site Verification API |
google_verification_status | TEXT | PENDING, VERIFIED, or FAILED |
google_sitemap_submitted_at | TIMESTAMPTZ | Last sitemap submission timestamp |
google_admin_account | TEXT | Which admin account owns this site (future) |
Troubleshooting
”Missing Google OAuth credentials”
Check that all three environment variables are set:
GOOGLE_CLIENT_ID_GSCGOOGLE_CLIENT_SECRET_GSCGOOGLE_REFRESH_TOKEN_GSC_ADMIN_1
”Failed to get access token”
The refresh token may be invalid. Re-run the OAuth Playground flow to get a new one.
”Domain verification failed”
- Check that the TXT record was added correctly
- DNS propagation can take up to 48 hours (usually 5-30 minutes)
- Verify the TXT record with:
dig TXT example.com
”Failed to add site to Search Console”
The domain may already be verified by another account. Check Search Console manually. 
