# FXNewsBias, https://fxnewsbias.com # AI-scored sentiment for the 8 major currencies, 15 forex pairs and gold # (XAU/USD), plus silver, oil, crypto and the US indices. # Developer API: https://fxnewsbias.com/developers # /api/ is the key-gated JSON API. It answers 401 to anyone without a key, # so crawling it only fills Search Console with "unauthorized" errors. Google # flagged exactly that on 16 Sept 2026 after the history endpoints were added # to the developers page structured data. Nothing under /api/ is a page. # /api/site/ is the exception: it is the read-only data the public pages load # their scores, labels and lists from, so crawlers that render pages must be # able to fetch it or they see "Loading..." where the data belongs. Its # answers carry X-Robots-Tag: noindex, so the JSON itself is never indexed. User-agent: * Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ # --------------------------------------------------------------------------- # AI and answer engines are welcome. Google-Extended and Applebot-Extended are # opt-out tokens, so listing them with Allow is the explicit signal that our # public pages may be used for AI grounding and answers. # --------------------------------------------------------------------------- User-agent: GPTBot Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: ChatGPT-User Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: OAI-SearchBot Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: ClaudeBot Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: Claude-User Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: Claude-SearchBot Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: anthropic-ai Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: PerplexityBot Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: Perplexity-User Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: Google-Extended Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: Applebot Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: Applebot-Extended Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: Amazonbot Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: meta-externalagent Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: CCBot Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: cohere-ai Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: DuckAssistBot Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: YouBot Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ User-agent: Bingbot Allow: / Disallow: /admin Disallow: /admin.html Disallow: /forecast-admin Disallow: /forecast-admin.html Disallow: /auth-action Disallow: /auth-action.html Disallow: /profile Disallow: /oauth/ Disallow: /*?checkout= Disallow: /*?paid= Disallow: /media/ Allow: /api/site/ Disallow: /api/ Sitemap: https://fxnewsbias.com/sitemap.xml