# Original Auto Styling, robots.txt # Full SEO + AEO crawling enabled. All public pages indexable. User-agent: * Allow: / # --- Google Ads landing pages (/lp/*) --- # # These are DELIBERATELY NOT disallowed here, and that is not an oversight. # # 1. They are kept out of search by a `noindex, nofollow` meta tag baked into # each page at prerender time (see scripts/seo-routes.mjs). A crawler has to # be able to FETCH a page to read its noindex. Blocking the path in robots.txt # would hide the noindex and can leave the URL indexed as a bare link anyway. # # 2. Google Ads verifies landing pages with AdsBot-Google. Disallowing an ad # destination causes "Destination not crawlable" disapprovals and hurts # Quality Score. AdsBot is explicitly allowed below for the same reason. # # They are also excluded from sitemap.xml and are not linked from anywhere on # the public site, so there is no organic discovery path to them. User-agent: AdsBot-Google Allow: / User-agent: AdsBot-Google-Mobile Allow: / # --- AI Search / Answer Engines --- # Explicit allow for the major AI crawlers that power AI Overviews, # ChatGPT Search, Perplexity, Claude, Gemini, and friends. User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: ClaudeBot Allow: / User-agent: anthropic-ai Allow: / User-agent: Claude-Web Allow: / User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / User-agent: Google-Extended Allow: / User-agent: Applebot Allow: / User-agent: Applebot-Extended Allow: / User-agent: Bingbot Allow: / User-agent: CCBot Allow: / User-agent: cohere-ai Allow: / User-agent: Bytespider Allow: / User-agent: Amazonbot Allow: / User-agent: DuckDuckBot Allow: / User-agent: FacebookBot Allow: / # Sitemap Sitemap: https://originalautostyling.com/sitemap.xml