User-agent: * Allow: Crawl-delay: 5 Allow: /ads.txt Disallow: /testing/ Disallow: /includes/ # API Routes Disallow: /api/* Sitemap: https://www.medindia.net/sitemap/index.xml User-agent: Mediapartners-Google* Allow: / User-agent: Googlebot-News Allow: / User-agent: Googlebot Allow: / User-agent: Bingbot Allow: / User-agent: AhrefsBot Crawl-Delay: 10 User-agent: SiteAuditBot Crawl-Delay: 10