User-agent: *
Allow: /
Disallow: /api/
Disallow: /docs
Disallow: /redoc
Disallow: /openapi.json

# Sitemap
Sitemap: https://scrapingsteps.com/sitemap.xml