Robots.txt Generator
Per-crawler Allow/Disallow rules and sitemap links, ready to paste
Why people choose us
Private by design
Your input is processed securely and never sold. Client-side tools never leave your browser at all.
Instant & free
No installs, no waiting. Results in seconds, right in your browser, on any device.
Free forever
No sign-up, no watermarks, no limits. All client-side tools are free to use as much as you like.
Frequently asked questions
What does an empty Disallow line mean?⌄
A bare "Disallow:" with nothing after it means "disallow nothing" — i.e. this crawler is allowed everywhere. That's what the Allow-all preset generates.
Does robots.txt actually stop pages from being indexed?⌄
Not reliably by itself — it only asks well-behaved crawlers not to fetch a path. A page can still get indexed (e.g. if linked from elsewhere) even if disallowed. To truly keep a page out of search results, use a noindex meta tag or HTTP header instead.
Is my content uploaded anywhere?⌄
No — everything runs in your browser with plain JavaScript. Nothing you type is ever sent to a server.
Related tools
Related tools
Meta Tags Generator
Title, description, keywords, canonical and robots meta tags with a live Google search preview
Social Share Preview
Open Graph + Twitter Card tags with a live Facebook/Twitter-style preview card
Sitemap Generator
Paste a list of URLs, get a ready XML sitemap or sitemap index