robots.txt Generator
A robots.txt file tells search engines what they can and can't crawl. Set your rules below and
this free tool builds a valid file you can copy or download and drop in your site root. No
signup — it runs in your browser.
Your robots.txt
Put the file at your site root so it's reachable at yoursite.com/robots.txt. Note:
robots.txt controls crawling, not privacy — don't use it to hide sensitive pages (use auth/noindex for that),
since the file itself is public.
Tips
- Most sites want "Allow all" plus a sitemap line — that's the default here.
- Always include your sitemap URL; it helps search engines find all your pages.
- Blocking all is for staging/private sites only — it removes you from search.
Launching a site? Pair this with the favicon generator, Open Graph tags, and your legal pages.