Technical SEO
Robots.txt Generator
Create a robots.txt file for WordPress or any website. Choose a preset, add your sitemap and download the file.
This tool runs in your browser. Nothing you type is uploaded.
Free tool made by Hridoy Ahmed. Found a bug or have a suggestion? Contact me. Need another free tool? Request it here.
How to use the Robots.txt Generator
- Pick a preset: WordPress, allow all, or block all.
- Add your sitemap URL and any paths to disallow.
- Optionally add rules for AI crawlers.
- Copy or download robots.txt and upload it to your site root, or paste it into your SEO plugin's robots editor.
Frequently asked questions
Does robots.txt remove a page from Google?
No. It controls crawling, not indexing. To keep a page out of search results use a noindex tag and keep the page crawlable.
Where does robots.txt go?
In the root of your domain, for example https://example.com/robots.txt.
Is "block all" safe?
Only for sites under development. On a live site it stops crawling of everything.
Need technical SEO done properly?
Manual, white-hat SEO. No shortcuts. Send me your site and I will review it by hand.