Robot Text File Generator

What This Tool Does

Generate a robot text file for your website that specifies which search engines are allowed to crawl your site. Excellent for new websites who want to restrict access.

How to Use

Simply check in which search engines you want crawling your website and then hit generate.

Please select which of the following web crawlers / engines you wish to allow access to your website. If you turn the engine ON, they will crawl and index the website. If you turn it OFF, they will not crawl nor index the website.

Here is the generated code. Please paste this into a blank htaccess file, or click the download button above.

Press Generate Code for the code to appear here.