About this tool: Generate a robots.txt file to control search engine crawlers and specify which parts of your website should be indexed.
How to use: Select user-agents, add allow/disallow rules, and click 'Generate Robots.txt' to get your file for download or copy.