# This is the /robots.txt file for http://www.usddelts.com User-Agent: * Disallow: /cgi-bin Disallow: style-delt.css Disallow: 401.shtml Disallow: 403.shtml Disallow: 404.shtml Disallow: 500.shtml Disallow: /Assets Disallow: /Documentation Disallow: /Library Disallow: /logs Disallow: /mailman Disallow: /secure Disallow: /Templates # That's all folks! # # Tips & Tricks: # To disallow files to certain crawlers, use this command: # User-Agent: Webcrawler # Disallow: ENTER PIECE HERE # # Here are the "PIECES" to add to describe components for non-index: # # To index nothing at all: / # Specific URL of a file in the root: /filename.extension # Specific URL of a file not in the root: /directoryname/filename.extension # Directory of files: /directoryname # For just the index.html or default.html in a directory: /directoryname/