User-agent: * Disallow: /backup-files/ User-agent: * Crawl-delay: 5