User-agent: * Allow: / Disallow: /js Disallow: /images Disallow: /img Disallow: /css Allow: /sitemap.html Sitemap: https://www.roshnipandit.com/sitemap.xml SetEnvIfNoCase User-Agent ^Httrack keep_away SetEnvIfNoCase User-Agent ^Offline Explorer keep_away SetEnvIfNoCase User-Agent ^psbot keep_away SetEnvIfNoCase User-Agent ^Teleport keep_away SetEnvIfNoCase User-Agent ^WebCopier keep_away SetEnvIfNoCase User-Agent ^WebReaper keep_away SetEnvIfNoCase User-Agent ^Webstripper keep_away