SEO

How enable robots.txt in website

How to Enable Robots.txt file in your website

Robots.txt is the first thing a crawler visit in your website. Robots.txt tells search engine bot (spider or crawler) that which url needs to be crawled or which does not.  Robots.txt Format Use the format below to create robots.txt file in notepad User-agent: * Allow:  Sitemap: https://prashantchauhan.com/sitemap.xml Disallow: /dashboard/retrieve-password/ Disallow: /wp-admin  ā€œ * ā€œ this …

How to Enable Robots.txt file in your website Read More »