There are many a blog post out there detailing the advantages and features of robots.txt files, but does your site even need one? And does it have any impact on search engine optimization strategy. Robots.txt is used to block search engine spiders from crawling certain pages or directories. On most sites private information is already blocked because […]

Read More