ghacks Technology News

  • Author: Martin
  • Published: May 20th, 2007
  • Comments: 1

Check your robots.txt at Google

A robots.txt tells the search engine robots about the directories and files that should be indexed on a website. A wrongly edited robots.txt file could mean that search engine robots are not crawling your website anymore with the result that new articles will not be indexed in the search engine index.

This can be quite devastating especially for webmasters who do earn their living from their websites. It takes up to two weeks until changes can be seen when you edit the robots.txt file which is way to long if you made a mistake.

Tags: , , , ,
Categories: Google, The Web

© 2005-2009 Ghacks.net. All Rights Reserved. Privacy Policy - About Us