Googlebot

The is the software used by the Google search engine to collect information about the content of websites. It is a search engine spider that tracks the web following links from one site to an other. Once a site is found it will try to read as much as possible on that site following internal links before it moves on the the next site. This is the reason why backlinks are so important, Googlebot finds your site only because it has followed a link from an other site to yours.

Googlebot has two versions, deepbot and freshbot. The Google deepbot, tries to follow every link on the web and download as many pages as it can to the Google indexing software. It completes this process about once a month. Freshbot crawls the web looking for fresh content. It visits websites that change frequently, according to how frequently they change. Ideally, freshbot would visit a daily newspaper's website every day and a weekly ezine would get crawled once every 7 days. This collection of information is also known as "The Google Dance".

Googlebot does not index your site, it mearly collects data handing it over to the indexing software. Googlebot adheres to the , meaning you can restrict asses to certain parts of your site to this bot by creating a robots.txt or adding a robots meta tag to your pages.