Search Engines Index 85% of Visible Web Content

From ISEDB.com:

According to a recent study co-written by Ask Jeeves Director of Advanced Products Antonio Gulli and University of Iowa's Alessio Signorini, the major search engines have managed to index nearly 85% of the estimated 11.5 billion documents that make up the "visible web". While there are likely over 500 billion documents on the web, most are invisible files hidden in databases and password protected regions search spiders don't travel. Collectively, the four major search engines are estimated to have indexed about 9.1 billion unique documents.

Google carries the most indexed content boasting 8.1 billion documents or 69.6% of the visible web in its search database. Yahoo comes in second with an estimated 6.6 billion or 57.4%. Ask Jeeves slips past MSN with an estimated 5.3 billion indexed documents (46.1%) to the 5.1 billion pages (44.3%) in MSN's database.

While the study relies heavily on estimates and on reporting from search engines, it is one of the only recent examinations of the actual size of the web and of search engine coverage.

Technorati tag: search engine