************************************************************** * * * CYBERSPACE * * A biweekly column on net culture appearing * * in the Toronto Sunday Sun * * * * Copyright 1999 Karl Mamer * * Free for online distribution * * All Rights Reserved * * Direct comments and questions to: * * * * * ************************************************************** Search Me The web is like a library that houses a 75 million volume collection in one mountainous pile. Fortunately, a number of, well, obsessive individuals began cataloguing the web shortly after its creation. In the software industry, obsession is frequently followed by wads cash. Yahoo, the best known web index, raised about $35 million in its initial stock offering. Money, of course, always generates competitors. There are so many indexes these days that there are now "meta-indexes," sites that index the indexes. I'm sure if an index turned around and tried to add a meta-index a paradox would emerge. I live in fear of the day someone discovers this, writes a Bertrand Russell bot, and wipes out the net. Before that day, I decided to test the "omniscience" of the bigger catalogs. I recorded the number of "hits" a site would report for two searches. The first was a simple search for the word "sun." For the second search, I chose something obscure. I went hunting for the Third Secret of Fatima (a Marian prophesy known only to a select few in the Catholic Church). I searched for sites that matched both "secret" and "Fatima." Yahoo - www.yahoo.com Test Hits: "sun" - 515, "secret Fatima" - 21 Compared to some other indexes, Yahoo offers a small number of links. That's not always a Bad Thing. Yahoo builds its links by letting webmasters describe their own sites and then slot them into a hierarchy of topics (e.g., politics, entertainment, etc.). Hits tend to be accurate since the person who best understands his/her page did the indexing. Lycos - www.lycos.com Test Hits: "sun" - 62,629, "secret Fatima" - 25 Lycos gets its name from the Latin word for "Wolf Spider." Like a spider, Lycos uses "web crawling" software that searches the web and builds a catalog from the text it finds. Lycos is usually my first choice in a web index. It's fast, it has a lot of links (in the tens of millions), and it's an easy URL to remember. Alta Vista - www.altavista.digital.com Test Hits: "sun" - 2,823,121 "secret Fatima" - 400 Alta Vista, another web crawler, boasts it has the largest number of links (over 30 million). Alta Vista returned the most hits in my test searches, so that isn't just talk from a clueless marketroid. I would probably use Alta Vista more often but I can never remember the URL. HotBot - www.hotbot.com Test Hits: "sun" - 1,375,404 "secret Fatima" - 70 The people who bring you Wired magazine now bring you this "beta" version of what may become the ultimate web crawler. HotBot's creators hope to index roughly 70% of the web. Right now it has about as many links as Alta Vista, although it's grievously slow in comparison. On the plus side, HotBot has a great search engine, letting you limit searches in a number of innovative ways. Deja News - www.dejanews.com Test Hits: "sun" - N/A, "secret Fatima" - N/A Finally, if the web bores you, consider the amazing Deja News. This catalog lets you search Usenet for nearly every message posted since March 1995.