Wednesday, September 26, 2007

A couple of Search Engines

While it may seem as if the entire world is covered by Google, and therefore one need not look elsewhere, Google is not the be-all and end-all of search engines, and anyone serious about finding good information on the Web should have at least two decent search engines in their bag of tricks.

Why do you need more than one search engine? Because of maths.

You see, no matter how smart and pretty computers may appear, they can still really only do one thing well: count. They're still giant calculators at heart. In order to "think" about web pages, they have to give every factor involved (the words in your search, how often they turn up on a page, where they turn up on the page) a numerical value. They then use a mathematical formula, or algorithm, to rank the pages in a list. Different search engines use different algorithms, which means you get different results.

Google and Yahoo! are both general search engines. Their algorithms are designed to find the most "popular" websites by canvassing all of the sites they know about, picking the ones with the most links pointing to them and ranking them according to factors like commercial relevance. They are the K-marts and Targets of search engines. By all means, use them - you'll probably find something that's just what you're looking for, but remember to shop around in the specialty stores as well.

A personal favourite of mine has always been Ask. Well, technically, it hasn't 'always' been Ask. It used to be about three different search engines, Hotbot, Teoma and Ask Jeeves, which were all available separately and via the Hotbot home page. Hotbot sold out to Google then curled up and died, while Teoma and Ask Jeeves merged into Ask.

Ask combines the best features of both search engines, using the Teoma algorithm and Ask Jeeve's ability to recognise questions. Where Google's algorithm goes for the most popular web sites across the board, Ask attaches a few more numbers to the web sites in it's database and looks for the sites which are most popular amongst their peers.

In addition, Ask also offers three sets of results. There's the normal "hit list" of links to web pages, and two areas where they offer search suggestions based on tallied results of other, similar questions: "Narrow Your Search" and "Expand Your Search". Depending on what you search for, you'll also see examples from the Images search in another part of the screen.

Another search engine I have discovered very recently is Vivisimo (which also seems to be Clusty - I'm not entirely sure what the exact relationship is between these two). Vivisimo/Clusty subdivides it's results and groups (clusters) them together based on common content, which makes it much easier to sort through the results and hone in on those which seem most relevant. I spent several hours looking for information through several other search engines which I managed to find in minutes using Vivisimo, so it's definitely worth adding to the toolbox.

Have a play with them and get a feel for the ways they complement each other. Try putting the same search terms (or question) into each, and see what comes up. And remember to look beyond the first ten results - you can often find just what you're looking for on page 5.