DEVONagent Rushes In Where Google Fears to Tread
As I keep telling my mother when she doesn’t know or can’t remember a fact: "No problem, this is why heaven gave you the Internet!" Isn’t it great? You don’t need a memory; the Internet is a giant encyclopedia. No matter what you want to know, someone, somewhere, has been nutty enough to be extremely interested in it and to have written it up in excruciating detail. And it’s just sitting there, waiting for you to find it.
But how? There’s the rub. Finding facts on the Internet is not easy. Google is nice (and don’t you wish you’d bought stock?), but what it gives you is just a list of links. Basically, Google alone doesn’t solve your problem unless the first two or three links happen to consist precisely of a complete discussion of exactly the question you had in mind, which rarely happens. Instead, Google just confronts you with a completely new task, namely, to start visiting all those links and perusing them, with hand and eye and brain, hunting for the information you want.
The truth is that when you have a question, what you want is not a list of links; what you want is content, and in particular, content that answers your question. What if you had an application that provided exactly that, by taking the second step for you? Using a search engine, it would perform the search, then visit all the top links, download the content, and filter that content by relevance and index it so that you could find the particular facts you were looking for, quickly and easily.
That is what DEVONagent does. The recent advent of version 1.5 is a good excuse to write about DEVONagent, but the fact is, I’ve been meaning to mention this wonderful program for a long time – ever since I first tried it, to learn the answers to questions like these: "Prior to the artificial revival of Hebrew as a spoken language in post-War Israel, when did Hebrew go extinct as a living vernacular?" and "What’s the story behind the Lisa Della Casa / George London recording of Richard Strauss’s ‘Arabella’?" In both cases I ended up learning exactly what I wanted to know, immediately, and I knew instantly that this program was a keeper.
Way and Means — The first few times I used DEVONagent, I didn’t even read the manual: I just started it up, did a search, and was amazed. If you do read the manual, you learn that DEVONagent is full of options. When you perform a search, you use what’s called a "search set," which is simply a collection of instructions for how the search is to be performed. These instructions involve such things as: What search engine(s) should DEVONagent use? (DEVONagent comes with lots of plug-ins that know how to interact with standard search engines and information sites such as Google, Yahoo, dictionaries, Wikipedia, and so forth.) How many results should we gather? Should we also follow links in the resulting content? Should DEVONagent "intelligently" filter out irrelevant or repeated material? Should we look for some particular kind of content, such as downloadable binaries, or movies? You can also, or instead, specify particular Web sites where DEVONagent should look. In this way, DEVONagent also functions as a reader of news feeds (DEVONagent understands RSS and so forth) or as a Web crawler.
In all probability, you won’t even attempt to configure a search set; you’ll just pick one from the pop-up menu in the search field, enter some search terms, and press the Start button. Let’s say you picked "Internet (Fast Search)" and your query is "formation of snow crystals". After a while, DEVONagent finishes the search, and you are presented with two panels of results.
The Pages panel lists all the found pages, ranked by relevancy, much like a set of Google results. The difference is that clicking a page listing shows you summaries of all relevant sentences or paragraphs from the content of that page, immediately. Thus you can quickly peruse the found pages to see which one looks promising. However, I almost never do this; I use the Digest panel instead.
The Digest panel lists "topics," which are keywords used often in the content results, ranked by frequency. This is where it gets interesting. When you click on a topic – or when you enter search terms of your own – you are shown a text digest of the actual content of the relevant pages, in order of relevance. Each text digest is several paragraphs long; it may well contain the very information you’re after. Even if it doesn’t, the text digest ends with a link to the Web page so you can go there – and the digest is full enough so that you’ll know whether you want to go there.
Your Web browser is DEVONagent itself, and within it you can navigate between digest links without returning to DEVONagent’s main window; especially noteworthy is the "See Also" drawer, which lists the related found pages with their relevancy rankings. Another nice feature is the "objects" drawer, which lists items within the current page by type: all links, for example, or all images, all email addresses, and so on. (I don’t understand why all browsers don’t work this way; on the other hand, it’s a pity that DEVONagent doesn’t do tabbed browsing.) You can also archive a page for later study, or send it off to DEVONthink for even more advanced indexing and cross-reference.
In my case, I clicked the topic "snowflakes," I read the digest for the first entry, I went there in the browser, and I never came back – it was Kenneth Libbrecht’s astounding SnowCrystals.com, which shot the next hour completely out of my day.
Note that, with the rather clumsy query I originally entered, I would probably never have found this page using Google; it’s there, but not on the first page, and not with a promising-looking listing. Thus DEVONagent, with its indexing, its intelligent relevance rankings, and its digest display, completely compensated for my somewhat infelicitous initial search construction. And that’s the point: time after time, DEVONagent tells you what you really wanted to know, even if you didn’t realize you wanted to know it.
Conclusions — Whatever you want to know, some kind soul has probably written you a long and detailed letter telling you all about it. Unfortunately, that letter has been purloined; it’s hiding in plain sight, amidst all the other Internet content clamoring for your attention. DEVONagent searches intelligently so that you don’t have to, and presents the results with crystal clarity, slicing through the murk of the Internet, showing you just what you wanted to know. This version has been cleaned and polished, with tremendous attention to detail; DEVONagent is now stunningly straightforward and easy to use, and the underlying technology is just brilliant.
DEVONagent has a number of features I haven’t even mentioned, such as scheduled searches, and scriptability via AppleScript; using these, it could be part of some newshound’s regular automated workflow. The best way to appreciate the program is to download the 2.5 MB installer and evaluate it for yourself. DEVONagent is resource-intensive while it’s at work, so a broadband Internet connection is a must, and a fairly peppy computer is helpful too. Mac OS X 10.3 Panther is recommended but not required (10.2.7 is the minimum system). The price is $35.