Spotlight on Spotlight
Much will be written about Spotlight, one of Tiger’s marquee features that takes system-wide search from a time-consuming annoyance to an efficient part of everyone’s workflow. In fact, Spotlight works so well that the idea of filing email, files, and other data will eventually disappear – but not quite yet.
You’ll read a lot about the general features of Spotlight: you can find any text in any file quickly, or use it to pinpoint menu items in System Preferences. I’d like to tell you quickly about how Spotlight works and then delve into areas you probably won’t hear as much about elsewhere. I’ll conclude with musings on how Spotlight might free us from the tedium of forcing organization on top of what we create.
Spotlight in a Nutshell — Spotlight’s approach is simple: everything is indexed quickly and efficiently in an ongoing manner. Install Tiger and reboot, and the first thing the operating system does is index your hard disk. In multiple test installations, I didn’t even notice the indexing taking place, although some users report 50 percent of their processing power devoted to the task. You can’t use Spotlight until this initial index is done, but clicking the blue Spotlight icon in the upper right of the system menu bar will reveal how long Tiger thinks it will take to be finished. A pulsating dot in the center of the magnifying glass icon lets you know indexing is taking place.
When it’s done, Tiger automatically modifies the index for every changed document and adds every new document to it. This happens quietly as well. Let me restate this in case it didn’t sink in: Spotlight doesn’t run a full re-index of your hard drive every night requiring you to leave your computer on or causing loud drive access noises in the wee hours. All other overlay indexing programs and previous Apple attempts required that kind of churn.
I haven’t stress-tested Tiger yet by, say, using Automator to create 1,000 one-megabyte-sized files of random text, but that would be a good way to see Spotlight’s ongoing indexing in action.
By integrating index updates into the operating system at the filesystem level, Tiger avoids patching the system at a low level (always dangerous), the above-mentioned overnight reindexing, and subset indexing that omits potentially useful data.
Apple also seems to have pulled off a neat trick: using some kind of optimized index to produce some results right away, Spotlight searches start running as soon as you start typing. By the time you finish typing, either through predictive word finding or sheer good programming, the search is almost done.
I’ve found Spotlight incredibly zippy on a 1 GHz 15-inch aluminum PowerBook G4 and a dual 1.25 GHz Power Mac G4. I’ll be curious to hear about how it feels on the lowest-end machines that Apple supports.
Spotlight is available at any time from the upper right by clicking its icon, or pressing Command-Space. It also appears in every Finder window by default, and, most critically, within any Open and Save dialog box. No more navigating to find files to open! No more navigating to find the right folder to save! I will still love and cherish Default Folder, but it will be much less important to my future workflow.
Apple has made Spotlight available from the command line, too. The mdls command lets you see the metadata associated with any file. The mdfind command is essentially a Spotlight search.
Narrowing Spotlight Searches — Spotlight rewards those that need more sophisticated searches by allowing you to refine phrases that constrain date and time, file names, and other metadata. Metadata is data that describes data, like the last modified time, the F-stop of a camera, a QuickTime movie’s format or length, or the photographer’s name embedded into a TIFF image’s header.
Most searches will start with keywords, but you will quickly want to drill into subsets if you have many results. Apple has built a nomenclature for searching that they haven’t yet exposed well – the special words that you can use to restrict searches. Unfortunately, these words aren’t currently documented anywhere on their site or within Spotlight Help in the release of Tiger.
You can experiment with restrictive phrases. Apple’s page on Spotlight suggests that you might add "Date:yesterday" after keywords to find just files created in the last day. If you wanted to find all images created yesterday you could enter "Date:yesterday Kind:image". I expect this nomenclature will be fully documented over time. These restrictive words will be especially useful in Open and Save dialog boxes, where Spotlight could produce daunting results.
The capability to make use of some of the increasingly rich metadata produced by digital media devices is a boon. Imagine finding all pictures you’ve taken on a particular Canon camera model at a specific resolution. Right now you need to use a cataloging program such as iView Media Pro and keep that catalog constantly up to date.
There’s another way to use these restrictive add-ons without knowing Apple’s secret narrowing words – via Smart Folders.
Folders as Search Results — A couple versions of Entourage ago, Microsoft added pseudo-mailboxes that were actually search parameters presented as a mailbox. Unfortunately, for those of us with zillions of messages, a search took an unbelievably long time with the search engine Microsoft used at the time.
Spotlight has taken that concept and extended it to the Desktop in the form of Smart Folders, which are essentially the live results of a set of search parameters you define. Spotlight’s performance is good enough that you don’t notice the fact that a Smart Folder is populated dynamically.
Along the way, Apple removed Panther’s advanced searching from the Finder; selecting Find from the Edit menu effectively creates a new Smart Folder (using the same dialog as the New Smart Folder command) that isn’t yet saved. To create a search that narrows down beyond keywords, you either learn the incompletely documented nomenclature described above, or use Smart Folders.
When creating a Smart Folder, the default parameters are Kind: Any, and Last Opened: Any Date. The buttons above the search parameters list Servers, Computer, Home, and Others. If you leave it set to Home, the search is restricted to the current user’s Home directory. I prefer setting it to Computer to take full advantage of Spotlight’s capabilities, and because I keep documents and other files stored throughout my hard drive, not just in my Home directory as Apple would prefer. (Click Others to add or remove specific folders or hard drives.)
You can create a Smart Folder, too, in any Finder window by typing a search in the Spotlight field. That Smart Folder doesn’t show the default scope of Kind and Last Opened, but you can click the plus sign at the upper right next to the Save button to add bounds.
Smart Folders let you mix the contents of the Spotlight field, in which you might enter keywords, with restricting conditions. Click the plus sign next to any condition to add more. Select the pop-up menu that’s the condition’s name and you can select one of several favorite conditions, or select Other.
In Other, you will see the full range of predefined metadata that’s supported in Spotlight. For instance, select URL and you can choose to find any document that contains that URL. Check the Add to Favorites box and that attribute now shows up in the condition pop-up menu.
I don’t want to turn this into 10,000 words on Smart Folders, but there’s more: you can show the top 5 or all results for a given document category; sort by date or kind; click the "i" button next to a file to see a summary of its information; view PDFs by a thumbnail of their first page; show images; and so forth.
Rethinking Filing — Filing is a tedious activity that computers were supposed to save us from, right? That’s why I was so excited to see Creo’s Six Degrees program a few years ago. Six Degrees integrated with certain email programs under Mac and Windows so that recipients, subject lines (discussion threads), and attachments were the three points of a triangle. You could rotate your email-world around to view it through the window of who you corresponded with, what you talked about, and what files were involved. (The product was sold to Ralston Technology Group and is now marketed as Clarity.)
Spotlight expands that notion far, far beyond those modest but significant goals. Six Degrees was trying to free people from ever having to decide in which mailbox an email message should be stored, and in which folder a file belonged.
I don’t think Spotlight yet allows us to break down all barriers and use one giant email folder to store all messages, and one giant Finder folder to store every file we create or receive. But, it is moving us closer to what I think people actually want from their computers: not to spend a good percentage of time categorizing.
Perhaps it will take some time yet, but I perceive the future of information to be much more amorphous. Instead of discrete information chunks, every graphic, letter, report, presentation, movie, or other project piece is just a blob in the middle of some kind of data medium that we navigate through in many different ways: by date, by content, by visual presentation, by keywords, by attributes.
That is, the interface to our data is no longer the worn-out metaphor of files and folders, but a rich interactive approach that mediates between an underlying structure we don’t need to understand and our desire to find things by the way we remember them. Say goodbye to descriptive file names, for instance.
I didn’t come up with this way of viewing the future of desktop information, nor did Apple. David Gelernter, a Yale University computer science professor, has been talking about this since at least 1991. Although a company he founded to implement these ideas seems to have disappeared, his ideas are well represented in a 2003 interview: read the section on Information Beams.
In that interview, he said, "When I acquire a new piece of ‘real-life’ (versus electronic) information – a new memory of (let’s say) talking to Melissa on a sunny afternoon outside the Red Parrot – I don’t have to give this memory a name, or stuff it in a directory. I can use anything in the memory as a retrieval key."
Spotlight is probably the first mainstream operating system or program to take a big step towards Gelernter’s humanist view that maps how we think to what we have stored.