AI Answer Engines Are Worth Trying
The search engine has been a key Internet tool since the early days. Before the Web was a thing, I fondly remember Archie (from “archive” without the “v”), which indexed filenames on FTP servers, followed by Jughead and Veronica (Very Easy Rodent-Oriented Net-wide Index to Computerized Archives), which extended the index-and-search concept to Gopher menu titles. Web search emerged through two distinct approaches: AltaVista pioneered automated full-text indexing that could rapidly crawl millions of Web pages, while Yahoo began as a human-curated directory that organized websites into categories. Google later took search to the next level by combining automated indexing with PageRank, an algorithm that determined a page’s importance based on how many other sites linked to it. Despite occasional challenges from competitors like Bing, Google has remained the dominant search engine for decades, to the point where the company name has become synonymous with search.
Although complaints about Google’s search quality decline are more recent, my disillusionment began a few years ago when I grew tired of the extent to which Google tracked and monetized everything I did. (I understand the tradeoff; I just hit a tipping point with everything else that was happening during the pandemic.) I tried the Ecosia search engine that Apple added to Safari’s search bar in 2020, but perhaps because it was based on Bing, I found its results lacking, just as I had with DuckDuckGo. Shortly thereafter, however, another new kid on the block, Brave Search, performed well for me and remained my go-to until early 2024 (see “Brave Search Public Beta Offers Alternative to Google,” 8 July 2021).
Although I wasn’t unhappy with Brave Search itself, over my last few years with it, I felt that the pages it was finding were of lower and lower quality. Many of my searches involve looking for confirmatory details about Apple-related topics I am at least roughly familiar with, so I can’t help but judge the results. I would often find myself reading more of the linked pages to find one whose information I trusted. Too many of them repeated what was written elsewhere or contained easily identified errors.
Looking for better results, I decided to try Perplexity. Unlike Brave Search’s limited AI summaries, Perplexity offers comprehensive AI-powered answers. It performs a search and then uses generative AI to respond to the query in narrative form, incorporating information from multiple sources. Perplexity also cites sources for each statement of fact, enabling you to confirm its claims and get more details if desired.
When OpenAI enabled ChatGPT to search the Web, I switched to using it on one of my machines, and it has performed similarly well. Then I came across You.com, which initially started as a search engine but has since evolved into a more comprehensive general AI assistant with search capabilities. Claude added Web search to its Pro accounts. And Google opened limited testing of its AI Mode, which replaces the list of links with AI-generated answers like the others. So many possibilities!
Search Engines Versus Answer Engines
To distinguish these tools from traditional search engines like Google and Bing, I’m calling them “answer engines.” Although they are performing live Web searches for you, the focus is on answering your question rather than displaying the results of the search. I see answer engines as the next step in networked knowledge acquisition because they fundamentally change how we find and absorb information online:
- From lists to synthesis: Early tools like Archie and Veronica simply showed you where files were stored. Web directories like Yahoo organized websites into categories. Google ranks Web pages by relevance. Answer engines go further by actively processing and combining information from multiple sources to provide direct answers to your questions.
- From keywords to natural language: Remember having to get your search terms exactly right? Early tools required precise text matches. Search engines have improved at handling misspellings and understanding related terms, but you must think like a search engine. Answer engines allow you to ask questions naturally, and they take the context embedded in your question into account.
- From isolated lookups to conversations: With traditional search, each query stands alone—researching a related question requires starting afresh. Answer engines maintain context throughout an entire conversation, allowing you to explore topics more deeply without needing to rephrase or restart.
- From raw results to smart summaries: Traditional search engines give you a reading assignment—a list of potentially relevant pages for you to investigate and synthesize. Answer engines do that work for you, analyzing multiple sources and presenting a coherent answer while still citing their sources so you can verify the information.
Procedural Differences Between Searches and Answers
To understand why answer engines represent such an improvement, let’s examine how their process differs from traditional search.
Since the rise of Google, we have become accustomed to searches returning a list of links that we hope will provide the information we want, but that experience is far from ideal. For every question you have, you must:
- Formulate your search: This step requires distilling your question into a set of keywords that will match the desired pages. We’ve all gotten decent at this, and it works relatively well with keywords that are relatively uncommon and unambiguous.
- Review the results list: You must review the list of returned links to determine which to investigate. Most people start at the top because search engines rank the results. However, if you prefer certain sources over others, you may need to look further down the list and read more closely.
- Evaluate the most likely source: Next, you have to open the first link that seems likely to answer the question, read or at least skim the page, and evaluate whether it’s helpful and accurate. People tend to skim very quickly, encouraging Web designers to emphasize page layouts that prioritize headings and easily parsed lists.
- Repeat as necessary: If the page lacks the information you want or you’re dubious about its accuracy, you must return to the list of links and repeat the process. Savvy searchers often open multiple result pages in new tabs, so it’s easier to switch from one to the next when comparing them.
- Recast the search: In the worst case, you may have to redo your search because it wasn’t sufficiently clear, perhaps because of choosing generic or ambiguous keywords. (For instance, finding information about Apple’s bundled apps has become more difficult now that they have generic names like Calendar and Photos instead of iCal and iPhoto.)
The process is different for answer engines:
- Ask your actual question: Instead of trying to come up with a set of keywords or worrying that multiple searches will be necessary to assemble data you can combine, go straight for your answer. For instance, I had a three-column Excel spreadsheet that I wanted to learn how to print “snaking” on one page (so the column stopped at the bottom of the page and restarted back at the top). Instead of searching on “Excel Mac print snaking columns” and sorting through videos and pages that addressed the topic in varying levels of accuracy and specificity, I asked, “In Excel for Mac, is there a way to print a thin column so it snakes from the bottom of the page back up to the top and down again?” (The answer is no, you can’t do that easily in Excel, but you can simulate it by moving the data to a Word table.)
- Verify source material: As always, AI-based summarization can make mistakes or be fooled by inaccurate sources. Before chatbots had Web access, asking them to cite their sources was sketchy at best. When their responses are informed by Web searches, however, they have no problem correctly linking to their sources. How often you verify sources depends on the stakes of your query—you might double-check sources for important technical information but not for background information queries.
- Refine or expand the search: In many cases, asking the question is all you need to do. However, because these answer engines evolved from the chatbot milieu, you can always continue the conversation. Perhaps you forgot to specify that you were referring to the Mac version of Excel, or you want to know if you can use Pages instead of Word to create snaking columns. All you have to do is continue the conversation. Perplexity goes beyond the others by suggesting further avenues of investigation so you can explore the topic space further with just a click.
The key benefit of answer engines is that they typically provide you with exactly the information you want, with no additional effort required. They’re a bit like Wikipedia in this way—you could fact-check statements with the listed sources in a Wikipedia article, and you can often learn more by following links to related topics, but most of the time, you’re happy to read the article and move on.
Of course, that’s only true if the answer engine actually answers your question. I’ve been using Perplexity or ChatGPT for about a year now, and I have been happy with the quality of the responses.
To quantify that opinion and compare against the others, I built a Keyboard Maestro macro that sends my questions to four answer engines simultaneously, opening each in a Split View pane in Arc for side-by-side comparison. I evaluate all the answers and then give each a score from 0 to 3 in a Google Sheet. (0 means the answer is completely wrong, 3 means it’s completely right, and I assign 1 or 2 for partial credit.) The ratings are still subjective and specific to my searches, not yours. However, ChatGPT and Perplexity are currently neck-and-neck with average scores of 2.63 and 2.61, respectively. Google AI Mode is third with 2.24, and You.com brings up the rear with 2.06. I don’t yet have enough data for Claude to feel confident in my results, but at 1.89, its early results are not promising.
Answer Engine Limitations
It’s important to realize that answer engines aren’t superior to traditional search engines in all situations. In particular, they struggle with navigational searches—when you know where you want to go but not the exact URL. In such cases, there’s no question for an answer engine to answer—the user simply wants a link to the destination.
For instance, if you’re in the market for an iPhone, a Google search for “iPhone” will present Apple’s page at the top for a quick click. When I sent that search to the answer engines, only ChatGPT gave me a link to that page. The more action-oriented search “buy iPhone 16” triggered the desired link in ChatGPT, Perplexity, and Google AI Mode, but Claude didn’t pick up on the desire for navigation.
This limitation explains the recent study conducted by Columbia University’s Tow Center for Digital Journalism, which strongly criticized the accuracy of the answer engines when searching for news. In fact, what the study tested was the answer engines’ ability, given an excerpt from an article, to identify the corresponding article’s headline, original publisher, publication date, and URL. In essence, they were asking for a specific page. That may have been necessary research methodology for reproducible results, but even the study authors admit that it doesn’t reflect typical user behavior. Additionally, although I haven’t confirmed this suspicion, I suspect that the answer engines aren’t nearly as quick as Google at indexing breaking news.
Other specialized searches that work better in traditional search engines include visual searches that return images instead of text, real-time information such as sports scores and flight status, searches for local businesses that benefit from integrated maps and business details, and other location-based information that requires an interactive map. Even if you switch to an answer engine for most searches, you may wish to keep a Google toolbar bookmark a click away for quick navigational searches.
Choosing an Answer Engine
Ultimately, only you can determine whether an answer engine will match your needs and workflow. It will depend on what you’re trying to achieve and how well you can refocus your brain to ask questions instead of performing keyword searches. Here are my recommendations, along with the details you’ll need to use each:
- Perplexity: For most people, I recommend Perplexity. Its success rate is high, and you can use it for free. I haven’t seen the need to subscribe to its $20-per-month Pro plan for more in-depth searches. Its search URL is
https://www.perplexity.ai/search?q=%s, or you can use the Perplexity Mac app. I found using the app more awkward than searching directly in Arc. - ChatGPT: ChatGPT is as good as or perhaps slightly better than Perplexity in terms of answer quality, but unlimited use of its search capabilities requires a $20 per month Plus subscription. I encourage you to try it on the free tier, but I suspect you will want a subscription for regular use. A search URL that works for me is
https://chatgpt.com?q=%s. OpenAI also offers a ChatGPT Mac app, but it requires a Mac with Apple silicon. - Google AI Mode: For now, the free Google AI Mode is only a Google Search Labs experiment, so you have to request access. I wasn’t impressed by Google AI Mode, though its connection to the Google index may make it better at breaking news. Once you have access, its URL is
https://www.google.com/search?udm=50&nord=1&q=%s. - You.com: Like Perplexity, You.com is free to use, or you can expand its capabilities with a $20-per-month Pro plan, but its success rate was sufficiently low for me that I wouldn’t recommend trying it first. Its URL is
https://you.com/search?q=%s. - Claude: I hear highly positive things about Claude as a chatbot, especially for coding, but its Web search capabilities are weak and require a $20-per-month subscription. Apart from the poor answers, Claude has usability issues. It requires the user to approve URL-filled prompts with a click and then click again to submit the search. Its URL is
https://claude.ai/new?q=%s, or you could try the Claude Mac app.
You’ll note that these answer engines, besides the experimental Google AI Mode, require or encourage a subscription. Currently, only Perplexity is experimenting with advertising using sponsored follow-up questions, which are easily ignored. (I hadn’t noticed the sponsored follow-ups in real-world usage.) The company says it will never share personal information with advertisers.
Configuring Browsers to Use Answer Engines
The leading browser makers—Google, Apple, and Mozilla—have never been enthused about adding alternative search engine options. It’s not surprising, given that Google Search accounted for $175 billion in 2024 (about 57% of Google’s revenue) and delivered roughly $20 billion to Apple and $500 million to Mozilla. Search is big business.
Nonetheless, with a little effort, you can change the default search engine in all three browsers on the Mac. Once changed, all searches you initiate from the location bar will go to your new answer engine rather than Google. Changing search engines is more difficult or impossible in Safari on the iPhone.
(Personally, I use Arc Search on the iPhone, which has its own Browse for Me answer engine and lets you select Perplexity as the default search engine. However, given The Browser Company’s lack of meaningful updates to Arc and Arc Search since late 2024, due to chasing the fever dream of a more mainstream browser, I can’t recommend Arc Search if you’re not already deeply invested in Arc.)
Google Chrome and Chromium Browsers
You can configure most of the answer engines directly within the search settings for Chrome or another Chromium-based browser like Arc, Brave, Edge, or Vivaldi. Navigate to Settings > Search Engine, click the Add button, and enter the details for your answer engine, replacing the query with %s—the necessary URLs are above. Click Add and, once back in the list, click the three-vertical-dot menu and choose Make Default.
This approach isn’t supposed to work for ChatGPT, although using https://chatgpt.com?q=%s worked fine for me. If necessary, you can instead make ChatGPT the default search engine by installing the ChatGPT search Chrome extension.
Safari
Apple doesn’t allow custom search engines in Safari. However, the Customize Search Engine extension lets you configure additional search engines using the same approach as Chrome. After you’ve installed and enabled the extension, click its toolbar icon to access its settings.
Firefox
It doesn’t appear that you can configure search engines directly in Firefox. Instead, you need to add a search extension. I’ve found unofficial extensions for Perplexity and ChatGPT, as well as an official one for You.com. Once installed, choose Firefox > Preferences > Search and choose the desired one from the Default Search Engine pop-up menu.
Will Answer Engines Hurt the Internet?
I’ve left the most controversial aspect of answer engines for last. Publishers have long been willing to let search engines index their websites in return for traffic, which they can monetize through ad revenue and subscriptions. Many publishers even allow Google and Microsoft crawlers to index paywalled information so it serves as a teaser for those who follow such links from search results.
(We should also remember that a vast amount of content comes from sources other than for-profit publishers. Consider government databases, university resources, personal blogs, open-source documentation, non-profit research publications, and public forums where people freely share their expertise.)
One of the criticisms of answer engines is that they send significantly less traffic back to the sites from which they obtain their source material. That’s indisputable—the entire point of an answer engine is that it answers your question rather than making you read all the sources independently.
At the heart of this issue is the conflict between what is best for the user versus what is best for businesses that generate revenue on the Internet. It’s the same argument publishers make against ad blockers and that Facebook makes against Apple’s Ad Tracking Transparency. Users hate ads, and most people understand that the surveillance advertising industry is morally and ethically bankrupt. Nonetheless, advertising remains the dominant model for many publishers.
Technological advances often disrupt long-standing business models. The rise of the Internet radically reshaped commerce, fundamentally altering how businesses interact with customers and disrupting brick-and-mortar stores. The music industry underwent a seismic shift due to the advent of digital music and streaming services, which hurt physical CD sales. Smartphones created an ecosystem of mobile apps that impacted numerous industries. Online advertising captured a significant portion of ad spending from traditional media outlets. Craigslist and similar platforms demolished conventional classified advertising, tanking newspaper revenue. We live in a constantly changing world, and everything must evolve or fade away.
Although not everyone has realized it yet, generative AI is a sea change on par with these previous advances. As the cognitive scientist Alison Gopnik has noted, large language models are cultural and social technologies, much like writing, print, and markets, as well as library card catalogs, Google, and Wikipedia. All of these technologies enable people to access, synthesize, and leverage information that others have created or accumulated. Given this trajectory, it’s inevitable that AI will become increasingly woven into our daily digital interactions.
So yes, I think business models predicated on eyeballs and attention will suffer, and companies that rely on such models will have to adjust their approaches to survive. That’s undoubtedly stressful, but given the multitude of ills surrounding advertising, perhaps this will encourage companies to refocus on serving customers, rather than exploiting them. It’s also likely that content licensing or revenue sharing of some sort will play a role, at least for larger publishers.
However, what I would prefer to see is a system that pays micro-royalties based on the materials used to generate responses. The technical, legal, and social hurdles to implementing such a system are significant, but some form of business collaboration between for-profit content creators and AI companies will be necessary in the long run.
For now, though, answer engines represent a real improvement in how we find and absorb information online. While traditional search engines remain best at taking you to specific pages, services like Perplexity and ChatGPT excel when you need a direct answer or want to explore a complex topic. The more you use them, the better you’ll become at formulating questions that get at the heart of what you want to know, and the more time you’ll save compared to traditional searching.
Thanks for the article and the links.
I tried asking a question using the Perplexity link. This is a question that I have researched extensively over the past few days. The answer provided by Perplexity was good and matched what I had found earlier. But I would not know that if I had not already visited the relevant pages.
Intrigued, I then asked about the Apple A1243 keyboard and Mac Studio issue that I just posted to TidBits. It returned much of the same information I had already found using a regular search…plus a whole lot more that I will dig through.
So 2-for-2 on testing. That’s a good start but I’ll have to convince myself by doing Internet Searches vs AI Answer Engines before I commit to the latter.
I’ve been using Perplexity Pro for about four months after being given six months’ access via a Logitech promotion. I’ve been very impressed and plan on continuing to use it after my Pro access expires. I ask it almost anything that can be answered based on facts, and I don’t remember ever stumping it. It really does beat doing a Google search and then wading through the top ten or twenty results. The answers are clear, complete, and to the point, and (as Adam mentions) link to their sources.
I’ve noticed that Google searches usually return an AI result as well, and those can be very good, if not as complete as Perplexity’s answers. I don’t remember requesting access.
Thanks Adam. This is probably the most comprehensive and useful article I’ve come across on TidBITS.
I have just one thing to add: For those queries where a search engine is preferred, people should try www.startpage.com. It uses Google as the underlying engine, but shields your browser from Google’s voracious privacy invasions. Plus, it won’t skew the results based on what it “knows” about you, as it doesn’t gather info on your searches and behaviors.
Thanks for the informative article. I installed the CES extension on Safari on my Macs, mobile devices, and AVP and pointed the default search engine to Perplexity as illustrated. I have a complementary 1-year subscription to Perplexity Pro (courtesy of Zoom), so I signed into Preplexity on all my devices to take advantage of that.
If you activate the Quick Search option in CES, you can easily force a traditional web search by prefacing your search with an appropriate acronym (g for Goolgle, ddg for DuckDuck Go, etc.). So you can easily do a web search from the browser without opeing a search page first.
“The company says it will never share personal information with advertisers.”
LOL. Remember “Don’t be evil”? How long did that last?
Business will adapt to this kind of interaction. What this will disrupt is the experience of serendipitous learning that can happen with traditional web searches. This weekend, I learned about the composition of the upper atmosphere in relation to the near earth one with respect to gases like argon, nitrogen and oxygen just by following some links. Some would call this a waste of time, others see it as one of the positive aspects of interacting with technology (there is research on this matter). Soon we will pine for the time when we went down internet rabbit holes, as the today’s elder generation pines for the times spent wandering the library stacks for inspiration.
I would actually argue the opposite based on my experience. It’s common for Perplexity or ChatGPT to provide more information about the topic than I need to know, so I often ask follow-up questions or use Perplexity’s query suggestions to continue the exploration.
With lists of links, the likelihood of my exploring random pages is pretty low.
When I was on vacation in Europe, I quickly exceeded my daily question allotment with ChatGPT when touring the Louvre. I decided to pay the $20 a month for the Plus service, and I haven’t looked back since. Not knowing a ton about AI searching, I did quickly learn that I could ask and receive very in-depth answers to myriad questions, without having to plow through websites in the hopes of gleaning the answer. I like the term “Answer Engine”, and am blown away by the responses that I’m receiving. And ChatGPT asks and suggests tons of follow-up questions that tend to inform and expand my curiosity.
For instance, I’ve recently had questions around cow’s milk in Europe vs US, as I can tolerate the milk in France but oftentimes not in the US. I asked both Perplexity and ChatGPT the same question, and they took completely different paths in answering. Perplexity talked about the processing methods, hormone use, fat content, etc., while ChatGPT immediately mentioned the difference in cows. The US primarily uses Holstein cows, which produce an A1 protein, while Europe uses Guernsey, Jersey, and some Asian or African cows, which produce an A2 protein. The A1 protein isn’t tolerated by many people, which leads to stomach discomfort and causes many, myself included, to believe that we’re lactose intolerant. The A2 protein is found in sheep, goat, and human milk and is tolerated. ChatGPT then went on to discuss the same things that Perplexity mentioned, though Perplexity never mentioned the A1/A2 protein difference. This was fascinating to me as well as being eye-opening!
Bravo, Adam. Excellent article.
They do? Hmmm. I wonder if I got grandfathered in to a free account or something, because I use ChatGPT all the time and I don’t pay them anything.
I agree completely that this is a sea change on the order of the examples you gave in the article (online shopping, digital music, etc.). I was just telling my wife the other day that she’d be able to tell our grandchildren of the days when you tried to answer a question by typing in some hopefully pertinent and distinguishing words, and then searched the list of pages that came back to see if anything resembling an answer appeared. I predicted they would laugh with incredulity, because they will have known only AI-assisted answer engines.
I mostly use Perplexity so I don’t know if this is still the case, but it wasn’t that long ago that the free version of ChatGPT only included data up through 2021, and to get the past few year’s data included in answers was what you got for the $20/month. Maybe image generation too.
You’re right! OpenAI recently added the search capabilities to the free version of ChatGPT. And the image generation. It is limited, so you may get frustrated when it times out for the day if you do a lot of searches, but it’s a lot easier to test.
https://help.openai.com/en/articles/9275245-using-chatgpt-s-free-tier-faq
I’ll update the article, since that makes ChatGPT even more of a contender.
Thank you, @ace, for yet another timely, thoughtfully researched, thoughtfully constructed and thoughtfully presented article. It will surely be useful both to answer engine “newbies” and to those of us who have been mucking around in them before they could even “see” the internet as it existed after 2021.
Your results — much more formally derived than mine, of course — match mine precisely.
I would only add this gentle advice to our TalkBITS colleagues — if you haven’t taken a course or two on using the new AI tools, I commend them to you, highly. I took several last year, offered online by Vanderbilt University via Coursera. They were virtual rocket fuel for the acceleration of my prompt structuring skills, and I think daily of what I learned, often referring to notes for those techniques not yet habits.
(A bit more info in these posts:
(first set of courses)
(second set of courses)
although I’m sure there are lots of newer courses to consider!)
(And for those of you not yet TidBITS members… where ya gonna find articles this incredibly good and immediately useful, really. Please consider membership!)
Maybe I’m just old. But I can’t see myself paying, yet another subscription fee to search the web. However I do agree this is the future of the internet. I’m glad you did point out that there’s different queries made. Sometimes “answering a question” isn’t what I’m after. Just links, please. I tend to look for images a lot. But as one person noted above there’s something to be said about going down rabbit holes on the internet. I’d surely miss that
Thanks, Adam, great article.
1 I’ve been experimenting on and off with Perplexity since you mentioned it in a piece a few months ago and I’ve found it gives excellent results, provided of course that you ask it an appropriate question. As you point out, there are occasions when a search engine is just as useful.
An aspect you don’t mention however is the cost to the planet of using answer engines – ten times as much as search engines! This is a real concern to those of us trying to follow a green agenda. You may be interested to read the following from Perplexity:
I feel we need to take a critical look at our usage, and use answer engines only when there’s a good reason to.
2 I was puzzled by your statement that “Changing search engines is more difficult or impossible in Safari on the iPhone.” On my iPhone, if I go to Settings/Apps/Safari, I can choose one of five search engines as my default.
I think Adam’s intended meaning was that it was difficult to change it to these ‘answer engines’.
Those of you who would like to be away from all of this might appreciate…
Google without the AI.
I have to say I’m very happy with Perplexity Pro, paying the annual fee (Zoom didn’t give me any offers… hmmm).
I’ve found the Deep Research mode available there to be really very impressive with what it can draw upon, including on topics I have a good knowledge of myself. I have found new information and insights on a regular basis. Recommended.
I’m sorry if I am missing something that explains why Grok isn’t included anywhere in this discussion. Please let me know.
Certainly just limited anecdotal evidence but Grok has been amazing in helping me consider a diagnosis algorithm, interpretation of test results and various treatment options for medical conditions.
For example, I uploaded extensive data from a recent cardiac ultrasound report (simple copy and paste) which provided valuable information not included in the report by the cardiologist who interpreted the data. I had this verified by a friend who is a cardiologist.
Next I added maximum heart rate (MHR) and resting heart rate (RHR) data along with rowing performance data from my Concept2 erg data and was further amazed how Grok considered all of that to both explain why I was a unicorn for my age due to a high MHR and offered how to best create a program to further improve my VO2Max, providing specific frequency, duration and exertion guidance. It even considered my naturally elevated testosterone (since I also uploaded a bunch of clinical blood test results), describing potential cardioprotective effects.
As I said, just anecdotal but Grok (and likely some other answer engines/AI) seem to have the capability to go well beyond what my Internist can both process and communicate.
fyi, Grok is owned by Elon Musk.
Thanks for an excellent article, @ace . I’ll recommend it even to non-Apple folks.
A few comments:
Perplexity is my primary generative AI engine, but I’ve found Grok to be quite easy to use and often to yield very good results. It’s my secondary engine.
I did have a sobering experience with Perplexity in recent weeks that highlights the need for caution when using generative AI. I’ve been working on a project where I’ve needed to generate around a dozen one-page information sheets for a subject about which I have significant first-hand knowledge.
The info sheets needed to be in a very specific format, and they each needed the equivalent of 10-15 bullet points each. Since I was under a lot of time pressure, I asked Perplexity to create a first draft for each topic.
On the positive side, Perplexity generated each document in seconds. Each was perfectly formatted and written to a very high, professional standard of vocabulary and grammar. Further, all were “directionally correct” in the sense that a non-expert of reasonable intelligence could be expected to understand them and, if asked about them a day or two later, could say a few things about each that, at a high enough level, matched reality.
On the negative side:
Unfortunately, I can’t share the specifics, but I can give an example of what would have been an embarrassing error if not caught:
If I hadn’t have caught the error, it would have been bad for a couple of reasons:
The sobering part is that the results were so well presented by Perplexity that I think a lot of people with casual knowledge of the subject would have been comfortable forwarding them as official documents with only minor corrections.
Overall, Perplexity saved me some time, especially since it provided links to source material that helped me to “fact check” easily, but it did take some real time and effort.
Bottom line: if the material matters, and especially if it is going to be shared on the Internet, DON’T SKIMP ON THE FACT CHECKING.
Just a caveat here for others - I got back into my Logi Options+ to look at my settings (I have a MX Master 3S), and the Perplexity promotion is available there. No indication of how long it might be there, but if anyone else were interested, and have a Logitech device, it might be worth checking to see if you can see the coupon code there. The MX Master 3S has been one of my better purchases. I absolutely love the way I can easily move back and forth between my work and personal Macs using the input select button on the bottom.
To be honest, I probably would have ignored the promotion if not for Adam’s article, as I’ve been avoiding AI for the most part (Microsoft keeps trying to force copilot across their entire product line that I use at work, and so far I’ve only found value in the ability to provide transcripts and reasonably accurate summarizations of video meetings). I use the Kagi search engine (and I’m happy to pay the $10/month for the increased accuracy and lack of marketing), and I’m finding that Perplexity works well in addition to Kagi. I still use Kagi for most of my targeted searches, but Perplexity is useful for more wide ranging searches where I don’t have enough info to form a targeted search.
I’m also using the Monarch launcher, and it was easy to set up a superlink for Perplexity - see the pictures below:
And in use:
oh sorry, I didn’t realize we were including ownership as pertinent to this topic.
In this case, I think it’s something many people would appreciate knowing.
I wonder what the consequences of this will be in a few years, assuming that web sites lose a significant number of their human viewers that turn to AI bots instead.
Sites like the New York Times. With many fewer human visitors to be bombarded with ads and enticed into subscriptions, it will have to rely more on AI licensing fees for survival. Increasing licensing fees will then cause consolidation and reduced competition among AI providers (which will probably happen anyway). Or will the owners scrap the whole news-producing part of the company and turn it into New York Wordle?
What about educational institutions like universities and museums that today make the effort to provide attractive and informative sites? Will they continue to do so simply to support the minority of web users who take @josehill’s wise advice to do fact checking?
What about sites produced by one or a few people that rely on some combination of ads and subscriptions? A site like the Online Etymology Dictionary, which I enjoy reading. Will they continue to make their information available to an audience of primarily AI bots?
I view social media and social media recommendation algorithms to be a bigger threat to general audience news publishers than generative AI “Answer Engines”. I also believe specialized news sources that serve professionals in a field, such as Bloomberg, will continue to operate in ways similar to today.
More broadly, I don’t think Answer Engines will affect website traffic and usage habits that much. Answer Engines are not that different from traditional search engines. As such, I don’t think clickthrough and follow-up habits will change to a great extent. People who rarely go beyond a Google result summary will simply read an Answer Engine summary and move on to another activity. Similarly, people who love to dig into results, look at pictures, and watch videos will continue to follow hyperlinks.
It’s pertinent to some of us. Bob, thanks for pointing it out.
Honestly, I had no idea that xAI’s Grok could do Web searches, which is why it’s not included. Your example, while compelling, is just standard chatbot behavior these days because it doesn’t require access to information retrieved from the Web. I wrote about doing that sort of thing with ChatGPT back in January.
That said, had I known that Grok could do Web searches, I wouldn’t have included it because I refuse to promote anything associated with Musk and X/Twitter, which is now owned by xAI in a corporate shell game.
One thing worth keeping in mind is that sites like The New York Times receive a significant amount of direct traffic. If you actually want to read the news, you’ll go to the site and see what’s there.
I suspect we’ll see a lot more emphasis on subscriptions and direct connections rather than relying on search traffic, which I always thought was a mistake anyway. It’s all too common for search providers to update their algorithms, causing previously popular destinations to suddenly see significantly less traffic.
I also expect news sources to have their own AI that will run through their own data to give a NYT or other answer. Medical journals are looking into this
Thanks, Adam, for the excellent discussion. FWIW, I’d support Scott Kuntzelman’s use of Kagi as a default search engine. I too will frequently use Perplexity Pro for complicated search questions (and use ChatGPT, Claude, Gemini, and Consensus as well) and like others have found that the recent LLM iterations have become less prone to hallucinations - though they definitely still do make mistakes.
My wife and I gave up on Google almost 2 years ago, since their search results had become increasingly cluttered with advertising. We’ve been using Kagi for well over a year via the duo professional option ($160/year for 2 people); a single is $10/month. They continue to incorporate some AI (see their AI Assistant). We’ve found it blissfully free of the forced ranking from advertisers that has made Google a trial. It’s powerful, fast, and very customizable (restricting searches by recency, advanced searching with OR, excluding words, academic sources only, etc.). Queries are anonymous, there’s no data retention and Kagi translate is excellent. If you are into privacy when browsing, their Orion browser is worth a try. John Gruber and Cory Doctorow are fans.
From Jason Koebler’s
404site:Kagi is our default browser search engine on all our computers, iPad, and phones. As for us, we are both very happy to pay $160/year to get a vastly improved search experience (I’ve no connection whatsoever to Kagi except as a happy user).
What blows me away is there was NO mention at all about something a pal turned me onto a few weeks ago. My friend and I also fed the same query to it and ChatGPT and both of us felt the answers far better in… Gemini. Maybe it’s the branded version of the GoogleAI mentioned. The only difference I spotted was ChatGPT can provide an image as the answer while Gemini can’t.
One example, I put in a general query about specifics of the monitor (1440p, mini led backlight, 27" size) I wanted and it came up with exact matches including the one I had searched for on Amazon for the last year or so (which probably also speaks to how god-awful their search engine is, I mean they should be embarrassed it’s so bad).
In the past 2 weeks it has actually answered a lot of queries I have had that the regular (google, tried duckduck, found it lacking) search engines could not find. Gemini’s answers are very well written, much more so than even software instruction manuals… yes, he and I have asked it questions about how to perform certain actions in various software programs and it always hit the nail on the head with complete and very thorough answers. Gemini not only supplies an answer, but explains how/why it came up with that answer. It’s 100% amazing.
Interesting, isn’t it, that you left the ‘most controversial aspect of answer engines for the last.’
"So yes, I think business models predicated on eyeballs and attention will suffer, and companies that rely on such models will have to adjust their approaches to survive.”
Many businesses are already suffering deep damage due to answer engines, are they not? It is well known that many AI companies are justifying their theft of user data, if not doubling down on it, by simply ignoring ‘do not index’ directives in robot.txt files by small publishers and even finding ways of breaching paywalls of commercial publications. Yes, The NY Times and big publications with deep pockets may eventually get paid somehow, but small hobby publishers seem to have zero recourse as of now, other than to charge subscriptions. Hence the proliferation of the substacks and the mediums and such platforms, where you essentially have no ownership of your data (unlike if you had your own website or blog).
The generative AI industry is even more destructive because they are positioning to essentially wipe out large segments of creatives including writers, artists, movie-makers and more (hence, the recent Hollywood strike).
And yet, here we are. Your suggestion of ‘micro-royalties’ is interesting but I wonder - once the AI ‘groks’ all the data it wants to ‘grok’ with impunity, what incentive does the company have to pay the small creatives even a single penny for their creative work? Who is going to stop them from ‘grokking’?
P.S. Congratulations on the 35th anniversary. I sincerely wish you many more. Thank you for your excellent coverage of all-things-Apple over the years!
I used Perplexity for a bit a year or so ago while also using Kagi as a regular search engine. After a while I found that I preferred Kagi because it was more of a traditional search engine, and I felt like the results were not so much opinionated as they were on Perplexity.
At the time I did not have the Kagi Ultimate package (which includes a bunch of AI engines). If you add the Ultimate subscription, you get more or less the same toolset as you get with Perplexity, but (for me at least) I feel more in control of my searches.
Thanks so much, @ace, for this article. Only yesterday, contemplating how to get a handle on AI and reflecting back on how I got a handle on the internet in the first place, I found myself wishing that I had something like The Internet Starter Kit for Mac to ease me into AI. (I still have the first edition of your book, complete with its 3.5in floppy.)
I did give Perplexity a test, which it pretty well failed, but I guess we need to be mindful of the limits of these tools and careful in phrasing questions. Over the weekend I managed to wipe out all my current tabs. I imagined that these could be recovered from a Time Machine backup of Safari preferences, like I used to do when accidentally wiping out all my open windows. But where exactly are those preferences now? I eventually found the answer in a Reddit thread after scrolling past three screens worth of Google pages telling me about ways to use the Safari History menu. When I tried with Perplexity it was so convinced by all the pages about the Safari History menu that it never bothered to present recovering preferences from a backup as a solution.
I nonetheless expect that the time has come when these answer engines may be useful for the likes of me. In addition to wanting a Starter Kit hand up, I was thinking that a structured course might be good, so thanks also to @jeff2 for suggesting Coursera.
PS Happy anniversary!
Just FYI @pbinderup " we’re excited to announce that Kagi Assistant is now available to all users across all plans, expanding from its previous exclusivity to Ultimate subscribers, as an added value to all Kagi customers, without increasing the price."
I hadn’t used Kagi Assistant until this week, so now I’m experimenting with it. The nice thing about Kagi Assistant is that you can choose the model from a single interface from a list (and if you have Ulitmate, you can choose premium models). I’m still fairly early in my experimenting with the tools, so I’ll continue comparing Perplexity Pro for the six months trial provided by Logitech, and experiment with the various models I can see from Kagi Assistant, and see how things go over time. It’s interesting when reading various blogs and comments on social media how many people seem to have a favorite model, but there’s quite a bit of variation. I suspect that part of it is that different people will phrase their questions differently, but it’s also likely to depend on the questions being asked and how well each model accesses the various sources that are relevant to the questions (I may not be stating this clearly, but LLM hasn’t been my focus due to my work responsibilities).
Deciding which chatbot and, within that, which model, you like most is tricky. As best I can figure out, the only real option is to see what feels best. In some respects, that shouldn’t be surprising, since working with an AI is much more like working with a person, and it’s difficult to know which person is “best” as well.
That said, I don’t have a feel for how the split between the Web search and what the model does with it breaks down. If an answer engine’s search is on the weaker side, that will affect the results, as does what the model decides to do with the results it has found.
There is a way to add arbitrary search sites to Firefox, but it is now hidden in the
about:configoptions. Add an entry forbrowser.urlbar.update2.engineAliasRefreshwith the boolean valuetrue.That will expose a new “Add” button in Firefox’s regular Search Preferences. You can then add a custom engine by name and URL as you can for other browsers, with the
%ssubstitution. It will appear as an option in the URL/Search field menu like any other engine.(Ironically, my main use for this is to add an AI-free version of Google; the search string for that is
https://www.google.com/search?udm=14&client=firefox&q=%s)Please be careful what you use AI tools at their current level of sophistication (training?) for. For financial advice, for example, they (to use a technical term) suck rocks. Finance Agent
Ha ha, I’d say gen-AI is no better or worse than influencers and random strangers on message boards and social media. After all, that’s where a lot of the data is scraped from!
:-)
I tried the CSE extension… I had duckduckgo configured as Allowed in Safari. I set up CSE to the Perplexity site. Now when I do a search it seems to go to duckduckgo then forward to perplexity… That slows things down. Is this correct?
Sorry, what is “the
%ssubstitution”?It’s the placeholder value in the URLs used to insert the text you enter when the query is sent off to the search engine. The
%sis replaced by whatever query you type.What I mean is, since all the examples in the article already use that same
%splaceholder, they will work in Firefox as they do in the other browsers mentionedIf you first activate Preplexity, it will take longer as you do an extensive search and write an organized footnoted summary. However, the second setting is a switch to turn on ‘Quick Search’. This allows you to send the prompt directly to a search engine by prefacing it with the appropriate abbreviation (list by tapping the ‘Quick Search Engines’ list). They’re pretty much what you would expect (‘g’ for Google, ‘b’ for Bing, ‘ddg’ for DuckDuckGo, etc.).
Verification is always essential when accuracy matters, but answer engines don’t change the situation at all.
I was recently conducting research on business cyber insurance and sought clarification on whether the statistics being shared applied to small and medium-sized businesses as well as large firms. (According to the 2024 IBM Cost of a Data Breach survey, the average cost of a data breach is $4.88 million globally and $9.36 million in the US, which are numbers that don’t mesh well with my view of what a small or medium-sized business could afford.)
The initial answer engine responses claimed there were specifics related to smaller companies, but when I followed them, I ended up at articles that summarized the IBM study and made claims about smaller firms. I went to the IBM study and found that, apart from a statement about over 600 companies of various sizes being surveyed, the researchers explicitly did not collect demographic information about the companies surveyed for privacy reasons.
In other words, the answer engines provided me with inaccurate data because people had generated incorrect information about the IBM study. When I asked ChatGPT about the mistakes and prompted it to examine the primary source, it acknowledged that no information about company size was provided.
Adam – Is there a reason you omitted Gemini as an AI answer engine? I have asked some complicated questions and the 2.0 Flash version has produced satisfactory results. AFAIK it’s free.
Based on my non-AI (no Assist used) supported DDG search: Gemini is an AI ‘product’ of Google/Meta/Alphabet/etc. So I’m sure it will “do no evil”.
As with Grok, I wasn’t aware that Gemini included real-time Web searches in its responses. It does seem to be able to do so—I could ask it “What has TidBITS written about text fragment linking?” and it knew about my article from yesterday and linked to it.
However, in no other query I’ve fed it from my spreadsheet of answer engine queries did it reference any sources or provide links to source material. I consider that an essential feature of an answer engine because otherwise there’s no way you can check what it’s saying.
A PR person sent me a link to this study about AI chatbots and search engines. It’s interesting, though it’s also clear that there’s no real competition here, either between Google and other search engines (Google has 1631.5 billion visits to Bing’s 60.1 billion, a 27x difference), or search engines and chatbots in general (1863 billion versus 55.2 billion, a 34x difference). The infographic lays it all out.
Realistically, I think answer engines will win out, but by all the traditional search engines moving toward providing answers, not by ChatGPT and Perplexity supplanting Google.
But it’s not a zero-sum game. There’s room for numerous players. It does make one wonder what Apple will do, though. The ChatGPT and possible Gemini integration with Siri is pretty minimal.
I’m curious to see what will happen as the Google antitrust matter plays out (this appears to be party agnostic - it’s been running since 2020). If Google is required to stop paying device manufacturers to be the default (or only?) search engine, how will that affect the overall landscape?
Interesting chart Adam. Extraordinary position Yahoo still maintains. Surprised.
Its quite something to see how ChatGPT has so rapidly achieved the reach it has. That rate of growth has very few precedents.
Yes, if Google can’t pay to be the default search engine anymore, that may change things. But a roughly 30x difference in traffic will take a long time to break down, and as I said, “googling” is a verb. No one ever talks about “binging” or “duckduckgoing” or “chatgpting”.
Something concerning I saw earlier today: Perplexity does not make their terms of service and privacy policies easy to find (no links on the site). Once you find them (the guy found them by guessing the URLs), they essentially say that they can track and profile you automatically, and share the data however they want, with whomever they want.
The guy who wrote the thread was Luke Mulks, the VP BizOps for the Brave browser. Unfortunately, the thread is on X. I can export it out to .pdf if requested. The context was Proton pointing out that the Perplexity CEO says it’s ok to track everything to serve “hyper personalized” ads. I’m trying not to get into the weeds, but privacy and security are important, so I watch various sources for things that may affect me.
I wouldn’t assume that Perplexity is the only AI Answer Engine in this boat, and it’s not like most of the search engines aren’t already harvesting data.
Interesting. Frederico Viticci on MacStories quotes a year old article in Wired which says that they don’t respect robots files blocking scraping either. What Siri Isn't: Perplexity's Voice Assistant and the Potential of LLMs Integrated with iOS - MacStories
Viticci’s piece is about how Perplexity does what Siri should be able to.
He wasn’t looking very hard—it took me two clicks. I scrolled to the bottom of the Perplexity page but didn’t see many links, so I clicked one that seemed likely to provide more information about the company (Blog). Then I scrolled to the bottom of the blog page and found the privacy link.
https://www.perplexity.ai/hub/legal/privacy-policy
Then I asked Perplexity about its privacy policy and to compare it with others, which is about the level that I had time to look into this.
https://www.perplexity.ai/search/please-recast-perplexity-s-pri-IfTkw_UUSLu9pAoqeqRgFg
Thanks, Adam! I was in the middle of looking for data for work when I found it, so I didn’t go down the rabbit hole myself. It doesn’t surprise me that the claims may have been “enhanced” for shock value. I’m still worried about privacy around the AI engines, although it’s not limited to Perplexity (and I’m still kicking the tires with it). I’m still kind of leaning towards using the Kagi Assistant, as it provides access to multiple models and emphasizes privacy (see below).
Adam, thanks for the great article. Would it be possible to share your Keyboard Maestro macro? Thanks!
Sure—here’s a copy. It’s nothing special, but there are a few little bits that might get you over a hump.
QuadAI Search.kmmacros (12.9 KB)
I decided to try out the Vanderbilt course “Prompt Engineering for ChatGPT”. I used several AI’s to search for courses, but this seems to be the best for me. It is taught by Dr. Jules White (Dr. Jules White, Instructor | Coursera) He has the best instructor’s voice and manner I have ever come across. Thanks for telling us about it!
Adam, thanks! Much appreciated.
My immediate take away from the free Vandy course I took is that I may never be able to trust anything “AI” tells me. The title just about explains major concern about these bots; ask a detailed, specificly taylored question and then be prepaired to ask multiple, ever more detailed “prompts”. While the responses are very impressive in their ‘computer sophistication’ that is simply shows my appreciation of the devs amazingly good programming skills.
My main concern is that the reliability and trustwortiness of a response is still dependent on the skill of the user to design the ‘perfect’ question(s).
I am still left with the ethical dilima of the language models built from the output of real people but without any respect or even acknowledgement of the original author nor any thought of recompense.
I don’t think I can name another app or software genre more open to abuse and damage. Perhaps that is something ChatGPT et all. can ‘halucinate’ for me.
Maybe I’m just too old (82 yo) for this change.
As I have said on many occasions, this is because these large-language models are not intelligent. Not at all. Not artificial, natural or otherwise. They generate text based on massive probability databases encoded into a neural net. These probabilities are 100% based on the corpus of text with which they were trained. But there is no understanding of anything in that text.
It would be like you trying to answer a question in Russian, when you don’t speak Russian, but have thousands of Russian books (which you can’t read) from which you can copy phrases based on how similar the question text resembles text from those books. You’ll produce an answer. Maybe it will be right, maybe it will be wrong, maybe it will be something in between. But you will never know, and you will have no clue what you were asked or what your answer means.
Nothing generated by these ML models (I refuse to call them “AI”) should be trusted without independent verification. And for any non-trivial question, the verification process could easily involve more work than finding an answer the old fashioned way.
Absolutely true. And because it will generate responses based on its training texts, it is trivially easy to create a chatbot biassed in any direction you want. Do you want to reinforce some opinion? Just omit all texts with dissenting opinions from the training data. And voilà, it will give the answers you want, even if they are provably wrong. And plenty of people will believe it because “it’s a computer” and is therefore assumed to be correct.
As I’ve mentioned in other threads, here’s when I use AI Answer Engines: if a traditional search site is going to be complicated to use for something (say, because I only have a vague recollection of something I’m looking for or I’m not sure of the exact wording I need to use), the conversational language prompts make searching easier and quicker.
I don’t view iterative prompts as a disadvantage in these cases because using, say, Google or DuckDuckGo will also require mutiple queries to narrow a search. As well, results returned by a legacy search engine need to be given a reality check, especially with the prevalence of sponsored links, links appearing due to SEO (search engine optimization) tactics, and content mills. So as long as an Answer Engine provides links to its sources, verifying information is about the same to my mind.
(I’m not trolling you here…your post led me to think about the issues you raised)
Isn’t that also the case with other ways of acquiring information online? On message boards, for example, posting “My Mac isn’t working” will generate a much different set of responses than “I just upgraded to Sequoia and now my Time Machine drive won’t connect”.
I think that’s an ongoing and long-time problem, no matter the online venue. People share links to paywall-bypass apps, web spiders and bots scrape content for all sorts of uses, articles are pasted into social media feeds, the list goes on and on…
How about social media, deep fake apps, and cryptocurrency? I believe, at this moment, these present more risks to society than Answer Engines.
If you’re interested in following nefarious uses of the Internet, an interesting website I learned about here on TidBITS is:
I’m not sure age is as much of a factor in attitudes towards finding and using information from various online methods than how trusting/skeptical one is in general and one’s feelings about the role of experts and expert knowledge.
There’s truth to this. What concerns me is that confronted with the maelstrom of contemporary media the average person is going to take the “tailored” reply from ChatGPT et. al. as more trustworthy because it sounds like ChatGPT is talking to them, personally. That’s not good. And they’ll readily take it because it’s free and fast as opposed to going down to the public library and asking a librarian to help them out with a question that might have real consequences. [All you maestros of web search might want to try a visit to a local or university library and chat with a librarian sometime. You might be surprised.]
This is an excellent summary of the problem. The probability machines can be astonishing and often useful but can they be trusted? No. Absolutely not.
Dave
This reminded me of this:
You can read the whole article here: Personality and Persuasion - by Ethan Mollick
This should be particularly frightening. That exact same tech, using a model trained on different texts could just as easily be used to gaslight people into believing misinformation.
And this doesn’t even have to be from a deliberate attempt to mislead. It could simply result from the biases of the engineers or project managers working on it, since they select what news sources are used for the training. If they include sources that are biased in one direction and omit sources biased in the other, maybe because they tend to share that bias, then the resulting chatbot is going to present similarly biased results.
And if you connect to a random chatbot that you didn’t personally develop, how could you possibly know whether or not this is taking place? You can’t.
Given how many news articles from a few years ago, from just about every outlet, have been later shown to be wrong (or for the more cynical, deliberately lying), do you really want to trust software trained on these news articles?
I agree the potential for persuasion can be a significant problem with online information channels, especially when the human needs for empathy, reciprocity, and community can be fulfilled. Dull, dry search results on Google for “is the Earth flat” cannot ever be as appealing as a podcast interview, an influencer video, social media conversations, or a seemingly infinitely patient and friendly (or even sycophantic) generative AI that gently pushes the idea that the Earth is flat.
For anybody interested in diving into persuasion and conspiracy theories, here are three books I recommend:
https://www.amazon.com/Influence-New-Expanded-Psychology-Persuasion-dp-0062937650/dp/0062937650/
https://www.amazon.com/People-Believe-Weird-Things-Pseudoscience/dp/0805070893/
https://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374533555
I’d argue that the chatbot is worse.
Everybody knows that printed articles, interviews, videos and social media conversations are the work of humans and express the opinions of humans. Most of the audience will quickly form an opinion about the person and filter their understandings based on that opinion.
But there is a perception that computers are unbiased because they’re not human, so people will be more likely to believe the chatbot. Or more precisely, the chatbot will have to be much more obviously wrong before people will discount it and stop trusting its output.
The adage “you can fool some of the people all of the time” is still completely true, but these chatbots are increasing the size of “some of the people”.
I don’t view beliefs as binary and permanently set. I’d say any communication channel, online and offline, that is able to make a person feel like somebody is listening to them, takes an empathetic approach, can be accessed repeatedly, and offers community can be extremely persuasive…even if a first impression is negative.
I’d also say that for a lot of people, trust is a bigger factor in accepting assertions than judgements about bias. The role (for older generations) belief-siloed cable news networks and (for younger generations) echo chamber social media algorithms play in how people view “reality” is an example of this. After all, a bias one agrees with feels “fair and balanced”, right?
Something I’ve been pondering of late is the fact that you can, to a large extent, control a chatbot’s personality.
The big realization I had recently is that nearly everything about how a chatbot tailors its response has to do with prompts. Even security vulnerabilities arise when people figure out how to word prompts in ways that evade safeguards. ChatGPT’s recent personality problems were the result of a prompt. The Lex.page word processor I’ve been using just introduced personalities—several canned, but you can make your own—for its built-in chatbot that you interact with while writing and editing.
In other words, we shouldn’t feel as though we’re lowly users at the mercy of the chatbot developers. We have power and agency, and we get to determine to a great extent the kinds of behavior we’ll see from the chatbots we use. ChatGPT even provides a place where you can give it instructions (I don’t yet fully understand the new + buttons; the ones you see are the ones I haven’t clicked). I seeded mine with this list of instructions from Seth Godin; I’ve subsequently tweaked it.
Yes, there are many ways of starting your interaction with the AI that can guide the way it answers you.
Two variants are called the Persona Pattern and the Audience Persona Pattern.
Then I change the way it answers.
Yes, and I view the ability to vary how a genAI creates results to be an important advantage over traditional search engines. Not everybody will use this capability as a way to be skeptical or as a way to explore different viewpoints on an issue but at least the capability is there.
But that doesn’t solve the real problem. I just replaces one with another.
So instead of needing to be a search-engine expert in order to find the results you want, you now need to be a chatbot expert.
You may prefer one over the other, but neither really solves the problem that it is hard to get reliable results from the Internet if you’re not already familiar with the subject matter.
And I would argue that this problem can not actually be solved, because the Internet is full of contradictory information, much of which can only be navigated by a domain expert.
It feels like being a “chatbot expert” now, but my belief right now is that it won’t last indefinitely. Where I think we’re going is personal assistants who know a lot about us, both personal context, like Apple keeps banging on about, and more amorphous things like education level, areas where we’re already experts, and areas where we have no knowledge at all, and so on.
Right now, that requires telling the chatbot all this information or prefacing specific conversations with the sort of prompts that @paal demoed above. But ChatGPT’s move toward using your past conversations as “memory” is a step in this direction, Gemini’s ability to use your Gmail as a data source is just starting to be helpful, and Apple’s push toward including personal context in Siri is going in the right direction, even if the reality isn’t here yet.
And yes, that will mean that a lot of people end up in bubbles of their own making, but that train left the station long ago.
Knowing those who own technology services companies is especially pertinent, especially in the case of Musk.
My comment originated from concern that valuable approximation of insightful objective assessment and evaluation, something we have come to appreciate from Adam’s efforts…in this case of the performance of the various services…risked being diminished when exclusion based upon judgment is involved.
I’m leaving it at that and Adam’s thoughtful comment…
“So I suspect that this article is something of a Rorschach test, revealing as much about the reader as it does about my opinions.”
While I agree that “because… Musk” could be seen as subjective judgement, there is data to call Grok’s utility into question:
And more recently:
Thanks for this Ron.
My very limited use case was related to a query about a fairly complex medical situation for which I had >30 different data points including clinical lab values.
I appreciated Adam’s response of “Your example, while compelling, is just standard chatbot behavior these days because it doesn’t require access to information retrieved from the Web” yet I can see I did a poor job of presenting my(n=1!) findings that Grok was superior in accuracy and clarity of presentation over other chatbots (ChatGPT, Copilot, Perplexity, Gemini) to which I submitted the exact same query. (some of the data calculations by Perplexity and Gemini were clearly hallucinations)
I ran all of the results by my Internist and Cardiologist and a particular insight offered only by Grok has subsequently helped them fine-tune an important aspect of my treatment. I thought my offering might be of some value to others and it was with this intention (and unexpressed caveat) that it was offered.
But that’s not what you’re addressing.
For me, and I’m not saying anyone else should be like me, all “data” everywhere and all the time is to be called into question including content from the articles you cited. I am forever biased from my psychoanalytic training which has taught me that no one is aware of what they don’t know aka…the problem with the unconscious is…it’s unconscious!
Most of us much of the time are operating from unconscious conditioned complexes which contaminate our beliefs. Our egos don’t like to hear this and will diminish and deny others not in alignment with those beliefs which often results in exclusionary behavior.
I prefer not to exclude those who may carry aspects of my shadow which does not equate to an endorsement. YMMV and I support and respect your’s and everyone’s right to do it their own way from whatever state of consciousness they can muster!
Have a lovely weekend!
Well here is the data. AI answers reduce clicks by about 2/3. There are other studies out there as well, but all seem to show that clicks are reduced by at least 1/3. It is clear that interacting with search will be fundamentally different.
Fascinating data, though it feels highly specific to Google’s current interface for the AI overviews. I suspect other interfaces would likely reduce click-through even more.
I was also struck by how “traditional” the example searches were. That’s how you use Google, but as I found when researching the AI answer engines, it’s more effective to ask for what you want to learn directly, rather than trying to guess keywords.
And, of course, the real question here is what the goal is. The goal of the person doing the search is to get their answer, not to provide click-through eyeballs to advertisers on the sites hosting information related to the search. So, another way to interpret these results is that AI overviews better meet user goals two-thirds of the time.
Google has now promoted AI Mode to the main interface.
I see they somewhat color ChatGPT with the “Browse with Bing” annotation…
I like Tufekci’s term “plausibility engine” for LLMs/generative AIs.
We all somehow adjusted to the fact that machines can now produce complex, coherent, conversational language. But that ability makes it extremely hard not to think about L.L.M.s as possessing a form of humanlike intelligence.
They are not, however, a version of human intelligence. Nor are they truth seekers or reasoning machines. What they are is plausibility engines. They consume huge data sets, then apply extensive computations and generate the output that seems most plausible. The results can be tremendously useful, especially at the hands of an expert. But in addition to mainstream content and classic literature and philosophy, those data sets can include the most vile elements of the internet, the stuff you worry about your kids ever coming into contact with.
Zeynep Tufekci - New York Times
While I’m unsurprised that Musk’s opinions may be granted undue importance in Grok responses, this whole thing does feel manufactured. A fake Twitter account, people asking leading questions, etc.
And more to the point of this article, it doesn’t appear that this situation has anything to do with enhancing Web searching with generative AI applied against search results. Whenever a response that doesn’t include data from the Web seems off, I strongly recommend saying, “Confirm with a search.” In my experience, that usually eliminates the confusion or mistakes.
I took up your suggestion. ‘Prompt Engineering for ChatGPT’ was very useful, but even I could see that it’s already out of date. Worthwhile, but out of date.
Interestingly, I just had a rather complete fail with ChatGPT. I have an old Canon Pixma IP110 portable printer that needs a new battery, so I went looking on Amazon. Searches for my printer name and “battery” turned up the LK-72 battery from Canon, works with the newer Canon Pixma TR150 printer. So I asked ChatGPT if they were compatible, and it was confident they were. But when I looked at the product pages for the LK-72 or third-party compatible versions, none listed the IP110 in the compatibility matrix. So I pushed harder, and eventually realized that ChatGPT was being confused by the fact that Amazon and eBay product pages often list many other related products and since the printers were similar, they listed products like ink cartridges for the IP110. That’s when I thought to check the physical dimensions and look at the actual photos, which revealed that they’re different sizes and have different connector orientations. So, not compatible.
However, Perplexity and Gemini and even Google’s AI answer all got the answer right. Sometimes you’re winners, and sometimes you’re losers…
And it drives me nuts. Amazon offers to auto-complete my search term for the exact item I’m seeking, and the resulting page does not contain the item that Amazon offered in its auto-complete. Argh.
Sorry for the digression. I’ll curtail the rest of my rant, which would otherwise be lengthy.
If Amazon were to offer a new state-of-the-art AmazonChatBot™ that presented you with exactly the product or the products meeting exactly the specs you specified and nothing else, now that would be a world-shaking AI development.
Dave
Yes, please.
It is still the best introduction to working with AI I was able to find. I used both AI and traditional search to find alternatives, and where trials were offered, I followed along with the free content.
I had a hard time writing this sentence so I asked AI to help me: “I have to admit that I excluded some courses due to either the presentation style or the variety of English, which, although comprehensible, required significant cognitive effort to process.”
The great thing about ‘Prompt Engineering for ChatGPT’ is that everything you learn still works and it get you into the right headspace for working with AI.
This isn’t specifically related to that article or to agentic web browsers, but I did recently use Amazon’s “Rufus” AI to help me in my purchase of a new iPad. I found it quite helpful, though not perfect.
So my situation is that I had a very old 13" iPad Pro (2018) that basically can’t be upgraded any more. It has Lightning connector and other limitations, such as the Logitech keyboard case I used with it breaking (2nd one – Logitech replaced the first for free). Since I only use this iPad for reading in bed at night, I haven’t been too bothered by the limitations, but I have considered upgrading my other iPad and “downshifting” it to night duty and getting rid of this old one.
Recently Apple announced the iPad M5 and that intrigued me, especially with Amazon having it on sale for Black Friday. But one reason I haven’t upgraded in the past is that a new iPad meant upgrading all my accessories: my Apple Magic Keyboard, Pencil, etc. wouldn’t work with the new iPad. Buying all of those again would be an extra $400.
Since I don’t buy iPads often, it’s hard for me to remember which Pencils and cases and keyboards work with which. The Pencils all sound like the same thing, but are very different, and each only works with certain iPads. Cases are a horror show of compatibility issues as they depend on the iPad size, type, and year. I got bogged down in the weeds trying to find compatible accessories and trying to figure out exactly how to proceed with the upgrade.
I was about to just give up, but I decided to try using Rufus to clarify exactly which Pencils and accessories worked with which devices. It was very helpful and would provide me with exact Amazon links for the recommended item. Once or twice there was some confusion where the AI contradicted itself, and I was a little worried it might be lying to me. But I found that if you ask the same question in two different ways you’ll get a more accurate answer. (It really is no different from talking with a salesperson at Circuit City back in the day, where the sales guy will tell you anything to make a sale, and you have to call him on it. “Why do you say this VCR has four heads when the box says it only has two?”)
Anyway, I ended up making the plunge, buying a Magic Keyboard clone from a third party manufacturer for $99 instead of Apple’s $350 one. I got a new Pencil on sale, too. I ended up springing for the 13" iPad M5 as my current one was the 11" where I wasn’t entirely happy with the size. Everything came and worked just as advertised, and really love the screen real estate of the 13". I moved the 2021 11" down to serve as my night iPad, where it works just fine and still gives me access to Apple Intelligence and other newer features. The old 13" iPad I sent to Apple as a trade-in (they say I’ll get about $100 back).
BTW, the third party keyboard case works great – the same “floating” magnetic design and the only difference between it and Apple’s is that it connects via bluetooth instead of the special connector on the back, so this keyboard has to be charged separately (via USB-C). But it lasts for weeks on a charge and connects in less than 2 seconds, so that’s not a big deal.
I’m delighted with everything and rather impressed with the Rufus AI (your mileage may vary).
If you have a few spare minutes and are interested, I’d be curious what you’d see if you asked ChatGPT, Perplexity, Claude, and Gemini the same basic questions. The issue is whether Rufus has some special Amazon knowledge that was helpful, or if any of them could have done it with access to the Web.
I haven’t had a chance to do that yet, but will sometime.
One fascinating thing I just noticed a few days ago is Rufus – Amazon’s chatbot – actually gave me purchase links for sites that were not Amazon!
I was looking for a quirky tee-shirt as a Christmas gift for a relative and it couldn’t find one like I wanted on Amazon and suggested some non-amazon sites.
Not really what I wanted, as I prefer the safety of Amazon for buying, but I was shocked Rufus did that. I had assumed it would be trained solely on Amazon products. I’m not sure why it does that: perhaps Amazon was worried they’d get anti-trust scrutiny if they only supported their own site? Or maybe Rufus is based on a general-purpose LLM and they couldn’t stop it from searching for things off-Amazon? Seems odd either way.
Here’s one I did with Google Gemini.
The task was to find what vintage Japanese drawing software was shown in the image.
To find it I uploaded the image to Google Gemini, asked the question, then had a short conversation with it during which I guided its assumptions, and pointed out the options for text outline and shadow. It then suspected it was Magic Palette, which I confirmed manually by doing a Google image search.
Chat transcript
https://g.co/gemini/share/a329328c3ba3
I then found the exact scan that had been cropped down to the image, in an old Japanese computer magazine on Internet Archive.