Brave to Launch Its Own Search Engine
Brave Software, which develops a Chromium-based, privacy-focused Web browser, has announced that it has acquired the open search engine Tailcat and will use it to launch its own search engine. Right now, the main search engine choices include 800-pound gorilla Google and distant second Microsoft Bing, with Bing also powering runners-up Yahoo, DuckDuckGo, and Ecosia to greater or lesser extents. Brave Search promises to emphasize privacy, with no tracking or profiling of users, and it will offer a paid, ad-free option in addition to free, ad-supported search. Brave says those ads won’t have the trackers usually associated with Web ads. Another unique aspect of Brave Search is that it promises to use open, community-curated ranking models to ensure diversity and prevent algorithmic biases and censorship. If you’re interested in testing Brave Search, you can sign up for the waitlist.
There’s an issue with competing against Google, and why it’s impossible to match Google.
When you have a web crawler read a website, there’s a robot.txt file that says what company can index what pages. Indexing a website takes a lot of bandwidth on that site. You maybe indexing tens of thousands of pages. Imagine indexing Wikipedia.
Most companies want Google to fully index them. They may also allow Bing too. However, most websites don’t want every web crawler crawling through every single one of their links, so they ban them. A web crawler might be a search engine, it might be a competitor, or maybe a hacker looking for a weakness. Websites will ban web crawler they don’t like.
The result is that no other search engine can build a web index as comprehensive as Google’s.
That’s the case today. There is absolutely no guarantee it will be this way in the future.
Google is very high profile in the news and they have been offiending quite a lot of people. Not just over privacy concerns but over censoring news/search results, biasing search results in favor of other Google-operated sites and other issues. Lots of people are moving to alternative search engines as a result.
Right now, people seem to be moving to Bing (and Duck Duck Go, which I believe uses Bing’s back-end). But there’s no guarantee about this either. Microsoft’s history hasn’t exactly been spotless either and I suspect many users (especially Duck Duck Go users) would prefer a privacy-focused search engine that doesn’t support them.
The number of web sites that allow Brave will increase as Brave’s search engine gets more popular because users will demand it. Similarly, the number of sites that allow Google and Bing may decrease in the future if these companies do things to tick off the operators of those sites.
DuckDuckGo has its own servers, but is using Bing’s web index because it had too many problems building its own index due to the issues I pointed out. People always allow Google to search their website. They usually allow Bing. DuckDuckGo? Who are they and what are they doing all over my website. I better block them!
But crawlers don’t have to honor robots.txt, and I would imagine they could also identify themselves as Googlebot if they wanted to get past any user-agent blocking. Not that I’m recommending such behavior, but robots.txt simply doesn’t seem like much of an obstacle. As long as the bot was otherwise well-behaved, most sites would be unlikely to notice and take alternative approaches to banning it.
The article neglected to mention that Startpage (a front-end anonymizer for Google) is also a search engine option with Brave.
If they come out with a widget for Android, I’ll certainly give it a try. Right now, I have DDG, Bing and Google on my home screen, with Google getting the least use. Brave’s mobile browser is my primary browser ever since Firefox dropped support for AdBlockPlus, so I’ll be happy to give the search engine a try.
Not sure how this translates to iOS - can you have multiple search widgets, or is there just one search widget where you have to switch between search engines manually?