Why is web search so bad these days?
Where I come from
Back in 2009, I wrote a blog post about using DuckDuckGo (DDG) as my go-to web search engine. The blog focused on the web at large, with a particular emphasis on search. While my old blog is now offline, the domain available and preserved only in archives, that’s irrelevant. What truly matters is the future of the web and navigating it effectively.
A lame duck I wanted to love
Why did DuckDuckGo gain popularity back then? For me, it stood out for its strong privacy features: no tracking, no personalized ads, and a clean, ad-free interface. It also appealed to those looking to avoid big tech dominance, offering consistent search results and unique tools like !bang commands. Its transparent privacy policies and minimal data collection added an extra layer of reassurance.
But it did not stick with me. After realizing that I started to add the !g bang command constantly to most of my DDG searches in order to jump to a Google SERP (search engine result page), I eventually went back to Google as my default web search engine and never looked back until recently. But why?
My fear of missing out
It is actually easy: I compared the Google search results with DDG results constantly, because I felt a strong FOMO not getting the most relevant, highest quality, most recent and complete set of results for my web search queries. This was true in 2009 and remained an issue for me in 2023. I even started to use LLM based services like ChatGPT as an alternative to fill the gap. But as we hopefully all know by now: LLMs are stupid text generating mathematical functions. They are not the web, they just had the web for breakfast and sometimes they have the hiccups.
Kagi fixed FOMO for me
Fast forward to 2024: Since May, I’ve been using Kagi for search, paying $11.90 per month. While the price might seem steep compared to, for example, what popular streaming services charge for delivering terabytes of media data for very similar prices, this comparison doesn’t fully account for the effort and resources required to provide web search engines: maintaining data pipelines, crawling the web, processing vast datasets, indexing, ranking information to deliver quality results and innovating on features.
From a consumer perspective, the cost feels more reasonable as a monthly fee for a superior service, one where quality and efficiency truly set it apart, unlike video streaming services that merely offer access to a catalog of static media files.
As someone with a search-heavy profile, regularly using advanced operators like site: and filetype: along with transactional and navigational queries to find "that specific GitHub repository with the unique fork of a fork of a neovim text editor plugin". Kagi usually delivers good results even for these sophisticated queries without bothering me with a single advertisement and without any tracking.
As I found out while testing Kagi, I do at least 50 searches daily, totaling 1,000 to 2,000 searches each month. Most on mobile devices by the way. Not only does this web search engine consistently deliver the results I need, it also stays out of my way. If you're interested, give it a try and compare it to your favorite web search. You might be surprised.
This is not about Kagi
This isn’t about promoting Kagi, and I have no affiliation with them. I’m not receiving anything for writing about them, and I don’t know enough about their service to recommend it over any other solution that might work better for you. I just pay them to keep my sanity.
This is about Google
This is about Google and how bad their web search has become in recent years. You might recall that Google, now part of the Alphabet Inc. holding, has been one of the world's most valuable companies for over a decade. However, commercial success alone doesn’t guarantee a better web search experience. In fact, one could argue that the relentless pressure to deliver shareholder value has only worsened it over time.
Workarounds and filter bubbles
Users increasingly voice concerns about Google's declining search quality, highlighting the prevalence of ads and less relevant results all over social media. This trend is proven by the growing use of the phrase <search term> reddit in queries, as users seek authentic, user-generated content, often driving substantial traffic to reddit from Google.
Shifting the habit from using general-purpose web searches to relying on "site-searching" social news aggregators like reddit is neither a solution nor an effective workaround. It is a double-edged sword, often moving users from one filter bubble to another, smaller and even more opinionated bubble.
A discussion on Hacker News from 2019 highlighted this trend years ago, but it’s far from an isolated case, as the issue continues to resurface. It’s highly likely this pattern began well before 2019, and with the recent rise of LLM-based AI text generation, the problem is poised to escalate rapidly.
The sell-out has already happened
Much of the content on Google now seems optimized for commercial value, offering little true value to the average user anymore. Using reddit is just a "navigational filter" to clean up Google SERP from irrelevant noise. But a very popular one. But using reddit is essentially just a DDG bang command which is now a workaround for fixing irrelevant Google result pages. I love the irony.
If you are not paying for the service, you're the product
I choose to pay Kagi directly for delivering quality search results, whereas Google is funded not by its users, but by advertisers and their data analysts.
Of course this has always been the case. However, while Google once invested significant effort in balancing relevance with commercial interests, in recent years, they have pushed this balance to the extreme. Only to the wrong side of the scale.
Web search and your life
It’s easy to passively accept poor web search results, much like we tolerate intrusive advertising, irrelevant news, or bizarre social media trends. However, it becomes truly concerning when there’s no viable alternative.
My strongest recommendation is to reflect on whom you trust with your hundreds or even thousands of monthly searches, the quality of the results you rely on, and how these results shape your daily decisions.
Why not stay curious and spend a few weeks experimenting with different search providers, even if they may not be perfect? I’m confident you’ll uncover some eye-opening insights and develop a better idea on how search affects your daily life.