I'm confused about how or why this is a new policy. My memory is inside Google we were discussing this risk back in 2003, probably earlier. Search quality was on it. I just assumed they'd lost the arms race, or that the parasites' ranking was justified for other reasons that were hard to tease apart. What are they doing new now?
I think often about Mahalo, the sleazy shovel content that was spamming the web back in 2007. Google shut that down somewhat fast, although it did take several years. These days with AI and more aggressive spammers it's a losing battle. The real problem is the financial incentives that make this kind of spamming profitable in the first place.
My tiny little blog gets about 3 requests a week for someone to "pay me to run a guest article". Going rate is $50-$200 and again, my blog is tiny.
Electricniko 13 days ago [-]
The air purifier review site Housefresh dug into why sites like theirs were seeing less traffic back in the spring, and it amounts to a handful of companies buying up popular magazine/blog brands and using them as affiliate farms that cross-post to sites within their networks of brands to boost visibility:
If an air purifier review site is publishing research on their search traffic and affiliate site rings, I feel it's safe to say that their specialty is in seo, not air purifiers.
viraptor 13 days ago [-]
I think that's unfair. They could've hired someone to analyse this, or maybe it's just run by people who work other gigs too. I can post about dance and in-depth http traffic optimisation - people know more than one thing.
pjc50 13 days ago [-]
Everything is like this. If you want to sell a nonzero quantity of a generic product, you have to be an expert in advertising and SEO.
masto 13 days ago [-]
The Google you remember from 2003 was murdered and its corpse is being worn by an entirely unrecognizable company.
giancarlostoro 13 days ago [-]
“Do all evil” its new unofficial slogan.
mint2 13 days ago [-]
Google’s unofficial/official core policies are great too!
Like the well know “all discussion must be self destructing or cc the attorneys so no one can find our evil plotting”
nurumaik 13 days ago [-]
Just manually review top K websites and ban such garbage?
Sometimes dumb, bruteforce and biased solution can work way better than any automation you can come up with
NelsonMinar 12 days ago [-]
I think that'd be a good approach. There was an idea in that time that everything had to be algorithmic, that hand-ranking or singling out individual sites was a bad idea. Ie: tweak a generic algorithm and then test what it does to top garbage sites, don't just penalize the site directly. I think that's not a bad principle but in practice it didn't seem to work well.
dotancohen 13 days ago [-]
[flagged]
danielheath 13 days ago [-]
Scale to what? You could manually look at the top million searches each year with four average college graduates.
carstenhag 13 days ago [-]
In all languages of the world? Is that realistic? How could you know whether something in Vietnamese is spam or not?
In my opinion it could work, but you'd definitely need 40-80 and not 4 people.
adgjlsfhk1 13 days ago [-]
Google has ~180k employees. that seems like a pretty small price to pay to make search work...
disgruntledphd2 13 days ago [-]
And they've been doing user testing of search results for well over a decade (FB started at least 10 years ago, and Google were well into it at that point).
consteval 13 days ago [-]
Not everything needs to be computerized and completely automated. You're allowed to just... use human labor.
Sure, if every phone call requires an operator that doesn't scale. Okay, so we now have automatic routing. But having a receptionist to route company requests, which may be complex and may not map nicely, still makes sense. It's a human operator, and you pay them. So what, it's a small expense.
dawnerd 13 days ago [-]
Seriously, they tackled this years ago with the panda update to kill off all the how to and similar seo spam. It's like after around that time they just stopped caring at all and let the best X sites take over.
lern_too_spel 13 days ago [-]
Because it became an embarrassing news story (https://larslofgren.com/forbes-marketplace/, also mentioned in this article). They would have lazily left it unfixed if everybody weren't laughing at them.
walterbell 13 days ago [-]
If so, the author deserves a Web Signal Award for succeeding where customers, regulators and execs failed despite years of trying.
mjr00 13 days ago [-]
> I'm confused about how or why this is a new policy.
My best guess is it's because they finally have a real competitor in ChatGPT.
> The real problem is the financial incentives that make this kind of spamming profitable in the first place.
Yeah, but the financial incentives exist on both ends. There's a gross symbiotic relationship between Google and SEO spammers, because Google also owns the ad network the spammers put on their page. If Google puts ad-laden SEO blogspam as the top result and a user clicks it, the user sees a bunch of ads from Google. Everyone wins: Google, the SEO spammers, and advertisers. Well, everyone except the user, but who cares about them?
My guess/hope is that ChatGPT has made someone who actually cares about the quality of search results actually step in and say things have gone too far.
NelsonMinar 13 days ago [-]
You're totally right about that symbiotic relationship. We were aware of that risk in the early days when AdSense launched, we saw some very innovative and gross exploitation and created some policies to rein it in. But ultimately if Google makes a buck coming and going, they will do that.
Wasn't there a big story last year in the wake of the DOJ antitrust investigation about Google manipulating search quality to boost ad revenue? I can't put my hands on a reference now, in part because Google is so bad at search these days I can't find anything more than a few months old.
Eisenstein 13 days ago [-]
> Wasn't there a big story last year in the wake of the DOJ antitrust investigation about Google manipulating search quality to boost ad revenue?
Define "wins". From what happening right now, it seems that google may lose much more than it earned by aligning with seo spammers
Maybe they need to start locking employee stock options for 100 years to prevent them optimizing short-term gains?
pjc50 13 days ago [-]
> locking employee stock options for 100 years
This is just a ban by another name. Besides, options are not the massive tax incentive that they used to be. The problems are locked into the nature of being publicly traded companies. If you want to do government search policy, do government search policy.
johnnyanmac 13 days ago [-]
Google's only taking the greedy approach. Spam sites on top, spam sites use google adsense, people click spam sites, they click google ads.
It works great, until it doesn't. But that's a problem for the next CEO.
Scoundreller 13 days ago [-]
iunno, I used to rank pretty well for things in my country like "$company's tech support number". Unlike every $company, my page had a nice clean URL like whatever.tld/Tech-Support-Number-for-$company and I'd just list of their phone numbers with a few paragraphs about how $company is shit. Maybe 50kb total.
Meanwhile $company's page was company.tld/234897234982-029823749823742-2340823492 and 3 pages down was a phone number if your browser didn't choke on the javascript.
For ISP ones, I recommended people print a copy so they can call if they can't get on the internet, which kinda backfired when a major ISP changed their tech support number (!) and it redirected to a toll-free squatter's sex chat line.
Turns out $company really hates it when you call their call (cost) centres.
I had maybe 50 pages for our different oligopolies and averaged $500/month revenue on adsense, so GOOG's cut was $250/month.
Today, for one $company, the first 9 results are different pages from $company.tld, each unhelpful with a phone number in their own way, and they don't run adsense!
chipsrafferty 13 days ago [-]
Because ChatGPT is dependent on good search when it searches the web? Or because it completes with Google when it provides a good answer without searching? Or what do you mean specifically?
ryandvm 13 days ago [-]
I would say the latter. For software dev questions, my Google searches and Stack Overflow visits have fallen off a cliff since I started paying for ChatGPT.
Ironically, I probably would have paid the same amount to Google for ad-free, old-style (accurate) Google searches, but no, they just wanted to keep cranking that ad dial up every year so that ship has sailed.
At this point, I'm enjoying watching the old guard of search scrambling to find a life jacket.
thephyber 13 days ago [-]
Stackoverflow visits fell[1] off a cliff since GPT became popular.
Google is getting destroyed by the chatbot workflow because it is no longer the start of a browser session and clickthrus (the things that earn the high sponsored link rates) are falling as more users get their queries answered faster with less effort.
StackOverflow has been dying a slow death since longer than before ChatGPT. Sure, ChatGPT is helping to accelerate it. The real data (leave aside social/community for a moment) issue with SO.com: Many answers don't age well. So, you have an answer from 8 years ago with 65 upvotes, but now the lang/lib was updated in 2023. A newer, more relevant answer is waaaaaaay down and only has one upvote. Personal note: I still pine for the old days when Joel Spolsky and Jeff Atwood were at the helm. They really knew how to build and sustain a vibrant community.
rockskon 13 days ago [-]
Are they, though? Inaccurate info is pretty common from LLMs.
thephyber 12 days ago [-]
Inaccurate info exists everywhere. StackOverflow contains inaccurate, outdated, incomplete info. Caveat Emptor wherever you are.
LLMs are like a knife. It is a tool that can hurt you if you misuse it, but it also has the capability to save LOTS to time if you use it well.
rockskon 10 days ago [-]
A knife's function is deterministic. LLMs are not.
They routinely misinterpret the information they've ingested and confidently spit out incorrect statements. Worse - they confidently spit out incorrect statements in ways we cannot anticipate.
This isn't comparable to a person. This isn't comparable to human intelligence. This isn't a problem that can be handwaved away by saying "people are sometimes wrong too!"
bigstrat2003 13 days ago [-]
> My best guess is it's because they finally have a real competitor in ChatGPT.
ChatGPT doesn't even fulfill the same function, to say nothing about the poor reliability inherent to the way it works. In no sense is it a real competitor to Google.
milesvp 12 days ago [-]
This is absolutely not my experience. My googlefu has gone to shit. I can no longer have some sense for what I’m looking for and find it. Something I know exists, I just don’t know the terms of art to pull up the wealth of knowledge. I used to be able to do a half dozen searches, and finally on page 4 or 5 finally find some clues as to the terms I need. Now, there is no page 4… like ever. Clearly not many people were going past page 3 so they stopped serving that content.
Now I use chatgpt for these kinds of queries, and it feels like using google circa 2004.
I know this is a small edge case, and ultimately I need to use google to crosscheck, but it hints at the rot that has taken over at google, and represents a potential shift. If I could get good reference links from chatgpt, I may be able to stop using google for an increasing number of my queries.
stcroixx 12 days ago [-]
All the developers I work with have stopped using Google search entirely in favor of ChatGPT. Even my wife prefers it after seeing how bad googles results are and full of ads.
resoluteteeth 13 days ago [-]
> My best guess is it's because they finally have a real competitor in ChatGPT.
My guess is it's because a bunch of articles about it have been posted to hn recently.
klabb3 13 days ago [-]
> My best guess is it's because they finally have a real competitor in ChatGPT.
Bingo. I always chuckle when people here say Google has lost it, and become incompetent. Well, they all make the mistake of assuming that they’re trying but failing, rather than that it’s deliberate simply due to boring economics.
Now look at how quickly decades-long problems, so big they have an entire cottage industry built around it, suddenly be cleaned up. Incompetence? Nah.
Of course, this does nothing to convince regulators and not even average HN user that innovation is harmed by these dominant players. Someone’s gotta think of the poor mega-corps.
xivzgrev 13 days ago [-]
I miss Google of 2003
What would it take for someone to make it today? No AI, only 1 on mobile, and sites with heavy ad loads are punished
Spivak 13 days ago [-]
You're mostly describing Kagi. They do have AI results but you have to explicitly ask for them. They have an "No AI" image search option as well.
I also like my "Before AI" lens I can click on to search the internet pre-2021. And you can downrank or fully block those garbage spam sites. They even have a "leaderboard" for most blocked/pinned sites you can use to get started.
Shorel 13 days ago [-]
I think it would be easy to make, with two decades of hardware improvements.
The problem is that the web of 2003 doesn't exist any longer.
Google existence changed the websites for better or worse. The Google of 2003 is no longer capable of dealing with today's web SEO dirty tricks.
oneshtein 13 days ago [-]
Install a spam filter for search engines, like uBlackList.
Use bunch of different search engines. In Firefox, enable search entry, then visit search engines and click green + in the entry, to add search engine.
NelsonMinar 12 days ago [-]
Kagi and Phind are both impressing me. They both use AI but in ways I like.
vikingerik 13 days ago [-]
It would take a benefactor who wants to pay for running it for its own sake and not for profit. As soon as there's a profit motive, enshittification sets in since you're serving whoever pays rather than your users.
nielsbot 13 days ago [-]
or maybe a government utility
openrisk 13 days ago [-]
Governments have been in recent decades completely hands off from anything tech related. For the longest time they followed the usual neoliberal economist trope that markets solve everything optimally. As a result they have created in the so-called "big tech" conglomerates a monster fit for the history books, the most broken and cornered market there ever was.
But a wholly government run search engine is not a solution. There are inherent biases in both constructing and presenting indices. You dont want to further stoke the anti-commons mistrust of polarized societies.
What the public sector could do is fund all the background techonologies to make it easy to have much larger numbers of search engines. Some of those assembled services could be completely open source, others could be value adding with various added services and customizations.
In any case the status quo is a disaster that has no future. Its effectively a forced dumbification of society as it artificially suppresses the flow of high-quality public information. Incidentally it also doesnt solve the problem that much of the world's information is private. Desktop search should become a thing again, in line with local AI etc.
theendisney 13 days ago [-]
Just traffic.
stackghost 13 days ago [-]
>My memory is inside Google we were discussing this risk back in 2003, probably earlier.
Yeah but that was before they hired the incompetent grifter Prabhakar Raghavan and eventually made him head of Search.
thephyber 13 days ago [-]
Seems like post hoc fallacy.
But people were complaining about the sAme issues under Matt Cutts. Also, there has been A Ton more money and work chasing the SEO farm game. Now big private equity companies have focused on buying a stable of big brands to do the same that used to be garage startups.
robk 13 days ago [-]
I think it's nick fox now and he's old school and as competent as they come
lumost 13 days ago [-]
Searching for python documentation was the worst, geeks for geeks and others would get the top slot for reskinning the pypi docs with ads.
The entire thing was so blatant and obvious that I assumed Google did not care due to ad revenue.
When ChatGPT launched search, you could immediately skip over all the crap. It made search nice again.
htrp 13 days ago [-]
>Google shut that down somewhat fast,
> although it did take several years.
You and I have very different definitions of fast
itsoktocry 13 days ago [-]
What's hilarious is when people boast about being "in Forbes" like it's the magazine from 20 years ago, and not this parasitic SEO operation that publishes garbage on anything.
dangus 13 days ago [-]
I took advantage of this in business school. A lot of my professors considered Forbes a reputable business magazine. It was amazing, I could easily cite a source for just about anything I wanted to say.
lotsofpulp 13 days ago [-]
Maybe the professors knew and were taking advantage to look the other way to make their job easier.
dangus 13 days ago [-]
haha maybe, I got my A’s and my piece of paper, and now I’m qualified to cut costs at your local private equity firm!
pimlottc 13 days ago [-]
Well, of course; “Forbes” still carries prestige and cachet. If it didn’t, this scam wouldn’t work.
maeil 13 days ago [-]
> If it didn’t, this scam wouldn’t work.
It would, as this scam relies on the domain authority, regardless of their "offline authority".
Ancalagon 13 days ago [-]
Hopefully this is a step in the right direction. Google's search results have gotten so bad - seems like even some of the simplest searches are just packed with AI generated and SEO garbage. I don't even want SearchGPT do take over the search market space because I'm almost sure it will still be garbage. Just bring back the google from 5-10 years ago please :(.
paxys 13 days ago [-]
> Just bring back the google from 5-10 years ago please
What you really want is the internet from 5-10 years ago (really even longer than that), and that's not coming back.
rockskon 13 days ago [-]
Search has degraded substantially in the past 5 years for reasons wholly, completely, and absolutely unrelated to SEO blogspam.
Google of 5 years ago didn't ignore words in almost every single query I made. Google of 5 years ago didn't constantly give me irrelevant garbage because they keep ignoring the words I use in my query.
This is a wholly separate issue from SEO crap. Ignoring search terms is 100% a Google issue and is 100% Google's fault!
Discordian93 13 days ago [-]
Yes they did. It has been getting steadily worse since 2014 or so.
rockskon 13 days ago [-]
It wasn't the majority of queries until the past 5 years though. What was once an annoyance became an active disruption to getting the information I'm looking for.
kevinmchugh 13 days ago [-]
Google search degraded in usefulness before the panda update, when spammers had filled the web with low quality content designed to exploit Google's algorithms. Google improved their search to punish the content farms, and people were happy with that search for many years.
karmonhardan 5 days ago [-]
People need to stop saying this. The internet from 5-10 years ago was already post-social media revolution; it's not substantially different from today's internet, on the consumer side. Google has gotten substantially worse, however.
firecall 13 days ago [-]
Agreed. That’s really my thoughts.
The internet or rather the crawlable WWW itself has changed.
The WWW content itself is no longer what it was.
chipsrafferty 13 days ago [-]
Do you think the proliferation of web frameworks makes it harder to crawl?
thwarted 13 days ago [-]
Probably has some influence. Because everything needs to be an app, implemented as an SPA, rather than just a web page. Hypertext is dead.
fragmede 13 days ago [-]
What I really want is me from 5-10 years ago. When can we get a pill that will do that?
grugagag 13 days ago [-]
You could get there somewhat. Do you exercise? Ready to put in some effort?
firecall 13 days ago [-]
You could probably take some pills that will make you think you are! LOL
I feel like testosterone therapy should be more readily available….
vasco 13 days ago [-]
I'm fine with this internet but the previous search.
Terr_ 13 days ago [-]
A close approximation might be a search-mode which penalizes results based on how many ads they have and how much of the page they cover...
nielsbot 13 days ago [-]
i would love for pages to also be ranked by readability…
seems like local news sites are the absolute worst in this area
NewJazz 13 days ago [-]
Exactly. In a constantly changing world, you need constantly changing policy to achieve the same outcomes. Even then you probably won't replicate the past universe perfectly.
viraptor 13 days ago [-]
It did degrade a bit. But Kagi and Perplexity proved (in very different ways) that you can get significantly better search from the same internet that Google uses.
viewtransform 13 days ago [-]
append ' before:2023' to your google searches.
dpkirchner 13 days ago [-]
I want the Internet from 5-10 years ago before Google incentivized this much SEO garbage. It wasn't awesome then but it was a lot better.
0xbadcafebee 13 days ago [-]
You can't go back to the way things were. The world moves forward and changes, and we have to adapt to it.
Web search has always been an extremely messy solution to many problems. Think about the premise: type in anything, and somehow it will read your mind, intuit who you are and what you really wanted, find the exact thing amid the morass of the whole web, and then give it to you?
That's impossible. So it uses tricks to make it seem like it worked. It uses information about you to refine results. It uses curated, human-edited search and result heuristics for the most common or difficult search queries. It uses a giant corups of data, and shows you things that are like what you wanted.
You don't notice that it isn't giving you the best result, because there are so many mediocre-but-acceptable results to look at. And it doesn't have to work perfectly every time, because we can "sift through" results and "refine" our search. Often we are flooded with results that are targeted at us, rather than what we want, because, remember: Google is an advertising company, and the entire Web is now a shopping mall, where either you're being sold-to, or you're just being sold.
You will get results, and they will sort-of seem like what you wanted, so you will just sort of sigh and accept it. Because what other option is there?
There are more intelligent, more accurate, more safe, ways to solve the problems people have, that are not "a search engine". It's time we start implementing them.
PhasmaFelis 13 days ago [-]
> You don't notice that it isn't giving you the best result
That's fine. It's always been fine. I don't need Google to read my mind and fulfill my dreams.
The problem isn't that they're not divinely perfect. The problem is that they used to be good enough, and now they're not.
> There are more intelligent, more accurate, more safe, ways to solve the problems people have, that are not "a search engine". It's time we start implementing them.
What solutions are there that fulfill all the use cases of a search engine, while definitively not being a search engine? An AI chatbot that gives me synopses of the same websites that I was searching for does not count.
13 days ago [-]
jerry1979 13 days ago [-]
> You can't go back to the way things were.
> type in anything, and somehow it will read your mind
I think we can go back to the way things were, which had nothing to do with mind reading. In the past, you could type in word, and google would offer 10 million results, and you could page through each of them. That was very powerful, and google does not do that today.
wbl 13 days ago [-]
I was in high school 15 years ago and Google absolutely read minds to conclude Briney Spears was not a search for pickles but rather a pop artist. This was significant enough for them to come to go talk about it.
PhasmaFelis 8 days ago [-]
Also a good point. One time I couldn't remember what Minute (multi-app installer) was called, so I googled "the thing what downloads all the things" and lo and behold, Google found it. Their algo used to be really clever.
PhasmaFelis 4 days ago [-]
Ugh, "Ninite." Thanks, autocorrect.
Ferret7446 13 days ago [-]
I don't think you know what you are asking. Do you really want 10 million pages of results, of which 99.999...% are SEO spam for Viagra et al, and on average you will need to browser ~9 million pages of results to find something that's actually "relevant"?
FactKnower69 13 days ago [-]
>Think about the premise: type in anything, and somehow it will read your mind, intuit who you are and what you really wanted, find the exact thing amid the morass of the whole web, and then give it to you?
I never once asked for anything remotely like this. Maybe you could just show me results for the fucking thing I typed? When I go to the library, the Dewey decimal system doesn't rearrange itself based on all the metadata the library has on me and people fitting my demographic criteria, it just shows me what I fucking searched for.
rrr_oh_man 12 days ago [-]
This is exactly how I talk to my ChatGPT assistant.
63stack 12 days ago [-]
Web search has never been about this, where are you getting this fable from? It was about finding pages that include certain words, why would you overmistify this.
bgun 13 days ago [-]
Not sure why you’re being downvoted, this is a pointed analysis of why crawl-based search is insufficient for an Internet of our current scale. There is no corporate-curated algorithm that is up to the task, especially when the primary purpose is to profit from advertising.
jfengel 13 days ago [-]
Google is remarkably effective at handling the scale. It doesn't seem up for handling the sheer army dedicated to misleading it. Especially now that they've been given tools for automating crap generation.
Ironically, Google itself was a key developer of that tech.
If there is any solution it would seem to involve removing the incentive to merely look at your page. That problem seems remarkably stubborn.
amanaplanacanal 13 days ago [-]
It "doesn't seem up for handling" it because it profits from showing you those ad laden sites. It's intentional, not incompetence.
jfengel 13 days ago [-]
I think that they'd rather show you the right answers, if they can. Those sites will often have exactly the same ads, and they won't make you think about jumping ship to Bing or Kagi. And advertisers will pay more for sites that actually have good reputations.
Search engines are the picks-and-shovels of the Internet gold rush. They profit either way. They want to do it in a way that keeps the gravy train going.
I'm no starry-eyed capitalist. I'm sure that Google would sell their own grandmothers for a few ad clicks. But occasionally the cynical thing to do is actually the right thing.
the_snooze 13 days ago [-]
>There is no corporate-curated algorithm that is up to the task, especially when the primary purpose is to profit from advertising.
I think this is the root cause of the problem. Google can easily put a big dent in this problem by allowing users to create their own importable/exportable filters and support the dissemination of something like "EasyList for search results." But that kills their golden goose of advertising influence.
warkdarrior 13 days ago [-]
> "EasyList for search results"
Who will be in charge of curating that list? We know that crowd-sourced stuff is easily abused (see Amazon reviews, see YouTube comments).
the_snooze 13 days ago [-]
It would indeed be crowd-sourced, but with a core set of maintainers. Wouldn't be all that different from EasyList or Steven Black's HOSTS file. They basically take in merge requests from the community and serve as an initial filter against garbage. [1]
And unlike Amazon reviews or YouTube comments, anyone can fork it if they think they can maintain it better.
[1] "The filter lists are currently maintained by four authors, Fanboy, MonztA, Khrin, Yuki2718 and PiQuark6046, who are ably assisted by an ample forum community." https://easylist.to/
conorcleary 13 days ago [-]
In before your comment ends up in QuickBooks search results for the rest of time :-)
gerdesj 13 days ago [-]
"seems like even some of the simplest searches are just packed with AI generated and SEO garbage"
I'll give you a concrete example of that and it is a right old pain.
There are loads more hits like the above and they are nearly all wrong. The RPI distribution is based on Debian Linux but has a few differences. Between those two versions of Debian, RPi changed things in /boot quite dramatically and failing to do that, you will end up with a weird chimera - I created several of these beasts until I fixed them: https://blog.scheib.me/2024/04/14/upgrade-raspberry-bullseye...
In this case it may actually be a blog matching the template of the AI clones! However, they do all look very similar.
rafram 13 days ago [-]
Google does perfectly on the latter search. It returns a relevant blog post written by an actual human, and a bunch of forum threads about that exact upgrade path.
DDG has never been very good.
shombaboor 13 days ago [-]
I was searching for a uniquely named company by exact name (think: verizon), and it was 80% of the way down the results page. Google knew exactly what I wanted to see and flooded my screen with alternatives who had paid them.
kmoser 13 days ago [-]
Were those organic results, or the paid ads that always appear first?
rty32 13 days ago [-]
My guess: shopping results, followed by sponsored ads, followed by 1-2 results that are not ads but you don't care, some combination of news/Twitter/Youtube videos, more shopping results, then finally real search results. If "AI summary" didn't appear at the top.
I have seen that so many times that I can scroll to the "correct" part of the Google search result page within 2 seconds without thinking.
Now that I write this down, I realized how horrible this is.
JumpCrisscross 13 days ago [-]
> bring back the google from 5-10 years ago
Given Kagi's abysmal adoption rates, it's clear that good search isn't worth it for most people.
rurp 13 days ago [-]
I tried Kagi but just didn't see notably better results than other search engines. Maybe if I spent more time on the power user tools, or if Kagi offered more of a trial period I would have, but adding yet another monthly subscription is a high bar for me and what I saw didn't clear it.
These days my default assumption is that any SAAS product will get worse and more expensive over time, so it has to be pretty good to justify reworking my online habits around, given that I don't know how long I'll keep using it. Hopefully Kagi will be the exception to that rule, but I wouldn't bet on it.
rjst01 13 days ago [-]
I was recently looking for an article I remember reading a bit over a year ago. I could even remember some exact phrases that appeared. I tried to find it on Google for more than 10 minutes, ultimately to no avail. I then went looking through chat histories and was able to find where I had shared it to someone.
I relayed this story to a friend who suggested I try Kagi. It was on the first page on my first attempt. I was also able to use it to find a different article I was sure I read even longer ago, that I didn't have as clear memory of.
karmonhardan 5 days ago [-]
I'm not sure what your positive experience has to do with GPs negative experience. Is the implication that it negates it? Because it doesn't.
pesus 13 days ago [-]
That subscription fee is just too big of an obstacle in a time when everything has a subscription and is still often degrading in quality. Seems like an unsolvable chicken and egg scenario though, since relying on advertising to make it free would just result in the same issues as everything else.
deprecative 13 days ago [-]
It's quite literally this. It costs more than free and people don't want that. We're poor and poorer and everyone is overburdened by subscriptions for everything. I get that HN is in a rich bubble but most folks can't afford rent, food, and a search engine.
JumpCrisscross 13 days ago [-]
> costs more than free and people don't want that. We're poor and poorer and everyone is overburdened by subscriptions for everything
But that’s also the answer on preference. Google is good enough for most people. For everyone else, there can be a paid premium layer. Similar to news, this might be the equilibrium, not an anomaly.
13 days ago [-]
1vuio0pswjnm7 13 days ago [-]
I still search Google and other search engines from the command line. There is no "AI" garbage in the results. The way HN commenters refer to Google search in this thread, one might conclude it is not possible anymore to search the web without a popular browser running Javascript (which is a prerequisite for this "AI" stuff). That conclusion would be incorrect. It is still possible; I am still doing it every day.
1vuio0pswjnm7 13 days ago [-]
This is how I do it
1. Make HTTP
2. Send HTTP via TCP
3. Perform text processing on the response body (I create own SERP instead of using Google's)
Personally, I use multiple programs, some I wrote myself in C, to perform these individual steps, connected by UNIX pipes and the shortest, simplest possible Bourne shell scripting
However there are countless ways to perform these steps in wide variety of programming languages; there is no need for UNIX or shell scripting, it is purely personal preference
As others noted, this has been an issue for years. What prompted Google to act now, using a manual override that was supposedly not feasible in the era of algorithms? Did the viral articles by Lars provide ammunition for teams within Google?
rjrdi38dbbdb 13 days ago [-]
One area where Google search is terribly broken is porn.
If your search for some specific term "$foo", nearly every result is just 'search site $bar for "$foo"', taking you to the site's search page, regardless of whether $foo is actually found on the site.
Suppafly 13 days ago [-]
I swear the only reason people started using bing is because the search there actually worked well for porn.
scarface_74 13 days ago [-]
On the other hand, when I use to search for anything near the city I lived in, porn showed up way too frequently
Seems like a referendum for a name change would be a good idea, considering the current typical usage of that term.
G1N 13 days ago [-]
Had an old roommate who moved here after getting married to be closer to family. Weirdly, the name never at any point came up so I think everyone is just kind of resigned to the fact that they live in a place called Cumming
kmoser 13 days ago [-]
What does that have to do with porn?
rjrdi38dbbdb 13 days ago [-]
I've never noticed that behavior from Google when searching for any other topic. Have you?
buryat 13 days ago [-]
how many people do you think search for $foo and the opposite?
rjrdi38dbbdb 13 days ago [-]
What do you mean by the opposite?
The search behavior is the same whether $foo is a popular generic term or something niche.
kiru_io 13 days ago [-]
Google should vibe out others as well.
If I search now "Best CBD Gummies", the first few results are:
vice.com
independent.co.uk
healthline.com
observer.com
How is Forbes worse than any of those shallow comparison pages?
glial 13 days ago [-]
FWIW, Kagi is a little better but not much. First results are:
* cornbreadhemp.com
* forbes.com
* healthline.com
al_borland 13 days ago [-]
Kagi does let the user adjust the rankings of these sites if they don't want them coming up. While it would be nice to have this done proactively for the link farms, at least the user does have some control.
Forbes did make it on the blocked and lowered leaderboards.
But I usually use site:reddit.com in my searches anyway so it doesn't matter to my personal habits.
vasco 13 days ago [-]
Probably all ran by the same people too.
akira2501 13 days ago [-]
An entire department was just rendered useless. I genuinely don't feel bad.
kasey_junk 13 days ago [-]
Was probably a whole company right? Pretty good argument that Forbes the traditional media property and Forbes the seo giant are 2 different things: https://larslofgren.com/forbes-marketplace/
fakedang 13 days ago [-]
And Forbes the SEO giant was going to buy out the legacy media property.
_DeadFred_ 13 days ago [-]
It was funny watching the warrior whatever site back in the day when Panda came along. Love when these people get their horrible business models kneecapped.
Now let's make corporate stock manipulation illegal again and ban corporate stock buybacks. Talk about a purely manipulative business strategy.
They are nothing but direct stock manipulation that was 'legalized' at the same time where executive compensation was moved from salary to... stock, so that you end up with a quasi-legal (stock manipulation by executives is supposed to be illegal) corrupt incentives system.
quickthrowman 13 days ago [-]
Two things:
-Stock buybacks are not manipulation, they’re simply a way to return cash to shareholders and then the shareholder decides when to incur tax liability. A company is well within its rights to issue additional shares or buy back and destroy shares at their discretion. It’s functionally equivalent to a dividend without a taxable event.
-Corporate boards award stock grants to executives because they want management to be aligned with shareholders. I think executive compensation is excessive, but stock grants do align management and shareholders.
chipsrafferty 13 days ago [-]
Dilution is immoral and unfair to investors. If a company wants to raise money they should have to sell shares they own, not print more and sell those.
l33t7332273 13 days ago [-]
I think it’s kind of up to the investors what is unfair for them.
quickthrowman 13 days ago [-]
That is your opinion and does not reflect reality.
_DeadFred_ 13 days ago [-]
If it's intent is to manipulate the value of the stock, then it is stock manipulation. You give a distinction without a difference.
notyourwork 13 days ago [-]
Nothing directly, it just sounds bad at face value.
_DeadFred_ 13 days ago [-]
'Stock manipulation is cool, especially when you change executives pay structure to be based purely on said manipulation. Totally creates healthy incentives not perverse ones.'
notyourwork 13 days ago [-]
Sorry, buy backs are not stock manipulation. Let's step back from emotions and political skew. A company is able to take their capital and deploy it how they see fit. This can include purchasing percentage ownership of their company back from stockholders. Whether or not you agree doesn't make it manipulation in the general sense. It's just a way for a company to use their money.
_DeadFred_ 13 days ago [-]
Why do they repurchase the stock? In order to impact the stock's value (also known as manipulating it's value). That is the definition of stock manipulation. No emotions involved, no need for the passive aggressive attack that I'm somehow being emotional.
It's not just a way for a company to use their money. It is a company intentionally using funds for stock manipulation, many times by executives who directly benefit from said manipulation. Companies even take out loans purely to re-purchase stock.
notyourwork 12 days ago [-]
You have a shallow understanding of economics and the stock market. Sorry but your cynical view is not something that generalizes to stock buybacks. And the example of taking a loan out can be financially sound. Using debt when cheap to deploy capital can be beneficial.
l33t7332273 13 days ago [-]
It’s a way to return money to share holders.
_DeadFred_ 13 days ago [-]
By manipulating the value of the stock that shareholders. In order to do what you claim it does, it has to.... manipulate the stocks value, aka stock manipulations. You give a distinction without a difference.
And shareholders only 'benefit' from this return if they sell their stock (ie give up being stock holders) versus the traditional method where stock holders receive and dividend and maintain their stock ownership. A dividend benefits all stock holdres, stock manipulation only benefits those that sell, a smaller arbitrary subset. Why chose a 'return' method that is only for some investors?
l33t7332273 12 days ago [-]
In what sense is it a manipulation? If I have a billion in cash and spend it all on a stock, that stock price will go up; that’s not manipulation. That’s supply and demand.
>Why chose a 'return' method that is only for some investors?
The investors control the company, so they get to decide that.
scarface_74 13 days ago [-]
The company is giving money back to shareholders. What exactly is wrong with that?
_DeadFred_ 13 days ago [-]
It is not. It is manipulating the value of the stock, which the shareholders can possible take advantage of by selling their stock. Giving money back to shareholders is called a dividend. Manipulating the price of a stock is manipulation, not returning money.
scarface_74 12 days ago [-]
What’s the practical difference? Either way the amount that ends up in your brokerage account should increase be the same amount. Stock buybacks are more tax efficient for the stock owner in the US
readyplayernull 13 days ago [-]
I wouldn't be surprised of the whole thing being automated.
trompetenaccoun 13 days ago [-]
I just used the exact prompt they give as an example and got the very spam they claimed to have removed. Vice, The Independent, Healthline, etc... all advertising those gummies in ads disguised as news articles.
So yeah, probably not actually rendered useless.
12907835202 12 days ago [-]
Is healthline spam? It's my go to whenever I have a medical question to add healthline to the search
al_borland 13 days ago [-]
I assume the threat to their business posed by OpenAI (and others) is what is getting them to start addressing these long standing issues. I'm glad they're doing it, but upset that they let users suffer with sub-par results for so many years.
warkdarrior 13 days ago [-]
How does OpenAI's searchGPT rank Forbes Advisor/Marketplace?
al_borland 13 days ago [-]
Good point. I just asked ChatGPT for the best CBD gummies, then asked what sources it used for the list. This was the first thing it said…
> Consumer reviews from trusted websites like Healthline, CNET, and Forbes Health that provide in-depth reviews and rankings based on effectiveness, ingredients, and customer feedback.
So the LLMs are now giving us affiliate link garbage, but we can’t easily see that was the source, and the affiliate links don’t even work. Everyone loses in this scenario.
citizenpaul 13 days ago [-]
Why doesn't google just manually block/derank all these massive content spam sites like forbes, business insider. Actually solving the problem even though its not some neat tech solution. This is like watching doctors theorize on how to save a bleeding out patient that dies because they are talking.
Seattle3503 13 days ago [-]
The article says Google took "manual action", which implies thats exaxtly what they did.
That page gives a good hint at how opaque these paid placements can be to an outsider like Google. Really tough to prevent too much collateral damage when going after bad actors like Forbes. Glad Google is working on it though.
pointing out the line between what you can get away with with SEO and what you can't get away with and what you can't get away with is making Google look stupid.
PhasmaFelis 13 days ago [-]
Not for more than five years or so, at least.
PaulHoule 13 days ago [-]
That blogger talks about his speculations which I agree with.
Google has been fighting against SEO spammers since the very beginning, in fact there was keyword stuffing even before there was Google.
Google at some point made a decision to privilege sites like Forbes because they were dealing with hundreds of subprime sites that looked even sketchier. Rather than playing whac-a-mole with new spammers every month as they change the rules they could put sites like Forbes and Wirecutter on top, ultimately discouraging other SEO spammers from even trying, and being able to manage the relationship. (e.g. if you see Forbes rank for "best CBD gummies" just once it looks a little sketchy but you get over it, you only really catch the bad smell after you've done 5,000 searches and see Forbes ranks so high on so many of them)
I've been working with a financial analyst to quantify the value of online engagement (what value does YC get from an HN comment? what value do users get from it?) and reviews are one of the clearest examples since a review adds or subtracts revenue from a vendor and gives customers a certain amount of satisfaction. It's an interesting area in my mind because the stakes are so high.
AlienRobot 13 days ago [-]
After scrolling a bit I looked at my scrollbar and I was scared.
Yes, this is /blog. The whole blog.
simonw 13 days ago [-]
An interesting intellectual exercise is to think about how a search engine could provide the best possible answer (from a user satisfaction perspective) to a query like "best CBD gummies".
A lot of people have a significant financial incentive to win at that search query.
What would the perfect top search result for that look like?
It would probably be an article written by professional writers in a trustworthy publication with a strong ethics policy, provably followed over the years, concerning whether they accept payment for promoting specific products in supposedly impartial reviews.
If you can figure out how to algorithmically detect that kind of content you could build a pretty great search engine!
johnfn 13 days ago [-]
I think I'd be pretty happy if Consumer Reports was on the top for queries like these (if they had the relevant data, of course). I think they follow your criteria pretty closely.
eviks 13 days ago [-]
Since "the best" doesn't exist, just like there is no magical professional that has unique insight into the mind of the user making the search, a search engine could become pretty great by simply not taking decades to remove scams like the one described in the article from the top of search results
kmoser 13 days ago [-]
There are many criteria for "best" that are acceptable to many people, e.g. lowest price, proven high quality ingredients, efficacy, etc. "Website with high reputation that happens to be running ads for company XYZ" is usually not how people define "best".
eviks 13 days ago [-]
By splitting one very ambiguous term into another "high quality" one you haven't resolved anything.
And people do transfer trust from the medium to the product, so neither is the second filter very robust when it comes to people perception
consf 13 days ago [-]
And the notion of "the best" is inherently subjective
ameliaquining 13 days ago [-]
Are we assuming that this search engine is only used by a few nerds, or is the idea to build something that remains good even if it gets popular enough that webmasters have strong financial incentives to game it like they currently do with Google Search? Because the latter sounds like a much, much harder problem, and in particular like it probably requires huge financial resources in order to win the ongoing cat-and-mouse game, if that's even possible.
simonw 13 days ago [-]
For the sake of this exercise I want to hear how people would solve that harder problem.
batch12 13 days ago [-]
I think it'd maybe a query for the best gummies would be based on reviews from users, but I guess that's the point. Having something understand what one means by best is hard.
johnfn 13 days ago [-]
Hm, I think that Amazon shows that just user votes might not be sufficient - e.g. because users can be paid off to give 5 star reviews, which bias the results.
kmoser 13 days ago [-]
Google has reams of company reviews, both those they've scraped and those they've solicited from the public. How hard could it be for them to downrank sites that advertise companies with relatively bad reviews, and uprank sites that advertise companies with relatively good reviews?
They could even scale the downranking so that the higher your site's reputation, the more it gets downranked if you're advertising poorly-reviewed companies. That would ding Forbes more than it would ding Joe's Little Blog, and prevent highly ranking sites (like Forbes) from having a monopoly on some search results.
fwip 13 days ago [-]
It sounds like they're just deranking the blog spam posts. In my opinion, Google should derank the whole domain/brand. If you purposely put your name on garbage, we should put your name in the trash.
bhartzer 13 days ago [-]
Google should have done this 5-10 years ago.
wslh 13 days ago [-]
My feeling as a Google user since the beginning is that the search engine doesn't matter anymore in terms of quality. That is why I wonder how Google supposely discover their own "bugs" so late.
MR4D 12 days ago [-]
Translation: Google doesn’t like companies managing their own traffic without Google’s “help”, so Google drops their ranking so the companies have to pay the master (Google).
Classic shakedown.
dmead 13 days ago [-]
What was the website they had to delist like 10 years ago that was like this? I'm drawing a blank.
PcChip 13 days ago [-]
After getting used to kagi i can’t imagine going back to google, that would be like trying to use the web without uBlock origin after getting used to it
jsemrau 13 days ago [-]
Google starts getting into the way too much. Recently I searched for German football clubs that "fan rivalries" and Google refused that and only gave me results for "fan friendships" .
tayo42 13 days ago [-]
I just tried this(German football fan rivalries) and only got one thing with friendly and it was a reddit post. The ai response on top was on topic too
jsemrau 13 days ago [-]
should have taken a screenshot.
scarface_74 13 days ago [-]
When I search for “best pet insurance”, US News & World Reports still shows up 5th.
When I search for “Best Delta credit card”, CNBC shows up 6th
urbandw311er 13 days ago [-]
Site that has historically abused its monopoly to game search profits clamps down on others doing exactly the same thing.
theendisney 13 days ago [-]
If you advertise with paid articles you are not spending as much as you could on adsense.
13 days ago [-]
consf 13 days ago [-]
How the blurred lines between content and commerce can erode trust
lofaszvanitt 13 days ago [-]
Just rotate results... If there are 10000 good articles for a query, why only show 10. 10 because almost noone goes beyond page 1.
I think often about Mahalo, the sleazy shovel content that was spamming the web back in 2007. Google shut that down somewhat fast, although it did take several years. These days with AI and more aggressive spammers it's a losing battle. The real problem is the financial incentives that make this kind of spamming profitable in the first place.
My tiny little blog gets about 3 requests a week for someone to "pay me to run a guest article". Going rate is $50-$200 and again, my blog is tiny.
https://housefresh.com/how-google-decimated-housefresh/
On HN here:
https://news.ycombinator.com/item?id=40239811
And their previous article mentioned in that post generated a lot of discussion on HN:
https://news.ycombinator.com/item?id=39433451
Like the well know “all discussion must be self destructing or cc the attorneys so no one can find our evil plotting”
Sometimes dumb, bruteforce and biased solution can work way better than any automation you can come up with
In my opinion it could work, but you'd definitely need 40-80 and not 4 people.
Sure, if every phone call requires an operator that doesn't scale. Okay, so we now have automatic routing. But having a receptionist to route company requests, which may be complex and may not map nicely, still makes sense. It's a human operator, and you pay them. So what, it's a small expense.
My best guess is it's because they finally have a real competitor in ChatGPT.
> The real problem is the financial incentives that make this kind of spamming profitable in the first place.
Yeah, but the financial incentives exist on both ends. There's a gross symbiotic relationship between Google and SEO spammers, because Google also owns the ad network the spammers put on their page. If Google puts ad-laden SEO blogspam as the top result and a user clicks it, the user sees a bunch of ads from Google. Everyone wins: Google, the SEO spammers, and advertisers. Well, everyone except the user, but who cares about them?
My guess/hope is that ChatGPT has made someone who actually cares about the quality of search results actually step in and say things have gone too far.
Wasn't there a big story last year in the wake of the DOJ antitrust investigation about Google manipulating search quality to boost ad revenue? I can't put my hands on a reference now, in part because Google is so bad at search these days I can't find anything more than a few months old.
This is the email chain you are looking for:
* https://www.justice.gov/d9/2023-11/417581.pdf
Define "wins". From what happening right now, it seems that google may lose much more than it earned by aligning with seo spammers
Maybe they need to start locking employee stock options for 100 years to prevent them optimizing short-term gains?
This is just a ban by another name. Besides, options are not the massive tax incentive that they used to be. The problems are locked into the nature of being publicly traded companies. If you want to do government search policy, do government search policy.
It works great, until it doesn't. But that's a problem for the next CEO.
Meanwhile $company's page was company.tld/234897234982-029823749823742-2340823492 and 3 pages down was a phone number if your browser didn't choke on the javascript.
For ISP ones, I recommended people print a copy so they can call if they can't get on the internet, which kinda backfired when a major ISP changed their tech support number (!) and it redirected to a toll-free squatter's sex chat line.
Turns out $company really hates it when you call their call (cost) centres.
I had maybe 50 pages for our different oligopolies and averaged $500/month revenue on adsense, so GOOG's cut was $250/month.
Today, for one $company, the first 9 results are different pages from $company.tld, each unhelpful with a phone number in their own way, and they don't run adsense!
Ironically, I probably would have paid the same amount to Google for ad-free, old-style (accurate) Google searches, but no, they just wanted to keep cranking that ad dial up every year so that ship has sailed.
At this point, I'm enjoying watching the old guard of search scrambling to find a life jacket.
Google is getting destroyed by the chatbot workflow because it is no longer the start of a browser session and clickthrus (the things that earn the high sponsored link rates) are falling as more users get their queries answered faster with less effort.
[1] https://x.com/altimor/status/1853893158368928124?s=46
LLMs are like a knife. It is a tool that can hurt you if you misuse it, but it also has the capability to save LOTS to time if you use it well.
They routinely misinterpret the information they've ingested and confidently spit out incorrect statements. Worse - they confidently spit out incorrect statements in ways we cannot anticipate.
This isn't comparable to a person. This isn't comparable to human intelligence. This isn't a problem that can be handwaved away by saying "people are sometimes wrong too!"
ChatGPT doesn't even fulfill the same function, to say nothing about the poor reliability inherent to the way it works. In no sense is it a real competitor to Google.
Now I use chatgpt for these kinds of queries, and it feels like using google circa 2004.
I know this is a small edge case, and ultimately I need to use google to crosscheck, but it hints at the rot that has taken over at google, and represents a potential shift. If I could get good reference links from chatgpt, I may be able to stop using google for an increasing number of my queries.
My guess is it's because a bunch of articles about it have been posted to hn recently.
Bingo. I always chuckle when people here say Google has lost it, and become incompetent. Well, they all make the mistake of assuming that they’re trying but failing, rather than that it’s deliberate simply due to boring economics.
Now look at how quickly decades-long problems, so big they have an entire cottage industry built around it, suddenly be cleaned up. Incompetence? Nah.
Of course, this does nothing to convince regulators and not even average HN user that innovation is harmed by these dominant players. Someone’s gotta think of the poor mega-corps.
What would it take for someone to make it today? No AI, only 1 on mobile, and sites with heavy ad loads are punished
I also like my "Before AI" lens I can click on to search the internet pre-2021. And you can downrank or fully block those garbage spam sites. They even have a "leaderboard" for most blocked/pinned sites you can use to get started.
The problem is that the web of 2003 doesn't exist any longer.
Google existence changed the websites for better or worse. The Google of 2003 is no longer capable of dealing with today's web SEO dirty tricks.
Use bunch of different search engines. In Firefox, enable search entry, then visit search engines and click green + in the entry, to add search engine.
But a wholly government run search engine is not a solution. There are inherent biases in both constructing and presenting indices. You dont want to further stoke the anti-commons mistrust of polarized societies.
What the public sector could do is fund all the background techonologies to make it easy to have much larger numbers of search engines. Some of those assembled services could be completely open source, others could be value adding with various added services and customizations.
In any case the status quo is a disaster that has no future. Its effectively a forced dumbification of society as it artificially suppresses the flow of high-quality public information. Incidentally it also doesnt solve the problem that much of the world's information is private. Desktop search should become a thing again, in line with local AI etc.
Yeah but that was before they hired the incompetent grifter Prabhakar Raghavan and eventually made him head of Search.
But people were complaining about the sAme issues under Matt Cutts. Also, there has been A Ton more money and work chasing the SEO farm game. Now big private equity companies have focused on buying a stable of big brands to do the same that used to be garage startups.
The entire thing was so blatant and obvious that I assumed Google did not care due to ad revenue.
When ChatGPT launched search, you could immediately skip over all the crap. It made search nice again.
> although it did take several years.
You and I have very different definitions of fast
It would, as this scam relies on the domain authority, regardless of their "offline authority".
What you really want is the internet from 5-10 years ago (really even longer than that), and that's not coming back.
Google of 5 years ago didn't ignore words in almost every single query I made. Google of 5 years ago didn't constantly give me irrelevant garbage because they keep ignoring the words I use in my query.
This is a wholly separate issue from SEO crap. Ignoring search terms is 100% a Google issue and is 100% Google's fault!
The internet or rather the crawlable WWW itself has changed.
The WWW content itself is no longer what it was.
I feel like testosterone therapy should be more readily available….
seems like local news sites are the absolute worst in this area
Web search has always been an extremely messy solution to many problems. Think about the premise: type in anything, and somehow it will read your mind, intuit who you are and what you really wanted, find the exact thing amid the morass of the whole web, and then give it to you?
That's impossible. So it uses tricks to make it seem like it worked. It uses information about you to refine results. It uses curated, human-edited search and result heuristics for the most common or difficult search queries. It uses a giant corups of data, and shows you things that are like what you wanted.
You don't notice that it isn't giving you the best result, because there are so many mediocre-but-acceptable results to look at. And it doesn't have to work perfectly every time, because we can "sift through" results and "refine" our search. Often we are flooded with results that are targeted at us, rather than what we want, because, remember: Google is an advertising company, and the entire Web is now a shopping mall, where either you're being sold-to, or you're just being sold.
You will get results, and they will sort-of seem like what you wanted, so you will just sort of sigh and accept it. Because what other option is there?
There are more intelligent, more accurate, more safe, ways to solve the problems people have, that are not "a search engine". It's time we start implementing them.
That's fine. It's always been fine. I don't need Google to read my mind and fulfill my dreams.
The problem isn't that they're not divinely perfect. The problem is that they used to be good enough, and now they're not.
> There are more intelligent, more accurate, more safe, ways to solve the problems people have, that are not "a search engine". It's time we start implementing them.
What solutions are there that fulfill all the use cases of a search engine, while definitively not being a search engine? An AI chatbot that gives me synopses of the same websites that I was searching for does not count.
> type in anything, and somehow it will read your mind
I think we can go back to the way things were, which had nothing to do with mind reading. In the past, you could type in word, and google would offer 10 million results, and you could page through each of them. That was very powerful, and google does not do that today.
I never once asked for anything remotely like this. Maybe you could just show me results for the fucking thing I typed? When I go to the library, the Dewey decimal system doesn't rearrange itself based on all the metadata the library has on me and people fitting my demographic criteria, it just shows me what I fucking searched for.
Ironically, Google itself was a key developer of that tech.
If there is any solution it would seem to involve removing the incentive to merely look at your page. That problem seems remarkably stubborn.
Search engines are the picks-and-shovels of the Internet gold rush. They profit either way. They want to do it in a way that keeps the gravy train going.
I'm no starry-eyed capitalist. I'm sure that Google would sell their own grandmothers for a few ad clicks. But occasionally the cynical thing to do is actually the right thing.
I think this is the root cause of the problem. Google can easily put a big dent in this problem by allowing users to create their own importable/exportable filters and support the dissemination of something like "EasyList for search results." But that kills their golden goose of advertising influence.
Who will be in charge of curating that list? We know that crowd-sourced stuff is easily abused (see Amazon reviews, see YouTube comments).
And unlike Amazon reviews or YouTube comments, anyone can fork it if they think they can maintain it better.
[1] "The filter lists are currently maintained by four authors, Fanboy, MonztA, Khrin, Yuki2718 and PiQuark6046, who are ably assisted by an ample forum community." https://easylist.to/
I'll give you a concrete example of that and it is a right old pain.
Let's try upgrading Debian Bullseye to Bookworm. Search "upgrade debian bullseye to bookworm" - first hit from DDG is: https://www.debian.org/releases/bookworm/amd64/release-notes... - YES - Debian documentation, staid, verbose, stolid and correct.
Now let's try to upgrade a Raspberry Pi from Bullseye to Bookworm: Search "upgrade raspberry pi bullseye to bookworm". First hit: https://raspberrytips.com/upgrade-raspberry-pi-os-bookworm/.
There are loads more hits like the above and they are nearly all wrong. The RPI distribution is based on Debian Linux but has a few differences. Between those two versions of Debian, RPi changed things in /boot quite dramatically and failing to do that, you will end up with a weird chimera - I created several of these beasts until I fixed them: https://blog.scheib.me/2024/04/14/upgrade-raspberry-bullseye...
In this case it may actually be a blog matching the template of the AI clones! However, they do all look very similar.
DDG has never been very good.
I have seen that so many times that I can scroll to the "correct" part of the Google search result page within 2 seconds without thinking.
Now that I write this down, I realized how horrible this is.
Given Kagi's abysmal adoption rates, it's clear that good search isn't worth it for most people.
These days my default assumption is that any SAAS product will get worse and more expensive over time, so it has to be pretty good to justify reworking my online habits around, given that I don't know how long I'll keep using it. Hopefully Kagi will be the exception to that rule, but I wouldn't bet on it.
I relayed this story to a friend who suggested I try Kagi. It was on the first page on my first attempt. I was also able to use it to find a different article I was sure I read even longer ago, that I didn't have as clear memory of.
But that’s also the answer on preference. Google is good enough for most people. For everyone else, there can be a paid premium layer. Similar to news, this might be the equilibrium, not an anomaly.
1. Make HTTP
2. Send HTTP via TCP
3. Perform text processing on the response body (I create own SERP instead of using Google's)
Personally, I use multiple programs, some I wrote myself in C, to perform these individual steps, connected by UNIX pipes and the shortest, simplest possible Bourne shell scripting
However there are countless ways to perform these steps in wide variety of programming languages; there is no need for UNIX or shell scripting, it is purely personal preference
Details here: https://larslofgren.com/cnn-usa-today-forbes-marketplace/
If your search for some specific term "$foo", nearly every result is just 'search site $bar for "$foo"', taking you to the site's search page, regardless of whether $foo is actually found on the site.
https://en.m.wikipedia.org/wiki/Cumming,_Georgia
Seems like a referendum for a name change would be a good idea, considering the current typical usage of that term.
The search behavior is the same whether $foo is a popular generic term or something niche.
How is Forbes worse than any of those shallow comparison pages?
* cornbreadhemp.com * forbes.com * healthline.com
Forbes did make it on the blocked and lowered leaderboards.
https://kagi.com/stats?stat=leaderboard
forbes.com healthline.com medicalnewstoday.com fortune.com
But I usually use site:reddit.com in my searches anyway so it doesn't matter to my personal habits.
Now let's make corporate stock manipulation illegal again and ban corporate stock buybacks. Talk about a purely manipulative business strategy.
https://hbr.org/2020/01/why-stock-buybacks-are-dangerous-for...
https://www.theatlantic.com/ideas/archive/2018/07/are-stock-...
-Stock buybacks are not manipulation, they’re simply a way to return cash to shareholders and then the shareholder decides when to incur tax liability. A company is well within its rights to issue additional shares or buy back and destroy shares at their discretion. It’s functionally equivalent to a dividend without a taxable event.
-Corporate boards award stock grants to executives because they want management to be aligned with shareholders. I think executive compensation is excessive, but stock grants do align management and shareholders.
It's not just a way for a company to use their money. It is a company intentionally using funds for stock manipulation, many times by executives who directly benefit from said manipulation. Companies even take out loans purely to re-purchase stock.
And shareholders only 'benefit' from this return if they sell their stock (ie give up being stock holders) versus the traditional method where stock holders receive and dividend and maintain their stock ownership. A dividend benefits all stock holdres, stock manipulation only benefits those that sell, a smaller arbitrary subset. Why chose a 'return' method that is only for some investors?
>Why chose a 'return' method that is only for some investors?
The investors control the company, so they get to decide that.
So yeah, probably not actually rendered useless.
> Consumer reviews from trusted websites like Healthline, CNET, and Forbes Health that provide in-depth reviews and rankings based on effectiveness, ingredients, and customer feedback.
So the LLMs are now giving us affiliate link garbage, but we can’t easily see that was the source, and the affiliate links don’t even work. Everyone loses in this scenario.
That page gives a good hint at how opaque these paid placements can be to an outsider like Google. Really tough to prevent too much collateral damage when going after bad actors like Forbes. Glad Google is working on it though.
I remember this guy
http://www.seobook.com/blog
pointing out the line between what you can get away with with SEO and what you can't get away with and what you can't get away with is making Google look stupid.
Google has been fighting against SEO spammers since the very beginning, in fact there was keyword stuffing even before there was Google.
Google at some point made a decision to privilege sites like Forbes because they were dealing with hundreds of subprime sites that looked even sketchier. Rather than playing whac-a-mole with new spammers every month as they change the rules they could put sites like Forbes and Wirecutter on top, ultimately discouraging other SEO spammers from even trying, and being able to manage the relationship. (e.g. if you see Forbes rank for "best CBD gummies" just once it looks a little sketchy but you get over it, you only really catch the bad smell after you've done 5,000 searches and see Forbes ranks so high on so many of them)
I've been working with a financial analyst to quantify the value of online engagement (what value does YC get from an HN comment? what value do users get from it?) and reviews are one of the clearest examples since a review adds or subtracts revenue from a vendor and gives customers a certain amount of satisfaction. It's an interesting area in my mind because the stakes are so high.
Yes, this is /blog. The whole blog.
A lot of people have a significant financial incentive to win at that search query.
What would the perfect top search result for that look like?
It would probably be an article written by professional writers in a trustworthy publication with a strong ethics policy, provably followed over the years, concerning whether they accept payment for promoting specific products in supposedly impartial reviews.
If you can figure out how to algorithmically detect that kind of content you could build a pretty great search engine!
And people do transfer trust from the medium to the product, so neither is the second filter very robust when it comes to people perception
They could even scale the downranking so that the higher your site's reputation, the more it gets downranked if you're advertising poorly-reviewed companies. That would ding Forbes more than it would ding Joe's Little Blog, and prevent highly ranking sites (like Forbes) from having a monopoly on some search results.
Classic shakedown.
When I search for “Best Delta credit card”, CNBC shows up 6th