that's very unsettling

The list of web crawler bots I "deny" in my robots.txt is long. Now if only bots respected that–I suspect many don't.

My website further requires a rewrite rule in its .htaccess file to prevent websites from reusing the images I host for their own purposes, as if my blog was an image server. While technically possible, it's unethical to consume someone else's storage and bandwidth without notice or agreement.

It's the wild west out here in the world wide web.

Exactly. I wait for the inevitable moment when bots chat to each other. Or even different instances of the same model. That may be fun to watch.

#InternetBots

A good reason to adopt a UX analytics solution, that is able to accurately filter out bot trafic by looking at what makes humans, well, humans 😀

The fakery can only have got worse since 2018, I imagine.

How much *energy* is that chewing up?

Would love to see an update to this analysis. Much has changed in the last four years, not necessarily for the better, and would guess that human traffic is less than 40% or worse

The article is drawing a bad conclusion based on bad data. It is easy to spot "bots" as they don't "linger" after "clicking" links.

Their analysis of how much web traffic is "human" is akin to sending people into a maze to calculate "traffic", and then counting the walls as people b/c they are also inside the room.

Wowzers! It's a wonder that any company advertises on the internet any more (although, looking a Twitter, they aren't there).

same is true of the American #stockmarket My financial advisors tell me it is often now 70% of the trading: high frequency algorithms based trades. 30% when I first started to ask in 2009. There are billions of decisions,but 70% or more made by machines. Happy for hedge funds and high frequency traders to have their own separate unequal exchange, leaving us out of their own weirdness: crypto contagion, credit default swap contagion, triple witching hours, etc.

I have had zero faith in online figures for over a decade. I'll give these examples....
1. I have a blog and hits are greater than page views. Suspicious.
2. I advertised on AdWords and Facebook with many hits on the adverts... Enough that I got charged. Hits on the webpage I set up... Nil. I got charged for fakery. I didn't advertise on either again because there was apparent dishonesty from Google and Facebook.
3. I uploaded a 17 second short video on YouTube in error. I am not sure what went wrong but it should have uploaded as 28 seconds. That video almost instantly got 2000 views. Something fishy there. It's a clearly incomplete video and way more than the dozen or so views my videos normally get.

And what is worse is how the traffic and the bots are designed to influence opinion or gather information.

I love this quote: "a full half of YouTube traffic was “bots masquerading as people,” a portion so high that employees feared an inflection point after which YouTube’s systems for detecting fraudulent traffic would begin to regard bot traffic as real and human traffic as fake. They called this hypothetical event “the Inversion.”"

Just so you know, when the internet first started, we at HP discussed that about 50% of the nascent web traffic was digital. A lot of B2B traffic. So the 60% number you are quoting is not necessarily all malicious bot traffic.

did you see this one @SiHalpin ?
Fascinating article with worrying implications

Human here, not a bot. I sometimes wish I was a bot though because then I wouldn't get tired and would write a whole lot faster.

They are tayloring the internet in a way that serves advertisers not people.

“…In the future, when I look back from the high-tech gamer jail in which President PewDiePie will have imprisoned me, I will remember 2018 as the year the internet passed the Inversion..”

🧐🤔😲 thank you for sharing this

Great and fascinating article. It's also from 2018, which I feel you should have mentioned.

soo...I keep seeing arguments regarding the current Gonsalez v Google case that these companies *must* be allowed to push dangerous and illegal content on people, because there's just so much content that they have no choice! But if over half the content and traffic is fake, and a significant portion of what is left is gonna be content farms like five minute crafts and AI generated garbage...well, maybe the only reason there's such an overwhelming amount of content is because the systems actively encourage posting complete garbage...

And if the algorithms optimize for clicks and ad revenue, and most of that is fake, then it's just bots optimizing for other bots. Should we really be fighting so hard to protect what must be entire server farms burning through literal tons of CO2 just for bots to appease other bots?Yet another reason it may not be the end of the world if these companies are actually held responsible for the content -- not merely hosting -- that they produce...

thank God I'm real

Am I real?

half of (online advertising) is wasted. If only we knew which half ...

"have created an environment where it makes more sense to be fake online — to be disingenuous and cynical, to lie and cheat, to misrepresent and distort — than it does to be real."
this is so true

I work in advertising tech. If you believe the people selling countermeasures 80% of the entire industry is fraud (bots).

which is in no small part why I've kicked Facebook and FB marketplace into the long grass...that and FB toxicity...

hunh. Well, that explains why so much of the advertising I see is BS - it's not actually intended for a human readership.