We’re seeing this at work too - our public git frontend is constantly getting scraped as well as our self hosted issue tracker. We had to spend days working on fail2ban and other kinds of tools to mitigate all the traffic that’s adding tons of load to our instances, which otherwise would easily be able to handle the handful of employees who actually use these systems.
Fuck AI
Looks like this is the written version of the video:
https://thelibre.news/foss-infrastructure-is-under-attack-by-ai-companies/
Ah, yeah, I keep being scatterbrained and impulsive when crossposting, and forgetting he sadly doesn’t even link direct links in the video description, instead of just to the website itself. Will add the link to the lemmy post bodies at least.
There are tarpits like Nepenthes but they use up your CPU resources and I imagine it would be pretty easy to update a scraper to recognize these generated pages, since they’re all structurally similar.
I’m no Sysadmin but to me it sounds like we need a botnet/scraper resistant application layer protocol to replace HTTPS.
So basically what Drew DeVault wrote about the other day I guess?
