website crawler

Open source devs are fighting AI crawlers with cleverness and vengeance

AI /

AI web crawling bots are the cockroaches of the internet, many software developers believe. Some devs have started fighting back in ingenuous, often humorous ways. While any website might be targeted by bad crawler behavior – sometimes taking down the site – open source developers are “disproportionately” impacted, writes Niccolò Venerandi, developer of a Linux […]

Open source devs are fighting AI crawlers with cleverness and vengeance Read More »

How OpenAI’s bot crushed this seven-person company’s web site ‘like a DDoS attack’

AI /

On Saturday, Triplegangers CEO Oleksandr Tomchuk was alerted that his company’s ecommerce site was down. It looked to be some kind of distributed denial-of-service attack.  He soon discovered the culprit was a bot from OpenAI that was relentlessly attempting to scrape his entire, enormous site.  “We have over 65,000 products, each product has a page,”

How OpenAI’s bot crushed this seven-person company’s web site ‘like a DDoS attack’ Read More »