Close Menu
TechBrunchTechBrunch
  • Home
  • AI
  • Apps
  • Crypto
  • Security
  • Startups
  • TechCrunch
  • Venture

Subscribe to Updates

Subscribe to our newsletter and never miss our latest news

Subscribe my Newsletter for New Posts & tips Let's stay updated!

What's Hot

Klarna CEO and Sutter Hill wins lap after Jony Ive's Openai deal

May 22, 2025

Bluesky begins to check for “notable” users

May 22, 2025

Microsoft says Lumma Password Stealer Malware found on 394,000 Windows PCs

May 22, 2025
Facebook X (Twitter) Instagram
TechBrunchTechBrunch
  • Home
  • AI

    OpenAI seeks to extend human lifespans with the help of longevity startups

    January 17, 2025

    Farewell to the $200 million woolly mammoth and TikTok

    January 17, 2025

    Nord Security founder launches Nexos.ai to help enterprises move AI projects from pilot to production

    January 17, 2025

    Data proves it remains difficult for startups to raise capital, even though VCs invested $75 billion in the fourth quarter

    January 16, 2025

    Apple suspends AI notification summaries for news after generating false alerts

    January 16, 2025
  • Apps

    Bluesky begins to check for “notable” users

    May 22, 2025

    Mozilla shuts down its Read-It-Later app pocket

    May 22, 2025

    Opening a Social Web Browser Surf makes it easy for anyone to create custom feeds

    May 22, 2025

    Anthropic's new Claude4 AI model can be inferred in many steps

    May 22, 2025

    Strava buys athletic training app – First Runna, and now Breakaway

    May 22, 2025
  • Crypto

    Starting from up to $900 from Ticep, 90% off +1 in 2025

    May 22, 2025

    Early savings for 2025 will end on May 25th

    May 21, 2025

    Coinbase says its data breach will affect at least 69,000 customers

    May 21, 2025

    There are 6 days to save $900 to destroy 2025 tickets

    May 20, 2025

    Save $900 to destroy 2025 tickets before prices rise on May 25th

    May 19, 2025
  • Security

    Microsoft says Lumma Password Stealer Malware found on 394,000 Windows PCs

    May 22, 2025

    Signal's new Windows update prevents the system from capturing screenshots of chat

    May 22, 2025

    Wyden: AT&T, T-Mobile and Verizon did not inform senators of surveillance requests

    May 21, 2025

    US students agree to plead guilty to hacking affecting tens of millions of students

    May 21, 2025

    The people in Elon Musk’s DOGE universe

    May 20, 2025
  • Startups

    7 days left: Founders and VCs save over $300 on all stage passes

    March 24, 2025

    AI chip startup Furiosaai reportedly rejecting $800 million acquisition offer from Meta

    March 24, 2025

    20 Hottest Open Source Startups of 2024

    March 22, 2025

    Andrill may build a weapons factory in the UK

    March 21, 2025

    Startup Weekly: Wiz bets paid off at M&A Rich Week

    March 21, 2025
  • TechCrunch

    OpenSea takes a long-term view with a focus on UX despite NFT sales remaining low

    February 8, 2024

    AI will save software companies' growth dreams

    February 8, 2024

    B2B and B2C are not about who buys, but how you sell

    February 5, 2024

    It's time for venture capital to break away from fast fashion

    February 3, 2024

    a16z's Chris Dixon believes it's time to focus on blockchain use cases rather than speculation

    February 2, 2024
  • Venture

    Klarna CEO and Sutter Hill wins lap after Jony Ive's Openai deal

    May 22, 2025

    Wild story of how Moxxie-led Intestinal Toilet Startup Sloan was registered as a gut toilet startup throne

    May 22, 2025

    Submitted submission raises $17 million to automate tax preparation dr voyages

    May 21, 2025

    In a busy VC landscape, Elizabeth Weil's graffiti venture shows that networks are still important

    May 21, 2025

    A comprehensive list of 2025 tech layoffs

    May 21, 2025
TechBrunchTechBrunch

How OpenAI's bots destroyed this seven-person company's website 'like a DDoS attack'

TechBrunchBy TechBrunchJanuary 10, 20255 Mins Read
Facebook Twitter Pinterest Telegram LinkedIn Tumblr WhatsApp Email
Share
Facebook Twitter LinkedIn Pinterest Telegram Email


On Saturday, Triplegang CEO Oleksandr Tomchuk received a call that his company's e-commerce site was down. This appeared to be some kind of distributed denial of service attack.

He soon discovered that the culprit was an OpenAI bot that was relentlessly trying to scrape his entire massive site.

“We have over 65,000 products, and each product has a page,” Tomchuk told TechCrunch. “Each page contains at least three photos.”

OpenAI was sending “tens of thousands” of server requests trying to download all of them, hundreds of thousands of photos, along with detailed descriptions.

“OpenAI used 600 IPs for data collection. We're still analyzing last week's logs, but it's probably much more than that,” he said. Regarding IP addresses, he said.

“Their crawlers were crushing our site. It was basically a DDoS attack,” he said.

The Triplegangers website is that business. The seven-employee company has spent more than a decade building the largest database of “human digital doubles,” or 3D image files scanned from real human models, on the web.

We sell 3D object files and photos (from hands to hair to skin to whole bodies) to 3D artists, video game makers, and anyone who needs to digitally recreate real human features.

Tomchuk's team is based in Ukraine but also has a license in the US from Tampa, Florida, and has a terms of service page on the site that prohibits bots from taking images without permission. But nothing happened. Your website must use a properly configured robot.txt file that includes a tag that tells OpenAI's bot, GPTBot, to leave your site. (According to OpenAI's crawler information page, OpenAI has several other bots with their own tags: ChatGPT-User and OAI-SearchBot.)

Robot.txt (also known as Robot Exclusion Protocol) was created to tell search engine sites what not to crawl when indexing the web. OpenAI says on its information page that it will respect such files if they have their own set of no-crawl tags set, but it will take up to 24 hours for bots to recognize updated robot.txt files. It also warns that this may occur.

If a site doesn't use robot.txt properly, as Tomchuk experienced, OpenAI and others take that to mean they can scrape to their heart's content. It is not an opt-in system.

To add insult to injury, not only was Triple Rangers taken offline by OpenAI's bots during US business hours, but all of the CPU usage and download activity from the bots resulted in high AWS bills. Tomchuk predicts.

Robot.txt is also not failsafe. AI companies will comply voluntarily. Another AI startup, Perplexity, was rather famously called out last summer by a Wired investigation because there was some evidence suggesting that Perplexity didn't respect itself.

Triple Ranger product pageEach of these is one product, and the product page contains more photos. Used with permission. Image credit: Triplegangers (Opens in new window)

I don't know exactly what was taken

By Wednesday, a few days after OpenAI's bots returned, Trilegangers had a properly configured robot.txt file in place to support GPBot, Barkrowler (SEO crawler), and Bytespider (TokTok crawler). Tomczak is also hopeful that he was able to block crawlers from other AI modeling companies. The site did not crash Thursday morning, he said.

But Tomchuk still doesn't have a reasonable way to determine exactly what OpenAI succeeded in doing or to remove that material. He hasn't found a way to contact OpenAI and ask. OpenAI did not respond to TechCrunch's request for comment. And, as TechCrunch recently reported, OpenAI has so far failed to deliver its long-promised opt-out tools.

This is a particularly troublesome problem for triple gangers. “We're in a business where rights are a pretty serious issue because we're scanning real people,” he said. Laws like Europe's GDPR “make it impossible to take and use a photo of someone on the web.”

The Triplegangers website was also a particularly tasty find for AI crawlers. Multi-billion dollar startups like Scale AI are founded where humans painstakingly tag images to train AI. The Triplegangers site features photos tagged with details such as ethnicity, age, tattoos and scars, and all body types.

Ironically, it was the OpenAI bot's greed that alerted Tripleganger to how exposed it was. If he had rubbed it more gently, Mr. Tomczak would never have noticed, he said.

“It's scary because there seem to be loopholes that these companies are using to crawl your data by saying, 'You can opt out by updating your robot.txt with our tag.' It puts the onus on business owners,” Tomczak said. Understand how to block them.

openai crawler logTriplegangers' server logs showed how the OpenAI bot was relentlessly accessing the site from hundreds of IP addresses. Used with permission.

He wants other small online businesses to know that the only way to discover if AI bots are stealing copyrighted material from your website is to actively investigate. That's what I think. He is not alone in their fear. Other website owners recently told Business Insider how OpenAI bots caused their sites to crash and resulted in high AWS bills.

The problem is even bigger in 2024. A new study from digital advertising firm DoubleVerify found that AI crawlers and scrapers caused an 86% increase in “general invalid traffic” in 2024, or traffic that doesn’t come from real users.

Still, “most sites remain unaware that they have been scraped by these bots,” Tomczak warns. “Going forward, log activity should be monitored daily to identify these bots.”

If you think about it, the whole model is a bit like a mafia shakedown. AI bots will take what they want unless you protect them.

“In addition to scraping data, you need to ask for permission,” Tomczak says.



Source link

Share. Facebook Twitter Pinterest LinkedIn Tumblr Email

Related Posts

OpenAI seeks to extend human lifespans with the help of longevity startups

January 17, 2025

Farewell to the $200 million woolly mammoth and TikTok

January 17, 2025

Nord Security founder launches Nexos.ai to help enterprises move AI projects from pilot to production

January 17, 2025

Data proves it remains difficult for startups to raise capital, even though VCs invested $75 billion in the fourth quarter

January 16, 2025

Apple suspends AI notification summaries for news after generating false alerts

January 16, 2025

Nvidia releases more tools and guardrails to help enterprises adopt AI agents

January 16, 2025

Leave A Reply Cancel Reply

Top Reviews
Editors Picks

7 days left: Founders and VCs save over $300 on all stage passes

March 24, 2025

AI chip startup Furiosaai reportedly rejecting $800 million acquisition offer from Meta

March 24, 2025

20 Hottest Open Source Startups of 2024

March 22, 2025

Andrill may build a weapons factory in the UK

March 21, 2025
About Us
About Us

Welcome to Tech Brunch, your go-to destination for cutting-edge insights, news, and analysis in the fields of Artificial Intelligence (AI), Cryptocurrency, Technology, and Startups. At Tech Brunch, we are passionate about exploring the latest trends, innovations, and developments shaping the future of these dynamic industries.

Our Picks

Klarna CEO and Sutter Hill wins lap after Jony Ive's Openai deal

May 22, 2025

Bluesky begins to check for “notable” users

May 22, 2025

Microsoft says Lumma Password Stealer Malware found on 394,000 Windows PCs

May 22, 2025

Subscribe to Updates

Subscribe to our newsletter and never miss our latest news

Subscribe my Newsletter for New Posts & tips Let's stay updated!

© 2025 TechBrunch. Designed by TechBrunch.
  • Home
  • About Tech Brunch
  • Advertise with Tech Brunch
  • Contact us
  • DMCA Notice
  • Privacy Policy
  • Terms of Use

Type above and press Enter to search. Press Esc to cancel.