Close Menu
TechBrunchTechBrunch
  • Home
  • AI
  • Apps
  • Crypto
  • Security
  • Startups
  • TechCrunch
  • Venture

Subscribe to Updates

Subscribe to our newsletter and never miss our latest news

Subscribe my Newsletter for New Posts & tips Let's stay updated!

What's Hot

Uber Eyes B2B Logistics pushes in India through a state-backed open commerce network

May 19, 2025

Crypto Elite is increasingly worried about their personal safety

May 18, 2025

Epic Games Ask Judges to Force Apple to Approve Fortnite

May 17, 2025
Facebook X (Twitter) Instagram
TechBrunchTechBrunch
  • Home
  • AI

    OpenAI seeks to extend human lifespans with the help of longevity startups

    January 17, 2025

    Farewell to the $200 million woolly mammoth and TikTok

    January 17, 2025

    Nord Security founder launches Nexos.ai to help enterprises move AI projects from pilot to production

    January 17, 2025

    Data proves it remains difficult for startups to raise capital, even though VCs invested $75 billion in the fourth quarter

    January 16, 2025

    Apple suspends AI notification summaries for news after generating false alerts

    January 16, 2025
  • Apps

    Uber Eyes B2B Logistics pushes in India through a state-backed open commerce network

    May 19, 2025

    Epic Games Ask Judges to Force Apple to Approve Fortnite

    May 17, 2025

    Google I/O 2025: What to expect including Gemini and Android 16 updates?

    May 16, 2025

    After adding your own billing option to iOS, Apple asks Patreon to go to an external browser

    May 16, 2025

    The epic game says Apple is blocking Fortnite from the US and EU app stores

    May 16, 2025
  • Crypto

    Crypto Elite is increasingly worried about their personal safety

    May 18, 2025

    Robinhood expands its footprint in Canada by getting Wonderfi

    May 13, 2025

    Stripe unveils AI Foundation model for payments, revealing a “deeper partnership” with Nvidia

    May 7, 2025

    Movie Pass explores the daily fantasy platform of film buffs

    May 1, 2025

    Speaking on TechCrunch 2025: Application is open

    April 24, 2025
  • Security

    American man spiked the price of Bitcoin hacked SEC X account and sentenced to prison

    May 16, 2025

    Coinbase says that customer's personal information was stolen in a data breach

    May 15, 2025

    White House Scrap plans to block data brokers from selling sensitive American data

    May 14, 2025

    Xai's promised safety report is MIA

    May 13, 2025

    Seven things we learned from WhatsApp vs. NSO Group Spyware Litigation

    May 13, 2025
  • Startups

    7 days left: Founders and VCs save over $300 on all stage passes

    March 24, 2025

    AI chip startup Furiosaai reportedly rejecting $800 million acquisition offer from Meta

    March 24, 2025

    20 Hottest Open Source Startups of 2024

    March 22, 2025

    Andrill may build a weapons factory in the UK

    March 21, 2025

    Startup Weekly: Wiz bets paid off at M&A Rich Week

    March 21, 2025
  • TechCrunch

    OpenSea takes a long-term view with a focus on UX despite NFT sales remaining low

    February 8, 2024

    AI will save software companies' growth dreams

    February 8, 2024

    B2B and B2C are not about who buys, but how you sell

    February 5, 2024

    It's time for venture capital to break away from fast fashion

    February 3, 2024

    a16z's Chris Dixon believes it's time to focus on blockchain use cases rather than speculation

    February 2, 2024
  • Venture

    Build, not bind: Accel's Sonali de Rycker on European AI Crossroads

    May 17, 2025

    How Silicon Valley's influence in Washington benefits high-tech elites

    May 16, 2025

    Red Point raises $650 million three years from the last big early stage fund

    May 15, 2025

    Lip Ring vs Deal Unpacking: Corporate Spy and $16.8 billion Plot Twist

    May 14, 2025

    A $2.5 billion treasured chime file for IPO reveals a $33 million deal with the Dallas Mavericks

    May 13, 2025
TechBrunchTechBrunch

OpenAI claims “new telemetry service'' is to blame for massive ChatGPT outage

TechBrunchBy TechBrunchDecember 13, 20243 Mins Read
Facebook Twitter Pinterest Telegram LinkedIn Tumblr WhatsApp Email
Share
Facebook Twitter LinkedIn Pinterest Telegram Email


OpenAI blamed the longest outage in its history on a failure of its “new telemetry service.”

On Wednesday, ChatGPT, OpenAI's AI-powered chatbot platform, was announced. That video generator, Sora. and its developer API experienced a massive disruption starting around 3:00 PM Pacific Time. OpenAI immediately recognized the problem and began working on a fix. However, it will take approximately three hours for the company to restore all services.

In a post-mortem published late Thursday, OpenAI wrote that the outage was not caused by a security incident or recent product launch, but rather by a telemetry service it introduced on Wednesday to collect Kubernetes metrics. are. Kubernetes is an open source program that helps manage containers, or packages of apps and related files, that are used to run software in isolated environments.

“Because the telemetry service has such a wide footprint, this new service configuration inadvertently caused… resource-intensive Kubernetes API operations,” OpenAI wrote in a post-mortem. . “[Our] The Kubernetes API servers became overloaded and the Kubernetes control plane went down on most of the large servers. [Kubernetes] cluster. ”

This is jargon-heavy, but essentially the new telemetry service impacted OpenAI's Kubernetes operations. This includes resources where many of the company's services rely on DNS resolution. DNS resolution converts IP addresses to domain names. This is why you can type “Google.com” instead of “142.250.191.78”.

The DNS cache used by OpenAI maintains information about previously searched domain names (such as website addresses) and their corresponding IP addresses, complicating the issue with “latency.”[ing] OpenAI writes: [of the telemetry service] Need to proceed before the full extent of the problem is understood. ”

OpenAI said it was able to detect the issue “minutes before” it ultimately started impacting customers, but was unable to implement a fix quickly because it had to work around an overabundance of Kubernetes servers.

“This was due to multiple systems and processes failing simultaneously and interacting in unexpected ways,” the company wrote. “Our tests did not capture the impact that the changes were having on the Kubernetes control plane.” [and] Repairs were delayed significantly due to the lockout. ”

To prevent similar incidents from occurring in the future, OpenAI is improving its phased rollouts to better monitor infrastructure changes and ensuring that OpenAI engineers can access its Kubernetes API servers under any circumstances. It said it would adopt a number of measures, including new mechanisms to ensure this.

“We apologize for the impact this incident has had on all of our customers, from ChatGPT users to developers to businesses that rely on OpenAI products,” OpenAI said in a statement. “We fell short of our expectations.”

TechCrunch has a newsletter focused on AI. Sign up here to get it delivered to your inbox every Wednesday.



Source link

Share. Facebook Twitter Pinterest LinkedIn Tumblr Email

Related Posts

OpenAI seeks to extend human lifespans with the help of longevity startups

January 17, 2025

Farewell to the $200 million woolly mammoth and TikTok

January 17, 2025

Nord Security founder launches Nexos.ai to help enterprises move AI projects from pilot to production

January 17, 2025

Data proves it remains difficult for startups to raise capital, even though VCs invested $75 billion in the fourth quarter

January 16, 2025

Apple suspends AI notification summaries for news after generating false alerts

January 16, 2025

Nvidia releases more tools and guardrails to help enterprises adopt AI agents

January 16, 2025

Leave A Reply Cancel Reply

Top Reviews
Editors Picks

7 days left: Founders and VCs save over $300 on all stage passes

March 24, 2025

AI chip startup Furiosaai reportedly rejecting $800 million acquisition offer from Meta

March 24, 2025

20 Hottest Open Source Startups of 2024

March 22, 2025

Andrill may build a weapons factory in the UK

March 21, 2025
About Us
About Us

Welcome to Tech Brunch, your go-to destination for cutting-edge insights, news, and analysis in the fields of Artificial Intelligence (AI), Cryptocurrency, Technology, and Startups. At Tech Brunch, we are passionate about exploring the latest trends, innovations, and developments shaping the future of these dynamic industries.

Our Picks

Uber Eyes B2B Logistics pushes in India through a state-backed open commerce network

May 19, 2025

Crypto Elite is increasingly worried about their personal safety

May 18, 2025

Epic Games Ask Judges to Force Apple to Approve Fortnite

May 17, 2025

Subscribe to Updates

Subscribe to our newsletter and never miss our latest news

Subscribe my Newsletter for New Posts & tips Let's stay updated!

© 2025 TechBrunch. Designed by TechBrunch.
  • Home
  • About Tech Brunch
  • Advertise with Tech Brunch
  • Contact us
  • DMCA Notice
  • Privacy Policy
  • Terms of Use

Type above and press Enter to search. Press Esc to cancel.