Close Menu
TechBrunchTechBrunch
  • Home
  • AI
  • Apps
  • Crypto
  • Security
  • Startups
  • TechCrunch
  • Venture

Subscribe to Updates

Subscribe to our newsletter and never miss our latest news

Subscribe my Newsletter for New Posts & tips Let's stay updated!

What's Hot

Sarah Smith launches a $16 million fund and says AI can “unlock” for solo GPS like herself

July 11, 2025

Medium CEO explains what it takes to stop losing $2.6 million a month

July 11, 2025

CISA confirms that hackers are actively taking advantage of the critical “Citrix Bleed 2” bug

July 11, 2025
Facebook X (Twitter) Instagram
TechBrunchTechBrunch
  • Home
  • AI

    OpenAI seeks to extend human lifespans with the help of longevity startups

    January 17, 2025

    Farewell to the $200 million woolly mammoth and TikTok

    January 17, 2025

    Nord Security founder launches Nexos.ai to help enterprises move AI projects from pilot to production

    January 17, 2025

    Data proves it remains difficult for startups to raise capital, even though VCs invested $75 billion in the fourth quarter

    January 16, 2025

    Apple suspends AI notification summaries for news after generating false alerts

    January 16, 2025
  • Apps

    Belkin will no longer support most WEMO devices and their WEMO apps

    July 10, 2025

    Mockly has actually created a fake DM generator that is user-friendly

    July 10, 2025

    YouTube removes its trending pages and now trend list

    July 10, 2025

    As X loses CEO, daily use is decreasing and competition is growing

    July 10, 2025

    Google adds inter-image generation capabilities to VEO 3

    July 10, 2025
  • Crypto

    Bitcoin surpasses $118K at the second highest high in 24 hours

    July 11, 2025

    Vitalik Buterin reserves for Sam Altman's global project

    June 28, 2025

    Calci will close a $185 million round as rival Polymeruk reportedly seeks $200 million

    June 25, 2025

    Stablecoin Evangelist: Katie Haun's Battle of Digital Dollars

    June 22, 2025

    Hackers steal and destroy millions of Iran's biggest crypto exchanges

    June 18, 2025
  • Security

    CISA confirms that hackers are actively taking advantage of the critical “Citrix Bleed 2” bug

    July 11, 2025

    AI chatbot's simple “123456” password was at risk of revealing personal data from millions of McDonald's job seekers

    July 11, 2025

    French police arrest Russian basketball player accused of ransomware: Report

    July 10, 2025

    Authorities arrest four hackers related to UK retail hacking

    July 10, 2025

    Jack Dorsey says his “safe” new bitchat app hasn't been tested for security

    July 9, 2025
  • Startups

    7 days left: Founders and VCs save over $300 on all stage passes

    March 24, 2025

    AI chip startup Furiosaai reportedly rejecting $800 million acquisition offer from Meta

    March 24, 2025

    20 Hottest Open Source Startups of 2024

    March 22, 2025

    Andrill may build a weapons factory in the UK

    March 21, 2025

    Startup Weekly: Wiz bets paid off at M&A Rich Week

    March 21, 2025
  • TechCrunch

    OpenSea takes a long-term view with a focus on UX despite NFT sales remaining low

    February 8, 2024

    AI will save software companies' growth dreams

    February 8, 2024

    B2B and B2C are not about who buys, but how you sell

    February 5, 2024

    It's time for venture capital to break away from fast fashion

    February 3, 2024

    a16z's Chris Dixon believes it's time to focus on blockchain use cases rather than speculation

    February 2, 2024
  • Venture

    Sarah Smith launches a $16 million fund and says AI can “unlock” for solo GPS like herself

    July 11, 2025

    Medium CEO explains what it takes to stop losing $2.6 million a month

    July 11, 2025

    All stages in Boston and 4 days until lowest ticket prices disappear

    July 11, 2025

    Learn how to raise seed rounds from top VCS in 2025

    July 10, 2025

    Save up to $475 for 5 days to all stages before prices rise

    July 10, 2025
TechBrunchTechBrunch

AI21 Labs' new AI model can handle more context than most AI models

TechBrunchBy TechBrunchMarch 28, 20244 Mins Read
Facebook Twitter Pinterest Telegram LinkedIn Tumblr WhatsApp Email
Share
Facebook Twitter LinkedIn Pinterest Telegram Email


The AI ​​industry is increasingly moving towards generative AI models with longer context. However, models with large context windows tend to be computationally expensive. Or so Dagan, his lead for products at AI startup AI21 Labs, argues, that doesn't necessarily have to be the case, and his company has released a generative model to prove it.

Context, or context window, refers to input data (such as text) that a model considers before producing output (additional text). Models with small context windows tend to forget even the most recent conversations, whereas models with large contexts avoid this pitfall and have the added benefit of better visibility into the data flow they ingest.

AI21 Labs' Jamba, a new text generation and analysis model, can perform many of the same tasks as models such as OpenAI's ChatGPT and Google's Gemini. Trained on a combination of public and proprietary data, Jamba can produce text in English, French, Spanish, and Portuguese.

Jamba can process up to 140,000 tokens while running on a single GPU (such as a high-end Nvidia A100) with at least 80 GB of memory. This equates to about 105,000 words, or 210 pages, making it a decent-sized novel.

By comparison, Meta's Llama 2 has a 32,000-token context window (on the small side by today's standards), but only requires a GPU with about 12 GB of memory to run. (Context windows are typically measured in tokens, which are bits of raw text or other data.)

At first glance, Jumba is unremarkable. There are a number of generative AI models available for free download, from Databricks' recently released DBRX to his aforementioned Llama 2.

But what makes Jamba unique is what's under the hood. It uses a combination of his two model architectures: transformers and state-space models (SSM).

Transformers are an ideal architecture for complex inference tasks, powering models such as GPT-4 and Google's Gemini, for example. Trance has several unique characteristics, but by far the most distinctive feature of trance is its “attention mechanism.” For each input data (such as a sentence), the transformer evaluates the relevance of all other inputs (other sentences) and extracts from them to produce an output (a new sentence).

SSM, on the other hand, combines several characteristics of older types of AI models, such as recurrent neural networks and convolutional neural networks, to create a more computationally efficient architecture that can process long data sequences.

Now, SSM has its limits. However, some of the earlier reified models, including an open-source model called Mamba by researchers at Princeton University and Carnegie Mellon University, were able to handle larger inputs than their transformer-based counterparts and were less effective at language generation tasks. Delivers excellent performance.

In fact, Jamba uses Mamba as part of its core model, which Dagan claims delivers three times the throughput in long contexts compared to transformer-based models of comparable size.

“There are some early academic examples of SSM models, but this is the first commercial-grade, production-scale model,” Dagan said in an interview with TechCrunch. “This architecture offers great efficiency and throughput potential, in addition to being innovative and interesting for further research by the community.”

Jamba is currently released under the Apache 2.0 License, an open source license with relatively few usage restrictions, but Dagan emphasizes that this is not a research release intended for commercial use. This model has no safeguards to prevent the generation of harmful text or mitigations to address potential bias. A tweaked and ostensibly “more secure” version is expected to be available in the coming weeks.

However, Dagan argues that Jamba is demonstrating the promise of SSM architecture even at this early stage.

“The added value of this model is that it can be easily attached to a single GPU, both due to its size and innovative architecture,” he said. “We believe that further tweaks to Mamba will further improve performance.”



Source link

Share. Facebook Twitter Pinterest LinkedIn Tumblr Email

Related Posts

OpenAI seeks to extend human lifespans with the help of longevity startups

January 17, 2025

Farewell to the $200 million woolly mammoth and TikTok

January 17, 2025

Nord Security founder launches Nexos.ai to help enterprises move AI projects from pilot to production

January 17, 2025

Data proves it remains difficult for startups to raise capital, even though VCs invested $75 billion in the fourth quarter

January 16, 2025

Apple suspends AI notification summaries for news after generating false alerts

January 16, 2025

Nvidia releases more tools and guardrails to help enterprises adopt AI agents

January 16, 2025

Leave A Reply Cancel Reply

Top Reviews
Editors Picks

7 days left: Founders and VCs save over $300 on all stage passes

March 24, 2025

AI chip startup Furiosaai reportedly rejecting $800 million acquisition offer from Meta

March 24, 2025

20 Hottest Open Source Startups of 2024

March 22, 2025

Andrill may build a weapons factory in the UK

March 21, 2025
About Us
About Us

Welcome to Tech Brunch, your go-to destination for cutting-edge insights, news, and analysis in the fields of Artificial Intelligence (AI), Cryptocurrency, Technology, and Startups. At Tech Brunch, we are passionate about exploring the latest trends, innovations, and developments shaping the future of these dynamic industries.

Our Picks

Sarah Smith launches a $16 million fund and says AI can “unlock” for solo GPS like herself

July 11, 2025

Medium CEO explains what it takes to stop losing $2.6 million a month

July 11, 2025

CISA confirms that hackers are actively taking advantage of the critical “Citrix Bleed 2” bug

July 11, 2025

Subscribe to Updates

Subscribe to our newsletter and never miss our latest news

Subscribe my Newsletter for New Posts & tips Let's stay updated!

© 2025 TechBrunch. Designed by TechBrunch.
  • Home
  • About Tech Brunch
  • Advertise with Tech Brunch
  • Contact us
  • DMCA Notice
  • Privacy Policy
  • Terms of Use

Type above and press Enter to search. Press Esc to cancel.