Close Menu
TechBrunchTechBrunch
  • Home
  • AI
  • Apps
  • Crypto
  • Security
  • Startups
  • TechCrunch
  • Venture

Subscribe to Updates

Subscribe to our newsletter and never miss our latest news

Subscribe my Newsletter for New Posts & tips Let's stay updated!

What's Hot

What should the founder think about if they are trying to raise the series c?

August 2, 2025

Venture company CRV raises $750 million and returns capital to investors, then downsizing

August 1, 2025

Kleiner Perkins has a really good week

August 1, 2025
Facebook X (Twitter) Instagram
TechBrunchTechBrunch
  • Home
  • AI

    OpenAI seeks to extend human lifespans with the help of longevity startups

    January 17, 2025

    Farewell to the $200 million woolly mammoth and TikTok

    January 17, 2025

    Nord Security founder launches Nexos.ai to help enterprises move AI projects from pilot to production

    January 17, 2025

    Data proves it remains difficult for startups to raise capital, even though VCs invested $75 billion in the fourth quarter

    January 16, 2025

    Apple suspends AI notification summaries for news after generating false alerts

    January 16, 2025
  • Apps

    TrueCaller's call recording function will not work on iPhones from September 30th

    August 1, 2025

    TrueCaller's call recording function will not work on iPhone from September 30th

    August 1, 2025

    Tiktok has incorporated “Tiktok Pro” in Europe and has incorporated a charity program

    August 1, 2025

    Instagram now requires users to live with at least 1,000 followers

    August 1, 2025

    Reddit will be integrated with the search interface as it appears to be a search engine

    August 1, 2025
  • Crypto

    Telegram's Crypto Wallet will be released in the US

    July 22, 2025

    Indian Crypto ExchangeCoindCX confirms $44 million stolen during hack

    July 21, 2025

    North Korean hackers blamed record-breaking spikes in 2025

    July 17, 2025

    Bitcoin surpasses $118K at the second highest high in 24 hours

    July 11, 2025

    Vitalik Buterin reserves for Sam Altman's global project

    June 28, 2025
  • Security

    Commerce Dept.'s backlog reportedly has suspended Nvidia's H20 chip license

    August 1, 2025

    The Commerce Department's backlog reportedly stalling Nvidia's H20 chip license

    August 1, 2025

    Sex toy maker Ravens threatens legal action after fixing security flaws that published user data

    August 1, 2025

    Authorities seize servers of black suit ransomware gang

    August 1, 2025

    Google is experimenting with machine learning power age estimation technology in the US

    July 31, 2025
  • Startups

    7 days left: Founders and VCs save over $300 on all stage passes

    March 24, 2025

    AI chip startup Furiosaai reportedly rejecting $800 million acquisition offer from Meta

    March 24, 2025

    20 Hottest Open Source Startups of 2024

    March 22, 2025

    Andrill may build a weapons factory in the UK

    March 21, 2025

    Startup Weekly: Wiz bets paid off at M&A Rich Week

    March 21, 2025
  • TechCrunch

    OpenSea takes a long-term view with a focus on UX despite NFT sales remaining low

    February 8, 2024

    AI will save software companies' growth dreams

    February 8, 2024

    B2B and B2C are not about who buys, but how you sell

    February 5, 2024

    It's time for venture capital to break away from fast fashion

    February 3, 2024

    a16z's Chris Dixon believes it's time to focus on blockchain use cases rather than speculation

    February 2, 2024
  • Venture

    What should the founder think about if they are trying to raise the series c?

    August 2, 2025

    Venture company CRV raises $750 million and returns capital to investors, then downsizing

    August 1, 2025

    Kleiner Perkins has a really good week

    August 1, 2025

    Details will be revealed about how Windsurf's VCS and Founders were paid from Google transactions

    August 1, 2025

    Basic Research Lab nabs over $30 million to build AI agents vertically

    August 1, 2025
TechBrunchTechBrunch

Nvidia's new tool lets you run GenAI models on your PC

TechBrunchBy TechBrunchFebruary 13, 20243 Mins Read
Facebook Twitter Pinterest Telegram LinkedIn Tumblr WhatsApp Email
Share
Facebook Twitter LinkedIn Pinterest Telegram Email


Always keen to encourage the purchase of the latest GPUs, Nvidia is releasing a tool that allows owners of GeForce RTX 30-series and 40-series cards to run AI-powered chatbots offline on their Windows PCs.

The tool, called Chat with RTX, allows users to customize GenAI models along OpenAI's ChatGPT by connecting them to documents, files, and notes that can be queried.

“Rather than searching through notes or saved content, users can simply enter a query,” Nvidia wrote in a blog post. “For example, you can ask, 'What restaurant did your partner recommend to you while we were in Las Vegas?'” Chat with RTX scans the user-specified local His file and provides context-filled provide answers. ”

Chat with RTX defaults to the open source model from AI startup Mistral, but it also supports other text-based models such as Meta's Llama 2. Nvidia warns that downloading all the necessary files will consume a significant amount of storage (50 GB to 100 GB, depending on the model) selected.

Currently, Chat with RTX works with text, PDF, .doc, .docx, and .xml formats. When you specify a folder that contains supported files in the app, the files are loaded into the model's fine-tuning dataset. Additionally, Chat with RTX can retrieve the URL of a YouTube playlist and load the transcripts of the videos in the playlist, allowing selected models to query that content.

There are some limitations to keep in mind here, which to Nvidia's credit are outlined in the how-to guide.

Chat with RTX

Image credits: Nvidia

Chat with RTX can't remember context, so the app won't consider previous questions when answering follow-up questions. For example, if you ask “What is a common bird in North America?” followed by “What color is it?” Chat with RTX knows you're talking about birds. not.

Nvidia also acknowledges that the relevance of an app's responses can be affected by a variety of factors, including the phrasing of the question, the performance of the selected model, and the size of the fine-tuning dataset. , some are easier to control. Asking for facts covered in several documents is likely to yield better results than asking for an overview of one or a set of documents. Also, response quality generally improves with larger datasets. Similarly, you'll be able to see more content on specific subjects in chat with RTX, he says Nvidia.

Therefore, Chat with RTX is more of a toy than something for production use. Still, there's something to be said for apps that make it easy to run AI models locally. This is kind of a growing trend.

In a recent report, the World Economic Forum predicted a “dramatic” increase in affordable devices that can run GenAI models offline, including PCs, smartphones, Internet of Things devices, and networking equipment. The WEF says the reasons are clear advantages. Not only are offline models inherently more private, offline models ensure that the data being processed never leaves the device on which it is running, resulting in lower latency and more cost-effectiveness than cloud-hosted models. .

Of course, democratizing the tools for running and training models opens the door to malicious attackers. A quick Google search will bring up a long list of models that have been tweaked based on malicious content on the web. But proponents of apps like Chat with RTX argue that the benefits outweigh the harm. We'll have to wait and see.



Source link

Share. Facebook Twitter Pinterest LinkedIn Tumblr Email

Related Posts

TrueCaller's call recording function will not work on iPhone from September 30th

August 1, 2025

TrueCaller's call recording function will not work on iPhones from September 30th

August 1, 2025

Tiktok has incorporated “Tiktok Pro” in Europe and has incorporated a charity program

August 1, 2025

Instagram now requires users to live with at least 1,000 followers

August 1, 2025

Reddit will be integrated with the search interface as it appears to be a search engine

August 1, 2025

More about Twitter's previous trust and safety head

July 31, 2025

Leave A Reply Cancel Reply

Top Reviews
Editors Picks

7 days left: Founders and VCs save over $300 on all stage passes

March 24, 2025

AI chip startup Furiosaai reportedly rejecting $800 million acquisition offer from Meta

March 24, 2025

20 Hottest Open Source Startups of 2024

March 22, 2025

Andrill may build a weapons factory in the UK

March 21, 2025
About Us
About Us

Welcome to Tech Brunch, your go-to destination for cutting-edge insights, news, and analysis in the fields of Artificial Intelligence (AI), Cryptocurrency, Technology, and Startups. At Tech Brunch, we are passionate about exploring the latest trends, innovations, and developments shaping the future of these dynamic industries.

Our Picks

What should the founder think about if they are trying to raise the series c?

August 2, 2025

Venture company CRV raises $750 million and returns capital to investors, then downsizing

August 1, 2025

Kleiner Perkins has a really good week

August 1, 2025

Subscribe to Updates

Subscribe to our newsletter and never miss our latest news

Subscribe my Newsletter for New Posts & tips Let's stay updated!

© 2025 TechBrunch. Designed by TechBrunch.
  • Home
  • About Tech Brunch
  • Advertise with Tech Brunch
  • Contact us
  • DMCA Notice
  • Privacy Policy
  • Terms of Use

Type above and press Enter to search. Press Esc to cancel.