A New GPU and Now, a New AI Chatbot? Nvidia is on a Roll, Launches ‘Chat with RTX’

Nvidia is on a roll! Yesterday, the company unveiled a whole new GPU, the RTX 2000 Ada. Now, the company has just come out with an early version of a new app called Chat with RTX, and it’s pretty exciting for anyone with a newer Nvidia graphics card. This app is all about letting your computer do the heavy lifting when it comes to working with AI. You can throw YouTube videos and documents at it, and it’ll help you make sense of them, right there on your machine. The best part? You only need an Nvidia RTX 30- or 40-series GPU to get started.

which is a demo that lets users personalize a chatbot with their content on Windows PCs.

The custom generative AI is now free to download, and it requires an RTX 30 or 40-series GPU with at least 8GB of VRAM. This means users will have the capability to upload their documents to create summaries and receive relevant answers based on their data.

Chat with RTX can let you search YouTube URLs and transcripts for specific mentions or summarize an entire video.

Users will be able to connect local files on a PC as a dataset to an open-source large language model like Mistral or Llama 2, enabling queries for relevant answers. According to Nvidia’s blog, Chat with RTX “uses retrieval-augmented generation (RAG), NVIDIA TensorRT-LLM software, and NVIDIA RTX acceleration to bring generative AI capabilities to local, GeForce-powered Windows PCs.”

RAG is essentially an assistant that searches through the data, especially if the model set is particularly large. The tool supports various file formats, including .txt, .pdf, .doc/.docx, and .xml.

The company, which became the fourth most valuable U.S. corporation this week, said by utilizing the power of local GeForce-equipped Windows PCs, users can enhance their experience and take advantage of generative AI with unparalleled speed and privacy.

“Rather than relying on cloud-based LLM services, Chat with RTX lets users process sensitive data on a local PC without the need to share it with a third party or have an internet connection,” it added.

Nvidia also invited developers to discover how RTX GPUs can potentially speed up large language models by consulting the TensorRT-LLM RAG developer reference project on GitHub.

Nvidia’s ongoing success

In recent months, the company has experienced an extraordinary surge in its value. In December, Nvidia’s stock value tripled, outperforming every other company in the S&P 500. Its chips have proved crucial at a time when there is a global shortage.

However, there has been controversy over its sale of chips to Chinese military bodies and state-affiliated groups, despite a U.S. ban on the export of the commodity to China.