How to Use Chat with RTX from NVIDIA

Mytips.id - What is Chat with RTX from NVIDIA? Here is a comprehensive explanation that has been summarised for you by Mytips.id.

It is undeniable that the technology called chatbots is currently being used by millions of people worldwide every day. This technology is supported by cloud servers. However, there is good news for Windows users, as there are now chatbots supported by NVIDIA RTX for local, fast, and customizable generative AI. These chatbots are named Chat with RTX from NVIDIA.

What is Chat with RTX from NVIDIA? As the name suggests, Chat with RTX is a conversation with an RTX supported by NVIDIA. It is a technology demo that allows users to personalise chatbots with their own content. The chat process is very realistic because the results are accelerated by a local NVIDIA GeForce RTX 30 Series GPU or higher with a minimum of 8 GB of video random access memory (VRAM).

This chat with RTX uses retrieval-augmented generation (RAG), NVIDIA TensorRT-LLM software, and NVIDIA RTX acceleration to bring generative AI capabilities to local Windows PCs supported by GeForce. Users can quickly and easily connect local files on their PC as a dataset to large open-source language models such as Mistral or Llama 2. This will enable queries to receive quick and relevant answers in a contextual manner. Instead of searching through stored notes or content, users simply type in their question.

For example, someone might ask, "Which restaurant did my partner recommend while in Las Vegas?" and Chat with RTX will scan the user-designated local files and provide contextually relevant answers. Rest assured, this tool supports various file formats, including .txt, .pdf, .doc/.docx, and .xml, so you can use the file formats you prefer. Simply direct the application to the folder containing these files, and the tool will load them into its library in just seconds. Users can also input information from videos and YouTube playlists. Adding a video URL to Chat with RTX allows users to integrate this knowledge into their chatbot for contextual questions.

How to download Chat with RTX

The first step is to download and configure Chat with RTX, which is actually slightly more complicated than you might think. All you need to do is run the installer, but the installer is prone to failure, and you must meet some minimum system requirements.

You will need an RTX series GPU 40 or 30 with a minimum of 8 GB VRAM, 16 GB system RAM, 100 GB disk space, and Windows 11. Step 1: Download the Chat with RTX installer from the Nvidia website. This compressed folder will be 35GB in size, so it may take some time to download.

Step 2: Once downloaded, right-click on the folder and select Extract all. Step 3: Inside the folder, you will find several files and folders. Select setup.exe and run the installer.

Step 4: Before the installation begins, the installer will ask where you want to save Chat with RTX. Make sure you have at least 100 GB of disk space in the location you choose, as Chat with RTX actually downloads AI models.

Step 5: The installer may take up to 45 minutes to complete, so do not worry if you see it paused momentarily. It may also slow down your PC, especially when configuring AI models, so we recommend stepping away for a while once the installation is complete.

Step 6: Installation might fail. If so, rerun the installer, select the same location for data as before, and the installer will resume from its last point.

Step 7: Once installation is complete, you will get a shortcut to Chat with RTX on your desktop, and the application will open in a browser window.

Using Chat with RTX with your data

An interesting aspect of Chat with RTX is that you can use your own data. It utilises something called retrieval-augmented generation, or RAG, to flip through documents and give you answers based on those documents. Instead of answering any question, Chat with RTX excels in answering specific questions about a particular dataset.

Nvidia provides some sample data for you to try out the tool, but you need to add your own data to unlock the full potential of Chat with RTX.

Step 1: Create a folder where you will store your dataset. Note its location, as you will have to direct Chat with RTX to that folder. Currently, Chat with RTX supports .txt, .pdf, and .doc files.

Step 2: Open Chat with RTX and select the pen icon in the Dataset section. Step 3: Navigate to the folder where you stored your data and select it. Step 4: In Chat with RTX, select the refresh icon in the Dataset section. This will retrain the model based on the new data. You should refresh the model whenever you add new data to the folder or select a different dataset.

Step 5: Once your data is added, select the AI model you want to use in the AI model section. Chat with RTX includes Llama 2 and Mistral, with Mistral as the default. Experiment with both, but for new users, Mistral is recommended.

Step 6: From there, you can start asking questions. Nvidia notes that Chat with RTX does not consider context, so previous responses do not influence future responses. Additionally, specific questions generally yield better results than general questions. Lastly, Nvidia notes that Chat with RTX sometimes references incorrect data when providing responses, so keep that in mind.

Step 7: If Chat with RTX stops working, and restarting does not fix the issue, Nvidia says you can delete the preferences.json file to resolve the problem. 

It can be found in C:\Users\\AppData\Local\NVIDIA\ChatWithRTX\RAG\trt-llm-rag-windows-main\config\preferences.json.