A few questions on a simple RAG app

Help
1

@mrunal-modi asked in Discord:

  1. Does vast data allow for vector search index too? So we can avoid both external vector db and search index solutions?
  2. Context Assembly i.e. process of gathering relevant data, such as retrieved vector search results and other contextual information, before sending a query to the LLM for response generation can be hosted inside the VAST data platform too or is on the API server?

image
image1590×1124 103 KB

image
image1612×762 83.9 KB

2 Likes
2

@brian.verkley responded:
There is actually a range of answers here. First, as you appear to have a good handle on, VAST can host the data, and will process the vector embedding creation based on triggers as data is added or changed in the system. These vectors can be stored in our vector database. When we come to RAG, VAST doesn’t make any of the search, index, or context assembly software, but we use and can run NVIDIA NIMS in our kubernetes cluster. Our VAST InisghtEngine with NVIDIA product is an example of this fully managed solution, where those NIMS are available to you out of the box. But if you have specific needs where you need different software for any component, we’re happy to adapt.

1 Like