The importance of memory.

 Well, in a RAG chain we have always several important pieces:

  • the Embeddings Model, which translates texts in dense vectors, enabling Semantic Search
  • the Vector Store, where you safely save your texts and vectors, giving you a fast way to find all relevant docs
  • A Reranker, which helps to refine your search
  • The Large Language Model (LLM)
In the beginning, you could think that the LLM is only useful at the end of the chain, when all the docs retrieved are put inside the context of the prompt, together with your request. There, it synthetizes the answer.

But all the evolution we see today is often based on ideas on how to use more and more of the incredible power we have in current LLMs.



Let us consider one important feature we want to have in a Knowledge Assistant: we want the assistant to keep the memory of all the previous questions and answers (message history) and use it to enable a more natural kind of conversation.

For example, imagine that one of your questions is: "What is Long COVID?" (I'm working on a demo based on nice documentation from NIH).
Your next question could be: "What are the symptoms?" (and you don't repeat the subject).

Now, when you do the search inside the Vector Store you cannot simply use "What are the symptoms?" The symptoms of what? Every disease in your knowledge base?

You have to rephrase your question based on the message history.

One approach, well supported for example in Llama-index, is called "condense_plus_context": here you're using the LLM twice:
  • first, you take your last question + the message history and ask the LLM to create a condensed question (for example: "What are the symptoms for Long covid")
  • Then, you do a search in the Vector Store using the condensed question
  • and, only at the end, you send all the retrieved docs from the condensed question, plus the question, to the LLM to synthesize the answer
If you have built a chain using Llama-index or LangChain, adding memory and such an approach means a few lines of code.

If you want to see more details, have a look here: chat with memory


Commenti

Posta un commento

Post popolari in questo blog

Immagine
Improving the quality of your Generative AI Solution using RAG and Oracle OCI Data Science. Since the launch of ChatGPT , in November 2022, we have seen widespread interest in its innovative technology, particularly in its GPT-4 iteration. Since that moment, we have come to understand the capabilities of Large Language Models (LLMs) and we have entered a new era, where the Generative AI potential of these models can significantly improve both productivity and quality, across numerous professions We have learned that: LLM can answer our questions LLM can write emails for our customers LLM can perform copy-editing of our papers (like this one) LLM can quickly generate code snippets for prototype .... But, although LLMs have shown great promises, they also have limitations, especially in the Enterprise sector. At the beginning of this year, a really  good review paper was published, as usual, on Arxiv. In the paper they highlight, among others, these limitations: LLMs have limited k...
Continua a leggere
Immagine
Correre troppo avanti a volte non conviene... (TLDR). Questa volta voglio raccontarvi una storia un pizzico diversa (ed infatti lo faccio nella mia bella lingua nativa). Ma comunque una storia che ha a che fare con l'innovazione e l'AI. Parto da lontano: quando sono venuto a Roma per la prima volta, dopo la laurea, dividevo la camera di albergo con un carissimo Ingegnere di Bologna: Leo (eh, avessi seguito i suoi consigli, sopratutto in un certo campo... ). Era un genio dell'elettronica digitale! Dopo un po',  andò a lavorare in una multinazionale dell'elettronica, un gigante dell'epoca. Ma in un posto sperduto, caro a I. Silone. Quando ci incontravamo mi regalava le calcolatrici che usavano per i test di "resistenza", le lanciavano da X metri di altezza e dovevano sopravvivere (non i lanciatori, le calcolatrici). Un giorno mi racconto', un po' depresso, che il suo responsabile gli aveva detto di andare più piano. Di non essere sempre cosi avan...
Continua a leggere