Original Paper: https://arxiv.org/abs/2407.16908
By: Georgios Kollias, Payel Das, Subhajit Chaudhury
Addressing the issue of hallucinations in large language models (LLMs) is a critical challenge.
As the cognitive mechanisms of hallucination have been related to memory, here we explore hallucination for LLM that is enabled with explicit memory mechanisms.
We empirically demonstrate that by simply scaling the readout vector that constrains generation in a memory-augmented LLM decoder, hallucination mitigation can be achieved in a training-free manner.
Our method is geometry-inspired and outperforms a state-of-the-art LLM editing method on the task of generation of Wikipedia-like biography entries both in terms of generation quality and runtime complexity.
Figure: Larimar pipeline for processing (prompt, input) pairs. Here model refers explicitly to Larimar decoder. Larimar encoder is implicitly involved in converting tokens in write and the query prompt (prompt bracketed by [CLS], [SEP] tokens) into latent vectors.
Large Language Models (LLMs) have revolutionized the field of natural language processing, boasting impressive capabilities in language generation and machine translation.
However, they are not without flaws; one of the most notable issues is hallucination, where the model generates text that is factually incorrect or nonsensical.