Researchers at Los Alamos National Laboratory have developed a new tool called the Prelim Attention Score, or PAS, to help detect when a vision-language model’s output is grounded in the image and when it may be relying too heavily on its own generated text.
The system is being viewed as a means to enhance vision-language model safety and trustworthiness.
“The PAS is a real-time, plug-and-play metric that acts as an internal monitor for the AI,” said Manish Bhattarai, a Los Alamos computer scientist.
“The system works with major existing vision-language models and requires minimal additional computational overhead, making it an efficient way to detect potential hallucinations. PAS achieves state-of-the-art accuracy in catching hallucinations, offering developers a practical path toward safer and more trustworthy multimodal AI systems,” he added.
Most commonly used vision-language models are autoregressive. They generate each new token, or word, partly by relying on the words they have already produced.
While this process helps the model form coherent responses, it can also cause the system to lean too strongly on its own previous output rather than the image itself.
PAS monitors a vision-language model’s prediction of each token. By doing so, it helps identify where the model is drawing information from and where hallucinations are likely to occur. The tool then presents a score that alerts users to the possible presence of...
A German court has ruled that Google can be held directly responsible for false answers produced by its AI Overview feature. This decision could have important consequences for tech companies that...