Validating Large Language Models

Large language models (LLMs) are a class of artificial intelligence (AI) models that are designed to understand and generate human language. These models have been applied to language translation tasks and content recommendations, the latter of which relies on analyzing a user’s interests and preferences.

George Amvrosiadis, an associate research professor of electrical and computer engineering, describes LLMs as resources with great industry potential while also recognizing that AI must continue to improve its sophistication.

“In essence, large language models serve as versatile tools that can automate and enhance various language-related tasks, making communication, information processing, and content creation more efficient and accessible,” said Amvrosiadis. 

“They have already demonstrated their versatility in addressing diverse real-world applications. However, they also raise ethical and societal concerns, such as bias in language generation and the responsible use of AI in decision-making processes and content attribution.”

One of the biggest challenges in dealing with language models is validating the information they generate. Amvrosiadis and Virginia Smith, an assistant professor of machine learning in the School of Computer Science, co-advised former Ph.D. student Michael Kuchnik, who took the lead on a research project to design an automated tool to audit LLM responses. The team’s research paper, “Validating Large Language Models with ReLM,” won the Outstanding Paper Award at the MLSys Conference in May. 

Validation, in the context of AI, refers to making sure that the responses generated by a language model adhere to certain legal, ethical, and quality standards, while also being appropriate, accurate, and fair for the given context. 

Considering language models are trained using vast amounts of text data from the internet, as Amvrosiadis explained, validating the responses they generate is important not only for factual accuracy but also to watch for and remediate biases. LLMs can inherit and propagate biases present in their training data, leading to the possible generation of discriminatory or offensive content. 

In order for a tool to be practical, it must be based on the prompt-answer format used to query LLMs. The validation tool should be able to constrain the responses provided by the language model to a predetermined format. For example, when trying to determine George Washington’s birthday, the format of acceptable answers would be: “George Washington was born on <month> <day> <year>.” 

Language models are essentially graphs with more connections than the neurons in a human brain, Amvrosiadis noted. To that end, getting a birthday right doesn’t prevent a model from spreading misinformation about George Washington in general. “Our tool, ReLM, can issue many questions per second to increase coverage against misinformation.” 

There are still many advancements that Amvrosiadis foresees as LLMs continue to develop. “We also need tools that will allow us to act once we detect that a model has biases. LLMs are expensive to retrain, so we would have to find a way to mitigate responses that are inappropriate. 

“I am hoping that ReLM is only the beginning in devising these tools, because LLMs are poised to become part of our everyday lives and affect the way we interact with the world.”