Large Language Models Struggle to Describe the Haystack without Human Help: Human-in-the-loop Evaluation of LLMs

We measure knowledge users acquire with topic models on two datasets, finding that LLM-based methods generate human-readable but overly generic topics for domain-specific data.

arXiv

Recommended citation: Zongxia Li, Lorena Calvo-Bartolomé, Alexander Hoyle, Paiheng Xu, Daniel Stephens, Alden Dima, Juan Francisco Fung, Jordan Lee Boyd-Graber. (2025). "Large Language Models Struggle to Describe the Haystack without Human Help." ACL.
Download Paper