Teaching AI agents to ask better questions by playing “Battleship”

Source Domain: news.mit.edu

Artificial intelligence agents show promise in executing well-defined tasks through language models, but struggle with complex problem-solving in fields like medical diagnosis and scientific discovery.
Researchers from MIT’s CSAIL and Harvard’s SEAS designed a “Collaborative Battleship” game to study how language models can better inquire and answer complex, natural language questions.
Their results indicated that large language models outperform humans in the game, while smaller models struggle, showing that many models lack the ability to form useful questions.
By integrating a Monte Carlo inference strategy, researchers improved both large and small language models’ question-asking effectiveness, dramatically boosting smaller ones’ performance.
The researchers converted questions into executable commands using Python, aiding smaller models to provide more accurate answers and improving the overall performance of all tested language models.
Although models showed progress in games like “Guess Who?,” they still lag behind expert human performance in complex scenarios.
The researchers believe this improved information-seeking capability could make language models excellent assistants for complex scientific discoveries, though they plan to test them in more complex, real-world settings.
The study highlights the need for AI agents to improve their pragmatic reasoning and ability to collaborate effectively with humans for solving tougher problems.

Teaching AI agents to ask better questions by playing “Battleship” | MIT News