Libraries to Start Trials of AI Scholarly Search Tools on Jan. 20

Starting Jan. 20, WSU Libraries will run trials on three artificial intelligence (AI)-powered research platforms: Consensus, Scite.ai, and Undermind. The platforms help researchers analyze citation context, conduct deep literature searches, and synthesize findings across scientific papers.

The Consensus trial will run through the spring semester. Scite.ai and Undermind will be available to evaluate for 6-8 weeks. To learn more about the AI tools, please refer to the library guide created by Emily Cukier, Suzanne Fricke, Alex Merrill, and Jon Schoenfelder, members of the libraries’ AI working group. To be a tester or to receive more information as the trials roll out, please email Merrill.

“We are highly interested in getting student, faculty, and staff feedback about the AI platforms,” Cukier said. “It may not be feasible or cost effective for us to maintain subscriptions to all of them, so we want help deciding which of them (if any) the libraries should subscribe to.”

Working group members want to hear from people across disciplines which of the tools and features are most valuable to their work and learning, and how, Cukier said, as well as any concerns, such as accuracy or reliability, potential for misuse, or loss of learning opportunities.

“We’d also like feedback on what kind of education and support we can give to help people make the best use of the tools,” she added.

Advantages and disadvantages of using AI scholarly search tools

AI tools can be helpful for academic literature searches in several ways. They allow people to use “natural language” rather than controlled vocabulary, keywords, and Boolean operators; conduct more in-depth searches; find and rank results according to meaning and/or metadata; generate summaries and extract information; and pose clarifying questions to zero in on a good research starting point.

However, these tools have their disadvantages as well. First, they are limited to open access articles and abstracts, unless the AI platform has a licensing agreement with an academic publisher to pull journal information from behind a paywall. Generated summaries may ignore or omit context, misattribute content, or reflect or even amplify bias. Evaluating or critically appraising reference quality is limited. Finally, the same search question will yield different results each time it is run.

Cukier said the libraries are taking a nuanced approach to exploring AI-enabled research platforms. While not perfect, they are tailored for finding and summarizing information specifically from articles and books, rather than the web at large.

“This is a big advantage when you want to focus on scholarly sources, as we often do in academia,” she added.

All university users sought for AI trials

From early investigations, Cukier and other working group members suspect that different university user groups will have different preferences among the AI platforms, she said. Some may do better in specific fields than others. Faculty will use the tools for different purposes than students. Even students at different points in their educational journeys will need them for different things.

“So it’s very important to us to get feedback from across the university community to know how well they work and for whom,” Cukier said. “We want to ensure students get the opportunity to develop their skills using these tools in a guided setting and to let faculty explore how these might empower them in finding and analyzing prior research. But we also want to make sure the tools are providing good value and to know their limitations.”

AI literature search and summarization create new challenges for information—the tools can be “wrong” in new and unexpected ways—so WSU librarians want to work alongside testers to help them understand what they can trust and where they may need to be more careful, Cukier said. “This feels like our best shot to help students come to terms with information searching in the age of AI as this new reality emerges.”

“During lean budget years, I am grateful to our many donors, especially Mary Kay Fowler, whose gifts to the Library Excellence Fund are supporting these AI trials,” said Trevor Bond, WSU Libraries’ dean.