Newsrooms’ Insights
To get firsthand feedback, we conducted 30-minute interviews with each newsroom following the chatbot’s live period. These conversations began with open-ended questions from us and allowed time for participants to share their thoughts, concerns, surprises, and observations in their own words.
None of the four participating newsrooms had previously adopted similar AI tools. However, they all expressed interests in leveraging AI to facilitate newsroom operation, either through chatbots or other automation tasks. From the initial onboarding conversations, CISLM clearly explained how the chatbot functioned, including its limitations and privacy safeguards. All four newsrooms actively engaged in testing and refining the bot beyond the first demo.
Shared Reflections
While each newsroom had distinct feedback, several common reflections emerged.
All four newsrooms said that they were surprised about how quickly the chatbot can be built. After brainstorming meetings with each newsroom, we spent about a week building the demo chatbot. Newsrooms started testing and provided feedback within three days, after which revisions were made. Some also noted the low cost of implementation, approximately $40 per month, as a surprising advantage.
All four newsrooms said that the search function on their website was not satisfying. Most had a basic keyword-based search bar that frequently failed to return the most comprehensive results. Chapelboro and Henrico Citizen specifically said that they hoped the chatbot could be used as an alternative to the internal search bar for the website. Both teams felt that their existing tools performed poorly, even when compared to external engines like Google Search. They saw the chatbot’s potential in helping readers surface archived stories and related articles more effectively.
A common downside of the chatbot mentioned by all newsrooms was that it couldn’t respond to all questions. As mentioned above, all chatbots were configured to rely solely on their internal knowledge bases and not draw from external web sources to lower the chance of hallucination. When relevant information was missing, the chatbot returned a default “I don’t know” message. While this approach helped avoid hallucinations, it limited the bot’s range of responses.
Three newsrooms said automatically updating the chatbot’s knowledge base with new content as articles are published was valuable. Since none of the participating teams had in-house technologists or staff available to maintain the chatbot after launch, updates were made only when specific errors were reported. As each newsroom continued to publish a significant volume of content during the program, the information in the chatbot’s database quickly became outdated. The failure to integrate the most up-to-date information can cause misinformation and harm the credibility of the chatbot’s answers.
Two newsrooms, Henrico Citizen and The News Reporter, said that time and efforts for maintaining and updating the chatbot posed a significant challenge. The most time-consuming aspect was organizing archives, putting together resources, and updating the knowledge base. None of the four newsrooms had a standalone archive of their reporting, but their websites functioned as both publishing platforms and databases. As a result, older stories not currently featured on the homepage or topic pages were difficult to locate and integrate into the chatbot.
Two newsrooms, Chapelboro and Henrico Citizen, found the chatbot useful for basic, static information like navigating guidance on the website. They said that navigation guidance, submission instructions, and FAQs are often difficult for users to find. But the chatbot was able to provide clear answers quickly by drawing on information spread across multiple pages.