We know that there's interest in experimenting with AI in local news, yet not a lot of great insights as to how audiences actually are engaging with AI. To help news organizations experiment with chatbots, and to help us better understand how news consumers are interacting with AI, we developed a research project, the Local NewsBot Studio, that tackled both.
- Bots addressing FAQs, customer service, or archive navigation provided the clearest value; broad “ask anything” bots struggled.
- About 32% of conversations triggered “I don’t know,” and bots often failed with time-sensitive queries like “latest stories.”
- 185 inquiries across four bots in 45 days; content-focused bots saw more follow-up questions than service bots.
We’re not just interested in whether these tools work — we want to understand how news audiences would like to engage with chatbots, and thus AI, and why. That’s the research question behind this project.
I have questions — Who can I contact?
For general program questions, please reach out to CISLM Project Manager Sarah Vassello (svassello@unc.edu). For AI and chatbot specific questions, please reach out to Local News Researcher Yanan Sun (yanans@live.unc.edu). For research questions, please reach out to Interim Director Jessica Mahone (jamahone@email.unc.edu).
FAQ
We know that there's interest in experimenting with AI in local news, yet not a lot of great insights as to how audiences actually are engaging with AI. To help news organizations experiment with chatbots, and to help us better understand how news consumers are interacting with AI, we're kicking off a pilot program, the Local NewsBot Studio, that we hope will tackle both.
What will this program entail?
CISLM Local News Researcher Yanan Sun and Project Manager Sarah Vassello will consult with four news orgs to determine the best chatbot product for the audience.
- We'll be working with different types of media organizations (ex: print, public radio, student media and TV) to test with different audiences.
After getting a sense of the desired product, Yanan and Sarah will build, install the Zapier-based chatbot on the news org's site (and/or provide instructions on how to install) and train the news organization on how to use/how it's built.
- This can be an in-person or virtual training, depending on newsroom needs.
CISLM will collect the user journeys from the Zapier bot during the month-long run time of the bot.
After a month, our research team will start writing their findings, based on the results.
We’re a journalism organization — how can we ensure this is accurate information?
We work at a journalism school, so accurate information is paramount to us! The biggest concern in AI products is the possibility for hallucinations, in which false information is given mistakenly. Our chatbots only pull from the Knowledge Source, a custom spreadsheet with information provided by the newsroom for each of these projects. This limits the possibility of hallucinations. While the bots are live, both CISLM and newsrooms will have access to the query data, which will help both parties monitor accuracy and correct hallucinations in a timely manner. Further, prior to launch, the CISLM team will be sure to train and correct any hallucinations that are occurring.
Why CISLM?
Our unique team structure makes this program standout. With our strong news research focus, we're able to test and measure the effectiveness of these tools for communities. Additionally, Local News Researcher Yanan Sun and Project Manager Sarah Vassello both have extensive product experience, especially in AI:
- Sun is a former entrepreneurial fellow at the Brown Institute for Media Innovation and has co-founded an AI product that helps users read mobile apps’ privacy policies.
- Vassello created the Local News Audience Assistant chatbot for CISLM and has product development experience with the Institute for Nonprofit News and The Daily Tar Heel.
- Jessica Mahone, CISLM's interim director, is a celebrated researcher in the local news field.
- Both Sun and Vassello have spoken at conferences about product development and AI, and have been invited to participate in AI-specific workshops.