Could machine learning models help the public better judge the quality of the health news they consume? Recent research by two University of New Hampshire professors suggests they can.
Ermira Zifla 和 Burcu Eke Rubini他是该校决策科学助理教授 彼得?. 保罗经济与商业学院, recently trained machine learning models to evaluate the quality of health news stories about new medical treatments.
他们的研究成果发表于 决策支持系统, found that the machine learning models outperformed laypeople evaluations in assessing the quality of these health stories.
The research tackles the complex challenge of determining the reliability of news that can be more nuanced – instances where the whole story isn’t being told 但 doesn’t fall into the category of fake news.
This challenge can be more pronounced with the quick 和 wide dissemination of news stories 和 press releases about new medical treatments because such stories can feature inflated claims 和 suppression of associated risks. 同时, most ordinary people don’t have the medical expertise to underst和 some of these complexities.
“The way most people think about fake news is something that's completely fabricated, 但, 尤其是在医疗保健领域, 它不需要是假的. 这可能是他们没有提到什么，”Zifla说. “In the study, we’re not making claims about the intent of the news organizations that put these out. 但如果遗漏了一些东西，应该有一种方法来看待它.”
在研究中, Zifla 和 Eke Rubini utilized a data set from 健康 News Review that included news stories 和 press releases on new healthcare treatments published in various outlets from 2013 to 2018.
These articles were already evaluated by a panel of healthcare experts – medical doctors, healthcare journalists 和 clinical professors – based on 10 different evaluation criteria the experts had developed. 标准包括治疗或测试的成本和收益, 任何可能的伤害, 争论的质量, the novelty 和 availability of the procedure 和 the independence of the sources.
The researchers then developed an algorithm based on the same expert criteria, 并训练机器模型对新闻故事的各个方面进行分类, 将标准匹配为“满意”或“不满意”." The expert evaluations were the benchmark against which the machine-learning models were trained 和 tested.
“We had this great data set that had news that was evaluated on different criteria, 和 that is rare because it's costly 和 requires a lot of time 和 expertise to do,Zifla说. “We figured we could leverage that data set 和 machine learning to automate the process.”
他们的方法, 采用多准则专家评价, contrasts with previous studies that typically rely on a binary true-false framework for detecting fake news, Eke Rubini补充道.
The model's performance was compared against layperson evaluations obtained through a survey where participants rated articles as "Satisfactory" or "Not Satisfactory" based on the same criteria. 调查显示出一种“乐观偏见”,在254名参与者中，大多数人对文章的评价是满意的, 明显不同于模型中更为关键的评估.
“We can speculate as to why, 但 we don't test that in the paper,Zifla说. “相信新闻或医疗信息可能是一种普遍倾向.”
随着公众继续通过社交媒体快速消费健康新闻, Eke Rubini 和 Zifla believe it would benefit social media companies to use multi-criteria machine learning models like the one they developed to create digital nudges to help consumers assess these stories.
“每次有永利app新版本官网地址健康新闻的新闻报道, it would run through the algorithms 和 then give results based on whether it satisfies or doesn't satisfy the different criteria, 这可以自动整合到网站中,艾克·鲁比尼说.
Future research could focus on developing new models with different criteria or explore the public’s receptiveness to machine learning evaluations if implemented by social media platforms.
“这是一个非常困难的挑战. We hope to start a conversation about evaluating news based on multiple criteria. I can't emphasize enough that we should move away from the binary thinking fake news or not fake news,Zifla说. “这些模型可以适应更好的标准和更好的功能. 这一点总是可以改进的.”
-
写的:
亚伦桑伯恩 |彼得·T. 保罗工商经济学院| 亚伦.sanborn@tincyn.net
