Development and validation of a tool for detecting misinformation risk in diet, nutrition, and health content (Diet-MisRAT)
The figure above illustrates the development and validation of a tool for detecting misinformation risk in diet, nutrition, and health content (Diet-MisRAT). University College London
PAPER PUBLISHED IN
NATURE Scientific Reports
PRODUCED BY
University College London
AUTHORS
Alex Ruani
Michael J. Reiss
Anastasia Z. Kalea
ABSTRACT
Misinformation in diet and nutrition is recognized as a major public health threat, with the potential to misguide dietary choices and contribute to preventable harm. To address this, we developed a Misinformation Risk Assessment Model (MisRAM), grounded in the World Health Organization’s hazard risk assessment principles. MisRAM conceptualizes misleading content traits as stratifiable agents of informational adverse effects, weighed by their severity and likelihood of increasing recipient susceptibility. Building on this model, we designed the Diet-Nutrition Misinformation Risk Assessment Tool (Diet-MisRAT), a structured instrument that evaluates medium-to-long form content across four risk dimensions (inaccuracy, incompleteness, deceptiveness, health harm), yielding five-tier risk estimates from very low to very high. Validation involved five rounds: expert reviewers, trainee dietitians, postgraduate nutrition students, highly experienced nutrition professionals, and zero-shot prompt-based generative-AI risk detection. Results showed strong to very strong alignment with expert-derived benchmarks, supporting the tool’s interpretability and concurrent validity. ChatGPT demonstrated high test–retest reliability, accuracy, precision, sensitivity, and F1 scores under blinded untuned conditions, suggesting that adequately constructed, expert-designed prompting tools may help overcome training-dataset limitations. Diet-MisRAT offers a scalable, graded alternative to binary detection. Domain-calibrated risk stratification could guide proportionate interventions in content oversight, regulation, education, misinformation inoculation, and infodemic mitigation.
READ THE REST OF THE RESEARCH PAPER IN ITS ENTIRETY
RELATED RESEARCH PAPERS FROM THE INTEGRITY PROJECT
