Using Restaurant Review Websites To Identify Unreported Complaints Of Foodborne Illness

BACKGROUND: The NYC Department of Health and Mental Hygiene (DOHMH) receives approximately 2,800 complaints about restaurants annually through the City’s non-emergency call number and website known as “311.” During a recent outbreak, DOHMH noted that many patrons had reported illness on restaurant review website X but had not notified 311. To detect outbreaks not previously identified and potentially prevent further illness, DOHMH collaborated with website X and Columbia University on systematically analyzing restaurant reviews for foodborne illness complaints. 

METHODS: Using website X’s NYC data, we trained a program to automatically analyze the text of reviews for foodborne illness complaints (e.g., sick, vomiting, diarrhea), and other criteria including number of people ill and relevant incubation period (>10 hours). DOHMH used this program to download and analyze new reviews weekly. Flagged reviews were then examined to determine if they 1) indicated an episode of gastrointestinal illness related to a restaurant, 2) suggested the illness occurred recently (within four weeks of the review if a time period was provided), and 3) warranted further investigation because two or more people were ill or severe illness occurred. For all reviews with a recent illness or no time period specified, the reviews were compared with the 311 database to identify duplicate complaints. For reviews requiring further investigation, DOHMH contacted the reviewers through website X’s messaging service.

RESULTS: From July 2012 - November 2012, 309 reviews were flagged by the classifier program; 165 (53%) described an episode of gastrointestinal illness, and 150 reviews suggested the illness had occurred recently or did not provide a time period. Comparison of these 150 reviews with the 311 database revealed that only five (3%) reviewers had also submitted a complaint to 311. Thirty-one reviews required further investigation, and six (19%) of these reviewers completed a phone interview. In response to the six complaint interviews, representing approximately 13 illnesses, DOHMH reviewed the food inspection history of all six restaurants and conducted environmental investigations at two restaurants.

CONCLUSIONS: Many restaurant-related foodborne illness episodes are not reported to city agencies, but are posted to restaurant review websites. By analyzing reviews on a regular basis, DOHMH was able to detect unreported restaurant-related illnesses and further investigate these establishments. Although no additional outbreaks were detected, DOHMH will continue to explore this tool by refining the mechanics used in the classifier program and expanding the analysis to include additional review websites.