Web-Crawling Program ID's Disease Outbreaks

Scientists are searching within the virtual world and finding real viruses.

Every hour, HealthMap, an infectious disease-tracking Web site, culls through news Web sites, public health list servs, the World Health Organization's online pages, and other Web sites in six different languages to pinpoint outbreaks of disease that real-world doctors can then act on.

"We were originally thinking about how we could expand disease surveillance and pick up outbreaks earlier than traditional methods," said John Brownstein of Harvard Medical School and Children's Hospital Boston, who created HealthMap in September of 2006 with Clark Friefeld, a software developer at Harvard Medical School.

"It was a pilot project, a side gig for us," explained Brownstein.

About nine months ago, HealthMap came to the attention of Google.org, the philanthropic arm of the Internet search giant, which began to fund the team. Then, as Brownstein recalled, "all of a sudden, it just took off."

HealthMap gathers information from the Internet and filters it, removing, for example, duplicated or irrelevant information. It can pinpoint an incident of bubonic plague in Siberia, for example, while ensuring that a "plague" of home foreclosures in northern California doesn't show up on the free access Google Maps.

So far the program identifies about 95 percent of all disease outbreaks, sometimes days before the World Health Organization or the Centers of Disease Control announce them.

The most recent example of this is the ongoing salmonella outbreak in the United States that has sickened more than 1,000 people and the cause of which is still unknown. HealthMap detected the outbreak before the CDC announced anything.

"This will definitely save lives," said Larry Madoff, editor of ProMED, an infectious disease monitoring Web site run by the International Society for Infectious Disease.

"This is a good step forward," said Madoff. "[HealthMap] helps us predict how disease outbreaks will happen."

ProMed, unlike HealthMap, uses human moderators, not mathematical algorithms, who specialize in geographic regions or specific areas of disease to identify public health outbreaks.

HealthMap isn't just for doctors, specialists and public health officials, however. If travelers are heading to Paraguay they can see if there is an instance of Yellow Fever, for instance, and get vaccinated before they leave.

There is room for improvement however, says Madoff -- and Brownstein agrees. The HealthMap team is expanding operations and increasing the amount of detailed information for each particular outbreak and incorporating more "noisy" sources of information, such as blogs and chat rooms.

These sources can be less reliable than traditional sources but could alert authorities to outbreaks much sooner than traditional detection methods.

"We hope to improve that score of 95 percent by picking up that needle in the haystack, the quiet, early indication of potentially serious outbreaks," said Brownstein.