DeDeo is not the only researcher grap­pling with these chal­lenges. Across every dis­ci­pline, data sets are get­ting bigger and more com­plex, whether one is dealing with med­ical records, genomic sequencing, neural net­works in the brain, astro­physics, his­tor­ical archives, or social net­works. Alessandro Vespig­nani, a physi­cist at North­eastern Uni­ver­sity who spe­cial­izes in har­nessing the power of social net­working to model dis­ease out­breaks, stock market behavior, col­lec­tive social dynamics, and elec­tion out­comes, has col­lected many ter­abytes of data from social net­works such as Twitter, nearly all of it raw and unstruc­tured. “We didn’t define the con­di­tions of the exper­i­ments, so we don’t know what we are cap­turing,” he said.

 

Read the article at Wired →