Social media, cameras, sensors and more generate huge amounts of data that can overwhelm analysts sifting through it all for meaningful, actionable information to provide decision-makers such as political leaders and field commanders responding to security threats.
Sandia National Laboratories researchers are working to lessen that burden by developing the science to gather insights from data in nearly real time.
“The amount of data produced by sensors and social media is booming – every day there’s about 2.5 quintillion (or 2.5 billion billion) bytes of data generated,” said Tian Ma, a Sandia computer scientist and project co-lead. “About 90% of all data has been generated in the last two years – there’s more data than we have people to analyze. Intelligence communities are basically overwhelmed, and the problem is that you end up with a lot of data sitting on disks that could get overlooked.”