Have you recently logged in to your Twitter or Facebook accounts? Or visited an online retailer? If so, welcome to the world of Big Data!
You may not have chosen to join this club, but the moment you log in to a social media site, send a text message from your cell phone, or order something online, you are an unwitting contributor to Big Data. Information from hundreds of millions of people like you is helping create a vast data pool, which enables companies to pick over your shopping and spending habits, what you like to read and which YouTube videos you ‘like’. Governments, corporations and data-analysis firms trawl through the oceans of data to spot trends, identify problems and launch viral marketing campaigns. Big Data is proving an amazingly valuable tool to manage a complex consumerist planet, but it is also creating a sinister world of Orwellian ‘big brothers’.
The 4 billion people on their cell phones (450 million of them access the web), 750 million Facebook users and 500 million Twitter account-holders are generating a roaring stream of data about themselves — their likes, dislikes, their concerns and their location. Add to that the vast amount of data on citizens and their lives that governments and corporations have been collecting as well as data gathered by sensors of all kinds. According to IBM, 2.5 quintillion (that is 17 zeroes after 2.5) bytes of data are created every day. Increasingly, powerful algorithms developed to scan these disparate data from the Internet, social media, and sensors find correlations and project trends, providing unprecedented insight to users.
At the Tallberg Foundation’s annual forum in Sweden, I ran into Mikael Hagstrom, an executive vice-president of SAS Institute Inc., a leading analytics company. He flips open his iPad, types a few commands and bar graphs generated by SAS analytics appear on the screen to show what is being discussed on Twitter and where. He pulls up another application to show me visualised data of Twitter hotspots in Jakarta forming a virtual grid of Indonesia’s capital. That data snapshot was taken when Jakarta residents were tweeting about symptoms indicating the outbreak of an infectious disease. On another occasion, data showed many tweets about ripples in their cups, which might be an early warning of an earthquake. Tracking the twitter use of special codes for drugs, Hagstrom says, is allowing authorities in some Latin American countries to go after the pushers.
Big Data has given marketers a powerful tool to read the pulse of consumers in real time. With 68 million bloggers worldwide, reviewing products and making recommendations, companies are deploying Big Data analytics to monitor blogs and tweets to gauge and shape market responses in real time.
Ever growing data is also being increasingly used as a predictor. Analysing the timing and location of flu-related searches, Google has been able to identify possible flu outbreaks two weeks earlier than national governments. SAS has merged a variety of databases in North Carolina to accurately predict the areas in which there is a greater chance of child molestation; this enables social workers to intervene more effectively. Meanwhile, the ‘Blue Crush’ preventative analytics system developed by IBM has helped the city of Memphis, Tennessee, to achieve a substantial decline in serious crime. Analysing Big Data is allowing corporations to prevent and detect fraud and improper payments. The Securities and Exchange Board of India (Sebi), which processes more than 200 million transactions a day, relies on high-performance analytics to detect and prevent fraud.
But growing use of Big Data is also causing shivers of concern in civil society. Given the massive amount of personalised data private companies and government agencies hold — admittedly some from what people write about themselves on social media — an individual’s life can become an open book. Big Data enables marketers to manipulate customers’ purchasing decisions and governments to violate an individual’s privacy. Predictive insight from the mountain of personal data collected might help government in some areas, but it also brings closer the dystopian vision of an updated Big Brother in Philip Dick’s Minority Report — where computer analysis allows governments to identify and arrest individuals even before a ‘crime’ is committed.
Unless checks on abuse of personal data are instituted, Big Data to Big Brother may only be a short step.