The world and its dog has been shocked by the Prism news story. Early in June, we found out that the US National Security Agency (NSA) had developed a secret data-gathering mechanism to steal all our data and store it in a large data warehouse.
We are outraged that it is being mined, searched and otherwise prodded. But do we really think that big data security problems stop at Google, Facebook, Microsoft and Fort Meade?
The private sector has been collecting data on all of us for ages. It is stored in massive data sets, often spread between multiple sources. What makes us think this is any more secure? At least the NSA is well trained in keeping it all under lock and key.
What does “big data” mean, anyway? Some describe it – wrongly – as simply a lot of data in a relational database. But if that were the case, then the security challenges would be the same as for conventional databases. And they aren’t.
Others view it as data sets so large that they cannot be handled by traditional relational tools. But we have had that kind of thing for years, in the form of data warehouses.