Welcome! GovernYourData.com is an open peer-to-peer community of data governance practitioners, evangelists, thought leaders, bloggers, analysts and vendors.
The goal of this community is to share best practices, methodologies, frameworks, education, and other tools to help data governance leaders succeed in their efforts.
To appreciate what it takes to get the most out of Big Data let’s look at what Big Data is and “information density”. Information density is the amount of valuable information per byte of data.
What is Big Data?
Big data is typically data from sources that are collecting interactions of people or things.
What is “small data” i.e. not Big Data
small data is the data used in typical business processes
Relative Information density
Because small data is built for specific business processes Byte for Byte “small data” has more direct value to the business. This is why business applications have focused on this data. It may take less work but governing this data is still not done well in many businesses.
Big data is less dense so more work is needed to obtain value, e.g. processing text to derive business context. Because big data is not typically focused on business processes it also has a higher noise to information ratio and needs more analysis/filtering to obtain business information.
But … Pound for pound there is more big data available
So working on Big Data can add tremendous value, even if it is more work. This is why businesses are so interested in Big Data.
The other But … The value of Big Data is strongest when tied to small data
To really understand what profile of customers have what preferences requires tying together all the master data and transactions in about the customer. Knowing the sentiment around specific products / vendors requires knowing the relationship between your customers and products.
So to obtain the biggest gains form Big Data it is important to realize more work and filtering needs to be done on the Big Data. And that your Big Data needs to be integrated with well governed small data.
More about Jeff