Turning data into insights.
IDC tells us that 90 per cent of the world’s total data has been created in the past two years. Thanks to the sharp rise of social networking and mobile computing, data is now being created as fast as it is being collected. IDC predicts the digital universe will expand this year by almost 50 per cent to just under 4 trillion gigabytes.
What is “Big Data”?
Big Data generally refers to the collection of very large and complex data sets with sizes beyond the ability of commonly used software tools to process. Whilst the term is commonly used to reference the volume of the data itself, it may also refer to the technology required to manage and interpret the large amounts of data. The challenge is that most of today’s data is unstructured and in a variety of types (e.g. media feeds, images, streaming video, text files, documents) so Big Data needs to read and interpret everything from documents and electronic forms to emotion and expression.
Big Data in action
There are many examples of how Big Data helps big business and government bodies. Law enforcement agencies use Big Data to identify crime hotspots by crowd sourcing to take pre-emptive action that in the end contributes towards a better quality of life in the community. The latest cars on our roads can generate up to 25 Gigabytes of data per hour which can then be used by the manufacturers to understand driving behaviour and vehicle wear and tear. This data is used to reduce accidents and lower maintenance costs.
The problem facing big business
In a recent IDC study, which polled 300 medium-large organisations from all industries across Australia, 47 per cent of respondents revealed they do not have the skill sets required to manage big data.
The problem is that the skills required to manage Big Data are not those a traditional Database Administrator (DBA) can up-skill for. The DBA is familiar with manipulating data that is neatly indexed and filed in rows and columns. Therefore, they quickly find themselves out of their comfort zone when they have to start mapping and contextualising large volumes of data of different types and sources. Big Data work is more suited to people skilled in mathematics and physics and it gives rise to such titles as data scientist, data modeler or analyst.
There is an abundance of DBAs in the market but there are only a limited number of experts trained in Big Data. Finding a talented Big Data expert is tricky and this leaves many organisations exposed when it comes to dealing with the new unstructured data coming into their environment.
The problem facing small business
Small business also faces the complexity of interpreting varieties of data to help them improve their services, create new markets and deploy new products. The problem is that small business owners typically can’t justify the cost of extra resources to provide Big Data insights and there are no “Big Data in-a-box” solutions for a small business budget. In short, the biggest issue for small business is simply that they can’t afford Big Data analysis for the time being so they must continue to rely on discrete data sources, such as their CRM and Google Analytics.
What next?
The concept of Big Data is still quite new and there is still a lot of market hype and confusion. If Big Data is something that you want to consider for your business then drop us a line below or give us a call on 02 9386 2900 and we’ll explore your options.