The concept of data lakes are largely an analogous one because no one has really seen a lake full of data. A lake full of water; yes, a lake full of liquid substances; yes, but a lake full of data; no. Not really.
But in the world of ICT, anything is possible. Like, for example, there are concepts such as data clouds, server farms, data mining, artificial intelligence, predictive and prescriptive analytics and the strangest one yet – machine learning.
The terms, as sophisticated as they may sound – are actually pretty straightforward – as a technology. They are conceptualised and brought to life mostly for the purpose of dealing with massive amounts of data that digital devices are generating through operational transactions. The wealth of data that today’s global population generates far exceeds anything the world has ever seen before, and is likely to continue growth in only one direction – and that is upwards.
To make sure that data is not lost forever, enterprises and organisations invest in large data storage facilities, thus giving rise to technology concepts such as databases, data centers, data warehouses and today – data lakes.
But data alone is nothing if it cannot be made sense of. The real value of data emerges when it is processed, analysed and structured to provide meaningful insights. Such insights can prompt decision makers to justify and execute strategic business transactions that provide long and short term benefits to their organisations.
For example, consider the case of a small sized, yet fast growing ecommerce retail start up in a developing economy that has a healthy mobile and internet penetration rate. Small startups are usually faced with challenges in scaling due to costly overheads such as staffing and warehousing of products.
With a business predictor module, the owners of such a startup are able to make justified decisions with regard to which products to hold in stock, as well as make use of timely sales predictions so that their warehouses are not holding products that aren’t selling – and are able to base decisions on mathematical and logical predictions and invest warehouse space for products that are more saleable and take up less space. Such decisions to drive more product turnover not only makes it more profitable for a business but also drives a higher number of customers to the store – which in turn drives lower customer acquisition costs, more happy customers, and overall positive branding for the startup store.
Similarly, enterprises too are exploring the fields of artificial intelligence in the form of machine learning to drill down into the wealth of data that their connected devices draw back into their data storage facilities. Large enterprises that cater to large markets tend to generate large amounts of transactional data. Note that transactional data is not limited to merely financial transactions. Transactional data can mean any data that is generated when one or more devices or personnel communicate any sort of information with one another. Consider the case of a large enterprise such as a fast food chain.
A few typical data transactions that are regularly generated are as follows:
- Customer profile data
- Customer order history
- Individual customer preferences
- Staff availability (a.k.a employee check in/out times)
- Daily stock keeping and purchases
- Order delivery statuses
- Housekeeping logs, etc…
At this point, enterprises are faced with two options. Ignore the flood of incoming data generated through interactions between customers, devices, systems and employees, or – accumulate incoming data in a secure suitable data storage facility and then figure out what to do with the continuously accumulating data.
Next — Part 2 (to be published soon and is being written as you read this)
Enterprises that are not curious about the dynamics of their customer acquisitions, business growth and prevention of failures – will overlook the flood of incoming data and will most likely be forever reactive in the nature of their strategic operations. Other enterprises that are more inclined to observe feedback from their business processes and analyse the dynamics of operations are more likely to emerge as proactive and opportunistic in their business development activities.
- Introduction to Praedictio – machine learning and prediction model
Follow us as we explore the newest frontiers in ICT innovation, and we apply such technologies to solving real world problems faced by enterprises, organisations and individuals. Thank you so much for reading! 🙂