Big Data 101

May 7, 2021 - 5 Minutes Read - By Acha Ouma

The term ‘big data’ refers to a collection of data that is large and complex. It is data that grows drastically with time and is difficult to process or store using the traditional data management systems (relational database and data warehouses). Although the storage of large sects of information can be traced to earlier decades, the term ‘big data’ was first mentioned in late 1990s and only gained momentum in the early 2000s. 

Dimensions of Data

In 2001, Doug Laney, an analyst at Meta Group coined what is now known as the three dimensions of big data. That is, volume, velocity and variety (the 3Vs) that have become the generally acceptable definitions and characteristics of big data. 

Over the years, the volume and speed of data generated has become immeasurable and beyond the intellectual capacity of human beings.  For instance, in 2010, International Data Corporation (IDC) estimated that 1.2 zettabytes (ZB) of new data were generated. In 2018, it was estimated that new data generated would grow from 33 ZB to 175 ZB in 2025. 

Currently, more than 5 billion consumers interact with data on their day to day lives. IDC estimates that by 2025, this will grow to about 75% of the world’s population, creating over 90ZB of data in 2025. At this point, it is estimated that each person will have at least a data interaction every 18 seconds. This is attributed to devices that connect wirelessly to a network including computer devices and wireless sensors among others.

There are three types of big data. That is Structured, unstructured and semi-structured. In structured, any data can be stored, processed and accessed in a fixed format. Unstructured data is huge and possesses a number of challenges in processing it. The semi-structured data can contain both the structured and unstructured forms of data.

Data & The 3Vs

Volume

Data in organizations is collected from various sources such as business transactions, smart devices, social media and industrial equipment among others. The size of the data is enormous, hence the term ‘big data’. 

Size of the data plays an important role in determining the significance and potential insight of the data as well as whether it is big data or not. In the past, storage of this kind of data would have been challenging. It is however easier to store this data with new technologies such as Data Lakes that have eased the burden.

Velocity

The speed of data from various sources is fast and unprecedented. It is massive and continuous. It must also be dealt with in a timely fashion, almost in real-time in order to cater for the demands and challenges.  How fast the data is streamed and processed, determines the potential of the data.

Variety

This refers to the diverse sources and nature of data from structured – numeric data to unstructured – email texts, financial transactions and videos. The diverse nature of the unstructured data pose challenges for storage, mining and analyzing. 

Basically, the data comes in all types and formats. While earlier technologies had the capacity to deal with structured data efficiently, the change of structured data to semi-structured and unstructured present a challenge to the earlier technologies. The modern big data technologies are thus developed and advanced to handle –store, analyze and process the data that is generated at a higher velocity, volume and that is in all types and formats.  

Other Vs

Veracity – this describes the value of the data. As the data originates from different sources, it is a challenge to link and match the data across systems. Businesses dealing with Big data need to link and show a correlation otherwise, the data may run out of control.

 Variability – this describes the changing nature of the data. It is unpredictable and changes often and is sometimes inconsistent. Businesses need to know what is trending on social media and how they can manage daily, seasonal or event triggered trending matters that are characterized with high peak data loads.

Importance of Big Data

The importance of big data is not pegged on how much data one has but what they do with it. Analysing and processing big data may lead to finding solutions that lead to cost and time reduction, product development and enhanced offerings. A business that processes big data effectively may leap the following benefits:

  • Detecting fraudulent behaviour
  • Improving customer service
  • Better operational efficiency
  • Identification of potential risks and determining the root cause of failures and issues.
  • Better decision making

Although big data has its good side, it poses some challenges in representativeness, harmonization, generalizability and data overload among others.

The Point of Big Data

In essence, it’s not the amount of data that a business has that matters but what the organization does with the data. Big data can be analyzed and processed for insights that lead to efficient and effective strategic moves.  The ability to harness big data is a key determinant of how successful a business will be in the digital era.

Spread the love

    Who is Who

  • Safaricom confirms Fawzia Ali as Chief Consumer Business Officer

    Safaricom CEO Peter Ndegwa has appointed Fawzia Ali as Chief Consumer Business Officer effective 15th March 2023. Ms Ali has held this role in an acting capacity since July 2022 and has provided exemplary leadership and inspiration to the team during this period. In a

    Spread the love
    More ..
  • CHANGES AT THE HELM OF THE KENYA REVENUE AUTHORITY (KRA)

    The Kenya Revenue Authority on Thursday, 23rd February 2023 communicated the new management team after the resignation of Githii Mburu. In addition to his departure, other Commissioners were replaced. These include; Lilian Nyawanda (Commissioner, of Customs and Border Control), Terra Saidimu (Commissioner, Intelligence & Strategic

    Spread the love
    More ..
  • New appointment at Safaricom board

    Mr. Adil Khawaja (MBS) has been appointed the Chairman of the Board of Safaricom PLC, having been appointed director at the same entity earlier. Mr. Khawaja currently serves as the Managing Partner at Dentons Hamilton Harrison & Mathews, and is a leading name in the

    Spread the love
    More ..