Big Data Analytics-The way ahead

By Dr Vartika Dutta

Organizations are increasingly trying to start or revamp their digital transformation journeys. Consequently, “big data and analytics” seems to have become the focal point in every business deliberations. According to a prediction by CISCO, the global IP traffic by the year 2021 will be nothing less than 3.3 ZB per year. Approximately, the devices connected to IP networks will be 3 times the global population by 2021, while Gartner predicts around 1Mn IoT devices will be sold by the hour. It is witnessed globally that the speed with which the increasing digital footprint will change the lives of people is phenomenal. This is the power of Big Data.

The trend started picking up globally since 2000 when the companies were collecting a large variety of data from different sources-both structured and unstructured. Due to fast-paced technological innovation and increased impetus on digitization, there was also an unprecedented speed at which the data got generated. The challenge was really in uncovering this stockpile of data to make data-driven, fact-based business decisions using various analytical tools. Using Big Data Analytics, the organization could delve deeper into the data and find answers to complex business problems and predict the right course for the organization.

The B-school Education with its only focus on conventional MBA’s also needs to be revamped. There is a need for sector-specific MBA’s which focus more on upskilling and reskilling of professionals. B-schools are increasingly starting new programmes in data analytics. These include one-year certification courses, blended programmes combining classroom and online teaching and two-year full-time residential PGDM programme. Goa Institute of Management in its 25th year of imparting quality management education has also come up with a unique two-year full-time residential PGDM program in Big Data Analytics (PGDM-BDA). What explains this spurt in sectoral, domain-specific programmes being offered by management institutes?

Prevalence of Big Data Analytics in India

The prevalence of big data in Indian organizations and start-ups is pretty evident. The traditional tactical tool-centric organizations are now looking for strategic integration of architecture-centric data and analytics programs. Most organizations are making headways in finding Artificial Intelligence (AI) based solutions to improve their business value chain. Big data analytics is helping organizations to make accurate predictions in fairly less time. Most of the innovations are trying to bridge the gaps between cloud and hybrid data management and the emergence of data lakes and data hubs.

The real challenge lies in finding the right skilled professionals who can drive companies towards the adoption of such newer methods and techniques. The key issue that the managers would be facing is developing a robust analytics strategy and building an effective data-driven organization. There is a massive influx of big data through a various medium like social media, customer data, Internet clickstream data etc. However, companies lack the resources and capabilities to manage and analyse such data. Poor data literacy has been repeatedly cited as the biggest hindrance for the organizations foraying into data analytics.

Apart from private companies, Government of India has shown a keen interest in leveraging data analytics for improved decision making, real-time monitoring of various departments, transparency, reducing corruption, improved the reach of welfare schemes to poor, generating better insights for better governance and administration. Project Insight which was launched in 2017, post demonetization, was implemented to catch the tax evaders used data mining techniques. AI and Machine Learning techniques were used to find patterns and trends in bank account transactions. Ministry of Corporate Affairs (MCA) has deregistered more than 2.26 lakh companies by these methods. Another example would be that Goods and Service Tax network (GSTN) was used to study the transaction level data to analyze the flow of goods across India. Similarly, labour migration across the country is being analyzed using the big data generated from the railway passenger data. Various state governments like Andhra Pradesh, Karnataka, Odisha, Telangana etc. are also adopting data analytics for governance in a big way. The Smart City project initiated by the Government of India shows a huge scope of the prevalence of IoT technologies to solve major infrastructural issues of big cities.

Shifts in the Job market

There is a huge shift that the job market is also experiencing with an increased stress on competencies focussing on future technologies. Rapid automation, the confluence of AI and Machine learning in businesses worldwide is paving way for the professionals to update there learning and skill sets.

Fresher hiring has been on an upward trend. This year nearly 16000 freshers were hired by the analytics sector, an increase of 33% compared to last year. This spurt in hiring is on the back of shifting trend in the analytics domain. Earlier primarily the large businesses were using big data analytics for data-driven decision making. Today, several medium to smaller enterprises are also relying on big data for business insights. Cloud-based platform for big data has led to smaller businesses to increasingly take advantage of the latest technology. This has also led to an increase in the demand for the position of Chief data officers (CDO) in organizations to promote data competency and literacy.

Challenges related to Big Data Analytics

Technology has given us tools to harness large volumes of data, which has the potential to transform lives. Companies frequently collect user data to improve customer experience by providing personalized services, for the detection of fraud, efficient use of resources etc.

However, with the growth in the use of big data, the concerns regarding data privacy and confidentiality have also started rising. Consequently, ethical issues in analytics are being debated frequently. The recent allegations on Facebook and Cambridge Analytica of harvesting and using personal data for political purposes to influence election outcomes are a case in point. Similarly, data theft can lead to sensitive personal information being used for fraudulent financial transactions.

Ironically, the solution for such data theft has also emerged from big data analytics wherein “behavioural biometrics” is used to detect fraud. Behavioural biometrics are unique individual identifiers, that captures the way an individual interacts with their devices, the way they scroll, swipe, click, tap, on phone screen or keyboard. They help establish the identity of the digital user and detect and block attempts on impersonation. However, such extensive personal user data being captured by the organizations in the name of fraud detection is also unsettling and further raises privacy concerns.

These ethical dilemmas reaffirm the need to have a stronger policy regarding data privacy. We need new technology to improve life, however, it should be consistent with the basic moral framework of society. Data privacy day is celebrated on 28th January to promote awareness of data protection and privacy.

Aspirants planning to make analytics as their career need to be cognizant about these issues and be prepared for tackling them. At the same time, B-schools also have a responsibility to inculcate high moral integrity among their graduates so that they are able to navigate confidently in this tricky world of data analytics. The opportunities in this sector are boundless and all one needs to be successful is a curious mind and an appetite for uncovering patterns from large data sets. At Goa Institute of Management, we constantly believe in our philosophy “Learning never Stops at GIM” and we push our students to be lifelong learners. GIM strives hard to innovate by coming up with such unique program offerings, which arrests the need of this hour.

About the Author:

Dr. Vartika Dutta earned her PhD in Management from IIT Kharagpur. Her research interests are in the area of Organizational Behavior, Educational Leadership, Understanding Millenials, Virtual Learning Environments and Social Data Analysis. She comes with a rich experience in training, research and consulting and has held major positions in the training and development vertical of reputed financial and skill development firms. She is a core member of the Big Data Analytics programme at Goa Institute of Management.