ABSTRACT
Chapter 4 explores the processes that are necessary for curating, standardising, and performing analytics at one level and the manner in which analysis has been done at another. The outputs from the analytics are then discussed, with results exhibited through various visualisation techniques. The use of such information for decision-making is then explored, while some of the limitations of different algorithms are raised. This chapter looks at issues related to differential privacy, multi-party computation, adversarial machine learning, and algorithmic fairness. These are concerns that have now become a part of the discourse in applying big data for problem-solving. This chapter concludes with how technology and regulations have to work in tandem to ensure that privacy and transparency are both attained to a reasonable degree in big data analytics. Data is an unusual commodity. Individual data points on their own typically have little value. Value in data is created and multiplied only when the individual data points are aggregated, cleaned, joined with other data sets, and then analysed with the right set of statistical and computational tools to answer the right questions and generate actionable insights. Through a workflow of data processing and analysis involving both humans and machines, the raw data is turned into data products in which the value of the data becomes greater than the sum of its parts.
