ABSTRACT
In this chapter, I present a brief overview on my early experiences with data analysis, followed by case illustrations covering specific projects. I started back in the 1970s with punch cards and a central computer, conducting simple multivariate analyses utilising what—by today’s standards—are small data sets. As time went on, data sets got larger and represented complex interrelationships, requiring a multi-layered approach to analysis. But the most dramatic disruption came with the availability of online data and, in particular, real-time online data. And right now we are facing a massive disruption brought about by the spread of Artificial Intelligence (AI), replacing statistical analysis in a range of optimisation challenges.
Going through these changes required constant learning and a high degree of agility. Importantly, there was also a need to gather domain-specific experience and expertise to be able to develop an effective protocol for meaningful and relevant analysis and interpretation of the results. The case study I provide on Implicit Goal Segmentation is intended to highlight this requirement. But the role of analysis is not always to identify meaningful relationships and patterns. The second case study shows how analysis allows us to validate and operationalise a theory.
Finally, I provide a brief introduction to the use of AI as an optimisation tool and highlight the key differences between the traditional statistical approach and AI from a user’s perspective.
The key points are summarised in the “Troubleshooting” section at the end of this chapter, followed by suggested exercises.
