ABSTRACT
It is ironic that the purpose of data and analytics is to support the business in making better decisions, yet often the data engineers and technologists do not follow the same approach. One of the key parts of the DataOps methodology is to instrument data to build up enough metadata to help drive design and implementation decisions. Capturing the right metadata is crucial to being able to make decisions on the right technologies to use and to understand where the data fits into the overall solution. People find that in many projects, profiling is seen as an unnecessary step in the project lifecycle, yet often the data that is captured in this key activity and the knowledge gained is often the difference between the success and failure of a project. It is important that the right data is captured and not just a brain dump of the requirements.
