ABSTRACT
Heroes exist; they put themselves out for a cause, often working all hours to solve problems and help things to move smoothly. However, sometimes this culture highlights underlying flaws in an approach. By using clever design and thinking in the first place, then maybe there is no need for heroics. This is what DataOps is about. One of the key challenges with data is that it is constantly flowing and changing. One of these key patterns is monitoring at all stages of the data pipeline and building in continuous improvement. This includes improvement in the monitoring and improvement in the diagnosis tools, as well as improvement in the ability to respond to changing requirements and needs. The meeting with Justin was set up for the following morning. Jennifer had spent much of the previous evening reading the documentation on the existing architecture. This included the cost justification for the data lake and the other technology selections in the solution stack.
