ABSTRACT
The evaluation of productivity, quality and agility requires concrete metrics to be evaluated prior to the introduction of ALIGNED tools. An important aspect of system evaluation that has cross-cutting impacts on quality, productivity, and agility is the well-developed concept of usability and ALIGNED performed usability evaluations on all tools developed within the project. This chapter provides a set of guidelines followed by the coordinators of ALIGNED pilot studies and trials. Specific instructions are provided for each step in the life cycle of these pilot studies that involves ethical considerations. Agility for our use cases will often be measured with respect to parallel co-development of software, schema and datasets as agility is a cross-cutting concern. For instance, a change to a schema or ontology will generally require both migrations of datasets, as well as changes to the programme interface to consumption of the data.
