ABSTRACT
Science has long placed an emphasis on revisiting and reusing past results: reproducibility is a core component of the scientific process. Testing and extending published results are standard activities that lead to practical progress: science moves forward using past work and allowing scientists to “stand on the shoulders of giants.” In natural science, long tradition requires experiments to be described in enough detail so that they can be reproduced 34by other researchers. This standard, however, has not been widely applied for computational experiments. Researchers often have to rely on tables, plots, and figure captions included in papers. Consequently, it is difficult to verify and reproduce many published results [43], and this has led to a credibility crisis in computational science [17].
