ABSTRACT
This chapter proposes concrete methods for specifying the editorial behavior of news recommendation systems through collaboration between journalists and technologists to create metrics, data sets, feedback methods, and evaluation protocols. So far, the desired behavior of news recommenders has mostly been specified in terms of principles or guidelines. I argue that natural language specifications are inadequate because the translation to software must subsequently be undertaken by technical specialists, a process which requires consequential values-related decisions. Instead, I propose the specification of recommender editorial values through the collaborative creation of specific value-laden technical artifacts already used in contemporary engineering. These artifacts are much more precise than principles, yet do not require the technical understanding necessary to create novel algorithms.
