ABSTRACT
Economists use utility functions to model human preferences (Varian 1992). A utility function basically specifies how much happiness an individual would receive from different possible choices. Rational people, by assumption, maximize their utility. Surprisingly, utility functions are a useful tool for artificial intelligence (AI) theorists to model how a future computer superintelligence might behave (see Bostrom 2014; Omohundro 2008; Soares et al. 2015; Yampolskiy 2016; Yudkowsky 2011). This chapter mines a small subset of economic utility function analysis, the theory of rational addiction (Becker and Murphy 1988; Stigler and Becker 1977), to gain insight into how a superintelligence might act.
