Two of the hallmarks of librarianship are that we believe we are better searchers than most patrons and that we have rigorous tests for the authority of the information we find. The first is part of the impetus for providing a chat reference service at all, but when our digital reference exit surveys tell us our patrons are satisfied 85 percent of the time, the second forces us to question our data and to make sure we have an accurate way of measuring our performance. This article describes our efforts to measure librarian performance in chat reference by using use transcript analysis.