chapter  9
30 Pages

Information Retrieval Methods

A desirable measure of retrieval performance would have the following proper­ ties. First, it would express solely the ability of a retrieval system to distinguish between wanted and unwanted items-that is, it would be a measure of “effec­ tiveness” only, leaving for separate consideration factors related to cost or “effi­ ciency.” Second, the desired measure would not be confounded by the relative willingness of the system to emit items-it would express discrimination power independent of any “acceptance criterion” employed, whether the criterion is characteristic of the system or adjusted by the user. Third, the measure would be a single number-in preference, for example, to a pair of numbers which may covary in a loosely specified way, or a curve representing a table of several pairs of numbers-so that it could be transmitted simply and apprehended imme­ diately. Fourth, and finally, the measure would allow complete ordering of differ­ ent performances, indicate the amount of difference separating any two perfor­ mances, and assess the performance of any one system in absolute terms-that is, the metric would be a scale with a unit, a true zero, and a maximum value. Given a measure with these properties, we could be confident of having a pure and valid index of how well a retrieval system (or method) was performing the function it was primarily designed to accomplish, and we could reasonably ask questions of the form, “Shall we pay X dollars for Y units of effectiveness?”