Please use this identifier to cite or link to this item:
http://hdl.handle.net/1942/763
Title: | A universal method of information retrieval evaluation: the "missing" link M and the universal IR surface | Authors: | EGGHE, Leo | Issue Date: | 2004 | Source: | Information Processing & Management, 40(1). p. 21-30 | Abstract: | The paper shows that the present evaluation methods in information retrieval (basically recall R and precision P and in some cases fallout F) lack universal comparability in the sense that their values depend on the generality of the IR problem. A solution is given by using all "parts" of the database, including the non-relevant documents and also the notretrieved documents. It turns out that the solution is given by introducing the measure M being the fraction of the not-retrieved documents that are relevant (hence the "miss" measure). We prove that - independent of the IR problem or of the IR action - the quadruple (P,R,F,M) belongs to a universal IR surface, being the same for &l IR-activities. This universality is then exploited by defining a new measure for evaluation in IR allowing for unbiased comparisons of all IR results. We also show that only using 1, 2 or even 3 measures from the set {P,R,F,M} necessary leads to evaluation measures that are nonuniversal and hence not capable of comparing different IR situations. | Keywords: | Universal IR surface; miss measure; precision; recall; fallout; silence; evaluation | Document URI: | http://hdl.handle.net/1942/763 | ISSN: | 0306-4573 | e-ISSN: | 1873-5371 | DOI: | 10.1016/S0306-4573(02)00094-8 | ISI #: | 000187738300002 | Category: | A1 | Type: | Journal Contribution | Validations: | ecoom 2005 |
Appears in Collections: | Research publications |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
universal.pdf | Non Peer-reviewed author version | 357.29 kB | Adobe PDF | View/Open |
universal 1.pdf Restricted Access | Published version | 256.97 kB | Adobe PDF | View/Open Request a copy |
SCOPUSTM
Citations
10
checked on Sep 2, 2020
WEB OF SCIENCETM
Citations
9
checked on Jul 18, 2024
Page view(s)
54
checked on Sep 6, 2022
Download(s)
204
checked on Sep 6, 2022
Google ScholarTM
Check
Altmetric
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.