Application of aboutness to functional benchmarking in information retrieval.
Wong, Kam-Fai; Song, Dawei; Bruza, Peter; Cheng, Chun-Hung
Experimental approaches are widely employed to benchmark the performance of an information retrieval (IR) system. Measurements in terms of recall and precision are computed as performance indicators. Although they are good at assessing the retrieval effectiveness of an IR system, they fail to explore deeper aspects such as its underlying functionality and explain why the system shows such performance. Recently, inductive (i.e., theoretical) evaluation of IR systems has been proposed to circumvent the controversies of the experimental methods. Several studies have adopted the inductive approach, but they mostly focus on theoretical modeling of IR properties by using some metalogic. In this article, we propose to use inductive evaluation for functional benchmarking of IR models as a complement of the traditional experiment-based performance benchmarking. We define a functional benchmark suite in two stages: the evaluation criteria based on the notion of "aboutness," and the formal evaluation methodology using the criteria. The proposed benchmark has been successfully applied to evaluate various well-known classical and logic-based IR models. The functional benchmarking results allow us to compare and analyze the functionality of the different IR models.
WONG, K.-F., SONG, D., BRUZA, P. and CHENG, C.-H. 2001. Application of aboutness to functional benchmarking in information retrieval. ACM transactions on information systems, 19(4), pages 337-370. Available from: https://doi.org/10.1145/502795.502796
|Journal Article Type||Article|
|Acceptance Date||Oct 31, 2001|
|Online Publication Date||Oct 31, 2001|
|Publication Date||Oct 31, 2001|
|Deposit Date||Aug 20, 2009|
|Publicly Available Date||Aug 20, 2009|
|Journal||ACM transactions on information systems|
|Publisher||ACM Association for Computing Machinery|
|Peer Reviewed||Peer Reviewed|
|Keywords||Functional benchmarking; Aboutness; Logic based information retrieval; Inductive evaluation|
WONG 2001 Application of aboutness to functional
Publisher Licence URL
You might also like
Predicting emotional reaction in social networks.
Early fusion and query modification in their dual late fusion forms.
You have e-mail, what happens next? Tracking the eyes for genre.