====== L: 30/10/2020 ======

**Master in Informatics and Computing Engineering\\
Information Description, Storage and Retrieval\\
Instance: 2020/2021**
\\
---
\\

====== Lecture #6 :: 30/10/2020 ======


===== Goals =====

By the end of this class, the student should be able to:
  * describe a test collection: documents, topics, relevance judgements;
  * explain the need for specific evaluation methods in information retrieval;
  * define precision, recall and their variants;
  * interpret a precision vs recall graph;

===== Topics =====

  - Evaluation in Information Retrieval
    * Test collections
    * IR measures: precision, recall, P@n, F, MAP
    * Evaluation of ranked results

===== Materials =====

  * {{.:dapi2021-evaluation-ir.pdf|Evaluation in Information Retrieval}}

  * W. Bruce Croft, Donald Metzler, Trevor Strohman, //Search Engines: Information Retrieval in Practice//, [[http://www.search-engines-book.com/slides/|Slides for chapters 2, 7, 8]], accessed October 2019
  * Ricardo Baeza-Yates, Berthier Ribeiro-Neto, //Modern Information Retrieval: The Concepts and Technology behind Search//, [[http://grupoweb.upf.edu/mir2ed/slides.php|Slides for Chapter 3: Modeling, Chapter 4: Retrieval Evaluation]], accessed October 2020
  * Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, //Introduction to Information Retrieval//, [[http://nlp.stanford.edu/IR-book/html/htmledition/irbook.html|Chapter 6, 7, 8, 11, 12]], accessed October 2020
  * Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, //Introduction to Information Retrieval//, [[https://nlp.stanford.edu/IR-book/newslides.html|Slides for chapters 6, 7, 8, 11, 12]], accessed October 2020

===== Tasks =====

  * Experiment with full-text indexing tools.
  * Milestone #2: define information needs for the full-text search task.
  * Milestone #2: perform indexing experiments in the project collection.

===== Summary =====

  * Evaluation in Information Retrieval. Test collections. IR measures: precision, recall, P@n, F, MAP.
  * Evaluation of non-ranked results. Evaluation of ranked results. Relevance judgements.
  * Work on Milestone 2 in the course project (Information Retrieval).
  * Indexing experiments on the working collection.
  * Definition of retrieval features for the domain.


 --- //MCR, JCL, SSN//

[[05|« Previous]] | [[index|Index]] | [[07|Next »]]