Home

Awesome

OSIRRC Docker Image for Galago

Build Status

Michael Zarozinski

This is the docker image for Galago v3.15 conforming to the OSIRRC jig for the Open-Source IR Replicability Challenge (OSIRRC) at SIGIR 2019. This image is available on Docker Hub.

This image is based on the Indri image by Claudia Hauff (https://github.com/osirrc/indri-docker)

Quick Start

First, clone the jig and follow its setup instructions - in particular, do not forget to also clone and install trec_eval inside the jig directory.

The following jig command can be used to index TREC disks 4/5 for robust04:

python run.py prepare \
  --repo osirrc2019/galago \
  --tag v0.0.2 \
  --collections robust04=/path/to/disk45=trectext

The following jig command can be used to perform a retrieval run on the collection with the robust04 test collection.

python run.py search \
  --repo osirrc2019/galago \
  --tag v0.0.2 \
  --output ./output \
  --top_k 1000 \
  --qrels qrels/qrels.robust04.txt \
  --topic topics/topics.robust04.txt \
  --collection robust04 \
  --opts out_file_name="output_robust04.txt" 

Passing Parameters to Galago

Since both the jig and Galago specify parameters using a double dash, we can't use the double dash when specifying a Galago parameter because the jig will think it's a jig parameter. We also can't use an equal sign when specifying a Galago parameter value due to the way the jig parses key/value pairs. So, to specify a Galago parameter we use the search_options field of the --opts parameter. For example, if you wanted to change the default value of mu when running Galago outside of the jig you would specify --mu=1000. Using the jig, you would specify --opts search_options="mu:1000".

Retrieval Methods

The Galago image supports the following retrieval method:

Expected Results

robust04

Galago Parameter(s)MAPP@10P@30NDCG@10NDCG@30
QL0.19480.38880.26590.40380.3573
QL (--opts search_options="mu:1000")0.19620.39160.26630.40770.3582
BM25 (--opts search_options="scorer:bm25")0.18250.37990.25810.38890.3439
SDM (--opts search_options="operatorWrap:sdm")0.19840.38880.22510.39970.3610