Python Client

For programatic access to TIRA, we provide a python API, the “Python Client”, which serves two use-cases:

1. Interfacing with the TIRA platform

2. Integrating with third party APIs (e.g., ir-datasets) that would not work otherwise due to TIRA’s sandbox.

Important

TIRA sandboxes all the evaluation runs to avoid publicizing evaluation data and maintain comparability beyond the initial shared task. For you, this means that you can’t access arbitrary external resources. Since you should not, of course, have to abstain from using hugging-face, ìr_datasets or other crucial dependencies, we integrated some of these into the sandbox.

• TIRA Client
• Pyterrier Integration
• IR-Datasets Integration