Connectionists: Labs CLEF 2025: Call for Participation

[Damiano Spina]

RMIT Classification: Trusted


CLEF 2025
Conference and Labs of the Evaluation Forum
https://clef2025.clef-initiative.eu/
9-12 September 2025, Madrid, Spain

*** CALL FOR PARTICIPATION ***

CLEF 2025 is the 16th CLEF conference continuing the popular CLEF campaigns which have run since 2000 contributing to the systematic evaluation of information access systems, primarily through experimentation on shared tasks.
Building on the format first introduced in 2010, CLEF 2025 consists of an independent peer-reviewed conference on a broad range of issues in the fields of multilingual and multimodal information access evaluation, and a set of labs and workshops designed to test different aspects of mono and cross-language Information retrieval systems. Together, the conference and the lab series will maintain and expand upon the CLEF tradition of community-based evaluation and discussion on evaluation issues.

*** LABS REGISTRATION ***
Registration for participation in labs is now open and available at
https://clef2025.clef-initiative.eu/index.php?page=Pages/registration.html


*** 2025 TASKS ***
BioASQ: A Challenge in Large-scale Biomedical Semantic Indexing and Question Answering
https://www.bioasq.org/workshop2025
The aim of the BioASQ Lab is to push the research frontier towards systems that use the diverse and voluminous information available online to respond directly to the information needs of biomedical scientists.

CheckThat!: Lab on Subjectivity, Fact-Checking, Claim Extraction & Normalization, and Retrieval
https://checkthat.gitlab.io/clef2025/
The eighth edition of the CheckThat! lab at CLEF presents a diverse set of challenges aimed at advancing technology to support and enhance the journalistic verification process. This edition revisits core tasks in the verification pipeline while also introducing auxiliary tasks such as subjectivity identification, claim normalization, and fact-checking numerical claims, with a particular emphasis on scientific web discourse. These tasks pose complex classification and retrieval problems at both the document level, including in multilingual contexts.

ELOQUENT lab for evaluation of generative language model quality
https://eloquent-lab.github.io/
The ELOQUENT lab for evaluation of generative language model quality and usefulness addresses high-level quality criteria through a set of open-ended shared tasks implemented to require minimal human assessment effort.

eRisk: Early Risk Prediction on the Internet
https://erisk.irlab.org/
eRisk explores the evaluation methodology, effectiveness metrics and practical applications (particularly those related to health and safety) of early risk detection on the Internet.

EXIST: sEXism Identification in Social neTworks
https://nlp.uned.es/exist2025/
EXIST aims to capture and categorize sexism, from explicit misogyny to other subtle behaviors, in social networks. In 2024 the EXIST campaign included multimedia content in the format of memes, stepping forward research on more robust techniques to identify sexism in social networks. Following this line, in 2025 we will focus on TikTok videos in the challenge, thus including in the dataset the three most important sources of sexism spreading: text, images and videos. Consequently, it is essential to develop automated multimodal tools capable of detecting sexism in text, images, and videos, to raise alarms or automatically remove such content from social network because platforms’ algorithms often amplify content that perpetuates gender stereotypes and internalized misogyny. This lab will contribute to the creation of applications that identify sexist content in social media across all three formats.

ImageCLEF: Multimodal Challenge in CLEF
https://www.imageclef.org/
ImageCLEF 2025 focuses on evaluating technologies for annotating, indexing, classifying, retrieving and generating multimodal data, providing access to large datasets across a veriety of scenarios, including medical, social media, and internet-based applications. Building on the success of recent editions, it encourages interdisciplinary methods by engaging participants in diverse domains, providing large amounts of challenging multimodal data and providing am evaluation platform for a large number of use cases.

JOKER: Humour in the Machine
https://www.joker-project.com/clef-2024/
JOKER aims to foster interdisciplinary approaches to the (semi-)automatic analysis and processing of humour and wordplay.

LifeCLEF: Challenges on Species Presence Prediction and Identification, and Individual Animal Identification
https://www.imageclef.org/LifeCLEF
The LifeCLEF 2025 lab focuses on advancing AI-driven solutions for biodiversity monitoring through challenges on species and individuals recognition and prediction.

LongEval: Longitudinal Evaluation of Model Performance
https://clef-longeval.github.io/
The goal of this task is to ignite the development of Information Retrieval systems that can handle temporal data evolution. The retrieval systems evaluated in this task are expected to be persistent in their retrieval efficiency over time, as Web documents and Web queries evolve. To evaluate such features of systems, we rely on collections of documents and queries, corresponding to real data acquired from actual Web search engines.
PAN Lab on Stylometry and Digital Text Forensics
https://pan.webis.de/
PAN is a series of scientific events and shared tasks on digital text forensics and stylometry whose goal is to advance the state of the art and provide for an objective evaluation on newly developed benchmark datasets in those areas.

QuantumCLEF
https://qclef.dei.unipd.it/
The second edition of the QuantumCLEF lab is composed of three tasks and aims at: (i) Discovering and evaluating Quantum Annealing approaches compared to their traditional counterpart; (ii) Identifying new ways of formulating Information Retrieval and Recommender Systems algorithms and methods, so that they can be solved with Quantum Annealing; (iii) Establishing collaborations among researchers from different fields to harness their knowledge and skills to solve the considered challenges and promote the usage of Quantum Annealing. This lab allows participants to use real quantum computers provided by CINECA, one of the most important computing centers worldwide.

SimpleText: Simplify Scientific Text
https://simpletext-project.com/
The SimpleText track aims at improving accessibility to scientific information for everyone, developing corpora, evaluation measures, and new IR/NL models able to reduce scientific text complexity with strict faithfulness to the original text.

TalentCLEF: Skill and Job Title Intelligence for Human Capital Management
https://talentclef.github.io/talentclef/
TalentCLEF aims to drive technological advancement in Human Capital Management by establishing a public benchmark for NLP models that facilitates their application in real-world Human Resources (HR) scenarios, incorporating evaluation criteria incluiding multilingualism, fairness, and cross-industry adaptability. The lab also seeks to build a community for researchers and practitioners to generate, evaluate, and discuss ideas on the use of AI in Human Resources, pushing the state-of-the-art of NLP applications for Human Resources.

Touché: Argumentation Systems
https://touche.webis.de/
Touché is a series of scientific events and shared tasks on computational argumentation and causality.


*** IMPORTANT DATES ***
(may vary depending on the task)
18 November 2025: Registration opens
25 April 2025: Registration closes
10 May 2025: End of Evaluation Cycle [submission of runs]
30 May 2025: Submission of Participant Papers [CEUR-WS]
30 May–27 June 2025: Review process of participant papers
27 June 2025: Notification of Acceptance for Participant Papers [CEUR-WS]
7 July 2025: Camera Ready Participant Papers and Extended Lab Overviews [CEUR-WS] due
9-12 September 2025, CLEF 2025 Conference, Madrid, Spain

*** OVERALL COORDINATION ***
General Chairs
Laura Plaza, Universidad Nacional de Educación a Distancia (UNED), Spain
Jorge Carrillo de Albornoz, Universidad Nacional de Educación a Distancia (UNED), Spain
Julio Gonzalo, Universidad Nacional de Educación a Distancia (UNED), Spain
Alba García Seco de Herrera, Universidad Nacional de Educación a Distancia (UNED), Spain

Lab Chairs
Paolo Rosso, Universitat Politècnica de València, Spain
Damiano Spina, RMIT University, Australia

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20241216/c3062e5d/attachment.html>