Connectionists: Call-for-Participation: TalentCLEF Shared Task (CLEF) - Skill and Job Title Intelligence for Human Capital Management

[Luis Gasco]

(* apologies for cross-posting *)

Call for Participation TalentCLEF Shared Task (CLEF 2025)

Skill and Job Title Intelligence for Human Capital Management


https://talentclef.github.io/talentclef/

TalentCLEF is an initiative to advance Natural Language Processing (NLP) in
Human Capital Management (HCM). It aims to create a public benchmark for
model evaluation and promote collaboration to develop fair, multilingual,
and flexible systems that improve Human Resources (HR) practices across
different industries.

Key information:

   -

   Web: https://talentclef.github.io/talentclef/
   -

   Data: https://doi.org/10.5281/zenodo.14002665
   -

   Registration: https://clef2025-labs-registration.dei.unipd.it/

Motivation
In today’s rapidly changing socio-technological landscape, industries and
workplaces are transforming quickly. Technological advancements, such as
task automation and Artificial Intelligence (AI), are reshaping the labor
market by creating new roles that demand specialized skills, often
difficult to source. The rise of remote hiring, fueled by technological
innovation, has expanded the labor market to a global and multilingual
scale. Simultaneously, social progress is narrowing ethnic and gender
disparities within companies, fostering more inclusive workplaces.

Integrating Natural Language Processing (NLP) into Human Capital Management
(HCM) enhances key areas such as sourcing and hiring, onboarding and
training, strategic workforce planning, and career development. Despite
these benefits, challenges persist in managing multilingual information,
ensuring fair AI models, and developing systems  flexible enough to work
across industries.

The TalentCLEF organizers expect that participation in the shared task will
contribute to establishing a public benchmark for multilingual job title
matching and skill prediction, enabling the evaluation and comparison of
different approaches. This initiative will also provide a foundation for
measuring gender bias in job-related NLP tasks and lay the groundwork for
future benchmarks in other areas of Human Capital Management, fostering
fairness, transparency, and adaptability in AI-driven workforce analysis.

The inaugural TalentCLEF shared-task aims to tackle these challenges
through two key tasks:

   -

   Task A - Multilingual Job Title Matching: Participants will develop
   systems to identify and rank job titles most similar to a given one. For
   each job title in a test set, systems must generate a ranked list of
   similar titles from a predefined knowledge base. Evaluation will be
   conducted in English, Spanish, German, and Chinese, covering both
   monolingual and cross-lingual (between English and the other languages)
   matching.
   -

   Task B - Job Title-Based Skill Prediction: This task focuses on
   retrieving relevant skills associated with a given job title. Participants
   will develop systems that predict and extract key skills based on job
   titles. The evaluation will be conducted in English.

Schedule

   -

   20th January 2025 - Training data available for Tasks A and B
   -

   17th February 2025 – Start of Task A with the release of the development
   data
   -

   17th March 2025 – Start of Task B with the release of the development
   data
   -

   21st April 2025 – Test set release
   -

   21st April - 5th May 2025 – Evaluation period of Task A and B
   -

   7th May 2025 – Publication of Official Results
   -

   30th May 2025 – Submission of CLEF 2025 Participant Working Notes
   (CEUR-WS)
   -

   27th June 2025 - Notification of Acceptance for Participant Papers


Publications and CLEF 2025 workshop
Teams participating in TalentCLEF will be invited to submit a system
description paper for the CLEF 2025 Working Notes proceedings, published on
CEUR-WS. Additionally, they will have the opportunity to present a brief
overview of their approach at the CLEF 2025 workshop, which will take place
in Madrid, Spain, from September 9th to 12th, 2025.


Main Organizers

   -

   Luis Gascó, Avature, Spain
   -

   Hermenegildo Fabregat, Avature, Spain
   -

   Laura García-Sardiña, Avature, Spain
   -

   Daniel Deniz Cerpa, Avature, Spain
   -

   Paula Estrella, Avature, Spain
   -

   Álvaro Rodrigo, Universidad Nacional de Educación a Distancia (UNED),
   Spain
   -

   Rabih Zbib, Avature, Spain


Scientific Committee

   -

   Eneko Agirre - Full Professor of the University of the Basque Country
   UPV/EHU - ACL Fellow
   -

   David Camacho - Full Professor of the Technical University of Madrid
   (UPM)
   -

   Debora Nozza - Assistant Professor of Bocconi University
   -

   Jens-Joris Decorte - Lead AI Scientist at TechWolf
   -

   David Graus - Lead Data Scientist at Randstad Group
   -

   Mesutt Kayaa - Postdoctoral Researcher at Jobindex A/S and IT University
   Copenhagen
   -

   Jan Luts - Senior Data Scientist at NTT Data & ESCO
   -

   Elena Montiel-Ponsoda - Professor at the Technical University of Madrid
   (UPM) - AI4Labour project
   -

   Javier Huertas Tato - Assistant Professor of the Technical University of
   Madrid (UPM)
   -

   Patricia Martín Chozas - Postdoctoral Researcher at the Ontology
   Engineering Group (UPM) - AI4Labour project
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/connectionists/attachments/20250218/4e3dce00/attachment.html>