iDPP CLEF Workshop – 21st September 2023
Agenda
Session 1, 11:30-13:00 EEST (90 mins)
Welcome and Introduction (5 mins)
Overview of the Lab (20 mins, including questions)
Participant Presentations (15 mins, including questions)
Lunch Break, 13:00-14:00 EEST
Session 2, 14:00-15:30 EEST (90 mins)
Participant Presentations (15 mins, including questions)
Discussion and closing (5 mins)
Motivation
Important dates
- Registration closes: April 28, 2023
- Runs submission deadline: May 10, 2023
- Evaluation results out: May 26, 2023
- Participant and position paper submission deadline: June 5, 2023
- Notification of acceptance for participant and position papers: June 23, 2023
- Camera-ready participant papers submission: July 7, 2023
- iDPP CLEF Workshop: September 18-21, 2023 during the CLEF Conference
The goal of iDPP@CLEF is to design and develop an evaluation infrastructure for AI algorithms able to:
Better describe disease mechanisms.
Stratify patients according to their phenotype assessed all over the disease evolution.
Predict disease progression in a probabilistic, time dependent fashion.
Tasks
Overall iDPP is targeting two kinds of activities: (a) tasks on disease progression prediction; (b) position papers on the impact of the exposure to pollutants on the prediction algorithms. Overall, this mix will provide participants with the opportunity to make some hands-on experience with these data and provide feedback about the task design as well as to brainstorm on how to evaluate this kind of algorithms and, in particular, assess their explainability.
In this iteration of iDPP@CLEF we will focus on MS for the prediction tasks and on ALS for the position papers task.
In particular, we offer the following activities:
Task 1 –
Predicting Risk of Disease Worsening (Multiple Sclerosis):
it will focus on ranking subjects based on the risk of worsening, setting the problem as a survival analysis task. More specifically the risk of worsening predicted
by the algorithm should reflect how early a patient experience the event “worsening”. Worsening is defined based on the Expanded Disability Status Scale (EDSS), accordingly to clinical standards. In particular we will consider two different definitions in subtasks 1a and 1b, respectively.
• Subtask 1a: the patient crosses the threshold EDSS ≥ 3 at least twice within one year interval;
• Subtask 1b: the second definition of worsening depends on the first recorded value accordingly to current clinical protocols. If Baseline EDSS < 1, worsening event occurs when an increase of EDSS by 1.5 points is first observed; if 1 ≤ Baseline EDSS < 5.5, worsening event occurs when an increase of EDSS by 1 point is first observed; if baseline EDSS ≥ 5.5, worsening event occurs when an increase of EDSS by 0.5 points is first observed.
In both cases the occurrence of the worsening event and the time of occurrence will be pre-computed by the challenge organizers
Task 2 –
Predicting Probability of Worsening (Multiple Sclerosis):
it will refine Task 1 asking participants to explicitly assign a probability of worsening at different time windows (e.g., between years 4 and 6, 6 and 8, 8 and 10 etc.). Worsening will be defined in two different ways in subtasks 2a and 2b as for Task 1.
Position Papers Task 3 –
Impact of Exposition to Pollutants (Amyotrophic Lateral Sclerosis):
we will evaluate proposals of different approaches to assess if exposure to different pollutants is a useful variable to predict time to Percutaneous Endoscopic Gastrostomy (PEG), Non-Invasive Ventilation (NIV) and death in ALS patients.
Information on the Datasets
We will provide retrospective, fully anonymized MS and ALS clinical data including demographic and clinical characteristics, coming from clinical institutions in Italy, Portugal, and Spain.
For Task 1 and Task 2 we will release a brand new dataset with MS data consisting of about 1,800 patients.
For Position Paper Task 3 we will re-use the ALS dataset developed in iDPP@CLEF 2022, consisting of about 2,250 patients, and will extend it with environmental and pollution data.
Organizers
- Helena Aidos, University of Lisbon, Portugal
- Roberto Bergamaschi, University of Pavia, Italy
- Paola Cavalla, “Città della Salute e della Scienza”, Turin, Italy
- Adriano Chio’, University of Turin, Italy
- Arianna Dagliati, University of Pavia, Italy
- Barbara Di Camillo, University of Padua, Italy
- Mamede Alves de Carvalho, University of Lisbon, Portugal
- Nicola Ferro, University of Padua, Italy
- Piero Fariselli, University of Turin, Italy
- Jose Manuel García Dominguez, Gregorio Marañon Hospital in Madrid, Spain
- Sara C. Madeira, University of Lisbon, Italy
- Eleonora Tavazzi, IRCCS Foundation C. Mondino in Pavia, Italy