Intelligent Disease Progression Prediction at CLEF - iDPP@CLEF 2022

 iDPP CLEF Workshop – 5th September 2022

Agenda

Session 1, 15:20-16:50 (90 mins)

Welcome and Introduction (5 mins)

Overview of the Lab (30 mins, including questions)

Overview of iDPP@CLEF 2022: The Intelligent Disease Progression Prediction Challenge
Alessandro Guazzo, Isotta Trescato, Enrico Longato, Enidia Hazizaj, Dennis Dosso, Guglielmo Faggioli, Giorgio Maria Di Nunzio, Gianmaria Silvello, Martina Vettoretti, Erica Tavazzi, Chiara Roversi, Pietro Fariselli, Sara C. Madeira, Mamede de Carvalho, Marta Gromicho, Adriano Chiò, Umberto Manera, Arianna Dagliati, Giovanni Birolo, Helena Aidos, Barbara Di Camillo, Nicola Ferro

Paper

h

Slides

Repository

m

Position Papers (15 mins each, including questions)

Explaining Artificial Intelligence Predictions of Disease Progression with Semantic Similarity
Susana Nunes, Rita T. Sousa, Filipa Serrano, Ruben Branco, Diogo F. Soares, Andreia S. Martins, Eleonora Auletta, Eduardo N. Castanho, Sara C. Madeira, Helena Aidos, Catia Pesquita

Paper

h

Slides

Evaluation of XAI on ALS 6-months mortality prediction
Tommaso Mario Buonocore, Giovanna Nicora, Arianna Dagliati, Enea Parimbelli

Paper

h

Slides

Participant Presentations (25 mins, including questions)

Multi-Event Survival Prediction for Amyotrophic Lateral Sclerosis
Corrado Pancotti, Giovanni Birolo, Tiziana Sanavia, Cesare Rollo, Piero Fariselli

Paper

h

Slides

Repository

m

Break, 16:50-17:10

Session 2, 17:10-18:40 (90 mins)

Participant Presentations (25 mins, including questions)

Hierarchical Modelling for ALS Prognosis: Predicting the Progression Towards Critical Events
Ruben Branco, Diogo F. Soares, Andreia S. Martins, Eleonora Auletta, Eduardo N. Castanho, Susana Nunes, Filipa Serrano, Rita T. Sousa, Catia Pesquita, Sara C. Madeira, Helena Aidos

Paper

h

Slides

Repository

m
Baseline Machine Learning Approaches To Predict Amyotrophic Lateral Sclerosis Disease Progression
Isotta Trescato, Alessandro Guazzo, Enrico Longato, Enidia Hazizaj, Chiara Roversi, Erica Tavazzi, Martina Vettoretti, Barbara Di Camillo

Paper

h

Slides

Repository

m
Predicting the Risk of & Time to Impairment for ALS patients
Aidan Mannion, Thierry Chevalier, Didier Schwab, Lorraine Goeuriot

Paper

h

Slides

Repository

m

Discussion and planning for next year (15 mins)

Motivation

Amyotrophic Lateral Sclerosis (ALS) and Multiple Sclerosis (MS) are chronic diseases characterized by progressive or alternate impairment of neurological functions (motor, sensory, visual, cognitive). Patients have to manage alternated periods in hospital with care at home, experiencing a constant uncertainty regarding the timing of the disease acute phases and facing a considerable psychological and economic burden that also involves their caregivers. Clinicians, on the other hand, need tools able to support them in all the phases of the patient treatment, suggest personalized therapeutic decisions, indicate urgently needed interventions.

Important dates

  • Registration closes: April 22, 2022
  • Runs submission deadline: May 6, 2022
  • Evaluation results out: May 20, 2022
  • Participant and position paper submission deadline: May 27, 2022
  • Notification of acceptance for participant and position papers: June 13, 2022
  • Camera-ready participant papers submission: July 1, 2022
  • iDPP CLEF Workshop: September 5-8, 2022 during the CLEF Conference

The goal of iDPP@CLEF is to design and develop an evaluation infrastructure for AI algorithms able to:

Better describe disease mechanisms.

Stratify patients according to their phenotype assessed all over the disease evolution.

Predict disease progression in a probabilistic, time dependent fashion.

Tasks

Overall iDPP is targeting two kinds of activities: (a) preliminary and exploratory pilot tasks on disease progression prediction; (b) position papers on the explainability of the prediction algorithms. Overall, this mix will provide participants with the opportunity to make some hands-on experience with these data and provide feedback about the task design as well as to brainstorm on how to evaluate this kind of algorithms and, in particular, assess their explainability. For this initial iteration we focus on ALS progression prediction; future cycles will be extended to MS as well.

In particular, we offer the following activities:

Pilot Task 1 –

Ranking Risk of Impairment

It focuses on ranking of patients based on the risk of impairment in specific domains. More in detail, we will use the ALSFRS-R scale to monitor speech, swallowing, handwriting, dressing/hygiene, walking and respiratory ability in time and will ask participants to rank patients based on time to event risk of experiencing impairment in each specific domain.

Pilot Task 2 –

Predicting Time of Impairment

It refines Task 1 asking participants to predict when specific impairments will occur (i.e. in the correct time-window). In this regard, we assess model calibration in terms of the ability of the proposed algorithms to estimate a probability of an event close to the true probability within a specified time-window.

Position Papers Task 3 –

Explainability of AI algorithms

We call for proposals of different visualization frameworks able to show the multivariate nature of the data and the model predictions in an explainable, possibly interactive, way.

Information on the Datasets

The iDPP@CLEF 2022 challenge will share three valuable datasets which are used in the tasks above described for both training and testing algorithms to predict the progression of ALS and/or to showcase approaches for the explainability of such algorithms.

These datasets come from two clinical institutions, one in Lisbon (Portugal), and the other in Turin (Italy) and contain data about real patients, fully anonymized. These datasets are highly curated. They are produced from an ontology developed by the BRAINTEASER project which ensures the consistency of the data represented. Moreover, several checks have been performed to ensure that all the instances are clean, contain proper values in the expected ranges, and do not have contradictions.

Dataset A: is intended for the prediction of NIV – Non-invasive ventilation (or the competing event Death) and consists of 1,804 patients and 6,002 visits (ALSFRS-R questionnaires, Spirometry, etc.).
Dataset B: is intended for the prediction of PEG – Percutaneous Endoscopic Gastrostomy (or the competing event Death) and consists of 2,145 patients and 7,180 visits (ALSFRS-R questionnaires, Spirometry, etc.).
Dataset C: is intended for the prediction of Death and consists of 2,250 patients and 7,536 visits (ALSFRS-R questionnaires, Spirometry, etc.).

All the datasets are split into a training and test set according to a (approximately) 80%-20% ratio.

Register Here
Find out more

Organizers

  • Adriano Chiò, University of Turin, Italy
  • Arianna Dagliati, University of Pavia, Italy
  • Barbara Di Camillo, University of Padua, Italy
  • Mamede Alves de Carvalho, University of Lisbon, Portugal
  • Nicola Ferro, University of Padua, Italy
  • Piero Fariselli, University of Turin, Italy
  • Sara C. Madeira, University of Lisbon, Portugal
Back to Open evaluation challenges
Skip to content