Details¶

Registration¶

• Please note that this challenge is now closed.
• Participants need to register a team on the challenge website.
• Anonymous registration is not allowed. Real name, affiliation and institutional email address should be provided.
• To finalize the registration and to get access to the test data, a signed data use agreement should be emailed to knoap2020@gmail.com
• Participants from the institution organizing this challenge (Erasmus MC) can participate in this challenge. Their results will be evaluated but will not be included in the final ranking.

Data¶

• Image data, clinical variables, and ground truth labels of 30 knees from the same study than the test set will also be released at the same time than the test set. This data can be used to optimize the algorithms.
• The images (X-ray and MR images) will be provided via XNAT and are in nifti-format.
• In addition to the provided data, methods can be trained and tuned on any suitable data (for example OAI data).
• The provided data may only be used for preparing an entry to be submitted to this challenge.

Submissions¶

• Classification outcomes (probabilities) on the test data set (Table 1) are submitted via the challenge web site. You should use the submission template when submitting the predictions. Please do not alter the format of the template, because we use automatic evaluation.
• Submission of the prediction for each sample is obligatory.
• The organizers will write a paper on the results on the test set, with active participants that provide sufficient information as co-authors.
• Each individual KNOAP2020 challenge participant may contribute to a maximum of five submissions. This is to minimize statistical problems arising from multiple comparisons. To avoid gaming the system by dividing a group of researchers working with the same method into separate submission groups, separate groups submitting the same methods may be asked to combine if the methods are sufficiently similar. We do not intend to force distinct research teams genuinely working on similar methods to merge.
• Each submission must include a maximum four-page (A4, 1cm margins, 11pt Times New Roman font) written description of the methods used. A template can be downloaded here. The following items should be detailed:
  -    Training data set: Describe what training data was used and the results on the training/validation data (ROC AUC, BACC). We recommend to include also the results on the OAI leaderboard test data.
  -    Image data: Which imaging modalities/sequences were used?
  -    Features: Which and how many features were used? Were the features selected automatically or manually?
  -    Missing data: How the method deals with missing data?
  -    Processing time: How much time does it on average take to make a prediction for one knee? What are computation times and requirements for training?
  -    Description of the methods: Provide a detailed full description of the methods.

Table 1. An example of the format of predictions.