From cbb875c479697e3d1b1e91e0fc444f2f61e37c1a Mon Sep 17 00:00:00 2001 From: Luke Zappia Date: Tue, 8 Oct 2024 05:15:49 +0200 Subject: [PATCH] Add foundation models page (#354) * Add foundation models page * Update CHANGELOG * Update CHANGELOG.md --------- Co-authored-by: Robrecht Cannoodt --- CHANGELOG.md | 8 +- results/foundation_models/index.qmd | 25 + results/foundation_models/thumbnail.svg | 3034 +++++++++++++++++++++++ 3 files changed, 3064 insertions(+), 3 deletions(-) create mode 100644 results/foundation_models/index.qmd create mode 100644 results/foundation_models/thumbnail.svg diff --git a/CHANGELOG.md b/CHANGELOG.md index 89c673ba..2273ca3c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,8 @@ * Update base images documentation (PR #342). +* Add foundation models benchmark page (PR #354) + ## BUG FIXES * Fix error in summary figure when not all method info ids are present in the results (PR #330). @@ -34,12 +36,12 @@ ## NEW FUNCTIONALITY -* Add gloassary function to pages and a glossary page (PR #300). +* Add glossary function to pages and a glossary page (PR #300). ## MAJOR CHANGES * Update `fundamentals` pages (PR #300). - + * Refactor the reference view (PR #300). * Update `reference` documentation (PR #300). @@ -85,7 +87,7 @@ * Update neurips 2021 datasets data with new dataset_id (PR #315). * Update to funkyheatmapjs v0.2.5 (PR #316). - + * Add CSS to fix not active tab color in dark mode (PR #319). * Improvements to the documentation (PR #317). diff --git a/results/foundation_models/index.qmd b/results/foundation_models/index.qmd new file mode 100644 index 00000000..2c962e8c --- /dev/null +++ b/results/foundation_models/index.qmd @@ -0,0 +1,25 @@ +--- +title: "Foundation models" +subtitle: "Modelling of single-cells to perform multiple tasks." +image: thumbnail.svg +page-layout: full +css: ../_include/task_template.css +engine: knitr +fig-cap-location: bottom +citation-location: document +toc: false +--- + +Recent developments in deep-learning have led to the creation of several "foundation models" for single-cell data. +These are large models that have been trained on data from millions of cells and am to fully capture the variability in the single-cell landscape. +Typically, they use a transformer architecture and undergo self-supervised pre-training using masking of parts of the input data. +Trained foundation models can then be applied to a variety of downstream tasks, either by directly feeding new data into the model or by fine-tuning to better fit a new dataset or to produce a specific output. +The general nature of single-cell foundation models and the large amount of data they have been trained on makes them potentially powerful tools for single-cell analysis but their performance is yet to be fully established. + +Open Problems plans to build on the existing evaluation of foundation models by incorportating them into our continuous benchmarking framework. +Models will be included in existing tasks where they will be compared to more traditional methods. +We also plan to compare foundation models across tasks to evaluate their flexibility and compare them to each other. + +If you are interested in evaluating foundation models for single-cell data please fill in the form below to get in touch. + + diff --git a/results/foundation_models/thumbnail.svg b/results/foundation_models/thumbnail.svg new file mode 100644 index 00000000..a3f9b62e --- /dev/null +++ b/results/foundation_models/thumbnail.svg @@ -0,0 +1,3034 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + dim-2 + dim-1 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + dim-2 + dim-1 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + A + B + C + D + E + F + G + H + 2 + 1 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + 10 + 11 + 12 + + + + + Foundationmodel + + + + +