Initial commit

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,24 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: [bug]
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1 @@
+blank_issues_enabled: true

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: [enhancement]
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,17 @@
+## Describe your changes
+
+<!-- Describe your changes  -->
+
+## Checklist before requesting a review
+- [ ] I have performed a self-review of my code
+
+- Check the correct box. Does this PR contain:
+  - [ ] Breaking changes
+  - [ ] New functionality
+  - [ ] Major changes
+  - [ ] Minor changes
+  - [ ] Bug fixes
+
+- [ ] Proposed changes are described in the CHANGELOG.md
+
+- [ ] CI Tests succeed and look good!

.github/workflows/build.yaml

@@ -0,0 +1,21 @@
+name: Build
+
+on:
+  push:
+    branches: [ 'main' ]
+  workflow_dispatch:
+    inputs:
+      version:
+        description: |
+          The version of the project to build. Example: `1.0.3`.
+          
+          If not provided, a development build with a version name
+          based on the branch name will be built. Otherwise, a release
+          build with the provided version will be built.
+        required: false
+
+jobs:
+  build:
+    uses: openproblems-bio/actions/.github/workflows/build.yml@main
+    with:
+      version: ${{ github.event.inputs.version }}

.github/workflows/test.yaml

@@ -0,0 +1,11 @@
+name: Test
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+
+jobs:
+  build:
+    uses: openproblems-bio/actions/.github/workflows/test.yml@main

.gitignore

@@ -0,0 +1,10 @@
+resources
+resources_test
+work
+.nextflow*
+target
+.vscode
+.DS_Store
+output
+trace-*
+.ipynb_checkpoints

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "common"]
+	path = common
+	url = https://github.com/openproblems-bio/common_resources.git

CHANGELOG.md

@@ -0,0 +1,24 @@
+# task_template x.y.z
+
+## BREAKING CHANGES
+
+<!-- * Restructured `src` directory (PR #3). -->
+
+## NEW FUNCTIONALITY
+
+* Added `control_methods/true_labels` component (PR #5).
+
+* Added `methods/logistic_regression` component (PR #5).
+
+* Added `metrics/accuracy` component (PR #5).
+
+## MAJOR CHANGES
+
+* Updated `api` files (PR #5).
+
+## MINOR CHANGES
+
+* Updated `README.md` (PR #5).
+
+## BUGFIXES
+

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Open Problems in Single-Cell Analysis
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,37 @@
+# Task Template
+
+This repo is a template to create a new task for the OpenProblems v2. This repo contains several example files and components that can be used when updated with the task info.
+
+> [!WARNING] 
+> This README will be overwritten when performing the `create_task_readme` script.
+
+## Create a repository from this template
+
+> [!IMPORTANT] 
+> Before creating a new repository, make sure you are part of the OpenProblems task team. This will be done when you create an issue for the task and you get the go ahead to create the task.
+> For more information on how to create a new task, check out the [Create a new task](https://openproblems.bio/documentation/create_task/) documentation.
+
+The instructions below will guide you through creating a new repository from this template ([creating-a-repository-from-a-template](https://docs.github.com/en/repositories/creating-and-managing-repositories/creating-a-repository-from-a-template#creating-a-repository-from-a-template)).
+
+
+* Click the "Use this template" button on the top right of the repository.
+* Use the Owner dropdown menu to select the `openproblems-bio` account.
+* Type a name for your repository (task_...), and a description.
+* Set the repository visibility to public.
+* Click "Create repository from template".
+
+## Clone the repository
+
+To clone the repository with the submodule files, you can use the following command:
+
+```bash
+git clone --recursive [email protected]:openproblems-bio/<repo_name>.git
+```
+>[!NOTE]
+> If somehow there are no files visible in the submodule after cloning using the above command. Check the instructions [here](common/README.md).
+
+## What to do next
+
+Check out the [instructions](https://github.com/openproblems-bio/common_resources/blob/main/INSTRUCTIONS.md) for more information on how to update the example files and components. These instructions also contain information on how to build out the task and basic commands.
+
+For more information on the OpenProblems v2, check out the [documentation](https://openproblems.bio/documentation/).

_viash.yaml

@@ -0,0 +1,64 @@
+viash_version: 0.9.0-RC6
+
+# Step 1: Change the name of the task.
+# example: task_name_of_this_task
+name: task_template
+organization: openproblems-bio
+version: dev
+# Step 2: Update the description to a short description of the task.
+description: |
+  An OpenProblems benchmark task.
+license: MIT
+# Step 3: Add keywords to describe the task.
+keywords: [single-cell, openproblems, benchmark]
+# Step 4: Update the `task_template` to the name of the task from step 1.
+links:
+  issue_tracker: https://github.com/openproblems-bio/task_template/issues
+  repository: https://github.com/openproblems-bio/task_template
+  docker_registry: ghcr.io
+# Step 5: Update the info fields to the text from the task issue. 
+info:
+  # A unique, human-readable, short label. Used for creating summary tables and visualisations.
+  label: Template
+  description: |
+    Provide a clear and concise description of your task, detailing the specific problem it aims
+    to solve. Outline the input data types, the expected output, and any assumptions or constraints.
+    Be sure to explain any terminology or concepts that are essential for understanding the task.
+  summary: A one sentence summary of purpose and methodology. Used for creating an overview tables.
+  motivation: |
+    Explain the motivation behind your proposed task. Describe the biological or computational 
+    problem you aim to address and why it’s important. Discuss the current state of research in
+    this area and any gaps or challenges that your task could help address. This section 
+    should convince readers of the significance and relevance of your task.
+  image: The name of the image file to use for the component on the website.
+  # Step 6: Replace the task_template to the name of the task in `info.name`.
+  test_resources:
+    - type: s3
+      path: s3://openproblems-data/resources_test/task_template/
+      dest: resources_test/task_template
+    - type: s3
+      path: s3://openproblems-data/resources_test/common/
+      dest: resources_test/common
+
+# Step 7: Update the authors of the task.
+authors: 
+  # Full name of the author, usually in the name of FirstName MiddleName LastName.
+  - name: Kai Waldrant 
+    # Role of the author. Possible values:
+    # 
+    # * `"author"`: Authors who have made substantial contributions to the component.
+    # * `"maintainer"`: The maintainer of the component.
+    # * `"contributor"`: Authors who have made smaller contributions (such as code patches etc.).
+    roles: [ "author", "maintainer" ]
+    # Additional information on the author
+    info: 
+      github: KaiWaldrant
+      orcid: 0009-0003-8555-1361
+      email: ...
+      twitter: ...
+      linkedin: ...
+# Step 8: Remove all of the comments of the steps you completed
+# Step 9: High five yourself!
+
+config_mods: |
+  .runners[.type == "nextflow"].config.labels := { lowmem : "memory = 20.Gb", midmem : "memory = 50.Gb", highmem : "memory = 100.Gb", lowcpu : "cpus = 5", midcpu : "cpus = 15", highcpu : "cpus = 30", lowtime : "time = 1.h", midtime : "time = 4.h", hightime : "time = 8.h", veryhightime : "time = 24.h" }

main.nf

@@ -0,0 +1,3 @@
+workflow {
+    print("This is a dummy placeholder for pipeline execution. Please use the corresponding nf files for running pipelines.")
+}

nextflow.config

@@ -0,0 +1 @@
+process.container = 'nextflow/bash:latest'

scripts/.gitignore

@@ -0,0 +1,3 @@
+add_a_method.sh
+add_a_control_method.sh
+add_a_metric.sh

scripts/create_readme.sh

@@ -0,0 +1,5 @@
+#!/bin/bash
+
+common/create_task_readme/create_task_readme \
+    --task_dir src \
+    --output README.md

scripts/create_test_resources.sh

@@ -0,0 +1,38 @@
+#!/bin/bash
+
+# get the root of the directory
+REPO_ROOT=$(git rev-parse --show-toplevel)
+
+# ensure that the command below is run from the root of the repository
+cd "$REPO_ROOT"
+
+set -e
+
+RAW_DATA=resources_test/common
+DATASET_DIR=resources_test/task_template
+
+mkdir -p $DATASET_DIR
+
+# process dataset
+echo Running process_dataset
+nextflow run . \
+  -main-script target/nextflow/workflows/process_datasets/main.nf \
+  -profile docker \
+  -entry auto \
+  --input_states "$RAW_DATA/**/state.yaml" \
+  --rename_keys 'input:output_dataset' \
+  --settings '{"output_train": "$id/train.h5ad", "output_test": "$id/test.h5ad"}' \
+  --publish_dir "$DATASET_DIR" \
+  --output_state '$id/state.yaml'
+
+# run one method
+viash run src/methods/logistic_regression/config.vsh.yaml -- \
+    --input_train $DATASET_DIR/pancreas/train.h5ad \
+    --input_test $DATASET_DIR/pancreas/test.h5ad \
+    --output $DATASET_DIR/pancreas/denoised.h5ad
+
+# run one metric
+viash run src/metrics/accuracy/config.vsh.yaml -- \
+    --input_predicition $DATASET_DIR/pancreas/predicted.h5ad \
+    --input_solution $DATASET_DIR/pancreas/solution.h5ad \
+    --output $DATASET_DIR/pancreas/score.h5ad

scripts/download_resources.sh

@@ -0,0 +1,9 @@
+#!/bin/bash
+
+set -e
+
+echo ">> Downloading resources"
+
+# the sync_resources script uses the test_resources S3 URI's in the _viash.yaml to download the resources.
+common/sync_resources/sync_resources \
+  --delete

scripts/run_benchmark.sh

@@ -0,0 +1,23 @@
+#!/bin/bash
+
+RUN_ID="run_$(date +%Y-%m-%d_%H-%M-%S)"
+publish_dir="s3://openproblems-data/resources/task_template/results/${RUN_ID}"
+
+# make sure only log_cp10k is used
+cat > /tmp/params.yaml << HERE
+input_states: s3://openproblems-data/resources/task_template/datasets/**/state.yaml
+rename_keys: 'input_train:output_train;input_test:output_test'
+output_state: "state.yaml"
+publish_dir: "$publish_dir"
+HERE
+
+tw launch https://github.com/openproblems-bio/task_template.git \
+  --revision build/main \
+  --pull-latest \
+  --main-script target/nextflow/workflows/run_benchmark/main.nf \
+  --workspace 53907369739130 \
+  --compute-env 6TeIFgV5OY4pJCk8I0bfOh \
+  --params-file /tmp/params.yaml \
+  --entry-name auto \
+  --config common/nextflow_helpers/labels_tw.config \
+  --labels task_template,full

scripts/run_benchmark_test.sh

@@ -0,0 +1,19 @@
+#!/bin/bash
+
+cat > /tmp/params.yaml << 'HERE'
+input_states: s3://openproblems-data/resources_test/task_template/**/state.yaml
+rename_keys: 'input_train:output_train;input_test:output_test'
+output_state: "state.yaml"
+publish_dir: s3://openproblems-nextflow/temp/task_template/
+HERE
+
+tw launch https://github.com/openproblems-bio/task_template.git \
+  --revision build/main \
+  --pull-latest \
+  --main-script target/nextflow/workflows/run_benchmark/main.nf \
+  --workspace 53907369739130 \
+  --compute-env 6TeIFgV5OY4pJCk8I0bfOh \
+  --params-file /tmp/params.yaml \
+  --entry-name auto \
+  --config common/nextflow_helpers/labels_tw.config \
+  --labels task_template,test

scripts/test_all_components.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+
+# Test all components in a namespace (refer https://viash.io/reference/cli/ns_test.html)
+viash ns test --parallel