From b4b664062d1acc9f55a54880086680e258da822d Mon Sep 17 00:00:00 2001 From: Charles Reid Date: Tue, 17 Dec 2019 11:13:41 -0800 Subject: [PATCH 01/13] move explore scanpy examples --- .../ENSG_to_name.clean.csv | 0 .../README.md | 0 .../f9e363cd-7fa5-4349-add2-1c9bd86d10c8.loom | Bin .../notebooks_hca_demo_scanpy.ipynb | 0 .../requirements.txt | 0 .../terra.sh | 0 .../ENSG_to_name.csv | 0 .../README.md | 4 ++-- .../hca_demo_scanpy.ipynb | 0 .../pancreas.loom | Bin .../requirements.txt | 0 11 files changed, 2 insertions(+), 2 deletions(-) rename {Explore an HCA Data Set in Scanpy (May 2019) => explore-hca-dataset-scanpy-may-2019}/ENSG_to_name.clean.csv (100%) rename {Explore an HCA Data Set in Scanpy (May 2019) => explore-hca-dataset-scanpy-may-2019}/README.md (100%) rename {Explore an HCA Data Set in Scanpy (May 2019) => explore-hca-dataset-scanpy-may-2019}/f9e363cd-7fa5-4349-add2-1c9bd86d10c8.loom (100%) rename {Explore an HCA Data Set in Scanpy (May 2019) => explore-hca-dataset-scanpy-may-2019}/notebooks_hca_demo_scanpy.ipynb (100%) rename {Explore an HCA Data Set in Scanpy (May 2019) => explore-hca-dataset-scanpy-may-2019}/requirements.txt (100%) rename {Explore an HCA Data Set in Scanpy (May 2019) => explore-hca-dataset-scanpy-may-2019}/terra.sh (100%) rename {Explore an HCA Data Set in Scanpy (Nov 2018) => explore-hca-dataset-scanpy-nov-2018}/ENSG_to_name.csv (100%) rename {Explore an HCA Data Set in Scanpy (Nov 2018) => explore-hca-dataset-scanpy-nov-2018}/README.md (96%) rename {Explore an HCA Data Set in Scanpy (Nov 2018) => explore-hca-dataset-scanpy-nov-2018}/hca_demo_scanpy.ipynb (100%) rename {Explore an HCA Data Set in Scanpy (Nov 2018) => explore-hca-dataset-scanpy-nov-2018}/pancreas.loom (100%) rename {Explore an HCA Data Set in Scanpy (Nov 2018) => explore-hca-dataset-scanpy-nov-2018}/requirements.txt (100%) diff --git a/Explore an HCA Data Set in Scanpy (May 2019)/ENSG_to_name.clean.csv b/explore-hca-dataset-scanpy-may-2019/ENSG_to_name.clean.csv similarity index 100% rename from Explore an HCA Data Set in Scanpy (May 2019)/ENSG_to_name.clean.csv rename to explore-hca-dataset-scanpy-may-2019/ENSG_to_name.clean.csv diff --git a/Explore an HCA Data Set in Scanpy (May 2019)/README.md b/explore-hca-dataset-scanpy-may-2019/README.md similarity index 100% rename from Explore an HCA Data Set in Scanpy (May 2019)/README.md rename to explore-hca-dataset-scanpy-may-2019/README.md diff --git a/Explore an HCA Data Set in Scanpy (May 2019)/f9e363cd-7fa5-4349-add2-1c9bd86d10c8.loom b/explore-hca-dataset-scanpy-may-2019/f9e363cd-7fa5-4349-add2-1c9bd86d10c8.loom similarity index 100% rename from Explore an HCA Data Set in Scanpy (May 2019)/f9e363cd-7fa5-4349-add2-1c9bd86d10c8.loom rename to explore-hca-dataset-scanpy-may-2019/f9e363cd-7fa5-4349-add2-1c9bd86d10c8.loom diff --git a/Explore an HCA Data Set in Scanpy (May 2019)/notebooks_hca_demo_scanpy.ipynb b/explore-hca-dataset-scanpy-may-2019/notebooks_hca_demo_scanpy.ipynb similarity index 100% rename from Explore an HCA Data Set in Scanpy (May 2019)/notebooks_hca_demo_scanpy.ipynb rename to explore-hca-dataset-scanpy-may-2019/notebooks_hca_demo_scanpy.ipynb diff --git a/Explore an HCA Data Set in Scanpy (May 2019)/requirements.txt b/explore-hca-dataset-scanpy-may-2019/requirements.txt similarity index 100% rename from Explore an HCA Data Set in Scanpy (May 2019)/requirements.txt rename to explore-hca-dataset-scanpy-may-2019/requirements.txt diff --git a/Explore an HCA Data Set in Scanpy (May 2019)/terra.sh b/explore-hca-dataset-scanpy-may-2019/terra.sh similarity index 100% rename from Explore an HCA Data Set in Scanpy (May 2019)/terra.sh rename to explore-hca-dataset-scanpy-may-2019/terra.sh diff --git a/Explore an HCA Data Set in Scanpy (Nov 2018)/ENSG_to_name.csv b/explore-hca-dataset-scanpy-nov-2018/ENSG_to_name.csv similarity index 100% rename from Explore an HCA Data Set in Scanpy (Nov 2018)/ENSG_to_name.csv rename to explore-hca-dataset-scanpy-nov-2018/ENSG_to_name.csv diff --git a/Explore an HCA Data Set in Scanpy (Nov 2018)/README.md b/explore-hca-dataset-scanpy-nov-2018/README.md similarity index 96% rename from Explore an HCA Data Set in Scanpy (Nov 2018)/README.md rename to explore-hca-dataset-scanpy-nov-2018/README.md index 4c9029b..e7de869 100644 --- a/Explore an HCA Data Set in Scanpy (Nov 2018)/README.md +++ b/explore-hca-dataset-scanpy-nov-2018/README.md @@ -1,6 +1,6 @@ # HCA Scanpy Demo -The vignette in this folder demonstrates the ability to load an expression matrix downloaded +The vignette in this folder demonstrates the ability to load an expression matrix downloaded from the HCA browser into scanpy and briefly explore the data. The downloaded file, `pancreas.loom`, is the result of end-to-end processing in the Data Coordination Platform workflow, and is included in this repo. The file `ENSG_to_name.csv` contains the Gencode identifiers for the genes in the HCA expression matrices, along with the more commonly used gene symbol (i.e. `ENSG00000115263.14,GCG`). This might be generally useful in your exploration of HCA expression matrices. @@ -8,7 +8,7 @@ The file `ENSG_to_name.csv` contains the Gencode identifiers for the genes in th ## Installation -This notebook assumes that you have installed python 3 on your system, +This notebook assumes that you have installed python 3 on your system, and that the `pip` executable installs packages for that python distribution. For some users, it may be necessary to use `pip3` instead. diff --git a/Explore an HCA Data Set in Scanpy (Nov 2018)/hca_demo_scanpy.ipynb b/explore-hca-dataset-scanpy-nov-2018/hca_demo_scanpy.ipynb similarity index 100% rename from Explore an HCA Data Set in Scanpy (Nov 2018)/hca_demo_scanpy.ipynb rename to explore-hca-dataset-scanpy-nov-2018/hca_demo_scanpy.ipynb diff --git a/Explore an HCA Data Set in Scanpy (Nov 2018)/pancreas.loom b/explore-hca-dataset-scanpy-nov-2018/pancreas.loom similarity index 100% rename from Explore an HCA Data Set in Scanpy (Nov 2018)/pancreas.loom rename to explore-hca-dataset-scanpy-nov-2018/pancreas.loom diff --git a/Explore an HCA Data Set in Scanpy (Nov 2018)/requirements.txt b/explore-hca-dataset-scanpy-nov-2018/requirements.txt similarity index 100% rename from Explore an HCA Data Set in Scanpy (Nov 2018)/requirements.txt rename to explore-hca-dataset-scanpy-nov-2018/requirements.txt From 11baab1d4a0ff7b23c0447853c3a4f084338f272 Mon Sep 17 00:00:00 2001 From: Charles Reid Date: Tue, 17 Dec 2019 11:16:09 -0800 Subject: [PATCH 02/13] move spark exapmle --- {SPARK Example => spark-example}/README.md | 2 +- {SPARK Example => spark-example}/img/emr1.png | Bin {SPARK Example => spark-example}/img/emr2.png | Bin {SPARK Example => spark-example}/img/emr3.png | Bin {SPARK Example => spark-example}/img/emr4.png | Bin {SPARK Example => spark-example}/img/launch_job.png | Bin {SPARK Example => spark-example}/pom.xml | 0 .../src/main/java/kmer/DataStoreClient.java | 0 .../src/main/java/kmer/HttpUtil.java | 0 .../src/main/java/kmer/Kmer.java | 0 .../src/main/java/kmer/Retriable.java | 1 - .../src/main/java/kmer/SparkUtil.java | 8 ++++---- .../src/test/java/kmer/DataStoreClientTest.java | 1 - .../src/test/java/kmer/HttpUtilTest.java | 0 .../src/test/java/kmer/RetriableTest.java | 2 -- 15 files changed, 5 insertions(+), 9 deletions(-) rename {SPARK Example => spark-example}/README.md (99%) rename {SPARK Example => spark-example}/img/emr1.png (100%) rename {SPARK Example => spark-example}/img/emr2.png (100%) rename {SPARK Example => spark-example}/img/emr3.png (100%) rename {SPARK Example => spark-example}/img/emr4.png (100%) rename {SPARK Example => spark-example}/img/launch_job.png (100%) rename {SPARK Example => spark-example}/pom.xml (100%) rename {SPARK Example => spark-example}/src/main/java/kmer/DataStoreClient.java (100%) rename {SPARK Example => spark-example}/src/main/java/kmer/HttpUtil.java (100%) rename {SPARK Example => spark-example}/src/main/java/kmer/Kmer.java (100%) rename {SPARK Example => spark-example}/src/main/java/kmer/Retriable.java (99%) rename {SPARK Example => spark-example}/src/main/java/kmer/SparkUtil.java (88%) rename {SPARK Example => spark-example}/src/test/java/kmer/DataStoreClientTest.java (99%) rename {SPARK Example => spark-example}/src/test/java/kmer/HttpUtilTest.java (100%) rename {SPARK Example => spark-example}/src/test/java/kmer/RetriableTest.java (99%) diff --git a/SPARK Example/README.md b/spark-example/README.md similarity index 99% rename from SPARK Example/README.md rename to spark-example/README.md index 4d4093f..8683816 100644 --- a/SPARK Example/README.md +++ b/spark-example/README.md @@ -193,7 +193,7 @@ d03e1fb5-0bd6-41ad-9744-c87af0fbdc33 40 TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTCT This is an extremely basic example of how to access HCA data on AWS using SPARK. A future iteration of this code could do something much more interesting. I'm -thinking of implementing what Titus Brown covers in this [blog post](http://ivory.idyll.org/blog/2016-sourmash.html). +thinking of implementing what Titus Brown covers in this [blog post](http://ivory.idyll.org/blog/2016-sourmash.html). Specifically it would be super cool to calculate MinHash signature for each of the biomaterials in HCA and use these to quickly compare samples by samples. Perhaps this would be useful in building a feature to ask "what cells are diff --git a/SPARK Example/img/emr1.png b/spark-example/img/emr1.png similarity index 100% rename from SPARK Example/img/emr1.png rename to spark-example/img/emr1.png diff --git a/SPARK Example/img/emr2.png b/spark-example/img/emr2.png similarity index 100% rename from SPARK Example/img/emr2.png rename to spark-example/img/emr2.png diff --git a/SPARK Example/img/emr3.png b/spark-example/img/emr3.png similarity index 100% rename from SPARK Example/img/emr3.png rename to spark-example/img/emr3.png diff --git a/SPARK Example/img/emr4.png b/spark-example/img/emr4.png similarity index 100% rename from SPARK Example/img/emr4.png rename to spark-example/img/emr4.png diff --git a/SPARK Example/img/launch_job.png b/spark-example/img/launch_job.png similarity index 100% rename from SPARK Example/img/launch_job.png rename to spark-example/img/launch_job.png diff --git a/SPARK Example/pom.xml b/spark-example/pom.xml similarity index 100% rename from SPARK Example/pom.xml rename to spark-example/pom.xml diff --git a/SPARK Example/src/main/java/kmer/DataStoreClient.java b/spark-example/src/main/java/kmer/DataStoreClient.java similarity index 100% rename from SPARK Example/src/main/java/kmer/DataStoreClient.java rename to spark-example/src/main/java/kmer/DataStoreClient.java diff --git a/SPARK Example/src/main/java/kmer/HttpUtil.java b/spark-example/src/main/java/kmer/HttpUtil.java similarity index 100% rename from SPARK Example/src/main/java/kmer/HttpUtil.java rename to spark-example/src/main/java/kmer/HttpUtil.java diff --git a/SPARK Example/src/main/java/kmer/Kmer.java b/spark-example/src/main/java/kmer/Kmer.java similarity index 100% rename from SPARK Example/src/main/java/kmer/Kmer.java rename to spark-example/src/main/java/kmer/Kmer.java diff --git a/SPARK Example/src/main/java/kmer/Retriable.java b/spark-example/src/main/java/kmer/Retriable.java similarity index 99% rename from SPARK Example/src/main/java/kmer/Retriable.java rename to spark-example/src/main/java/kmer/Retriable.java index 132c6e4..2b6ae14 100644 --- a/SPARK Example/src/main/java/kmer/Retriable.java +++ b/spark-example/src/main/java/kmer/Retriable.java @@ -47,4 +47,3 @@ static Optional runWithRetries( return runWithRetries(maxRetries, maxInterval, supplier, t -> true); } }; - diff --git a/SPARK Example/src/main/java/kmer/SparkUtil.java b/spark-example/src/main/java/kmer/SparkUtil.java similarity index 88% rename from SPARK Example/src/main/java/kmer/SparkUtil.java rename to spark-example/src/main/java/kmer/SparkUtil.java index f86f166..a13e6d8 100644 --- a/SPARK Example/src/main/java/kmer/SparkUtil.java +++ b/spark-example/src/main/java/kmer/SparkUtil.java @@ -4,14 +4,14 @@ import org.apache.spark.api.java.JavaSparkContext; /** - * This is a utility class to create JavaSparkContext - * and other objects required by Spark. There are many - * ways to create JavaSparkContext object. Here we offer + * This is a utility class to create JavaSparkContext + * and other objects required by Spark. There are many + * ways to create JavaSparkContext object. Here we offer * 2 ways to create it: * * 1. by using YARN's resource manager host name * - * 2. by using spark master URL, which is expressed as: + * 2. by using spark master URL, which is expressed as: * * spark://:7077 * diff --git a/SPARK Example/src/test/java/kmer/DataStoreClientTest.java b/spark-example/src/test/java/kmer/DataStoreClientTest.java similarity index 99% rename from SPARK Example/src/test/java/kmer/DataStoreClientTest.java rename to spark-example/src/test/java/kmer/DataStoreClientTest.java index 559201c..6922cb6 100644 --- a/SPARK Example/src/test/java/kmer/DataStoreClientTest.java +++ b/spark-example/src/test/java/kmer/DataStoreClientTest.java @@ -70,4 +70,3 @@ public void testSerialize() throws IOException { oos.writeObject(new DataStoreClient("https://somewhere.com/")); } } - diff --git a/SPARK Example/src/test/java/kmer/HttpUtilTest.java b/spark-example/src/test/java/kmer/HttpUtilTest.java similarity index 100% rename from SPARK Example/src/test/java/kmer/HttpUtilTest.java rename to spark-example/src/test/java/kmer/HttpUtilTest.java diff --git a/SPARK Example/src/test/java/kmer/RetriableTest.java b/spark-example/src/test/java/kmer/RetriableTest.java similarity index 99% rename from SPARK Example/src/test/java/kmer/RetriableTest.java rename to spark-example/src/test/java/kmer/RetriableTest.java index 2f5679e..65c005d 100644 --- a/SPARK Example/src/test/java/kmer/RetriableTest.java +++ b/spark-example/src/test/java/kmer/RetriableTest.java @@ -22,5 +22,3 @@ public void runWithRetriesShouldReturnSuccessfulValue() throws InterruptedExcept assertTrue(opt.get() == 2); } } - - From 11366676bb0fe54aac3e74b46048c7707f6f159d Mon Sep 17 00:00:00 2001 From: Charles Reid Date: Tue, 17 Dec 2019 11:18:34 -0800 Subject: [PATCH 03/13] move 10x seq tcell tutorial --- .../README.md | 10 +++++----- .../download-10x-seq-tcell.ipynb | 0 2 files changed, 5 insertions(+), 5 deletions(-) rename {Download 10x Seq T-cell Bundles => download-10x-seq-tcell}/README.md (98%) rename Download 10x Seq T-cell Bundles/Download All Bundles for T-cells Sequenced with 10x.ipynb => download-10x-seq-tcell/download-10x-seq-tcell.ipynb (100%) diff --git a/Download 10x Seq T-cell Bundles/README.md b/download-10x-seq-tcell/README.md similarity index 98% rename from Download 10x Seq T-cell Bundles/README.md rename to download-10x-seq-tcell/README.md index 2218061..1da2d53 100644 --- a/Download 10x Seq T-cell Bundles/README.md +++ b/download-10x-seq-tcell/README.md @@ -1,5 +1,5 @@ -# Download All Bundles for T-Cells Sequenced with 10x - -In this task, the goal is to download all bundles containing data on T-cells and 10x sequencing. - -See the [notebook](https://github.com/HumanCellAtlas/data-consumer-vignettes/blob/jk-download-t-cells-with-10x/tasks/Download%2010x%20Seq%20T-cell%20Bundles/Download%20All%20Bundles%20for%20T-cells%20Sequenced%20with%2010x.ipynb). +# Download All Bundles for T-Cells Sequenced with 10x + +In this task, the goal is to download all bundles containing data on T-cells and 10x sequencing. + +See the [notebook](https://github.com/HumanCellAtlas/data-consumer-vignettes/blob/jk-download-t-cells-with-10x/tasks/Download%2010x%20Seq%20T-cell%20Bundles/Download%20All%20Bundles%20for%20T-cells%20Sequenced%20with%2010x.ipynb). diff --git a/Download 10x Seq T-cell Bundles/Download All Bundles for T-cells Sequenced with 10x.ipynb b/download-10x-seq-tcell/download-10x-seq-tcell.ipynb similarity index 100% rename from Download 10x Seq T-cell Bundles/Download All Bundles for T-cells Sequenced with 10x.ipynb rename to download-10x-seq-tcell/download-10x-seq-tcell.ipynb From 998609b68129f47ee8507f7349cb6bec0d342e8b Mon Sep 17 00:00:00 2001 From: Charles Reid Date: Wed, 18 Dec 2019 11:29:55 -0800 Subject: [PATCH 04/13] move download any bam file tutorial --- {Download Any BAM File => download-any-bam-file}/README.md | 0 .../download-any-bam-file.ipynb | 0 {Download Any BAM File => download-any-bam-file}/requirements.txt | 0 3 files changed, 0 insertions(+), 0 deletions(-) rename {Download Any BAM File => download-any-bam-file}/README.md (100%) rename Download Any BAM File/Download Any BAM.ipynb => download-any-bam-file/download-any-bam-file.ipynb (100%) rename {Download Any BAM File => download-any-bam-file}/requirements.txt (100%) diff --git a/Download Any BAM File/README.md b/download-any-bam-file/README.md similarity index 100% rename from Download Any BAM File/README.md rename to download-any-bam-file/README.md diff --git a/Download Any BAM File/Download Any BAM.ipynb b/download-any-bam-file/download-any-bam-file.ipynb similarity index 100% rename from Download Any BAM File/Download Any BAM.ipynb rename to download-any-bam-file/download-any-bam-file.ipynb diff --git a/Download Any BAM File/requirements.txt b/download-any-bam-file/requirements.txt similarity index 100% rename from Download Any BAM File/requirements.txt rename to download-any-bam-file/requirements.txt From 94fd942592ce4eeb17d746f963261ad203d7360d Mon Sep 17 00:00:00 2001 From: Charles Reid Date: Wed, 18 Dec 2019 11:31:28 -0800 Subject: [PATCH 05/13] move login nb --- {Login to the DSS => login}/README.md | 0 Login to the DSS/Log In.ipynb => login/login.ipynb | 0 2 files changed, 0 insertions(+), 0 deletions(-) rename {Login to the DSS => login}/README.md (100%) rename Login to the DSS/Log In.ipynb => login/login.ipynb (100%) diff --git a/Login to the DSS/README.md b/login/README.md similarity index 100% rename from Login to the DSS/README.md rename to login/README.md diff --git a/Login to the DSS/Log In.ipynb b/login/login.ipynb similarity index 100% rename from Login to the DSS/Log In.ipynb rename to login/login.ipynb From 04b212b1c0308a89e190666f787aaec8d1fbd79e Mon Sep 17 00:00:00 2001 From: Charles Reid Date: Tue, 17 Dec 2019 14:39:54 -0800 Subject: [PATCH 06/13] move install nb --- {Install the HCA CLI tools => install-hca}/README.md | 0 .../Install.ipynb => install-hca/install-hca.ipynb | 0 2 files changed, 0 insertions(+), 0 deletions(-) rename {Install the HCA CLI tools => install-hca}/README.md (100%) rename Install the HCA CLI tools/Install.ipynb => install-hca/install-hca.ipynb (100%) diff --git a/Install the HCA CLI tools/README.md b/install-hca/README.md similarity index 100% rename from Install the HCA CLI tools/README.md rename to install-hca/README.md diff --git a/Install the HCA CLI tools/Install.ipynb b/install-hca/install-hca.ipynb similarity index 100% rename from Install the HCA CLI tools/Install.ipynb rename to install-hca/install-hca.ipynb From ac10e1c3cce3621d89d68d777c9773a7aba010c8 Mon Sep 17 00:00:00 2001 From: Charles Reid Date: Tue, 17 Dec 2019 14:40:37 -0800 Subject: [PATCH 07/13] move find cell type count nb --- .../Find Cell Type Count.ipynb | 0 .../README.md | 12 ++++++------ 2 files changed, 6 insertions(+), 6 deletions(-) rename {Find Cell Type Count => find-cell-type-count}/Find Cell Type Count.ipynb (100%) rename {Find Cell Type Count => find-cell-type-count}/README.md (97%) diff --git a/Find Cell Type Count/Find Cell Type Count.ipynb b/find-cell-type-count/Find Cell Type Count.ipynb similarity index 100% rename from Find Cell Type Count/Find Cell Type Count.ipynb rename to find-cell-type-count/Find Cell Type Count.ipynb diff --git a/Find Cell Type Count/README.md b/find-cell-type-count/README.md similarity index 97% rename from Find Cell Type Count/README.md rename to find-cell-type-count/README.md index ee3ec7c..923330a 100644 --- a/Find Cell Type Count/README.md +++ b/find-cell-type-count/README.md @@ -1,6 +1,6 @@ -# Find Cell Type Count - -In this task, you want to count the number of available cells of a certain cell type. For instance, -you might ask, "How many _hematopoietic system_ cells are available in the database?" - -See the [notebook](Find%20Cell%20Type%20Count.ipynb). +# Find Cell Type Count + +In this task, you want to count the number of available cells of a certain cell type. For instance, +you might ask, "How many _hematopoietic system_ cells are available in the database?" + +See the [notebook](Find%20Cell%20Type%20Count.ipynb). From 57c41285ff7de1ce3bf96c5be3a4a11b8fe3869d Mon Sep 17 00:00:00 2001 From: Charles Reid Date: Tue, 17 Dec 2019 14:41:38 -0800 Subject: [PATCH 08/13] move download all fastq files nb --- .../README.md | 0 .../download-all-fastq-files.ipynb | 0 2 files changed, 0 insertions(+), 0 deletions(-) rename {Download all .fastq files in a project => download-all-fastq-files}/README.md (100%) rename Download all .fastq files in a project/Download all .fastq files in a project.ipynb => download-all-fastq-files/download-all-fastq-files.ipynb (100%) diff --git a/Download all .fastq files in a project/README.md b/download-all-fastq-files/README.md similarity index 100% rename from Download all .fastq files in a project/README.md rename to download-all-fastq-files/README.md diff --git a/Download all .fastq files in a project/Download all .fastq files in a project.ipynb b/download-all-fastq-files/download-all-fastq-files.ipynb similarity index 100% rename from Download all .fastq files in a project/Download all .fastq files in a project.ipynb rename to download-all-fastq-files/download-all-fastq-files.ipynb From 9ce12d81fc55b4b8b5f8e8c094c6ddfa6c297d5c Mon Sep 17 00:00:00 2001 From: Charles Reid Date: Wed, 18 Dec 2019 12:01:29 -0800 Subject: [PATCH 09/13] move download smartseq2 expression matrix nb --- .../README.md | 10 +++++----- .../download-smartseq2-matrix-scanpy.ipynb | 0 2 files changed, 5 insertions(+), 5 deletions(-) rename {Download Expression Matrix for Scanpy => download-smartseq2-matrix-scanpy}/README.md (98%) rename Download Expression Matrix for Scanpy/Download SmartSeq2 Expression Matrix as Input to Scanpy.ipynb => download-smartseq2-matrix-scanpy/download-smartseq2-matrix-scanpy.ipynb (100%) diff --git a/Download Expression Matrix for Scanpy/README.md b/download-smartseq2-matrix-scanpy/README.md similarity index 98% rename from Download Expression Matrix for Scanpy/README.md rename to download-smartseq2-matrix-scanpy/README.md index e8879f6..50629d2 100644 --- a/Download Expression Matrix for Scanpy/README.md +++ b/download-smartseq2-matrix-scanpy/README.md @@ -1,5 +1,5 @@ -# Download SmartSeq2 Expression Matrix as an Input to Scanpy - -In this task, suppose you want to download an expression matrix for use in Python's scanpy module. - -See the [notebook](https://github.com/HumanCellAtlas/data-consumer-vignettes/blob/feature/download-expression-matrix/tasks/Download%20Expression%20Matrix%20for%20Scanpy/Download%20SmartSeq2%20Expression%20Matrix%20as%20Input%20to%20Scanpy.ipynb). +# Download SmartSeq2 Expression Matrix as an Input to Scanpy + +In this task, suppose you want to download an expression matrix for use in Python's scanpy module. + +See the [notebook](https://github.com/HumanCellAtlas/data-consumer-vignettes/blob/feature/download-expression-matrix/tasks/Download%20Expression%20Matrix%20for%20Scanpy/Download%20SmartSeq2%20Expression%20Matrix%20as%20Input%20to%20Scanpy.ipynb). diff --git a/Download Expression Matrix for Scanpy/Download SmartSeq2 Expression Matrix as Input to Scanpy.ipynb b/download-smartseq2-matrix-scanpy/download-smartseq2-matrix-scanpy.ipynb similarity index 100% rename from Download Expression Matrix for Scanpy/Download SmartSeq2 Expression Matrix as Input to Scanpy.ipynb rename to download-smartseq2-matrix-scanpy/download-smartseq2-matrix-scanpy.ipynb From b06ff0359b8b0648135b375f7c591f20cf7f6994 Mon Sep 17 00:00:00 2001 From: Charles Reid Date: Wed, 18 Dec 2019 12:44:48 -0800 Subject: [PATCH 10/13] rename notebook --- .../{Find Cell Type Count.ipynb => find-cell-type-count.ipynb} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename find-cell-type-count/{Find Cell Type Count.ipynb => find-cell-type-count.ipynb} (100%) diff --git a/find-cell-type-count/Find Cell Type Count.ipynb b/find-cell-type-count/find-cell-type-count.ipynb similarity index 100% rename from find-cell-type-count/Find Cell Type Count.ipynb rename to find-cell-type-count/find-cell-type-count.ipynb From eda85d9f6669be2d78d40822ee0a5f569c487950 Mon Sep 17 00:00:00 2001 From: Charles Reid Date: Thu, 19 Dec 2019 16:18:56 -0800 Subject: [PATCH 11/13] fix links in readme --- README.md | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index de1aea1..bd76a22 100644 --- a/README.md +++ b/README.md @@ -10,13 +10,12 @@ The DCP welcomes any contributed notebooks or other tutorials to the list below. ## Vignettes Table of Contents: -* [Explore an HCA Data Set in Scanpy (May 2019)](Explore%20an%20HCA%20Data%20Set%20in%20Scanpy%20(May%202019)/README.md) -* [Download Expression Matrix for Scanpy](Download%20Expression%20Matrix%20for%20Scanpy/README.md) -* [Explore an HCA Data Set in Scanpy (Nov 2018)](Explore%20an%20HCA%20Data%20Set%20in%20Scanpy%20(Nov%202018)/README.md) -* [SPARK Example](SPARK%20Example/README.md) -* [Login to the DSS](Login%20to%20the%20DSS/README.md) -* [Download all .fastq files in a project](Download%20all%20.fastq%20files%20in%20a%20project/README.md) -* [Download Any BAM File](Download%20Any%20BAM%20File/README.md) -* [Find Cell Type Count](Find%20Cell%20Type%20Count/README.md) -* [Install the HCA CLI tools](Install%20the%20HCA%20CLI%20tools/README.md) -* [Download 10x Seq T-cell Bundles](Download%2010x%20Seq%20T-cell%20Bundles/README.md) +* [install-hca](install-hca/install-hca.ipynb) +* [login](login/login.ipynb) +* [download-10x-seq-tcell](download-10x-seq-tcell/download-10x-seq-tcell.ipynb) +* [download-all-fastq-files](download-all-fastq-files/download-all-fastq-files.ipynb) +* [download-any-bam-file](download-any-bam-file/download-any-bam-file.ipynb) +* [download-smartseq2-matrix-scanpy](download-smartseq2-matrix-scanpy/download-smartseq2-matrix-scanpy.ipynb) +* [explore-hca-dataset-scanpy-may-2019](explore-hca-dataset-scanpy-may-2019/explore-hca-dataset-scanpy-may-2019.ipynb) +* [explore-hca-dataset-scanpy-nov-2018](explore-hca-dataset-scanpy-nov-2018/explore-hca-dataset-scanpy-nov-2018.ipynb) +* [find-cell-type-count](find-cell-type-count/find-cell-type-count.ipynb) From eb5df90621054206524c4804b388d292ae768f6d Mon Sep 17 00:00:00 2001 From: Charles Reid Date: Thu, 19 Dec 2019 16:19:22 -0800 Subject: [PATCH 12/13] remove tocgen script --- toc.md.j2 | 13 ------------- tocgen.py | 29 ----------------------------- 2 files changed, 42 deletions(-) delete mode 100644 toc.md.j2 delete mode 100644 tocgen.py diff --git a/toc.md.j2 b/toc.md.j2 deleted file mode 100644 index 8157c44..0000000 --- a/toc.md.j2 +++ /dev/null @@ -1,13 +0,0 @@ -# Data Consumer Vignettes - -[![Build Status](https://travis-ci.com/HumanCellAtlas/data-consumer-vignettes.svg?branch=master)](https://travis-ci.com/HumanCellAtlas/data-consumer-vignettes) - -## Overview - -Welcome to the DCP Vignette repository, containing walkthrough tutorials to help you get started with the DCP primarily via command-line access. For downstream application development, please refer to the [HCA DCP API documentation](https://prod.data.humancellatlas.org/apis). - -The DCP welcomes any contributed notebooks or other tutorials to the list below. You can create your own branch and submit a pull request. - -## Vignettes Table of Contents: - -{{toc}} diff --git a/tocgen.py b/tocgen.py deleted file mode 100644 index d672620..0000000 --- a/tocgen.py +++ /dev/null @@ -1,29 +0,0 @@ -from jinja2 import Environment, FileSystemLoader -import os, glob - -# Sets template to toc.md.j2 located at the root of project -env = Environment(loader=FileSystemLoader(os.path.dirname('./'))) -template = env.get_template('toc.md.j2') - -# Sets dir to root of project -rootDir = '.' - -# Gets a list of dirs with README -list_dirs = glob.glob("./**/*.md") - -content = "" - -# For list of items, perform split to get the wanted parts of the path and append it to -# content var -for items in list_dirs: - root, dirName, readMe = items.split("/") - link = os.path.join(dirName, readMe).replace(" ","%20") - if dirName != 'test': - content += "* [{}]({})\n".format(dirName, link) - -# Saves content to object that will be used to write to template -vars = {"toc": content} - -# Writes to README.md based on template -with open('README.md', 'w') as f: - f.write(template.render(**vars)) From 92abfe1a05a93aa8be92ef453dc6680d5648af7f Mon Sep 17 00:00:00 2001 From: Charles Reid Date: Wed, 8 Jan 2020 11:15:39 -0800 Subject: [PATCH 13/13] fix TOC --- README.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index bd76a22..7556db1 100644 --- a/README.md +++ b/README.md @@ -10,12 +10,12 @@ The DCP welcomes any contributed notebooks or other tutorials to the list below. ## Vignettes Table of Contents: -* [install-hca](install-hca/install-hca.ipynb) -* [login](login/login.ipynb) -* [download-10x-seq-tcell](download-10x-seq-tcell/download-10x-seq-tcell.ipynb) -* [download-all-fastq-files](download-all-fastq-files/download-all-fastq-files.ipynb) -* [download-any-bam-file](download-any-bam-file/download-any-bam-file.ipynb) -* [download-smartseq2-matrix-scanpy](download-smartseq2-matrix-scanpy/download-smartseq2-matrix-scanpy.ipynb) -* [explore-hca-dataset-scanpy-may-2019](explore-hca-dataset-scanpy-may-2019/explore-hca-dataset-scanpy-may-2019.ipynb) -* [explore-hca-dataset-scanpy-nov-2018](explore-hca-dataset-scanpy-nov-2018/explore-hca-dataset-scanpy-nov-2018.ipynb) -* [find-cell-type-count](find-cell-type-count/find-cell-type-count.ipynb) +* [Install the HCA CLI tools](install-hca/install-hca.ipynb) +* [Login to the DSS](login/login.ipynb) +* [Writing ElasticSearch Queries](elasticsearch-queries/elasticsearch-queries.ipynb) +* [Download 10x Liver Cell Data](download-10x-seq-tcell/download-10x-liver.ipynb) +* [Download All Fastq Files in a Project](download-all-fastq-files/download-all-fastq-files.ipynb) +* [Download Any BAM File](download-any-bam-file/download-any-bam-file.ipynb) +* [Download Smartseq2 Expression Matrix as an Input to Scanpy](download-smartseq2-matrix-scanpy/download-smartseq2-matrix-scanpy.ipynb) +* [Explore an HCA Data Set in Scanpy](explore-hca-dataset-scanpy/explore-hca-dataset-scanpy.ipynb) +* [Find Cell Type Count](find-cell-type-count/find-cell-type-count.ipynb)