olgabot/sourmash sig merge #117

olgabot · 2020-10-28T16:07:26Z

nf-core/kmermaid pull request

Many thanks for contributing to nf-core/kmermaid!

Please fill in the appropriate checklist below (delete whatever is not relevant).
These are the most common things requested on pull requests (PRs).

PR checklist

This comment contains a description of changes (with reason)
If you've fixed a bug or added code that should be tested, add tests!
If necessary, also make a PR on the nf-core/kmermaid branch on the nf-core/test-datasets repo
Ensure the test suite passes (nextflow run . -profile test,docker).
Make sure your code lints (nf-core lint .).
Documentation in docs is updated
CHANGELOG.md is updated
README.md is updated

Learn more about contributing: CONTRIBUTING.md

…y if run out of memory

…m single cells

github-actions · 2020-10-28T16:08:08Z

`nf-core lint` overall result: Failed ❌

Posted for pipeline commit c4f9521

+| ✅ 18 tests passed       |+
!| ❗  2 tests had warnings |!
-| ❌  1 tests failed       |-

❌ Test failures:

Test #1 - File must be removed: bin/markdown_to_html.r

❗ Test warnings:

Test #1 - File not found: .github/workflows/awstest.yml
Test #1 - File not found: .github/workflows/awsfulltest.yml

✅ Tests passed:

Test #1 - File found: nextflow.config
Test #1 - File found: nextflow_schema.json
Test #1 - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
Test #1 - File found: README.md
Test #1 - File found: CHANGELOG.md
Test #1 - File found: docs/README.md
Test #1 - File found: docs/output.md
Test #1 - File found: docs/usage.md
Test #1 - File found: .github/workflows/branch.yml
Test #1 - File found: .github/workflows/ci.yml
Test #1 - File found: .github/workflows/linting.yml
Test #1 - File found: main.nf
Test #1 - File found: environment.yml
Test #1 - File found: Dockerfile
Test #1 - File found: conf/base.config
Test #1 - File not found check: Singularity
Test #1 - File not found check: parameters.settings.json
Test #1 - File not found check: .travis.yml

Run details:

nf-core/tools version 1.11
Run at 2020-10-28 16:08:07

pranathivemuri · 2020-10-28T16:45:04Z

Will review once the tests are fixed

olgabot · 2020-10-29T17:15:21Z

Addresses #108

pranathivemuri · 2020-10-29T18:42:39Z

main.nf

    label "low_memory"
-    errorStrategy 'ignore'
+    errorStrategy { task.exitStatus in [143,137,104,134,139] ? 'retry' : 'ignore' }


this is cool!

pranathivemuri · 2020-10-29T18:48:34Z

CHANGELOG.md

@@ -20,6 +20,7 @@ Initial release of nf-core/kmermaid, created with the [nf-core](http://nf-co.re/
 barcode fastq
 * Add version printing for sencha, bam2fasta, and sourmash in Dockerfile, update versions in environment.yml
 * For processes translate, sourmash compute  add cpus=1 as they are only serial ([#107](https://github.com/nf-core/kmermaid/pull/107))
+* Add `sourmash sig merge` for aligned/unaligned signatures


could you write a detailed description? also sourmash_sig_merge

pranathivemuri · 2020-10-29T18:49:05Z

main.nf

+    sig = "${sig_id}.sig"
+    csv = "${sig_id}.csv"
+    """
+    sourmash compute \\


so we are doing sourmash compute twice? is this also dependent on skip_compute flag?

pranathivemuri · 2021-01-06T20:33:06Z

looks like the index and merge PRS test fail with similar errors

----------------------------------------------------
[9d/f0ba5a] Submitted process > get_software_versions
Channel `sourmash_sketches_nucleotide` has been used as an input by more than a process or an operator

 -- Check script 'main.nf' at line: 1499 or see '.nextflow.log' file for more details

olgabot · 2021-03-09T21:32:48Z

I'm going to be bad and merge this in with failing tests because I need some of this functionality in #131

olgabot added 3 commits October 28, 2020 09:00

Update extract_per_cell_fastqs to not say __aligned__aligned and retr…

df31610

…y if run out of memory

Initial commit for adding sourmash sig merge on aligned/unaligned fro…

700272d

…m single cells

Update changelog

c4f9521

Try to get grouptuple to work

7d3e215

olgabot added 3 commits October 29, 2020 10:05

Set minimum UMI per cell to be a default of 1000

533dc94

Set test min UMI per cell as 5

da84ba4

Remove unused --shard_size option

6d30732

olgabot added 3 commits October 29, 2020 10:18

Add option for skipping sig merge

8fc0c33

Update Dockerfile

63273e5

Add test for --skip_sig_merge

054d8b3

pranathivemuri reviewed Oct 29, 2020

View reviewed changes

olgabot and others added 7 commits January 4, 2021 16:26

Update changelog

f1304ca

Use more realistic scales and ksizes

a579175

regular test doesn't fail anymore

ef43907

Merge branch 'dev' into olgabot/sourmash-sig-merge

08c32ec

Update bam config

d0bec5c

Add dump ch_sourmash_sketches_mixed

000d6ca

Update schema

6bce013

Merge remote-tracking branch 'origin' into olgabot/sourmash-sig-merge

04e62d4

olgabot force-pushed the olgabot/sourmash-sig-merge branch from 544bba5 to 04e62d4 Compare January 6, 2021 22:07

olgabot added 4 commits January 7, 2021 08:58

Add params.ksizes to sketch output

ba765ff

Add peptide_molecules

ed5e72b

add check for skip_compute in sig merge logic

1718502

Add header

0029b51

olgabot added 25 commits January 11, 2021 08:30

Do per-ksize sourmash sig merge

8ba5db1

Add sourmash describe csvs to multiqc

c27b2b4

Update ProjectDir

05f6702

Properly save translate output

2fe4523

Add dump of sourmash sketches

27d95f0

Fixing sourmash sig merge

e68db00

Add ch_sourmash_sig_describe_nucleotides

7195eae

more if/else

9d090b4

Update changelog

5bea4c9

Getting "sig merge" to finally run

9682b03

Add option to skip sig merge

76b2ed7

Update validate_sketch_value to only allow a single value

3c95f82

Change sketch values to single value

1321968

peptide_molecule --> translate_peptide_molecule

dc0ecdd

add "translate_" to peptide ksize and jaccard threshold

1fe32b1

Do sig merge on individual moltypes

1297886

Add test_sig_merge

d8e764b

Add test_sig_merge to CI

232ac0d

Don't allow multiple sketch values

253fa83

Reduce bloom filter table size

67aec5e

Sig merge is working!

8c99f9f

Make test params more realistic

fccec83

Update default ksizes, add track abundance true

8de30e3

Update variables in merge_renamed_sigs.pyh

0aade17

Get sourmash compare to happen on correct ksizes and moltypes

ad36259

Merge branch 'dev' into olgabot/sourmash-sig-merge

43bbafd

olgabot merged commit b2acc1d into dev Mar 9, 2021

olgabot mentioned this pull request Mar 9, 2021

olgabot/sourmash sig merge v2 #132

Merged

5 tasks

olgabot mentioned this pull request Mar 16, 2021

Different max_hash values with Rust vs Python sourmash-bio/sourmash#1402

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

olgabot/sourmash sig merge #117

olgabot/sourmash sig merge #117

olgabot commented Oct 28, 2020

github-actions bot commented Oct 28, 2020

❌ Test failures:

❗ Test warnings:

✅ Tests passed:

Run details:

pranathivemuri commented Oct 28, 2020

olgabot commented Oct 29, 2020

pranathivemuri Oct 29, 2020

pranathivemuri Oct 29, 2020

pranathivemuri Oct 29, 2020

pranathivemuri commented Jan 6, 2021

olgabot commented Mar 9, 2021 •

edited

Loading

olgabot/sourmash sig merge #117

olgabot/sourmash sig merge #117

Conversation

olgabot commented Oct 28, 2020

nf-core/kmermaid pull request

PR checklist

github-actions bot commented Oct 28, 2020

nf-core lint overall result: Failed ❌

❌ Test failures:

❗ Test warnings:

✅ Tests passed:

Run details:

pranathivemuri commented Oct 28, 2020

olgabot commented Oct 29, 2020

pranathivemuri Oct 29, 2020

Choose a reason for hiding this comment

pranathivemuri Oct 29, 2020

Choose a reason for hiding this comment

pranathivemuri Oct 29, 2020

Choose a reason for hiding this comment

pranathivemuri commented Jan 6, 2021

olgabot commented Mar 9, 2021 • edited Loading

`nf-core lint` overall result: Failed ❌

olgabot commented Mar 9, 2021 •

edited

Loading