Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AberrantSplicing module error: wrong args for environment subassignment #581

Open
Wynandi opened this issue Oct 10, 2024 · 4 comments
Open

Comments

@Wynandi
Copy link

Wynandi commented Oct 10, 2024

Dear developers,

I have successfully run the AberrantExpression module but encountered issues with the AberrantSplicing module. My samples have been previously analyzed using OUTRIDER, FRASER, and FRASER2, and I am now transitioning to the DROP pipeline (v1.4.0). Given that the samples worked with the earlier tools, I believe they should not be the source of the problem. I have not yet tested with external example samples. Additionally, the drop_demo ran without any issues.

This is the error I keep getting:

Error in reducer$value.cache[[as.character(idx)]] <- values :
wrong args for environment subassignment
Calls: countSplitReads ... .bploop_impl -> .collect_result -> .reducer_add -> .reducer_add
In addition: Warning message:
In parallel::mccollect(wait = FALSE, timeout = 1) :
1 parallel job did not deliver a result
Execution halted

To tackle it I have checked the annotation file and the config file. In these, I am unable to find anything wrong. Additionally, I have checked my settings for the batch job for the command "snakemake aberrantSplicing --cores 40" (also ran interactively once). According to this list:

  1. As in AberrantExpression
  2. Changed batch memory to 100G
  3. Kept 100G and added ntasks to 2
    From now on, reduce number of files to 35, I have 70+
  4. Previous but on 2 nodes
  5. Added 150G, ntasks 2, nnodes 2 and cpus 40
  6. Add server network connection commands to batchjob (also cores were still 20 in the snakemake --cores command, updated now to 40)

Attatched below are all the relevant files. I hope you can aid me in figuring out this problem.
Thank you!

config.txt
fraser_annotation_patientIDsremoved.tsv.txt
job_err_32351_4294967294.txt

Br,
Victoria Lillback

@AtaJadidAhari
Copy link
Collaborator

Hi Victoria and thanks for using DROP!
We've seen this issue before and it is usually a problem with memory. It is easier if you just submit one job at a time and increase the memory up until it runs successfully. Let us know if the issue still persists.

@Wynandi
Copy link
Author

Wynandi commented Oct 10, 2024

Thanks for the response.

I have increased the limit to of 150GB, and after this I get server errors telling me that I exceeded limits.
If 150GB sounds too little I will try to request further access.

I just ran the FRASER2 separately to check once again everything works. It took me 3-4 hours for 70 samples to count the reads and generate splicing metrics with FRASER2 outside DROP.

Br,
Victoria

@vyepez88
Copy link
Collaborator

Hi Victoria, following up on this, did it work with more than 150GB?
When you run FRASER2 separately, how much memory did you provide?

@Wynandi
Copy link
Author

Wynandi commented Dec 11, 2024

Hello!

I realized my earlier message might have been unclear.

After discussing with colleagues, we discovered that running FRASER, whether inside or outside DROP, can require up to 300GB of memory, which exceeds my available resources.

Since FRASER offers greater flexibility outside DROP for adjustments, I started running it in serial mode. This approach takes about 3–4 hours to count reads per sample but is manageable, and 150GB of memory is sufficient (I have not tried with less) when parallel processing is disabled. Currently, I process batches of 30–90 samples with this setup.

Br,
Victoria

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants