scaling of SCOTCH?

[nick-youngblut]

Based on:

#! /bin/bash -l
#SBATCH --nodes=1
#SBATCH --ntasks=1 
#SBATCH --cpus-per-task=1
#SBATCH --mem=200G
#SBATCH --array=0-99
source ~/.bashrc
conda activate SCOTCH

bamfolder="path/to/tagged/bamfile"
outputfolder="path/to/output"

prepare.sh -t matrix -o $outputfolder --bam $bamfolder -match 0.2 --job_index ${SLURM_ARRAY_TASK_ID} --jobs 100

...it appears that the generate compatible matrix does not scale well. Do you have any stats on how well it scales, in regards to compute resources (especially mem & time) versus dataset size?

What is the default job time limit for your SLURM cluster? I don't see a time limit set in your sbatch job (above).