Update Calculate Amount scripts for more flexibility

[idalindegaard]

Changes are needed for the new Pacbio Revio workflow but they also affect many other steps that have different types of calculate amount automations using the same script

Added

Changed

For script calculate_amount_ng.py:

• From options.py, "subtract_volume_option" was substituted with "subtract_volume" as it offers more flexibility
• From options.py, "preset_volume" was added so that a volume specified in the cli can be used in the calculation

For script twist_qc_amount.py:

• From options.py, "subtract_volume" was added and substitutes the hardcoded volume subtraction in the script - the value can be modified in the cli command instead.
• Its pytest test_twist_qc_amount.py was modified to reflect the added option mentioned above

Fixed

Steps to consider while deploying

• Configuration changes:
• Documentation updates:
• Inform users by email:

Review:

• Code approved by
• Tests executed on stage by: (Document the test done with screen shots and description.)
• "Merge and deploy" approved by

This version is a:

• MAJOR - when you make incompatible API changes
• MINOR - when you add functionality in a backwards compatible manner
• PATCH - when you make backwards compatible bug fixes or documentation/instructions

[AbiDalby]

Looks good!

[idalindegaard]

Testing on stage

Several lims steps might be affected by the EPPs but here I have tested one step per new/updated EPP. The subtraction volumes now better reflect the lab operations than before.

EPP "2. Calculate Amount" in step CG002 - Quantit QC (DNA)

Script: calculate_amount_ng.py
Subtraction volume is 2 ul (instead of 3 ul as before)

bash -c -l "conda activate cg_lims && lims -c /home/glsai/.genologics.yaml epps -l {compoundOutputFileLuid0} -p {processLuid} udf calculate calculate-amount-ng --concentration-udf 'Concentration' --amount-udf 'Amount (ng)' --volume-udf 'Volume (ul)' --subtract-volume 2 -m"

EPP "Prehyb QC - Calc Amount and Assign QC Flag" in step CG002 - Qubit QC (Library Validation)

Script: calculate_amount_ng.py
Subtraction volume is 2 ul (instead of 0 ul as before)

bash -c -l "conda activate cg_lims && lims -c /home/glsai/.genologics.yaml epps -l {compoundOutputFileLuid0} -p {processLuid} udf calculate calculate-amount-ng --concentration-udf 'Concentration' --amount-udf 'Amount (ng)' --volume-udf 'Volume (ul)' --subtract-volume 2 -m && /home/glsai/miniconda3/envs/epp_master/bin/set_qc.py -p {processLuid} -l {compoundOutputFileLuid3} -t 'Minimum required amount (ng)' -u 'Amount (ng)'"

EPP "Calculate Amount for RNA" in step Qubit QC (RNA) v1

Script: calculate_amount_ng.py
Subtraction volume is 3 ul (same as it was before when it was named just Calculate Amount, but this EPP is specific for RNA)

bash -c -l "conda activate cg_lims &&  lims -c /home/glsai/.genologics.yaml epps -l {compoundOutputFileLuid0} -p {processLuid}  udf calculate calculate-amount-ng --concentration-udf 'Concentration' --amount-udf 'Amount (ng)' --volume-udf 'Volume (ul)' -m --subtract-volume 3"

Affected steps:

• Quantit QC (RNA) v1
• Qubit QC (RNA) v1

EPP "Calculate Amount" in step CG002 - Qubit QC (DNA)

Script: calculate_amount_ng.py
Subtraction volume is 2 ul (instead of 3 ul as before)

bash -c -l "conda activate cg_lims &&  lims -c /home/glsai/.genologics.yaml epps -l {compoundOutputFileLuid0} -p {processLuid}  udf calculate calculate-amount-ng --concentration-udf 'Concentration' --amount-udf 'Amount (ng)' --volume-udf 'Volume (ul)' -m --subtract-volume 2"

Affected steps:

• Quantit QC (DNA) (Dev) v1
• CG002 - Qubit QC (DNA)
• Quantit QC (DNA) (Dev) v2
• Qubit QC (DNA) WGS frag v1
• Quantit QC (DNA) (ONT)
• Quantit QC (DNA) WGS frag v1
• Qubit QC (DNA) (Dev) v3
• Qubit QC (DNA) (Dev) v4
• Quantit QC (DNA) WGS v1
• Qubit QC (DNA) (ONT)

EPP "Calculate amount Revio (from measurement)" in step Qubit QC (DNA) (Revio) v1

Script: calculate_amount_ng.py
Subtraction volume is 2 ul (instead of 0 ul as before)

bash -c -l "conda activate cg_lims &&  lims -c /home/glsai/.genologics.yaml epps -l {compoundOutputFileLuid0} -p {processLuid}  udf calculate calculate-amount-ng --concentration-udf 'Concentration' --amount-udf 'Amount (ng)' --volume-udf 'Volume (ul)' --subtract-volume 2 -m"

Affected steps:

• Qubit QC - Library validation (Revio) v1
• Quantit QC - Library validation (Revio) v1
• Quantit QC (DNA) (Revio) v1
• Qubit QC (DNA) (Revio) v1
• Qubit QC (post norm) (Revio) v1
• Quantit QC (post norm) (Revio) v1

EPP "Calculate Amount for 75 ul" in step DNA size selection (SRE) v2

Script: calculate_amount_ng.py
No subtraction volume, calculate amount with the preset volume 75 ul. New EPP for this specific step

bash -c -l "conda activate cg_lims &&  lims -c /home/glsai/.genologics.yaml epps -l {compoundOutputFileLuid0} -p {processLuid}  udf calculate calculate-amount-ng --concentration-udf 'Concentration' --amount-udf 'Amount (ng)' --volume-udf 'Elution Volume SRE' --subtract-volume 0 --preset-volume 75"

EPP "Calculate Amount for 47 ul" in step DNA Fragmentation (Revio) v1

Script: calculate_amount_ng.py
No subtraction volume, calculate amount with the preset volume 75 ul. New EPP for this specific step

bash -c -l "conda activate cg_lims &&  lims -c /home/glsai/.genologics.yaml epps -l {compoundOutputFileLuid0} -p {processLuid}  udf calculate calculate-amount-ng --concentration-udf 'Concentration' --amount-udf 'Amount (ng)' --volume-udf 'Volume (ul)' --subtract-volume 0 --preset-volume 47"

When missing a concentration value, we get an error message

The script calculates the amount with the preset volume of 47 ul regardless of values or missing values of Volume (ul)

EPP "Calculate Amounts" in step Qubit QC (DNA) TWIST v2

Script: twist_qc_amount.py
Subtraction volume is 2 ul (instead of 3 ul as before)

Affected steps:

• Qubit QC (DNA) TWIST v2
• Quantit QC (DNA) TWIST v2

bash -c -l "conda activate cg_lims &&  lims -c /home/glsai/.genologics.yaml epps -l {compoundOutputFileLuid0} -p {processLuid}  udf calculate twist-qc-amount --subtract-volume 2"

Conclusion

Amounts are calculated as expected. For calculate_amount_ng.py, when there is a preset volume in the cli command, this is used as the first choice, otherwise the script uses the specified volume udf. Volume subtractions work as intended for both scripts.

[Karl-Svard]

Fantastic work Ida! The change became much larger than initially expected, but you've structured the testing and re-configuring beautifully 👏

[AbiDalby]

The results look really great! Great work Ida! 🙌

[idalindegaard]

Updated with a final test of the "Calculate Amounts" EPP which is used in two TGA RC steps, I added it as the last test in my comment above