This repository has been archived by the owner on Jul 16, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
ef7b59f
commit 850b0d8
Showing
3 changed files
with
54 additions
and
123 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,32 +1,70 @@ | ||
# Workstation Housekeeping v1.11 | ||
## Workstation Cleaner (wscleaner) | ||
|
||
Scripts to manage data on the NGS workstation | ||
Workstation Cleaner (wscleaner) deletes local directories that have been uploaded to the DNAnexus cloud storage service. | ||
|
||
When executed, Runfolders in the input (root) directory are deleted based on the following criteria: | ||
|
||
## Workstation Cleaner (wscleaner) | ||
* A single DNAnexus project is found matching the runfolder name | ||
* All local FASTQ files are uploaded and in a 'closed' state | ||
* X logfiles are present in the DNA Nexus project /Logfiles directory (NB X can be added as a command line argument - default is 5) | ||
|
||
or if the run is identified as a TSO500 run, based on: | ||
* the bcl2fastq2_output.log file created by the automated scripts | ||
AND | ||
* Presence of `_TSO` in the human readable DNANexus project name | ||
|
||
A DNAnexus API key must be cached locally using the `--set-key` option. | ||
|
||
## Workstation Environment | ||
The directory `env/` in this repository contains conda environment scripts for the workstation. These remove conflicts in the PYTHONPATH environment variable by editing the variable when conda is activated. The conda documentation describes where to place these scripts under ['saving environment variables'](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#macos-and-linux). | ||
|
||
Delete local directories that have been uploaded to the DNAnexus cloud storage service. | ||
See wscleaner readme for more info | ||
## Install | ||
As descibed above, on the workstation 2 environments exist - wscleaner and wscleaner_test (for development work). | ||
You need to activate these environment before installing with pip (as below). | ||
|
||
## ngrok_start.sh | ||
|
||
Allow SSH access to the system by running ngrok as a background process. As of v1.11 supports dockerised ngrok instance. | ||
```bash | ||
git clone https://github.com/moka-guys/workstation_housekeeping.git | ||
pip install workstation_housekeeping/wscleaner | ||
wscleaner --version # Print version number | ||
``` | ||
|
||
### Installation | ||
## Automated usage | ||
The script `wscleaner_command.sh` is called by the crontab. This activates the enviroment and passes the logfile path (and any other non-default arguments). | ||
A development command script `wscleaner_command_dev.sh` can be used to call the test environment and provide testing arguments, eg --dry-run | ||
|
||
See knowledge base article for ngrok installation. | ||
|
||
### Usage | ||
## Manual Usage | ||
|
||
Non-dockerised ngrok: | ||
``` | ||
usage: wscleaner [-h] [--auth AUTH] [--dry-run] [--logfile LOGFILE] | ||
[--min-age MIN_AGE] [--logfile-count LOGFILE_COUNT] | ||
[--version] | ||
root | ||
`sudo bash ngrok_start.sh` | ||
positional arguments: | ||
root A directory containing runfolders to process | ||
Dockerised ngrok: | ||
optional arguments: | ||
-h, --help show this help message and exit | ||
--auth AUTH A text file containing the DNANexus authentication | ||
token | ||
--dry-run Perform a dry run without deleting files | ||
--logfile LOGFILE A path for the application logfile | ||
--min-age MIN_AGE The age (days) a runfolder must be to be deleted | ||
--logfile-count LOGFILE_COUNT | ||
The number of logfiles a runfolder must have in | ||
/Logfiles | ||
--version Print version | ||
``` | ||
|
||
`sudo bash ngrok_start.sh docker` | ||
## Test | ||
|
||
### output | ||
```bash | ||
# Run from the cloned repo directory after installation | ||
pytest . --auth_token DNA_NEXUS_KEY | ||
``` | ||
|
||
The script will output the ngrok connection details | ||
## License | ||
|
||
Developed by Viapath Genome Informatics |
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.