Implement support for uploading (and storing) workflow configuration files

[RalfG]

Next to structured and unstructured metadata, we would like to support the upload of configuration files (e.g., mqpar.xml) to be stored alongside the intermediate files and the metadata.

[enryH]

I had to parse a lot of parameter files for a submission into a table format. It is not yet a standalone module, but the functionality should be fairly general:

Link to script to parse maxquant parameter files

[enryH]

Aim for now:

• parse parameter files exported by search engines into a standardized format.
  • e.g. based on mqpar.xml of MaxQuant
• in case the parameter file cannot be parsed, describe the problem, but continue running (manuell editing)

[mlocardpaulet]

I am making a table with the list of parameters that we want to keep, and their description.
https://docs.google.com/spreadsheets/d/1Os3By8LYuVGcfH65X2-9tSjJnmtH-nL03tPYLtPOj-c/edit#gid=0

[enryH]

If you look at e.g. json and xml files in the test/params folder - https://github.com/Proteobench/ProteoBench/tree/parse_settings/test/params - an example mapping could be

our parameter name	MaxQuant
MS2 mass tolerance	'["msmsParamsArray"][0]["MatchTolerance"]'
minimum peptide length	'["minPepLen"]'

I think ideally the table will be build from a schema in the code.

Looking at the first example, I would store the maxquant information rather using
["msmsParamsArray"]["FTMS"]["MatchTolerance"]

[mlocardpaulet]

We noticed lack of consistency in the msfragger.params.toml file between the upper/lower tolerance bounds and the mass tolerance:

precursor_mass_lower = -10			# Lower bound of the precursor mass window.
precursor_mass_upper = 10			# Upper bound of the precursor mass window.
precursor_mass_units = 1			# Precursor mass tolerance units (0 for Da, 1 for ppm).
precursor_true_tolerance = 20			# True precursor mass tolerance (window is +/- this value).

We need to contact the developers to ask how to deal with it.

[mlocardpaulet]

precursor_mass_lower and precursor_mass_upper is defined for some pre-searching step. We'll ignore them

[mlocardpaulet]

For FragPipe we need to contact the developers to ask where to find the parameters. We'll send an email when we have a define set of parameters that we need.

[enryH]

I am making a table with the list of parameters that we want to keep, and their description. https://docs.google.com/spreadsheets/d/1Os3By8LYuVGcfH65X2-9tSjJnmtH-nL03tPYLtPOj-c/edit#gid=0

And here is the table Veit mentioned: link

Potentially it could be a good idea to take their parameter naming to keep the projects in sync.

[mlocardpaulet]

Of course! I totally agree

[enryH]

@RobbinBouwmeester Can you show me the current results.json? The one I linked in the main folder on the main branch is seven months old and did not look like the one you showed.

[RobbinBouwmeester]

Yes, here it is: https://github.com/Proteobench/Results_Module2_quant_DDA/blob/main/results.json

[enryH]

Discussion with Veit, Viki, Klemen, Julian, Witold and Nadja on a curate list of parameters. Based on that, I will

1. use as reference yaml of the common parameters the one in sdrf-pipelines
2. To keep it simple: For a task use the current one which is present when the task is defined -> Need to add a version or timestamp to the reference yaml file through a PR
3. If a new task needs other parameters: Sumbit a PR, get it accepted, bump version of reference yaml file

Ressources:

• table with overview in EXLIXIR of shared and categorized common parameters: link
• documentation on general parameters from Veit and David: link
• Bard and Marie's table of current parameters: link