conda + pip install create tmp files in yml file path

[yinshiyi]

Bug report

Expected behavior and actual behavior

when nextflow run use conda yml file which contains pip, (eg https://www.nextflow.io/docs/latest/conda.html#python-packages-from-pypi). The pip create tmp files (requirements.txt) in worflow dir under config dir ($baseDir/config/condaenv.sobah5ol.requirements.txt).

The problem only happens when pip is used in conda yml

For local pipeline, where only dev team has write access, downstream users of such pipeline without write permission to $baseDir worflow dir, will fail to run pipeline.

using conda directly install rather than pip install solved the error, but I hope to find a systematic fix

name: my-env-2
channels:
  - conda-forge
  - bioconda
dependencies:
  - numpy

Steps to reproduce the problem

use the official example, but make the worflow dir 744 or drwxr--r--, when execute from user account in the same group, will has write permission errors.
(eg https://www.nextflow.io/docs/latest/conda.html#python-packages-from-pypi)

name: my-env-2
channels:
  - conda-forge
  - bioconda
dependencies:
  - pip
  - pip:
    - numpy
    - pandas
    - matplotlib

Program output

(Copy and paste here output produced by the failing execution. Please highlight it as a code block. Whenever possible upload the .nextflow.log file.)

Environment

• Nextflow version: [?]
• Java version: [?]
• Operating system: [Linux]
• Bash version: (use the command $SHELL --version)

Additional context

I think the issue description is sufficient, if more details are needed, please let me know. Thank you.

[yinshiyi]

My current idea is
when there is pip in the yml file, the execution of conda env create -f path/to/test.yml involved the tmp file creation at the location of test.yml file folder.

1. i think the following is already true, but if not, I would like to try to adjust the nextflow code param so that the conda env create command is run in the work dir or launch dir where the user has full access.
2. i think i can adjust the nextflow code, so it copy the path/to/test.yml file to the work dir then create env.

If i have progress will submit a PR or follow up in this issue.
I will play around with conda.cacheDir also. But this is where the conda env are stored at the end, hopefully it could help with the tmp files created during the conda create process.

[yinshiyi]

I found the source of the problem

https://github.com/conda/conda/blob/fd89e90b81b196f9aff6a3971b2730da85216aa7/conda_env/installers/pip.py#L17

conda-env will copy the entries in the pip: YAML list and place them into a temporary pip requirements file.