with-elpy-rpc-virtualenv-activated incorrectly alters $PATH

[SergiyKolesnikov]

Summary

After with-elpy-rpc-virtualenv-activated is run in elpy-rpc--create-virtualenv, the PATH environment variable is incorrectly prepended with //bin. This makes elpy-rpc--create-virtualenv always use the system-wide Python binary (e.g., /bin/python) for creating the RPC virtual environment with virtualenv command.

Steps to reproduce

1. Install a Python distribution in a custom location. For example, install Anaconda to your home directory and run conda init, so that the PATH in your shell is always prepended with the path to the Anaconda's Python (e.g., /home/auser/anaconda3/bin:/home/auser/anaconda3/bin:/home/auser/anaconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games)
2. Run Emacs from the shell and install elpy from MELPA using package and configure it as described in the docs.
3. Restart Emacs from the shell and open a python source file. Elpy RPC virtual environment is created using the system-wide Python binary in /bin instead of the Anacondas binary.

The steps were completed in a freshly installed VM with a fresh install of Anaconda and elpy.

My configuration

OS

Debian 10 buster

Result of (elpy-config)

Emacs.............: 26.1
Elpy..............: 1.31.0
Virtualenv........: None
Interactive Python: python 3.7.4 (/home/auser/anaconda3/bin/python)
RPC virtualenv....: rpc-venv (/home/auser/.emacs.d/elpy/rpc-venv)
 Python...........: python 2.7.16 (/home/auser/.emacs.d/elpy/rpc-venv/bin/python)
 Jedi.............: Not found (0.15.1 available)
 Rope.............: Not found (0.14.0 available)
 Autopep8.........: Not found (1.4.4 available)
 Yapf.............: Not found (0.28.0 available)
 Black............: Not found (19.10b0 available)
Syntax checker....: flake8 (/home/auser/anaconda3/bin/flake8)

Elpy configuration in my init.el

(elpy-enable)

Additional info

I have added several (message (getenv "PATH")) to with-elpy-rpc-virtualenv-activated like this:

...
    (message (getenv "PATH"))  ;; <- Added
    (with-elpy-rpc-virtualenv-activated
     (message (getenv "PATH"))  ;; <- Added
     (cond
      ((= 0 (call-process elpy-rpc-python-command nil nil nil
                          "-m" "venv" "-h"))
       (with-current-buffer (get-buffer-create elpy-venv-buffname)
         (insert (concat "Running '" elpy-rpc-python-command " -m venv "
                         rpc-venv-path "':\n\n"))
         (setq success (call-process elpy-rpc-python-command nil t t
                                     "-m" "venv" rpc-venv-path))))
      ((executable-find "virtualenv")
       (message (getenv "PATH"))  ;; <- Added
...

and that is what is output to the Message buffer when the RPC environment is created:

Elpy is creating the RPC virtualenv (’/home/auser/.emacs.d/elpy/rpc-venv’)
/home/auser/anaconda3/bin:/home/auser/anaconda3/bin:/home/auser/anaconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
//bin:/home/auser/anaconda3/bin:/home/auser/anaconda3/bin:/home/auser/anaconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games [2 times]

A workaround for now is to set elpy-rpc-python-command in init.el to the absolute path of the required Python binary:

(setq elpy-rpc-python-command "/home/auser/anaconda3/bin/python")

[galaunay]

Elpy indeed uses the system environment to make its virtualenv.

It has been done that way because virtualenvs created with venv keep a link to the environment they have been created from.
And we don't want that, as Elpy's virtualenv could be used with other virtualenvs.
(Notably, it has raised issues with pip, see #1674).

Can I ask why it is a problem for you to have the RPC virtualenv created from the system environment ?

[SergiyKolesnikov]

Thanks for looking into this @galaunay. I do not use my system-wide Python for various reasons (primarily because of the old versions and limited selection of packages that come with it). Instead I use Anaconda (installed in my home directory) and would like elpy to use it too, because then I will have to manage only this one Python installation.

Maybe this deviating behavior of elpy with regard to creating RPC environment is worth mentioning in the documentation, because normally PATH is respected; and it was very surprising and took some time to find out why it was not respected in this particular case.

[galaunay]

Maybe this deviating behavior of elpy with regard to creating RPC environment is worth mentioning in the documentation

I definitely agree, I updated the documentation (#1716).

because normally PATH is respected; and it was very surprising and took some time to find out why it was not respected in this particular case.

I understand that it is confusing, but I couldn't find a better solution.
Setting elpy-rpc-python-command to a the full interpreter path is the right thing to do.
I mentioned it in the doc as well.