Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build instructions don't work #335

Closed
chsasank opened this issue Jan 23, 2024 · 11 comments
Closed

Build instructions don't work #335

chsasank opened this issue Jan 23, 2024 · 11 comments
Assignees
Labels
documentation Improvements or additions to documentation

Comments

@chsasank
Copy link
Contributor

I have followed the instructions in the readme and the build fails with the following error. MLIR and others got built successfully.

**** Configuring /home/sasank/code/triton-build/intel-xpu-backend-for-triton ****                                                                                                                                  
Defaulting to user installation because normal site-packages is not writeable                                                                                                                                      
Obtaining file:///home/sasank/code/triton-build/intel-xpu-backend-for-triton/python                                                                                                                                
  Installing build dependencies ... done                                                                                                                                                                           
  Checking if build backend supports build_editable ... done                                                                                                                                                       
  Getting requirements to build wheel ... done                                                                                                                                                                     
  Preparing metadata (pyproject.toml) ... done                                                                                                                                                                     
Requirement already satisfied: filelock in /home/sasank/.local/lib/python3.10/site-packages (from triton==3.0.0) (3.13.1)                                                                                          
Installing collected packages: triton                                                                                                                                                                              
  Attempting uninstall: triton                                                                                                                                                                                     
    Found existing installation: triton 2.1.0                                                                                                                                                                      
    Uninstalling triton-2.1.0:                                                                                                                                                                                     
      Successfully uninstalled triton-2.1.0                                                                                                                                                                        
  Running setup.py develop for triton
    error: subprocess-exited-with-error
     
    _ python setup.py develop did not run successfully.
    │ exit code: 1
    _─> [35 lines of output]
        copy /home/sasank/.triton/nvidia/bin/ptxas to /home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/../third_party/nvidia/backend/bin/ptxas ...
        copy /home/sasank/.triton/nvidia/bin/cuobjdump to /home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/../third_party/nvidia/backend/bin/cuobjdump ...
        copy /home/sasank/.triton/nvidia/bin/nvdisasm to /home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/../third_party/nvidia/backend/bin/nvdisasm ...
        running develop
        /usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
          warnings.warn(
        WARNING: The user site-packages directory is disabled.
        /usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
          warnings.warn(
        error: can't create or remove files in install directory
        
        The following error occurred while trying to add or remove files in the
        installation directory:
        
            [Errno 13] Permission denied: '/usr/local/lib/python3.10/dist-packages/test-easy-install-155469.write-test'
        
        The installation directory you specified (via --install-dir, --prefix, or
        the distutils default setting) was:
        
            /usr/local/lib/python3.10/dist-packages/
        
        Perhaps your account does not have write access to this directory?  If the
        installation directory is a system-owned directory, you may need to sign in
        as the administrator or "root" account.  If you do not have administrative

...

_ python setup.py develop did not run successfully.
│ exit code: 1
_─> [35 lines of output]
    copy /home/sasank/.triton/nvidia/bin/ptxas to /home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/../third_party/nvidia/backend/bin/ptxas ...
    copy /home/sasank/.triton/nvidia/bin/cuobjdump to /home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/../third_party/nvidia/backend/bin/cuobjdump ...
    copy /home/sasank/.triton/nvidia/bin/nvdisasm to /home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/../third_party/nvidia/backend/bin/nvdisasm ...
    running develop
    /usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    WARNING: The user site-packages directory is disabled.
    /usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    error: can't create or remove files in install directory
    
    The following error occurred while trying to add or remove files in the
    installation directory:
    
        [Errno 13] Permission denied: '/usr/local/lib/python3.10/dist-packages/test-easy-install-155469.write-test'
    
    The installation directory you specified (via --install-dir, --prefix, or
    the distutils default setting) was:
    
        /usr/local/lib/python3.10/dist-packages/


@whitneywhtsang
Copy link
Contributor

whitneywhtsang commented Jan 23, 2024

Are you building on a PVC machine with oneAPI 2024 installed?
Did you try building with virtualenv scripts/compile-triton.sh --venv?
Are you on branch llvm-target?

@chsasank
Copy link
Contributor Author

No, I am using arc 370m. Trying my luck 🤷🏽.

The issue is not the gpu (yet). Issue seems to be environment variables. Likely to do with venv. Yes, I am on the branch llvm-target.

Lemme retry with venv and report back results to you.

@chsasank
Copy link
Contributor Author

chsasank commented Jan 23, 2024

Hey, it actually compiles with venv! On my arc 370m system!

...

  adding '__editable__.triton-3.0.0.pth'
  creating '/tmp/pip-wheel-5a5dhopd/.tmp-zfq45dqu/triton-3.0.0-0.editable-cp310-cp310-linux_x86_64.whl' and adding '/tmp/tmpiolyuln_triton-3.0.0-0.editable-cp310-cp310-linux_x86_64.whl' to it
  adding 'triton-3.0.0.dist-info/METADATA'
  adding 'triton-3.0.0.dist-info/WHEEL'
  adding 'triton-3.0.0.dist-info/top_level.txt'
  adding 'triton-3.0.0.dist-info/RECORD'
  /tmp/pip-build-env-aum038bl/overlay/lib/python3.10/site-packages/setuptools/command/editable_wheel.py:347: InformationOnly: Editable installation.
  !!

          ********************************************************************************
          Please be careful with folders in your working directory with the same
          name as your package as they may take precedence during imports.
          ********************************************************************************

  !!
    with strategy, WheelFile(wheel_path, "w") as wheel_obj:
  Building editable for triton (pyproject.toml) ... done
  Created wheel for triton: filename=triton-3.0.0-0.editable-cp310-cp310-linux_x86_64.whl size=3081 sha256=507991c9d6791cbe4e2694f33fcdeeb2793f2f50d39d48179c7ea19c10877203
  Stored in directory: /tmp/pip-ephem-wheel-cache-pk1mumna/wheels/78/ec/db/266932a3b597eda259a1b7da6f4b004822eda0447b0d5fe1f3
Successfully built triton
Installing collected packages: triton, tomli, pyflakes, pycodestyle, pluggy, packaging, numpy, mccabe, isort, iniconfig, exceptiongroup, scipy, pytest, flake8, autopep8
  Attempting uninstall: triton
    Found existing installation: triton 3.0.0
    Uninstalling triton-3.0.0:
      Created temporary directory: /tmp/pip-uninstall-h63tty4s
      Removing file or directory /home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/lib/python3.10/site-packages/__editable__.triton-3.0.0.pth
      Removing file or directory /home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/lib/python3.10/site-packages/__editable___triton_3_0_0_finder.py
      Created temporary directory: /home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/lib/python3.10/site-packages/~_pycache__
      Removing file or directory /home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/lib/python3.10/site-packages/__pycache__/
      Created temporary directory: /home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/lib/python3.10/site-packages/~riton-3.0.0.dist-info
      Removing file or directory /home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/lib/python3.10/site-packages/triton-3.0.0.dist-info/
      Successfully uninstalled triton-3.0.0



  changing mode of /home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/bin/pyflakes to 755

  changing mode of /home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/bin/pycodestyle to 755



  changing mode of /home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/bin/f2py to 755


  changing mode of /home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/bin/isort to 755
  changing mode of /home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/bin/isort-identify-imports to 755




  changing mode of /home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/bin/py.test to 755
  changing mode of /home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/bin/pytest to 755

  changing mode of /home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/bin/flake8 to 755

  changing mode of /home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/bin/autopep8 to 755
Successfully installed autopep8-2.0.4 exceptiongroup-1.2.0 flake8-7.0.0 iniconfig-2.0.0 isort-5.13.2 mccabe-0.7.0 numpy-1.26.3 packaging-23.2 pluggy-1.3.0 pycodestyle-2.11.1 pyflakes-3.2.0 pytest-7.4.4 scipy-1.12.0 tomli-2.0.1 triton-3.0.0
Removed build tracker: '/tmp/pip-req-tracker-37hrwun3'

Lemme try importing triton:

$ cd /home/sasank/code/triton-build/intel-xpu-backend-for-triton
$ source .venv/bin/activate

# don't be in triton dir
$ cd ..

# install deps in venv
$ python -m pip install torch==2.1.0a0 torchvision==0.16.0a0 torchaudio==2.1.0a0 intel-extension-for-pytorch==2.1.10+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/


(triton) $ python3
>>> import triton
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/triton/__init__.py", line 8, in <module>
    from .runtime import (
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/triton/runtime/__init__.py", line 1, in <module>
    from .autotuner import (Autotuner, Config, Heuristics, OutOfResources, autotune, heuristics)
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/triton/runtime/autotuner.py", line 7, in <module>
    from ..testing import do_bench
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/triton/testing.py", line 7, in <module>
    from . import language as tl
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/triton/language/__init__.py", line 6, in <module>
    from .standard import (
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/triton/language/standard.py", line 3, in <module>
    from ..runtime.jit import jit
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/triton/runtime/jit.py", line 10, in <module>
    from ..runtime.driver import driver
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/triton/runtime/driver.py", line 1, in <module>
    from ..backends import backends
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/triton/backends/__init__.py", line 50, in <module>
    backends = _discover_backends()
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/triton/backends/__init__.py", line 44, in _discover_backends
    driver = _load_module(name, os.path.join(root, name, 'driver.py'))
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/triton/backends/__init__.py", line 12, in _load_module
    spec.loader.exec_module(module)
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/triton/backends/xpu/driver.py", line 10, in <module>
    import intel_extension_for_pytorch as ipex
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/lib/python3.10/site-packages/intel_extension_for_pytorch/__init__.py", line 123, in <module>
    from . import _inductor
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/lib/python3.10/site-packages/intel_extension_for_pytorch/_inductor/__init__.py", line 1, in <module>
    from . import xpu
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/lib/python3.10/site-packages/intel_extension_for_pytorch/_inductor/xpu/__init__.py", line 4, in <module>
    from .codegen.triton import XPUTritonScheduling
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/lib/python3.10/site-packages/intel_extension_for_pytorch/_inductor/xpu/codegen/triton.py", line 17, in <module>
    from torch._inductor.codegen.triton import (
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/lib/python3.10/site-packages/torch/_inductor/codegen/triton.py", line 26, in <module>
    from ..triton_heuristics import AutotuneHint
  File "/home/sasank/code/triton-build/intel-xpu-backend-for-triton/.venv/lib/python3.10/site-packages/torch/_inductor/triton_heuristics.py", line 44, in <module>
    from triton import Config
ImportError: cannot import name 'Config' from partially initialized module 'triton' (most likely due to a circular import) (/home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/triton/__init__.py)

Well something simple seems to be off.

@whitneywhtsang
Copy link
Contributor

Glad it works for you. I cannot reproduce the problem you see with importing Triton, is it reproducible by you? Can you share the output of your pip list?

@chsasank
Copy link
Contributor Author

Here’s the output of pip list of my env

Package                     Version            Editable project location
--------------------------- ------------------ ------------------------------------------------------------------
annotated-types             0.6.0
autopep8                    2.0.4
certifi                     2023.11.17
charset-normalizer          3.3.2
cmake                       3.28.1
exceptiongroup              1.2.0
filelock                    3.13.1
flake8                      7.0.0
fsspec                      2023.12.2
idna                        3.6
iniconfig                   2.0.0
intel-extension-for-pytorch 2.1.10+xpu
isort                       5.13.2
Jinja2                      3.1.3
MarkupSafe                  2.1.4
mccabe                      0.7.0
mpmath                      1.3.0
networkx                    3.2.1
ninja                       1.11.1.1
numpy                       1.26.3
packaging                   23.2
pillow                      10.2.0
pip                         22.0.2
pluggy                      1.3.0
psutil                      5.9.8
pycodestyle                 2.11.1
pydantic                    2.5.3
pydantic_core               2.14.6
pyflakes                    3.2.0
pytest                      7.4.4
requests                    2.31.0
scipy                       1.12.0
setuptools                  59.6.0
sympy                       1.12
tomli                       2.0.1
torch                       2.1.0a0+cxx11.abi
torchaudio                  2.1.0a0+cxx11.abi
torchvision                 0.16.0a0+cxx11.abi
triton                      3.0.0     
typing_extensions           4.9.0
urllib3                     2.1.0
wheel                       0.42.0

I will also share the data from 'Editable project location' of triton from above:

$ ls /home/sasank/code/triton-build/intel-xpu-backend-for-triton
bin  cmake  CMakeLists.txt  compile_commands.json  CONTRIBUTING.md  docs  include  lib  LICENSE  pyproject.toml  python  README.md  scripts  test  third_party  unittest  utils
$ ls /home/sasank/code/triton-build/intel-xpu-backend-for-triton/cmake/
FindLLVM.cmake  FindSPIRVToLLVMTranslator.cmake  llvm-hash.txt
$ ls /home/sasank/code/triton-build/intel-xpu-backend-for-triton/bin/
CMakeLists.txt            RegisterTritonDialects.h  triton-llvm-opt.cpp       triton-lsp.cpp            triton-opt.cpp            triton-reduce.cpp         
$ ls /home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/
build  examples  MANIFEST.in  pyproject.toml  setup.py  src  test  triton  triton.egg-info  tutorials
$ ls /home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/build/
cmake.linux-x86_64-cpython-3.10
$ ls /home/sasank/code/triton-build/intel-xpu-backend-for-triton/python/build/cmake.linux-x86_64-cpython-3.10/
bin          CMakeCache.txt  cmake_install.cmake    CTestTestfile.cmake    _deps  include  LLVMSPIRVLib.pc                test     third_party
build.ninja  CMakeFiles      compile_commands.json  DartConfiguration.tcl  docs   lib      tablegen_compile_commands.yml  Testing  unittest

@vlad-penkin vlad-penkin added the documentation Improvements or additions to documentation label Jan 24, 2024
@whitneywhtsang
Copy link
Contributor

I can reproduce the problem with importing Triton now. By doing import intel_extension_for_pytorch before import triton, the problem is resolved.
@quintinwang5 Can you please explain why is this needed after #239?

@quintinwang5
Copy link
Contributor

I can reproduce the problem with importing Triton now. By doing import intel_extension_for_pytorch before import triton, the problem is resolved. @quintinwang5 Can you please explain why is this needed after #239?

From the call stack above, you can see a circular import (triton -> IPEX -> triton). So we need to import IPEX before triton to break it.

@chsasank
Copy link
Contributor Author

Confirming that importing intel_extension_for_pytorch before triton doesn't throw error.

@etiotto
Copy link
Contributor

etiotto commented Jan 26, 2024

@chsasank given comment #335 (comment) can this issue be closed now ?

@chsasank
Copy link
Contributor Author

chsasank commented Jan 26, 2024 via email

@whitneywhtsang
Copy link
Contributor

whitneywhtsang commented Jan 26, 2024

Can we write a wiki or something detailing the process? I can volunteer.

That's appreciated, please open a pull request for reviewing when you are done.
@chsasank Can we track this effort in #327, and close this one?

@vlad-penkin vlad-penkin added this to the UT pass rate milestone Feb 9, 2024
pbchekin added a commit that referenced this issue Feb 12, 2024
As promised in #335, here are the instructions for building this fork of
triton. Can add a link to this doc in README if maintainers are ok :)

---------

Co-authored-by: Pavel Chekin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

5 participants