Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs for version compatibility #692

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 27 additions & 13 deletions src/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,27 +5,41 @@ It hides the complexity of the generation process and minimizes the amount of co

## Install OpenVINO™ GenAI

> **NOTE**: Please make sure that you are following the versions compatibility rules, refer to the [OpenVINO™ GenAI Dependencies](#openvino-genai-dependencies) for more information.

The OpenVINO™ GenAI flavor is available for installation via Archive and PyPI distributions.
To install OpenVINO™ GenAI, refer to the [Install Guide](https://docs.openvino.ai/2024/get-started/install-openvino.html).

To build OpenVINO™ GenAI library from source, refer to the [Build Instructions](https://github.com/openvinotoolkit/openvino.genai/tree/releases/2024/2/src/docs/BUILD.md).
To build OpenVINO™ GenAI library from source, refer to the [Build Instructions](https://github.com/openvinotoolkit/openvino.genai/tree/releases/2024/3/src/docs/BUILD.md).

### OpenVINO™ GenAI Dependencies

OpenVINO™ GenAI depends on [OpenVINO](https://github.com/openvinotoolkit/openvino) and [OpenVINO Tokenizers](https://github.com/openvinotoolkit/openvino_tokenizers).

When installing OpenVINO™ GenAI from PyPi, the same versions of OpenVINO and OpenVINO Tokenizers are used (e.g. `openvino==2024.3.0` and `openvino-tokenizers==2024.3.0.0` are installed for `openvino-genai==2024.3.0`).
If you update one of the dependency packages (e.g. install `openvino-nightly`), versions might be incompatible due to different ABI and running OpenVINO GenAI can result in errors (e.g. `ImportError: libopenvino.so.2440: cannot open shared object file: No such file or directory`).
Having packages version in format `<MAJOR>.<MINOR>.<PATCH>.<REVISION>`, only `<REVISION>` part of the full version can be varied to ensure ABI compatibility, while changing `<MAJOR>`, `<MINOR>` or `<PATCH>` parts of the version might break ABI.

GenAI, Tokenizers, and OpenVINO wheels for Linux on PyPI are compiled with `_GLIBCXX_USE_CXX11_ABI=0` to cover a wider range of platforms. In contrast, C++ archive distributions for Ubuntu are compiled with `_GLIBCXX_USE_CXX11_ABI=1`. It is not possible to mix different Application Binary Interfaces (ABIs) because doing so results in a link error. This incompatibility prevents the use of, for example, OpenVINO from C++ archive distributions alongside GenAI from PyPI.

If you want to try OpenVINO GenAI with different dependencies versions (**not** prebuilt packages as archives or python wheels), build OpenVINO GenAI library from source.

## Usage

### Prerequisites

1. Installed OpenVINO™ GenAI

> If OpenVINO GenAI is installed via archive distribution or built from source, you will need to install additional python dependencies (e.g. `optimum-cli` for simplified model downloading and exporting, it's not required to install [./samples/requirements.txt](./samples/requirements.txt) for deployment if the model has already been exported):
>
> ```sh
> # (Optional) Clone OpenVINO GenAI repository if it does not exist
> git clone --recursive https://github.com/openvinotoolkit/openvino.genai.git
> cd openvino.genai
> # Install python dependencies
> python -m pip install ./thirdparty/openvino_tokenizers/[transformers] --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/pre-release
> python -m pip install --upgrade-strategy eager -r ./samples/requirements.txt
> ```
> To use OpenVINO GenAI with models that are already in OpenVINO format, no additional python dependencies are needed. To
> convert models with optimum-cli and to run the examples, install the dependencies in [./samples/requirements.txt](./samples/requirements.txt):
```sh
# (Optional) Clone OpenVINO GenAI repository if it does not exist
git clone --recursive https://github.com/openvinotoolkit/openvino.genai.git
cd openvino.genai
# Install python dependencies
python -m pip install ./thirdparty/openvino_tokenizers/[transformers] --pre --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
python -m pip install --upgrade-strategy eager -r ./samples/requirements.txt
```

2. A model in OpenVINO IR format

Expand Down Expand Up @@ -289,8 +303,8 @@ For more examples of how metrics are used, please refer to the Python [benchmark

## How It Works

For information on how OpenVINO™ GenAI works, refer to the [How It Works Section](https://github.com/openvinotoolkit/openvino.genai/tree/releases/2024/2/src/docs/HOW_IT_WORKS.md).
For information on how OpenVINO™ GenAI works, refer to the [How It Works Section](https://github.com/openvinotoolkit/openvino.genai/tree/releases/2024/3/src/docs/HOW_IT_WORKS.md).

## Supported Models

For a list of supported models, refer to the [Supported Models Section](https://github.com/openvinotoolkit/openvino.genai/tree/releases/2024/2/src/docs/SUPPORTED_MODELS.md).
For a list of supported models, refer to the [Supported Models Section](https://github.com/openvinotoolkit/openvino.genai/tree/releases/2024/3/src/docs/SUPPORTED_MODELS.md).
200 changes: 159 additions & 41 deletions src/docs/BUILD.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,85 +3,203 @@
> **NOTE**: There is a known Python API issue with `ov::Tensor`. The issue is reproduced when building OpenVINO GenAI from sources while using OpenVINO from archives. Using `ov::Tensor` with OpenVINO GenAI fails. Possible errors: `TypeError: generate(): incompatible function arguments.`, `TypeError: __init__(): incompatible constructor arguments.`, `TypeError: Unregistered type : ov::Tensor`.
The preferred approach is to build both OpenVINO and OpenVINO GenAI from sources using the same build environment. Or to install prebuilt OpenVINO GenAI from [distribution channels](https://docs.openvino.ai/2024/get-started/install-openvino.html).

## Build for Linux Systems
## Software Requirements

### Software Requirements
### Linux

- [CMake](https://cmake.org/download/) 3.23 or higher
- GCC 7.5 or higher
- Python 3.8 or higher
- Git
Wovchena marked this conversation as resolved.
Show resolved Hide resolved

### Build Instructions
### Windows

- [CMake](https://cmake.org/download/) 3.23 or higher
- Microsoft Visual Studio 2019 or higher, version 16.3 or later
- Python 3.8 or higher
- Git for Windows
Wovchena marked this conversation as resolved.
Show resolved Hide resolved
- [NSIS](https://sourceforge.net/projects/nsis/)

### macOS

- [CMake](https://cmake.org/download/) 3.23 or higher
- [brew](https://brew.sh/) package manager to install additional dependencies:
```sh
brew install coreutils scons
```
- Clang compiler and other command line tools from Xcode 10.1 or higher:
```sh
xcode-select --install
```
- Python 3.8 or higher
- Git


## Build Instructions

### Build OpenVINO, OpenVINO Tokenizers, and OpenVINO GenAI From Source

1. Build and install OpenVINO from sources following the [instructions](https://github.com/openvinotoolkit/openvino/wiki#how-to-build).
The path to the openvino install directory is referred as <INSTALL_DIR> throughout the document.
The path to the OpenVINO install directory is referred as `<INSTALL_DIR>` throughout the document.
2. Clone OpenVINO GenAI repository and init submodules:
```sh
git clone --recursive https://github.com/openvinotoolkit/openvino.genai.git
cd openvino.genai
```
3. Build the project:
3. Set up the environment:

#### Option 1 - using OpenVINO `setupvars` script:

Linux and macOS:
```sh
source <INSTALL_DIR>/setupvars.sh
cmake -DCMAKE_BUILD_TYPE=Release -S ./ -B ./build/
cmake --build ./build/ --config Release --target package -j
cmake --install ./build/ --config Release --prefix ov
```

## Build for Windows Systems
Windows Command Prompt:
```cmd
call <INSTALL_DIR>\setupvars.bat
```

### Software Requirements
Windows PowerShell:
```cmd
. <INSTALL_DIR>/setupvars.ps1
```

- [CMake](https://cmake.org/download/) 3.23 or higher
- Microsoft Visual Studio 2019 or higher, version 16.3 or later
- Python 3.8 or higher
- Git for Windows
#### Option 2 - setting environment variables manually:

### Build Instructions
Linux:
```sh
export OpenVINO_DIR=<INSTALL_DIR>/runtime
export PYTHONPATH=<INSTALL_DIR>/python:./build/:$PYTHONPATH
export LD_LIBRARY_PATH=<INSTALL_DIR>/runtime/lib/intel64:$LD_LIBRARY_PATH
```

1. Build and install OpenVINO from sources following the [instructions](https://github.com/openvinotoolkit/openvino/wiki#how-to-build)
The path to the openvino install directory is referred as <INSTALL_DIR> throughout the document.
2. Clone OpenVINO GenAI repository and init submodules:
macOS:
```sh
git clone --recursive https://github.com/openvinotoolkit/openvino.genai.git
cd openvino.genai
export OpenVINO_DIR=<INSTALL_DIR>/runtime
export PYTHONPATH=<INSTALL_DIR>/python:./build/:$PYTHONPATH
export DYLD_LIBRARY_PATH=<INSTALL_DIR>/runtime/lib/intel64:$LD_LIBRARY_PATH
```

Windows Command Prompt:
```cmd
set OpenVINO_DIR=<INSTALL_DIR>\runtime
set PYTHONPATH=<INSTALL_DIR>\python;%CD%\build;%PYTHONPATH%
set OPENVINO_LIB_PATHS=<INSTALL_DIR>\bin\intel64\Release;%OPENVINO_LIB_PATHS%
set PATH=%OPENVINO_LIB_PATHS%;%PATH%
```
3. Build the project:

Windows PowerShell:
```sh
$env:OpenVINO_DIR = "<INSTALL_DIR>\runtime"
$env:PYTHONPATH = "<INSTALL_DIR>\python;$PWD\build;$env:PYTHONPATH"
$env:OPENVINO_LIB_PATHS = "<INSTALL_DIR>\bin\intel64\Release;$env:OPENVINO_LIB_PATHS"
$env:PATH = "$env:OPENVINO_LIB_PATHS;$env:PATH"
```

4. Build the project:
```sh
call <INSTALL_DIR>\setupvars.bat
cmake -DCMAKE_BUILD_TYPE=Release -S ./ -B ./build/
cmake --build ./build/ --config Release --target package -j
cmake --install ./build/ --config Release --prefix ov
cmake --build ./build/ --config Release -j
```

## Build for macOS Systems
5. Install OpenVINO GenAI:

### Software Requirements
#### Option 1 - using cmake:

The following command will store built OpenVINO GenAI artifacts along with OpenVINO in `<INSTALL_DIR>`:

- [CMake](https://cmake.org/download/) 3.23 or higher
- [brew](https://brew.sh/) package manager to install additional dependencies:
```sh
brew install coreutils scons
cmake --install ./build/ --config Release --prefix <INSTALL_DIR>
```
- Clang compiler and other command line tools from Xcode 10.1 or higher:

#### Option 2 - setting paths to built OpenVINO GenAI artifacts manually:

The path to the OpenVINO GenAI root directory is referred as `<GENAI_ROOT_DIR>` throughout the document.

Linux:
```sh
xcode-select --install
export PYTHONPATH=<GENAI_ROOT_DIR>/build/:$PYTHONPATH
export LD_LIBRARY_PATH=<GENAI_ROOT_DIR>/build/openvino_genai/:$LD_LIBRARY_PATH
```
- Python 3.8 or higher

### Build Instructions
macOS:
```sh
export PYTHONPATH=<GENAI_ROOT_DIR>/build:$PYTHONPATH
export DYLD_LIBRARY_PATH=<GENAI_ROOT_DIR>/build/openvino_genai:$DYLD_LIBRARY_PATH
```

1. Build and install OpenVINO from sources following the [instructions](https://github.com/openvinotoolkit/openvino/wiki#how-to-build)
The path to the openvino install directory is referred as <INSTALL_DIR> throughout the document.
2. Clone OpenVINO GenAI repository and init submodules:
Windows Command Prompt:
```cmd
set PYTHONPATH=<GENAI_ROOT_DIR>\build;%PYTHONPATH%
set PATH=<GENAI_ROOT_DIR>\build\openvino_genai;%PATH%
```

Windows PowerShell:
```sh
$env:PYTHONPATH = "<GENAI_ROOT_DIR>\build;$env:PYTHONPATH"
$env:PATH = "<GENAI_ROOT_DIR>\build\openvino_genai;$env:PATH"
```

To optimize the package size, you can reduce the ICU (International Components for Unicode) data size when OpenVINO Tokenizers are built as a submodule of OpenVINO GenAI.
For more information please refer to the [OpenVINO Tokenizers instructions](https://github.com/openvinotoolkit/openvino_tokenizers?tab=readme-ov-file#reducing-the-icu-data-size).


### Build OpenVINO GenAI Wheel

1. Clone OpenVINO GenAI repository and init submodules:
```sh
git clone --recursive https://github.com/openvinotoolkit/openvino.genai.git
cd openvino.genai
```
3. Build the project:
2. Set up the environment:
- Option 1 - using OpenVINO `setupvars.sh` script:
```sh
source <INSTALL_DIR>/setupvars.sh
```
- Option 2 - setting environment variables manually:
```sh
export OpenVINO_DIR=<INSTALL_DIR>/runtime
export PYTHONPATH=<INSTALL_DIR>/python:./build/:$PYTHONPATH
export LD_LIBRARY_PATH=<INSTALL_DIR>/runtime/lib/intel64:$LD_LIBRARY_PATH
```
3. Upgrade pip to ensure you have the latest version:
```sh
source <INSTALL_DIR>/setupvars.sh
cmake -DCMAKE_BUILD_TYPE=Release -S ./ -B ./build/
cmake --build ./build/ --config Release --target package -j
cmake --install ./build/ --config Release --prefix ov
python -m pip install --upgrade pip
```
4. Build the wheel in the `dist` directory:
```sh
python -m pip wheel . -w dist/ --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/pre-release
```

### Install OpenVINO GenAI From Source

1. Clone OpenVINO GenAI repository and init submodules:
```sh
git clone --recursive https://github.com/openvinotoolkit/openvino.genai.git
cd openvino.genai
```
2. Set up the environment:
- Option 1 - using OpenVINO `setupvars.sh` script:
```sh
source <INSTALL_DIR>/setupvars.sh
```
- Option 2 - setting environment variables manually:
```sh
export OpenVINO_DIR=<INSTALL_DIR>/runtime
export PYTHONPATH=<INSTALL_DIR>/python:./build/:$PYTHONPATH
export LD_LIBRARY_PATH=<INSTALL_DIR>/runtime/lib/intel64:$LD_LIBRARY_PATH
Wovchena marked this conversation as resolved.
Show resolved Hide resolved
```
3. Upgrade pip to ensure you have the latest version:
```sh
python -m pip install --upgrade pip
```
4. Install the package directly from source:
```sh
python -m pip install .
```
5. To verify the installation, run a simple Python script:
```python
import openvino_genai
print(openvino_genai.__version__)
```
Loading