Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature] Rework git workflow process #11

Merged
merged 2 commits into from
Sep 20, 2024
Merged

[feature] Rework git workflow process #11

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
199130b
[fea] Rework git workflow
Sep 13, 2024
b6dd43a
Review changes
Sep 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
147 changes: 94 additions & 53 deletions .github/workflows/deploy-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,84 +1,125 @@
name: Upload Release package
on:
push:
branches:
- master

jobs:
deploy:
runs-on: macos-latest
build:
name: Build project (ubuntu)
runs-on: ubuntu-22.04
env:
DEPLOY_BUILD_PYTHON_VERSION: '3.12'
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v3
- name: Install Reindexer
- name: Checkout PyReindexer repo
uses: actions/checkout@v4
- name: Checkout Reindexer repo
uses: actions/checkout@v4
with:
repository: Restream/reindexer
path: reindexer
- name: Install dependencies
run: |
brew tap restream/reindexer
brew install reindexer
shell: bash
- name: Build PyReindexer
sudo apt-get update -y
cd reindexer
sudo bash dependencies.sh
- name: Build Reindexer
run: |
python3 -m pip install wheel
python3 setup.py sdist bdist_wheel
shell: bash
- name: Deploy PyReindexer To Pypi
env:
PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
cd reindexer
mkdir -p build && cd build
cmake ..
make -j4
sudo make install
- name: Set up Python ${{ env.DEPLOY_BUILD_PYTHON_VERSION }}
uses: actions/setup-python@v5
with:
python-version: ${{ env.DEPLOY_BUILD_PYTHON_VERSION }}
- name: Install Python dependencies
run: |
python3 -m pip install twine
python3 -m twine upload --verbose dist/* -u __token__ -p pypi-$PYPI_TOKEN
# wait for deploy finnishing on test pypi server
sleep 240
shell: bash
python -m pip install --upgrade pip
python -m pip install build
- name: Build PyReindexer
run: python -m build --sdist
- name: Store build artifacts
uses: actions/upload-artifact@v4
with:
name: build
path: dist/*

test-macos:
runs-on: macos-latest
needs: deploy
deploy:
name: Upload to PyPI (ubuntu)
runs-on: ubuntu-22.04
needs: build
environment:
name: pypi
url: https://pypi.org/p/pyreindexer/
steps:
- uses: actions/checkout@v3
- name: Install Reindexer
run: |
brew tap restream/reindexer
brew install reindexer
shell: bash
- name: Install PyReindexer
run: python3 -m pip install -vvv pyreindexer
shell: bash
- name: Test
run: |
cd /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyreindexer
$GITHUB_WORKSPACE/.github/workflows/test.sh
shell: bash
- name: Fetch build artifacts
uses: actions/download-artifact@v4
with:
name: build
path: dist/
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
verbose: true
# user: __token__ ## used as default
password: pypi-${{ secrets.PYPI_TOKEN }}
repository-url: https://pypi.org/legacy/
MadSchemas marked this conversation as resolved.
Show resolved Hide resolved

test-ubuntu:
test:
strategy:
matrix:
os: [ubuntu-20.04,ubuntu-22.04]
os: [ubuntu-22.04,ubuntu-24.04,macos-14]
fail-fast: false
runs-on: ${{matrix.os}}
env:
OS: ${{matrix.os}}
# Python 3.12 issue found on MasOS arm64 (darwin) runners, actions/setup-python@v5 incorrect venv setup
BUILD_PYTHON_VERSION: ${{ startsWith(matrix.os, 'macos') && '3.8' || '3.12' }}
needs: deploy
defaults:
run:
shell: bash
steps:
- uses: actions/checkout@v3
- name: Install Reindexer-Dev
- name: Checkout PyReindexer
uses: actions/checkout@v4
- name: Install Reindexer (MacOS)
if: startsWith(matrix.os, 'macos')
run: |
brew tap restream/reindexer
brew install reindexer
- name: Install Reindexer-Dev\Server (Ubuntu)
if: startsWith(matrix.os, 'ubuntu')
run: |
curl https://repo.reindexer.io/RX-KEY.GPG | sudo apt-key add
if [[ $OS == ubuntu-20.04 ]]; then
echo 'deb https://repo.reindexer.io/ubuntu-focal /' | sudo tee -a /etc/apt/sources.list
else
if [[ ${{matrix.os}} == ubuntu-22.04 ]]; then
echo 'deb https://repo.reindexer.io/ubuntu-jammy /' | sudo tee -a /etc/apt/sources.list
MadSchemas marked this conversation as resolved.
Show resolved Hide resolved
elif [[ ${{matrix.os}} == ubuntu-24.04 ]]; then
echo 'deb https://repo.reindexer.io/ubuntu-noble /' | sudo tee -a /etc/apt/sources.list
else
echo "${{matrix.os}} not supported"
exit 1
fi
sudo apt-get update -y
sudo apt-get install -y libunwind-dev
sudo apt-get install -y reindexer-dev
shell: bash
sudo apt-get install -y reindexer-server
- name: Set up Python ${{ env.BUILD_PYTHON_VERSION }}
uses: actions/setup-python@v5
env:
PIP_DISABLE_PIP_VERSION_CHECK: 1
with:
python-version: ${{ env.BUILD_PYTHON_VERSION }}
- name: Install PyReindexer
run: |
python3 -m pip install setuptools
python3 -m pip install -vvv pyreindexer
shell: bash
- name: Install Reindexer-Server
run: sudo apt-get install reindexer-server
shell: bash
python -m pip install --upgrade --disable-pip-version-check pip importlib_metadata setuptools wheel
python -m pip install -vvv pyreindexer
- name: Prepare Test Environment
run: python -m pip install -r requirements.txt
- name: Test
run: |
cd $(python3 -m site --user-site)/pyreindexer
cd $(python -c "import sysconfig; print(sysconfig.get_path('purelib'))")/pyreindexer
$GITHUB_WORKSPACE/.github/workflows/test.sh
shell: bash
156 changes: 98 additions & 58 deletions .github/workflows/test-deploy-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,87 +1,127 @@
name: Upload Release package to TestPyPI
on:
pull_request:
types: [opened, reopened]

jobs:
deploy:
runs-on: macos-latest
build:
name: Build project (ubuntu)
runs-on: ubuntu-22.04
env:
DEPLOY_BUILD_PYTHON_VERSION: '3.12'
defaults:
run:
shell: bash
steps:
- name: Install Reindexer
- name: Checkout PyReindexer repo
uses: actions/checkout@v4
- name: Checkout Reindexer repo
uses: actions/checkout@v4
with:
repository: Restream/reindexer
path: reindexer
- name: Install dependencies
run: |
brew tap restream/reindexer
brew install reindexer
shell: bash
- uses: actions/checkout@v3
- name: Build PyReindexer
run: python3 setup.py sdist bdist_wheel
shell: bash
- name: Deploy PyReindexer To Test-Pypi
env:
TESTPYPI_TOKEN: ${{ secrets.TESTPYPI_TOKEN }}
sudo apt-get update -y
cd reindexer
sudo bash dependencies.sh
- name: Build Reindexer
run: |
python3 -m pip install twine
python3 -m twine upload --verbose --repository testpypi dist/* -u __token__ -p pypi-$TESTPYPI_TOKEN
# wait for deploy finnishing on test pypi server
sleep 240
shell: bash
cd reindexer
mkdir -p build && cd build
cmake ..
make -j4
sudo make install
- name: Set up Python ${{ env.DEPLOY_BUILD_PYTHON_VERSION }}
uses: actions/setup-python@v5
with:
python-version: ${{ env.DEPLOY_BUILD_PYTHON_VERSION }}
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
python -m pip install build
- name: Build PyReindexer
run: python -m build --sdist
- name: Store build artifacts
uses: actions/upload-artifact@v4
with:
name: build
path: dist/*

test-macos:
runs-on: macos-latest
needs: deploy
deploy:
name: Upload to TestPyPI (ubuntu)
runs-on: ubuntu-22.04
needs: build
environment:
name: testpypi
url: https://test.pypi.org/p/pyreindexer/
steps:
- name: Install Reindexer
run: |
brew tap restream/reindexer
brew install reindexer
shell: bash
- uses: actions/checkout@v3
- name: Install PyReindexer
run: python3 -m pip install -vvv --index-url https://test.pypi.org/simple/ --no-deps pyreindexer
shell: bash
- name: Prepare Test Environment
run: python3 -m pip install pytest==6.2.5 delegator envoy pyhamcrest==2.0.2
shell: bash
- name: Test
run: |
cd /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyreindexer
$GITHUB_WORKSPACE/.github/workflows/test.sh
shell: bash
- name: Fetch build artifacts
uses: actions/download-artifact@v4
with:
name: build
path: dist/
- name: Publish package distributions to TestPyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
verbose: true
# user: __token__ ## used as default
password: pypi-${{ secrets.TESTPYPI_TOKEN }}
repository-url: https://test.pypi.org/legacy/

test-ubuntu:
test:
strategy:
matrix:
os: [ubuntu-20.04,ubuntu-22.04]
fail-fast: false
matrix:
os: [ubuntu-22.04,ubuntu-24.04,macos-14]
runs-on: ${{matrix.os}}
env:
OS: ${{matrix.os}}
# Python 3.12 issue found on MasOS arm64 (darwin) runners, actions/setup-python@v5 incorrect venv setup
# Error: The version '3.12' with architecture 'arm64' was not found for macOS 14.4. In real version 3.9+ not found
# https://github.com/actions/setup-python/issues/654
# https://github.com/actions/setup-python/issues/865
BUILD_PYTHON_VERSION: ${{ startsWith(matrix.os, 'macos') && '3.8' || '3.12' }}
needs: deploy
defaults:
run:
shell: bash
steps:
- name: Install Reindexer-Dev
- name: Checkout PyReindexer
uses: actions/checkout@v4
- name: Install Reindexer (MacOS)
if: startsWith(matrix.os, 'macos')
run: |
brew tap restream/reindexer
brew install reindexer
- name: Install Reindexer-Dev\Server (Ubuntu)
if: startsWith(matrix.os, 'ubuntu')
run: |
curl https://repo.reindexer.io/RX-KEY.GPG | sudo apt-key add
if [[ $OS == ubuntu-20.04 ]]; then
echo 'deb https://repo.reindexer.io/ubuntu-focal /' | sudo tee -a /etc/apt/sources.list
else
if [[ ${{matrix.os}} == ubuntu-22.04 ]]; then
echo 'deb https://repo.reindexer.io/ubuntu-jammy /' | sudo tee -a /etc/apt/sources.list
elif [[ ${{matrix.os}} == ubuntu-24.04 ]]; then
echo 'deb https://repo.reindexer.io/ubuntu-noble /' | sudo tee -a /etc/apt/sources.list
else
echo "${{matrix.os}} not supported"
exit 1
fi
sudo apt-get update -y
sudo apt-get install -y libunwind-dev
sudo apt-get install -y reindexer-dev
shell: bash
- uses: actions/checkout@v3
sudo apt-get install -y reindexer-server
- name: Set up Python ${{ env.BUILD_PYTHON_VERSION }}
uses: actions/setup-python@v5
env:
PIP_DISABLE_PIP_VERSION_CHECK: 1
with:
python-version: ${{ env.BUILD_PYTHON_VERSION }}
- name: Install PyReindexer
run: |
python3 -m pip install setuptools
python3 -m pip install -vvv --index-url https://test.pypi.org/simple/ --no-deps pyreindexer
shell: bash
- name: Install Reindexer-Server
run: sudo apt-get install reindexer-server
shell: bash
python -m pip install --upgrade --disable-pip-version-check pip importlib_metadata setuptools wheel
python -m pip install -vvv --index-url https://test.pypi.org/simple/ --no-deps pyreindexer
- name: Prepare Test Environment
run: python3 -m pip install pytest==6.2.5 delegator envoy pyhamcrest==2.0.2
shell: bash
run: python -m pip install -r requirements.txt
- name: Test
run: |
cd $(python3 -m site --user-site)/pyreindexer
cd $(python -c "import sysconfig; print(sysconfig.get_path('purelib'))")/pyreindexer
$GITHUB_WORKSPACE/.github/workflows/test.sh
shell: bash
Loading