Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run E2E tests for custom streamlit version #76

Draft
wants to merge 19 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
39f98c1
Create .github/actions/run_e2e
sfc-gh-kbregula Aug 25, 2023
3e524ff
Disable a few tests
sfc-gh-kbregula Aug 25, 2023
67f6c11
Checkout all files to subdirectory
sfc-gh-kbregula Aug 25, 2023
dc427bd
fixup! Checkout all files to subdirectory
sfc-gh-kbregula Aug 25, 2023
7bc4227
Cache playwright and browsers
sfc-gh-kbregula Aug 25, 2023
da002c7
Modularize workflows
sfc-gh-kbregula Aug 25, 2023
ae66de2
fixup! Modularize workflows
sfc-gh-kbregula Aug 25, 2023
10beb53
Support development streamlit versions
sfc-gh-kbregula Aug 25, 2023
020b7f9
fixup! Support development streamlit versions
sfc-gh-kbregula Aug 25, 2023
1e63063
fixup! fixup! Support development streamlit versions
sfc-gh-kbregula Aug 25, 2023
c9648ca
fixup! fixup! fixup! Support development streamlit versions
sfc-gh-kbregula Aug 25, 2023
57fdc5c
fixup! fixup! fixup! fixup! Support development streamlit versions
sfc-gh-kbregula Aug 25, 2023
6b867b7
fixup! fixup! fixup! fixup! fixup! Support development streamlit vers…
sfc-gh-kbregula Aug 25, 2023
f7f3294
fixup! fixup! fixup! fixup! fixup! fixup! Support development streaml…
sfc-gh-kbregula Aug 25, 2023
8ae9747
fixup! fixup! fixup! fixup! fixup! fixup! fixup! Support development …
sfc-gh-kbregula Aug 25, 2023
bf42ed0
fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! Support devel…
sfc-gh-kbregula Aug 25, 2023
8a52bfe
Enable all tests
sfc-gh-kbregula Aug 25, 2023
d6a343c
Update dev.py
sfc-gh-kbregula Aug 28, 2023
de2c13f
Update dev.py
sfc-gh-kbregula Aug 28, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions .github/actions/build_component_wheels/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: 'Hello World'
description: 'Greet someone'
inputs:
custom_streamlit_component_lib_file:
required: false
description: ''
default: ''

outputs:
output_directory:
description: ''
value: ${{ steps.final_step.outputs.output_directory }}

runs:
using: "composite"
steps:
- name: Link develop version of streamlit-component-lib
if: inputs.custom_streamlit_component_lib_file != ''
working-directory: ${{ github.action_path }}/../../../
shell: bash
env:
STREAMLIT_COMPONENT_LIB_FILE: ${{ inputs.custom_streamlit_component_lib_file }}
run: |
find examples template template-reactless -name frontend -maxdepth 3 | while IFS= read -r line; do
(
cd "${line}";
npm install "${STREAMLIT_COMPONENT_LIB_FILE}"
)
done

- name: Install node dependencies
working-directory: ${{ github.action_path }}/../../../
shell: bash
run: ./dev.py all-npm-install

- name: Build frontend code
working-directory: ${{ github.action_path }}/../../../
shell: bash
run: ./dev.py all-npm-build

- name: Build wheel packages
working-directory: ${{ github.action_path }}/../../../
shell: bash
run: |
find examples template template-reactless -maxdepth 3 -name '__init__.py' |\
xargs -n 1 sed -i 's/_RELEASE = False/_RELEASE = True/';

./dev.py all-python-build-package

- name: Set outputs
working-directory: ${{ github.action_path }}/../../../
shell: bash
id: final_step
run: |
output_dir="$(readlink -e dist)"
echo "output_directory=${output_dir}" >> $GITHUB_OUTPUT
54 changes: 54 additions & 0 deletions .github/actions/build_streamlit_component_library/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: 'Hello World'
description: 'Greet someone'
inputs:
working_directory:
required: false
description: ""
default: "streamlit"

outputs:
output_file:
description: ''
value: ${{ steps.final_step.outputs.output_file }}

runs:
using: "composite"
steps:
- name: Check prerequisite
shell: bash
run: |
if ! command -v node > /dev/null
then
echo "Node is required to use this action"
exit 1
fi

- name: Checkout streamlit/streamlit
uses: actions/checkout@v3
with:
persist-credentials: false
repository: streamlit/streamlit
ref: develop
path: ${{ inputs.working_directory }}

- name: Install node dependencies for streamlit-component-lib
working-directory: ${{ inputs.working_directory }}/component-lib/
shell: bash
run: |
npm install -g yarn
yarn install

- name: Build streamlit-component-lib package
working-directory: ${{ inputs.working_directory }}/component-lib/
shell: bash
run: yarn run build && npm pack

- name: Link develop version of streamlit-component-lib
if: inputs.custom_streamlit_component_lib_file != ''
working-directory: ${{ inputs.working_directory }}/component-lib/
id: final_step
shell: bash
run: |
component_lib_tar_gz=$(find "./" -maxdepth 1 -name 'streamlit-component-lib-*.tgz')
component_lib_tar_gz=$(readlink -e "${component_lib_tar_gz}")
echo "output_file=${component_lib_tar_gz}" >> $GITHUB_OUTPUT
48 changes: 48 additions & 0 deletions .github/actions/run_e2e/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: 'Hello World'
description: 'Greet someone'
inputs:
python_version:
required: true
default: '3.8'
streamlit_version:
required: false
default: ''
streamlit_wheel_file:
required: false
default: ''

runs:
using: "composite"
steps:
- name: Check prerequisite
run: |
One and exactly one input is required: streamlit_version, streamlit_wheel_file
exit 1
if: inputs.streamlit_version == '' && inputs.streamlit_wheel_file == ''
shell: bash

- name: Build docker images
run: ./dev.py e2e-build-images "--streamlit-version=${{ env.STREAMLIT_VERSION }}" "--python-version=${{ env.PYTHON_VERSION }}"
working-directory: ${{ github.action_path }}/../../../
if: inputs.streamlit_version != ''
shell: bash
env:
STREAMLIT_VERSION: ${{ inputs.streamlit_version }}
PYTHON_VERSION: ${{ inputs.python_version }}

- name: Build docker images
run: ./dev.py e2e-build-images "--streamlit-wheel-file=${{ env.STREAMLIT_WHEEL_FILE }}" "--python-version=${{ env.PYTHON_VERSION }}"
working-directory: ${{ github.action_path }}/../../../
if: inputs.streamlit_wheel_file != ''
shell: bash
env:
STREAMLIT_WHEEL_FILE: ${{ inputs.streamlit_wheel_file }}
PYTHON_VERSION: ${{ inputs.python_version }}

- name: Run e2e tests
run: ./dev.py e2e-run-tests "--streamlit-version=${{ env.STREAMLIT_VERSION }}" "--python-version=${{ env.PYTHON_VERSION }}"
shell: bash
working-directory: ${{ github.action_path }}/../../../
env:
STREAMLIT_VERSION: ${{ inputs.streamlit_wheel_file == '' && inputs.streamlit_version || 'custom' }}
PYTHON_VERSION: ${{ inputs.python_version }}
146 changes: 93 additions & 53 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ jobs:
- uses: actions/checkout@v3
with:
persist-credentials: false
path: component-template

- name: Use Node.js ${{ env.NODE_VERSION }}
uses: actions/setup-node@v3
Expand All @@ -50,77 +51,116 @@ jobs:

- name: Check dependencies for examples
run: ./dev.py examples-check-deps
working-directory: ./component-template/

- name: Check e2e utils files
run: ./dev.py e2e-utils-check
working-directory: ./component-template/

- name: Checkout streamlit/streamlit
- name: Build Streamlit Component Library
uses: ./component-template/.github/actions/build_streamlit_component_library
if: matrix.component_lib_version == 'develop'
uses: actions/checkout@v3
id: streamlit_component_library

- name: Build components wheels
uses: ./component-template/.github/actions/build_component_wheels
id: component_wheels
with:
persist-credentials: false
repository: streamlit/streamlit
ref: develop
path: streamlit
custom_streamlit_component_lib_file: >-
${{ matrix.component_lib_version == 'develop' && steps.streamlit_component_library.outputs.output_file || '' }}

- name: Install node dependencies for streamlit-component-lib
if: matrix.component_lib_version == 'develop'
working-directory: ./streamlit/component-lib/
run: |
npm install -g yarn
yarn install
- name: Upload wheel packages
uses: actions/upload-artifact@v3
with:
name: all-wheel
path: ${{ steps.component_wheels.outputs.output_directory }}/*.whl
if-no-files-found: error

- name: Build streamlit-component-lib package
if: matrix.component_lib_version == 'develop'
working-directory: ./streamlit/component-lib/
run: yarn run build && npm pack
- name: Run E2E tests
if: matrix.node_version == '19.x'
uses: ./component-template/.github/actions/run_e2e
with:
python_version: ${{ env.PYTHON_VERSION }}
streamlit_version: ${{ env.STREAMLIT_VERSION }}

- name: Link develop version of streamlit-component-lib
if: matrix.component_lib_version == 'develop'
env:
COMPONENT_LIB_DIR: ${{ github.workspace }}/streamlit/component-lib/
run: |
component_lib_tar_gz=$(find "${COMPONENT_LIB_DIR}" -maxdepth 1 -name 'streamlit-component-lib-*.tgz')
component_lib_tar_gz=$(readlink -e "${component_lib_tar_gz}")
test-custom-streamlit-wheel:
runs-on: ubuntu-latest

strategy:
fail-fast: false
matrix:
node_version:
# # For details, see: https://nodejs.dev/en/about/releases/
# # Maintenance LTS. End Of Life: 2023-09-11
# - 16.x
# # Active LTS. End Of Life: 2025-04-30
# - 18.x
# Current version
- 19.x
component_lib_version:
- current
# - develop

find examples template template-reactless -name frontend -maxdepth 3 | while IFS= read -r line; do
(
cd "${line}";
npm install "${component_lib_tar_gz}"
)
done
env:
NODE_VERSION: ${{ matrix.node_version }}
PYTHON_VERSION: 3.8 # Oldest version supported by Streamlit
COMPONENT_LIB_VERSION: ${{ matrix.component_lib_version }}

- name: Install node dependencies
run: ./dev.py all-npm-install
name: Examples + Templates / node_version=${{ matrix.node_version }} / component_lib_version=${{ matrix.component_lib_version }}

- name: Build frontend code
run: ./dev.py all-npm-build
steps:
- uses: actions/checkout@v3
with:
persist-credentials: false
path: component-template

- name: Build wheel packages
run: |
find examples template template-reactless -maxdepth 3 -name '__init__.py' |\
xargs -n 1 sed -i 's/_RELEASE = False/_RELEASE = True/';
- name: Use Node.js ${{ env.NODE_VERSION }}
uses: actions/setup-node@v3
with:
node-version: ${{ env.NODE_VERSION }}

./dev.py all-python-build-package
- name: Check dependencies for examples
run: ./dev.py examples-check-deps
working-directory: ./component-template/

- name: Upload wheel packages
uses: actions/upload-artifact@v3
with:
name: all-wheel
path: dist/*.whl
if-no-files-found: error
- name: Check e2e utils files
run: ./dev.py e2e-utils-check
working-directory: ./component-template/

- name: Set up Docker Buildx
if: matrix.node_version == '19.x'
uses: docker/setup-buildx-action@7703e82fbced3d0c9eec08dff4429c023a5fd9a9 # v2.9.1
- name: Build Streamlit Component Library
uses: ./component-template/.github/actions/build_streamlit_component_library
if: matrix.component_lib_version == 'develop'
id: streamlit_component_library

- name: Build docker images
if: matrix.node_version == '19.x'
run: ./dev.py e2e-build-images "--streamlit-version=${{ env.STREAMLIT_VERSION }}" "--python-version=${{ env.PYTHON_VERSION }}"
- name: Build components wheels
uses: ./component-template/.github/actions/build_component_wheels
id: component_wheels
with:
custom_streamlit_component_lib_file: >-
${{ matrix.component_lib_version == 'develop' && steps.streamlit_component_library.outputs.output_file || '' }}

- name: Run e2e tests
if: matrix.node_version == '19.x'
run: ./dev.py e2e-run-tests "--streamlit-version=${{ env.STREAMLIT_VERSION }}" "--python-version=${{ env.PYTHON_VERSION }}"
- name: Upload wheel packages
uses: actions/upload-artifact@v3
with:
name: all-wheel
path: ${{ steps.component_wheels.outputs.output_directory }}/*.whl
if-no-files-found: error

- name: Download specific streamlit version
id: download_streamlit
run: |
set -x
cd "$(mktemp -d)"
pip download --no-deps "streamlit==1.23.0"
whl_file="$(readlink -e "$(find . -type f)")"
echo "whl_file=${whl_file}" >> $GITHUB_OUTPUT

- name: Run E2E tests
if: matrix.node_version == '19.x'
uses: ./component-template/.github/actions/run_e2e
with:
python_version: ${{ env.PYTHON_VERSION }}
streamlit_wheel_file: ${{ steps.download_streamlit.outputs.whl_file }}

build-cookiecutter:
runs-on: ubuntu-latest
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -68,3 +68,4 @@ template-reactless/**/package-lock.json
# VSCode
########################################################################
.vscode/
buildcontext/
Loading