Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(GHA): Run Java CI testing for MPL Latest Release #1605

Merged
merged 31 commits into from
Feb 7, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
126 changes: 126 additions & 0 deletions .github/workflows/ci_test_latest_released_mpl_java.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
# This workflow is for testing that the latest released version
# of the MPL is compatible with the current DB-ESDK Head
name: Test Latest Released MPL Java with DB-ESDK HEAD

on:
schedule:
- cron: "00 16 * * 1-5"
josecorella marked this conversation as resolved.
Show resolved Hide resolved
workflow_dispatch: # allows triggering this manually through the Actions UI
inputs:
run_test_vectors:
description: "Run Test Vectors?"
required: false
default: true
type: boolean

jobs:
getVersion:
# Don't run the cron builds on forks
if: github.event_name != 'schedule' || github.repository_owner == 'aws'
uses: ./.github/workflows/dafny_version.yml
getVerifyVersion:
if: github.event_name != 'schedule' || github.repository_owner == 'aws'
uses: ./.github/workflows/dafny_verify_version.yml
getMplDependencyJavaVersion:
if: github.event_name != 'schedule' || github.repository_owner == 'aws'
uses: ./.github/workflows/mpl_dependency_java_version.yml
testJava:
needs: [getVersion, getMplDependencyJavaVersion]
strategy:
max-parallel: 1
matrix:
java-version: [17]
os: [ubuntu-22.04]
runs-on: ${{ matrix.os }}
permissions:
id-token: write
contents: read
steps:
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-region: us-west-2
role-to-assume: arn:aws:iam::370957321024:role/GitHub-CI-DDBEC-Dafny-Role-us-west-2
role-session-name: DDBEC-Dafny-Java-Tests

- uses: actions/checkout@v3
with:
submodules: recursive

- name: Setup Java ${{ matrix.java-version }}
uses: actions/setup-java@v4
with:
distribution: "corretto"
java-version: ${{ matrix.java-version }}

- name: Setup Dafny
uses: dafny-lang/[email protected]
with:
dafny-version: ${{ needs.getVersion.outputs.version }}

- name: Regenerate code using smithy-dafny if necessary
if: ${{ inputs.regenerate-code }}
uses: ./.github/actions/polymorph_codegen
with:
dafny: ${{ env.DAFNY_VERSION }}
library: DynamoDbEncryption
diff-generated-code: false
update-and-regenerate-mpl: true

# The following two steps: "Build and deploy to maven local" and "Run Extensive Tests"
# mimic the tests in ./codebuild/staging/release-staging.yml
- name: Build and deploy to maven local
shell: bash
working-directory: ./DynamoDbEncryption
run: |
# Run transpile by itself. We don't want to locally build the MPL because
# we want to verify that the version pulled down from maven works correctly
make transpile_implementation_java
make transpile_test_java
make mvn_local_deploy
make test_java
- name: Run Extensive Tests
working-directory: ./DynamoDbEncryption
run: |
gradle -p runtimes/java clean
gradle -p runtimes/java test
# This makes sure that we are using the correct MPL version to test the DB-ESDK.
# If this contains a SNAPSHOT version, this will fail because'
# we are NOT building the MPL recursively but pulling from Maven.
- name: Update project.properties to use the correct MPL version (from project.properties in DB-ESDK)
working-directory: ./submodules/MaterialProviders/
run: |
sed "s/mplVersion=.*/mplVersion=${{needs.getMplDependencyJavaVersion.outputs.version}}/g" project.properties > project.properties2; mv project.properties2 project.properties
josecorella marked this conversation as resolved.
Show resolved Hide resolved
# The following three steps: "Transpile MPL Test Vectors without recursively building the MPL",
# "Run Test Vectors", and "Test Examples" mimic the tests in ./codebuild/staging/validate-staging.yml
- name: Transpile MPL Test Vectors without recursively building the MPL
working-directory: ./submodules/MaterialProviders/TestVectorsAwsCryptographicMaterialProviders
run: |
# Run transpile by itself. We don't want to locally build the MPL because
# we want to verify that the version pulled down from maven works correctly
make transpile_implementation_java
make transpile_test_java
make mvn_local_deploy
- name: Run Test Vectors
if: github.event_name == 'schedule' || (github.event_name == 'workflow_dispatch' && ${{inputs.run_test_vectors}})
working-directory: ./TestVectors
run: |
# Spin up ddb local
docker run --name dynamodb -d -p 8000:8000 amazon/dynamodb-local -jar DynamoDBLocal.jar -port 8000 -inMemory -cors *
# Run transpile by itself so we don't locally build the MPL.
make transpile_implementation_java
make transpile_test_java
gradle -p runtimes/java runTests
- name: Test Examples
working-directory: ./Examples
run: |
# Run Simple Examples
gradle -p runtimes/java/DynamoDbEncryption test
# Run Migration Examples
gradle -p runtimes/java/Migration/PlaintextToAWSDBE test
gradle -p runtimes/java/Migration/DDBECToAWSDBE test
25 changes: 25 additions & 0 deletions .github/workflows/mpl_dependency_java_version.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# This workflow reads the project.properties
# into the environment variables
# and then creates an output variable for `mplDependencyJavaVersion`
name: MPL Dependency Java Version

on:
workflow_call:
outputs:
version:
description: "The MPL Dependency Java version from project.properties"
value: ${{ jobs.getMplDependencyJavaVersion.outputs.version }}

jobs:
getMplDependencyJavaVersion:
runs-on: ubuntu-22.04
outputs:
version: ${{ steps.read_property.outputs.mplDependencyJavaVersion }}
steps:
- uses: actions/checkout@v4
- name: Read version from Properties-file
id: read_property
uses: christian-draeger/[email protected]
with:
path: "./project.properties"
properties: "mplDependencyJavaVersion"
6 changes: 3 additions & 3 deletions .github/workflows/mpl_head_version.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# This workflow reads the project.properties
# This workflow reads the project.properties in the MPL submodule
# into the environment variables
# and then creates an output variable for `dafnyVerifyVersion `
# and then creates an output variable for `mplVersion`
name: MPL HEAD Version

on:
Expand All @@ -13,7 +13,7 @@ on:
type: string
outputs:
version:
description: "The dafny version for verify"
description: "The MPL version"
value: ${{ jobs.getMplHeadVersion.outputs.version }}

jobs:
Expand Down
2 changes: 0 additions & 2 deletions codebuild/staging/release-staging.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,6 @@ phases:
- aws sts get-caller-identity
build:
commands:
# Validate the MPL submodule points to the correct release
- scripts/validate-mpl-submodule.sh
# Build and deploy to maven local
- cd DynamoDbEncryption/
- make transpile_implementation_java
Expand Down
Loading