Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial commit for wf-bb tests #11

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions test/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# from Aare's PyCharm - Python env
__pycache__/
*.py[cod]
*$py.class
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
dump.rdb
.DS_Store
.idea

# Django stuff:
*.log
*.log.*
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# pyenv
.python-version

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.venv
env/
venv/
venv.bak/

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

.pnp.*
.yarn/*
!.yarn/patches
!.yarn/plugins
!.yarn/releases
!.yarn/sdks
!.yarn/versions
1 change: 1 addition & 0 deletions test/.tool-versions
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
nodejs 18.10.0
10 changes: 10 additions & 0 deletions test/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Installation

1. `git clone <project_from_git>`
2. `cd <project_from_git>`
3. start the bb you want to test with
`cd ../examples/camunda && ./test_entrypoint.sh && cd ../`
4. `cd test`
5. `asdf install`
6. `yarn i`
7. `yarn run test`
5 changes: 5 additions & 0 deletions test/features/list-processes.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Feature: View a list of workflow processes

Scenario: Another GovStack user wants to see which workflow processes are available
When we request a list of processes from the api
Then we receive 3 processes
16 changes: 16 additions & 0 deletions test/features/support/steps.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
const assert = require('assert');
const { When, Then } = require('@cucumber/cucumber');

// TODO: is this the best way to allow insecure ssl?
// TODO: and do we want to?

process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = 0;

When('we request a list of processes from the api', async function () {
this.whatIHeard = await fetch('https://localhost/processes');
});

Then('we receive {int} processes', async function (expectedResponse) {
const data = await this.whatIHeard.json();
assert.equal(data.length, expectedResponse);
});
12 changes: 12 additions & 0 deletions test/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"name": "workflow-bb-tests",
"version": "0.1.0",
"scripts": {
"setup": "",
"test": "cucumber-js",
"shutdown": ""
},
"devDependencies": {
"@cucumber/cucumber": "^8.6.0"
}
}
71 changes: 61 additions & 10 deletions test/plan.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,65 @@
# Test plan for the _____________ building block.
# Test Plan for the Workflow BB

1. a
2. b
3. c
1. All `examples` must be runnable via `docker compose up`.

## Notes
2. Tests in `/test` will include the expected HTTP status codes for the 5
current endpoints.

At least three levels of testing...
3. Via a basic SSH script (maybe github actions? maybe a circleCI config?) we
will recurse through the `examples` directory, run `docker compose up` for
each example, wait until we get a running set of containers, and then execute
the 5 api endpoint tests for _that_ example.

1. can the BB be deployed via docker-compose?
2. can the BB interact with the IM?
3. can an adaptor be deployed alongside it to test API compliance?
4. do the required APIs respond to the required inputs and provide the required responses?
The output will be a ✅ or ❌ for _each_ example, for each of the 5 tests.

## Comments

@farai - How do we test an actual workflow process? (do we set up each example
with a "fixture" - a standard workflow that must be configured for each example
application.)

@aare - Several steps... first check endpoints. Next, specify a series of
workflows that the engine must be able to perform.

@taylor - if we define what we expect the output of a business process to be,
given a certain input and theoretical steps, could we ask the "owner" of each
candidate application to "create a process with id N, which when executed does
X, Y, and Z".

**Next Step:** come up with the "canonical workflow processes" that we want to
test and document them in a BPMN diagram.

- @aare - should we use yaml? json? we must have a textual definition of these
canonical worfklows

## Levels of Tests

1. ensure that application can be launched via docker with a adaptor and a
security server
2. check that all defined API endpoints in the openAPI-spec.yaml return proper
response codes
3. check input/output/result for canonical workflow processes

## Implementation

# Start

## For each `example` -> if we could box this up.

---

### Start it via docker and make sure it is running

1. every `example` must have a single "start" script which does all the setup
and config to create a passing instance for the tests.

### Run all the cucumber tests against that image

1. localhost:443/api/processes
2. localhost:443/api/processes/13
3. localhost:443/api/processes_instances
4. ...blah

### Shut down the container

---
Loading