Quick Start

Installation

Installing the latest development version:

git clone https://github.com/Linaro/test-definitions
cd ./test-definitions
. ./automated/bin/setenv.sh
pip install -r ${REPO_PATH}/automated/utils/requirements.txt

If the above succeeds, try:

test-runner -h

Running test

Running test script

linux

cd ./automated/linux/smoke/
./smoke.sh

Skip package installation:

./smoke.sh -s true

android

cd ./automated/android/dd-wr-speed/
./dd-wr-speed.sh

Specify SN when more than one device connected:

./dd-wr-speed.sh -s "serial_no"

Specify other params:

./dd-wr-speed.sh -i "10" -p "/dev/block/mmcblk1p1"

Using test-runner

single test run

test-runner -d ./automated/linux/smoke/smoke.yaml

skip package install:

test-runner -d ./automated/linux/smoke/smoke.yaml -s

running test plan

Run a set of tests defined in agenda file:

test-runner -p ./plans/linux-example.yaml

Apply test plan overlay to skip, amend or add tests:

test-runner -p ./plans/linux-example.yaml -O test-plan-overlay-example.yaml

Collecting result

Using test script

Test script normally puts test log and parsed results to its own output directory. e.g.

automated/linux/smoke/output

Using test-runner

test-runner needs a separate directory outside the repo to store test and result files. The directory defaults to $HOME/output and can be changed with -o <dir>. test-runner converts test definition file to run.sh and then parses its stdout. Results will be saved to results.{json,csv} by test. e.g.

/root/output/smoke_9879e7fd-a8b6-472d-b266-a20b05d52ed1/result.csv

When using the same output directory for multiple tests, test-runner combines results from all tests and save them to ${OUTPUT}/results.{json,csv}. e.g.

/root/output/result.json