CloudForms Applications Sanity Test Suite

Maintainer: Milan Falešník [email protected]

Documentation: http://mfalesni.github.io/cloud-app-sanity/

Introduction:

Test suite for validating deployed CloudForms applications. It can be also used for validating basically any kind of RHEL system. What can be validated? Current suite validates:

• filesystem (world-writable files, symlinks, ...)
• /etc/passwd (whether it contains only correct records for impotant accounts)
• /etc/inittab validity
• whether specified services are enabled in specified runlevels
• SSL key is enough long and does not use weak hashes
• sshd installed, activ and running
• /bin/bash and /bin/nologin in /etc/shells
• RPM packages signed, all files in packages are correct (hashes, symlinks, ...) and all binaries fortified
• System runs the latest kernel which is installed
• SElinux enabled and Enforcing
• and other (.bash_history, GRUB's menu.lst, ...)

Suite is compatible with these systems:

• RHEL6
• partially RHEL5 (it's not the main focus to support it, but anything I add into the suite runs also on RHEL5)

If you need some case-specific tuning, folder parametrized is used to store parametrization details which can be loaded using py.test switch --parametrize-file=somefile, which will load paraemtrized/somefile.yaml. If no parameter specified, file default.yaml is loaded.

Contribute!

This suite is not finished yet. If you have any idea which could extend the suite, feel free to fork, extend and make a pull request. Before making any changes, look into the test_* files to catch the basic principles and look also in the folder testsuite/plugins which is used to store system-manipulating functions to raise the level of abstraction.

And I really don't like the ninja comments, so please avoid them. If you see any ninja comment in my code, be sure it will disappear soon.

Writing tests:

This suite is based on py.test testing framework. There are no changes concerning test discovery. There is no need to write import pytest in each test module. There is an object Test available as a builtin. Some things from pytest are imported into it:

pytest.mark -> Test.Mark
pytest.fail -> Test.Fail
pytest.skip -> Test.Skip
pytest.fixture -> Test.Fixture
unittestzero.Assert -> Test.Assert
conftest -> Test.Fixtures

Take the already written tests as a reference.

Writing plugins:

This suite is equipped with a very simple and easy to use plugin system. Plugins are lazily loaded when called as Test.PLUGIN. If you call eg. Test.RPM, the lookup looks for a module RPM_plugin.py in plugins/ folder. If found, it imports it and looks for a varible export, which signifies the object to take as a plugin. Then the plugin is loaded and ready for use. Look in the plugins/ folder as a reference.

To-Do:

• extend, extend, ...

Prerequisities:

• python (v2.4+)
• python-setuptools
• make - For creating documentation

First two prerequisities are handled with the ./starter.sh script as they are necessary

Usage:

1. Checkout the git repository

git clone git://github.com/mfalesni/cloud-app-sanity.git

If you don't have git installed, use this command, which will download repository as a tarball and extract it

curl https://raw.github.com/mfalesni/cloud-app-sanity/master/tools/download_suite.sh | bash

2. Change into the repository directory

cd cloud-app-sanity

3. To run the suite using virtualenv and stuff, wrapper starter.sh is used. It wraps py.test call and passes all parameters to it.
4. Drink coffee

With current tests scheme, for example, one can run rpm tests by specifying keywordexpr (-k KEYWORD) as "RPM". Look into tests folder.