Apply Jitter to Load Test Setup

[rschwietzke]

Goal

Right now, XLT supports a user mode (traditional load test concept) and an arrival rate mode that is not feedback-based and therefore more realistic. Both modes will end up being relatively constant in the load picture at the end of the day, despite the ability to have a flexible load profile at hand. But this has to be set manually and is usually rather high-level picture (15 min or even more coarse).

So let's have the ability to introduce noise into the setup which can be defined easily.

Non-Goal

We would rather not create something fully random or partially intelligent. Keep in mind, that the setup is precalculated.

Requirements

Concept

Add a new definition besides arrivalRate, loadFactor, and users called jitter. Jitter is a value between 0.0 and 1.0, representing a percentage of 0.0 (default aka not applied) and 1.0. The jitter applied will be positive or negative.

The jitter represents a percentage deviation from the value that is valid at this point in time without a jitter applied. So when we would have 100 users and an arrivalRate of 10000, the jitter is 0.25. The min value will be 75u/7500 and the max will be 125u/12500.

You could also just see this as a jitter applied to the load function, which is by default 1.0, even when not set. We would just flex this to 0.75 to 1.25 as part of this definition.

In case we need a time resolution to apply these values, we would select the resolution we already have when defining the starting points for users over the course of the test runtime.

Extra Handling of Extremes

We will ensure that we will never get negative numbers or more than twice the initial value as a result of our calculation, aka due to rounding.

We will also round in favor of the base value aka 100 users and a jitter of 0.261 will be a min user count of 73.9 and max of 126.1, we round to 74 and 126.

Flexible Load Profile

The jitter should not be applied to the supporting points of a flexible load profile, such as: ....factor = 0/1 10m/10 20m/5 Only the points in between 0 10 min and 20 min will see the jitter applied. This will ensure we hit the defined values exactly and don't confuse the tester.

Random Generator

The random generator should use the default seed from the properties as the starting point per user series that is calculated (aka scenario) and an offset is added which is the hashcode of the scenario name. This will ensure different jitter per scenario but predictable jitter between runs (when desired).

We prefer a gaussian distribution to get a more natural behavior and less erratic behavior (but some will still be there). Make sure the gaussian function does not overshoot (hint gaussian is getting us numbers from -1 to 1!) https://docs.oracle.com/javase/8/docs/api/java/util/Random.html#nextGaussian--

Smoothing

We might get too much noise and up and downs, in that case, we might introduce a way to smooth things out. Imagine a sinus function applied to a time period where the sin max and min define the max and min deviation as mentioned above. The timer period can be set, such as 60 min or 5 min (the stretching of the sinus).

This is for a later iteration in case it is needed.