static stat analysis

[CorgiMan]

Part 1 for the cluster-test-orchestrator. One file contains the logic to scan a window of a file containing ringpop stats inserted with labels that indicate when the commands of a script are run. The other file contains analysis for: convergence time; number of converged checksums; counting of individual stats.

For reference, here is a small sample of what such a stat file would look like.

2016-06-16T16:22:57.795666563Z|ringpop.172_18_24_214_3000.make-alive:1|c
2016-06-16T16:22:57.795735453Z|ringpop.172_18_24_214_3000.membership-update.alive:1|c
2016-06-16T16:22:57.795883314Z|ringpop.172_18_24_214_3000.compute-checksum:0.016676|ms
2016-06-16T16:22:57.795897982Z|ringpop.172_18_24_214_3000.checksum:1565761574|g
2016-06-16T16:22:57.795919548Z|ringpop.172_18_24_214_3000.membership.checksum-computed:1|c
2016-06-16T16:22:57.795931518Z|ringpop.172_18_24_214_3000.changes.apply:1|g
2016-06-16T16:22:57.796057927Z|ringpop.172_18_24_214_3000.ring.checksum-computed:1|c
2016-06-16T16:22:57.796070776Z|ringpop.172_18_24_214_3000.ring.checksum:3321709178|g
2016-06-16T16:22:57.796082369Z|ringpop.172_18_24_214_3000.ring.server-added:1|c
2016-06-16T16:22:57.796091444Z|ringpop.172_18_24_214_3000.ring.server-removed:0|c
2016-06-16T16:22:57.796109218Z|ringpop.172_18_24_214_3000.ring.changed:1|c
2016-06-16T16:22:57.796119814Z|ringpop.172_18_24_214_3000.membership-set.alive:1|c
2016-06-16T16:22:57.796256752Z|ringpop.172_18_24_214_3000.updates:1|c
2016-06-16T16:22:57.796293995Z|ringpop.172_18_24_214_3000.max-piggyback:0|g

[motiejus]

Can you write unit tests?

Also, it's unclear how labels work. Looking at the code I see it, but it's neither in documentation of the pull request, nor there is a comment in the file.

Also, for every file, can you add a high-level summary on it's purpose, possibly examples?

I agree unit tests are not very visual, so it can be hard justification for this task. However, with doctest-style examples we could achieve two benefits: code snippets which show the functionality, and unit-like tests at the same time.

Thoughts?

[severb]

@motiejus I feel like testing the testing tool is maybe too much hustle?

[motiejus]

@severb I thought about it too, in this case, testing is not to assert correctness, but maintainability.

This is not a one-off code; I believe we will augment it and it will evolve. And having test + documentation (like discussed, can be in one!) will really save us a lot of time in the future.

[motiejus]

👍

[motiejus]

I really love your new tests. LGTM.