TimeSeriesDataset: support multiple aggregation methods

[epa095]

Now the TimeSeriesDataset supports an optional argument aggregation_methods
with default value mean. It can be either a single aggregation method or
a list of aggregation methods. Any aggregation method supported by pandas
is supported. If multiple aggregation methods are provided then the resulting
dataframe contains a multi-level column index with the tag-name as the top
level, and the aggregation method as the second level. If a single aggregation
method is provided then only the first level (tag-name) is used.

This closes #278

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (master@1c2c410). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master     #369   +/-   ##
=========================================
  Coverage          ?   89.32%           
=========================================
  Files             ?       48           
  Lines             ?     2211           
  Branches          ?        0           
=========================================
  Hits              ?     1975           
  Misses            ?      236           
  Partials          ?        0

Impacted Files	Coverage Δ
gordo_components/dataset/datasets.py	97.5% <100%> (ø)
gordo_components/dataset/base.py	85.71% <100%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1c2c410...ccb4562. Read the comment docs.

[flikka]

I like the idea, but I think the PR is WIP?

[epa095]

I think it is pretty good to go. Except I have not updated the documentation, I will do that as well.
But, are we good with how it works:

• If there is only a single resample method then the column names are the same as they were given, i.e. "Tag 1", "Tag 2" etc
• If there are several resample methods then the column names becomes "Tag 1_max", "Tag 1_mean", "Tag 2_max", "Tag 2_mean" etc. It is a bit ugly that it is different depending on how many aggregation methods we have, but it is maybe what the user would expect? Also it is backwards-compatible.

[epa095]

This PR does also not make sure that it actually works to do this in a workflow. I guess it makes sense to wait for #368 for that. It involves some dirty stuff, especially since the grafana dashboards will probably not work with several aggregation methods.

For now the full workflow works when selecting a single aggregation method (e.g. "max" instead of default "mean"), but the "multiple aggregation methods" is only useful when using the dataset locally.

[milesgranger]

In general, what do you think about using pd.MultiIndex for columns so we'd have things like this coming out:

                    tag1     tag2    
                    mean sum mean sum
2016-01-01 00:00:00    4  36    4  36
2016-01-01 01:00:00    9   9    9   9

or if a single agg:

                    tag1 tag2    
                    mean mean
2016-01-01 00:00:00    4    4
2016-01-01 01:00:00    9    9

Might result in more complex changes?

[epa095]

In general, what do you think about using pd.MultiIndex for columns so we'd have things like this coming out:

                    tag1     tag2    
                    mean sum mean sum
2016-01-01 00:00:00    4  36    4  36
2016-01-01 01:00:00    9   9    9   9

or if a single agg:

                    tag1 tag2    
                    mean mean
2016-01-01 00:00:00    4    4
2016-01-01 01:00:00    9    9

Might result in more complex changes?

I like it. I am a bit uncertain how big of an API change it is. Maybe not that big? At least it has a nicer consistency between the single and multiple aggregation methods.

[epa095]

So, now it is as follows: It uses multi-level dataframes when there are several aggregation methods, but the old-style single level ones in case of a single aggregation method. This is to make it backwards compatible, and make it a smaller PR. Two things:

• I must update the notebook (or just ditch that and just add documentation somewhere else. Proposal?)
• Maybe have an explicit tests in e.g. workflow generator which disallows multiple aggregation methods, since it is incompatible with the project at large (grafana particularly)?

[milesgranger]

Seems reasonable to me, I've added #382 so we can address using the multi level column dataframes and thus support multiple agg function throughout the workflow later on.

For the example, I think adding it in the https://github.com/equinor/gordo-test-project might be a good place?

[milesgranger]

Rebase, mahn.