Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make autoencoders take a y #368

Merged
merged 1 commit into from
Aug 12, 2019
Merged

Make autoencoders take a y #368

merged 1 commit into from
Aug 12, 2019

Conversation

milesgranger
Copy link
Contributor

@milesgranger milesgranger commented Jul 5, 2019

Will close #364

Will also close #371 (because it will no longer be relevant)

Allows an 'autoencoder' to take X
via a Pipeline for example, but then train itself to output
the same unscaled values to match y. Models will no longer implicitly train themselves for X (which was often scaled via previous steps in the Pipeline) Therefore the Pipeline/
preprocessing steps before the model are part of the encoding
process.

  • Make all autoencoders to take a y
  • Update workflow generator (gordo-infrastructure) to set target_tag_list if not defined
  • Update server output (ie. inverse-transformed-model-output doesn't make sense anymore)
  • Abstract logic for "anomaly" into it's own model.
  • Update readthedocs / documentation
  • Update Grafana according to new keys

PRs depending on this

PR depends on:

@codecov
Copy link

codecov bot commented Jul 5, 2019

Codecov Report

Merging #368 into master will increase coverage by 0.16%.
The diff coverage is 91.63%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #368      +/-   ##
==========================================
+ Coverage   89.32%   89.49%   +0.16%     
==========================================
  Files          48       52       +4     
  Lines        2211     2275      +64     
==========================================
+ Hits         1975     2036      +61     
- Misses        236      239       +3
Impacted Files Coverage Δ
gordo_components/model/base.py 75% <ø> (ø) ⬆️
gordo_components/client/client.py 85.89% <ø> (ø) ⬆️
gordo_components/client/utils.py 84.21% <ø> (ø) ⬆️
...ponents/model/factories/feedforward_autoencoder.py 95.34% <100%> (+0.11%) ⬆️
gordo_components/builder/build_model.py 98.9% <100%> (-0.02%) ⬇️
gordo_components/model/anomaly/__init__.py 100% <100%> (ø)
gordo_components/model/utils.py 100% <100%> (ø)
gordo_components/model/models.py 97.57% <100%> (+3.36%) ⬆️
...rdo_components/model/factories/lstm_autoencoder.py 95.34% <100%> (ø) ⬆️
gordo_components/server/model_io.py 76.92% <100%> (-12.37%) ⬇️
... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 07adbbf...55f0340. Read the comment docs.

gordo_components/model/factories/lstm_autoencoder.py Outdated Show resolved Hide resolved
gordo_components/server/views/base.py Outdated Show resolved Hide resolved
gordo_components/server/views/base.py Show resolved Hide resolved
tests/gordo_components/model/test_lstm_autoencoder.py Outdated Show resolved Hide resolved
@milesgranger milesgranger changed the title [WIP] Make autoencoders take a y Make autoencoders take a y Aug 7, 2019
docs/components/server/anomaly.rst Outdated Show resolved Hide resolved
docs/general/endpoints.rst Show resolved Hide resolved
docs/general/endpoints.rst Outdated Show resolved Hide resolved
gordo_components/model/anomaly/diff.py Outdated Show resolved Hide resolved
gordo_components/model/anomaly/diff.py Show resolved Hide resolved
gordo_components/model/anomaly/diff.py Show resolved Hide resolved
gordo_components/model/anomaly/diff.py Show resolved Hide resolved
docs/general/endpoints.rst Outdated Show resolved Hide resolved
gordo_components/model/anomaly/diff.py Outdated Show resolved Hide resolved
gordo_components/model/anomaly/diff.py Outdated Show resolved Hide resolved
gordo_components/model/anomaly/diff.py Outdated Show resolved Hide resolved
gordo_components/model/anomaly/diff.py Outdated Show resolved Hide resolved
gordo_components/model/anomaly/diff.py Show resolved Hide resolved
@epa095 epa095 mentioned this pull request Aug 9, 2019
Copy link
Contributor

@epa095 epa095 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting closer!

Copy link
Contributor

@epa095 epa095 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!!!!

Previously, y was implicitly set as a copy of X. This seperates
that logic such that anomaly calculations are done in a scikit-learn
like model. Therefore additional changes to the server, client
and other components in the codebase are in this commit.
@milesgranger milesgranger merged commit 463992e into equinor:master Aug 12, 2019
@milesgranger milesgranger deleted the autoencoders-take-y branch August 12, 2019 13:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants