Merge pull request #384 from autonomio/dev

v.0.6.3 to Master

.github/ISSUE_TEMPLATE/bug-report.md

@@ -1,24 +1,38 @@
 ---
 name: Bug Report
-about: something needs fixing
+about: I want to report something that is broken
 
 ---
 
-Thanks so much for coming here to raise an issue. Please take a moment to 'check' the below boxes:
+Thank you very much for reporting a bug on Talos. Before you do, please go through the below checklist carefully and make sure to prepare your bug report in a way that facilitates effective handling of the matter.
 
-- [ ] I'm up-to-date with the latest release:
-
-      pip install -U talos
+#### 1) Confirm the below
 
-- [ ] I've confirmed that my Keras model works outside of Talos.
+- [ ] My Python version is 3.5 or higher
+- [ ] I have searched through the issues [Issues](https://github.com/autonomio/talos/issues) for a duplicate
+- [ ] I've tested that my Keras model works as a stand-alone
 
-If you still have an error, please submit **complete trace** and a code with: 
+#### 2) Include the output of:
 
-- output of shape for x and y e.g. (212,12)
-- Talos params dictionary
-- The Keras model wired for Talos
-- Description of extra variables in the model
+`talos.__version__`
 
-You can provide the code in pastebin / gist or any other format you like. 
+#### 3) Explain clearly what you expect to happen
+
+*A description of what you tried to do and what you thought should happen.*
+
+#### 4) Explain what actually happened
+
+*A description of the issue in Talos that you had identified*
+
+#### 5) Provide a code-complete reference
+
+- [ ] My bug report includes an input model
+- [ ] My bug report includes a parameter dictionary
+- [ ] My bug report includes a `Scan()` command
+- [ ] My bug report question includes a link to a sample of the data
+
+NOTE: If the data is sensitive and can't be shared, [create dummy data](https://scikit-learn.org/stable/modules/classes.html#samples-generator) that mimics it.
+
+**A self-contained Jupyter Notebook, Google Colab, or similar is highly preferred and will speed up helping you with your issue.**
 
 -------------------------------------------------------------------------

.github/ISSUE_TEMPLATE/feature-request.md

@@ -0,0 +1,29 @@
+---
+name: Feature Request
+about: I want to suggest a new feature
+
+---
+
+Thanks a lot for suggesting a feature to Talos. Please take a moment to go through the below checklist to provide context in a way that makes it easy to take your request forward.
+
+#### 1) I think Talos should add...
+
+*A description of the feature with as much detail as you believe is valuable*
+
+#### 2) Once implemented, I can see how this feature will...
+
+*Explain how researchers will benefit from having this feature in Talos**
+
+#### 3) I believe this feature is... (choose one)
+
+- [ ] ...critically important
+- [ ] ...must have
+- [ ] ...nice to have
+
+#### 4) Given the chance, I'd be happy to make a PR for this feature...
+
+- [ ] ...definitely
+- [ ] ...possibly
+- [ ] ...unlikely
+
+-------------------------------------------------------------------------

.github/ISSUE_TEMPLATE/support-request.md

@@ -0,0 +1,43 @@
+---
+name: Support
+about: I want to ask for support
+
+---
+
+First off, make sure to check your [support options](https://github.com/autonomio/talos#-how-to-get-support).
+
+The preferred way to resolve usage related matters is through the [docs](https://autonomio.github.io/talos/#/) which are maintained up-to-date with the latest version of Talos.
+
+If you do end up asking for support in a new issue, make sure to follow the below steps carefully.
+
+#### 1) Confirm the below
+
+- [ ] I have looked for an answer in the [Docs](https://autonomio.github.io/talos)
+- [ ] My Python version is 3.5 or higher
+- [ ] I have searched through the issues [Issues](https://github.com/autonomio/talos/issues) for a duplicate
+- [ ] I've tested that my Keras model works as a stand-alone
+
+#### 2) Include the output of:
+
+`talos.__version__`
+
+#### 3) Explain clearly what you are trying to achieve
+
+*A description of your specific use-case and what you hope to achieve with it*
+
+#### 4) Explain what you have already tried
+
+*An outline of the steps that you have already taken so far*
+
+#### 5) Provide a code-complete reference
+
+- [ ] My support question includes an input model
+- [ ] My support question includes a parameter dictionary
+- [ ] My support question includes a `Scan()` command
+- [ ] My support question includes a link to a sample of the data
+
+NOTE: If the data is sensitive and can't be shared, [create dummy data](https://scikit-learn.org/stable/modules/classes.html#samples-generator) that mimics it.
+
+**A self-contained Jupyter Notebook, Google Colab, or similar is highly preferred and will speed up helping you with your issue.**
+
+-------------------------------------------------------------------------

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,14 @@
+## You want to make a PR to Talos
+
+Thanks so much :) First, please take a moment to carefully check through
+the below items:
+
+- [ ] Changes have gone through actual use testing
+- [ ] [Docs](https://autonomio.github.io/talos) are updated where relevant
+- [ ] Code is [PEP8](https://www.python.org/dev/peps/pep-0008/)
+- [ ] All local tests have passed (run ./test.sh in /talos)
+- [ ] Travis tests have passed
+- [ ] Open a pull request
+- [ ] PR is to daily-dev branch
+
+<hr>

docs/Examples_Generator.md

@@ -18,7 +18,7 @@ NOTE: In this example we will be using the `SequenceGenerator()` available in Ta
 
 ### Loading Data
 ```python
-x, y = ta.templates.datasets.iris()
+x_train, y_train, x_val, y_val = talos.templates.datasets.mnist()
 ```
 `x` and `y` are expected to be either numpy arrays or lists of numpy arrays.
 
@@ -28,7 +28,7 @@ x, y = ta.templates.datasets.iris()
 def mnist_model(x_train, y_train, x_val, y_val, params):
 
     model = Sequential()
-    model.add(Conv2D(32, kernel_size=(3, 3), activation=params['activation'], input_shape=input_shape))
+    model.add(Conv2D(32, kernel_size=(3, 3), activation=params['activation'], input_shape=(28, 28, 1)))
     model.add(Flatten())
     model.add(Dense(128, activation=params['activation']))
     model.add(Dropout(params['dropout']))
@@ -41,11 +41,11 @@ def mnist_model(x_train, y_train, x_val, y_val, params):
     out = model.fit_generator(SequenceGenerator(x_train,
                                                 y_train,
                                                 batch_size=params['batch_size']),
-                              epochs=params['epochs'],
-                              validation_data=[x_val, y_val],
-                              callbacks=[],
-                              workers=4,
-                              verbose=0)
+                                                epochs=params['epochs'],
+                                                validation_data=[x_val, y_val],
+                                                callbacks=[],
+                                                workers=4,
+                                                verbose=0)
 
     return out, model
 ```

docs/Examples_Generator_Code.md

@@ -14,7 +14,7 @@ x, y = ta.templates.datasets.iris()
 def mnist_model(x_train, y_train, x_val, y_val, params):
 
     model = Sequential()
-    model.add(Conv2D(32, kernel_size=(3, 3), activation=params['activation'], input_shape=input_shape))
+    model.add(Conv2D(32, kernel_size=(3, 3), activation=params['activation'], input_shape=(28, 28, 1)))
     model.add(Flatten())
     model.add(Dense(128, activation=params['activation']))
     model.add(Dropout(params['dropout']))
@@ -27,11 +27,11 @@ def mnist_model(x_train, y_train, x_val, y_val, params):
     out = model.fit_generator(SequenceGenerator(x_train,
                                                 y_train,
                                                 batch_size=params['batch_size']),
-                              epochs=params['epochs'],
-                              validation_data=[x_val, y_val],
-                              callbacks=[],
-                              workers=4,
-                              verbose=0)
+                                                epochs=params['epochs'],
+                                                validation_data=[x_val, y_val],
+                                                callbacks=[],
+                                                workers=4,
+                                                verbose=0)
 
     return out, model
 

docs/Monitoring.md

@@ -28,5 +28,9 @@ Scan(print_params=True)
 Epoch-by-epoch training data is available during the experiment using the `ExperimentLogCallback`:
 
 ```python
-
+model.fit(...
+          callbacks=[talos.utils.ExperimentLogCallback('experiment_name', params)])
 ```
+Here `params` is the params dictionary in the `Scan()` input model. Both
+`experiment_name` and `experiment_id` should match with the current experiment,
+as otherwise

docs/Scan.md

@@ -19,7 +19,7 @@ Argument | Input | Description
 `y` | array or list of arrays | prediction outcome variable
 `params` | dict | the parameter dictionary
 `model` | function | the Keras model as a function
-`experiment_name` | str | Used for experiment log
+`experiment_name` | str | Used for creating the experiment logging folder
 `x_val` | array or list of arrays | validation data for x
 `y_val` | array or list of arrays | validation data for y
 `val_split` | float | validation data split ratio
@@ -38,7 +38,8 @@ Argument | Input | Description
 `minimize_loss` | bool | `reduction_metric` is a loss
 `disable_progress_bar` | bool | Disable live updating progress bar
 `print_params` | bool | Print each permutation hyperparameters
-`clear_tf_session` | bool | Clear backend session between permutations
+`clear_session` | bool | Clear backend session between permutations
+`save_weights` | bool | Save model weights (increases memory pressure for large models)
 
 NOTE: `boolean_limit` will only work if its the last argument in `Scan()` and the following bracket is on a newline:
 

docs/Templates.md

@@ -30,6 +30,7 @@ talos.templates.datasets.breast_cancer()
 - telco_churn
 - titanic
 - iris
+- mnist
 
 <hr>
 

setup.py

@@ -16,7 +16,7 @@
 URL = 'http://autonom.io'
 LICENSE = 'MIT'
 DOWNLOAD_URL = 'https://github.com/autonomio/talos/'
-VERSION = '0.6.2'
+VERSION = '0.6.3'
 
 try:
     from setuptools import setup

talos/__init__.py

@@ -23,7 +23,7 @@
                 templates.pipelines]
 
 keep_from_templates = ['iris', 'cervical_cancer', 'titanic', 'breast_cancer',
-                       'icu_mortality', 'telco_churn']
+                       'icu_mortality', 'telco_churn', 'mnist']
 
 for sub in template_sub:
     for key in list(sub.__dict__):
@@ -34,4 +34,4 @@
 del commands, scan, model, metrics, key
 del sub, keep_from_templates, template_sub, warnings
 
-__version__ = "0.6.2"
+__version__ = "0.6.3"

talos/autom8/automodel.py

@@ -1,6 +1,6 @@
 class AutoModel:
 
-    def __init__(self, task, metric=None):
+    def __init__(self, task, experiment_name, metric=None):
 
         '''
 
@@ -19,12 +19,16 @@ def __init__(self, task, metric=None):
             If 'continuous' then mae is used for metric, if 'binary',
             'multiclass', or 'multilabel', f1score is used. Accuracy is always
             used.
+        experiment_name | str | Must be same as in `Scan()`
         metric : None or list
             You can also input a list with one or more custom metrics or names
             of Keras or Talos metrics.
         '''
 
+        from talos.utils.experiment_log_callback import ExperimentLogCallback
+
         self.task = task
+        self.experiment_name = experiment_name
         self.metric = metric
 
         if self.task is not None:
@@ -36,6 +40,7 @@ def __init__(self, task, metric=None):
 
         # create the model
         self.model = self._create_input_model
+        self.callback = ExperimentLogCallback
 
     def _set_metric(self):
 
@@ -79,7 +84,8 @@ def _create_input_model(self, x_train, y_train, x_val, y_val, params):
         elif params['network'] == 'dense':
             model.add(Dense(params['first_neuron'],
                             input_dim=x_train.shape[1],
-                            activation='relu'))
+                            activation='relu',
+                            kernel_initializer=params['kernel_initializer']))
 
         model.add(Dropout(params['dropout']))
 
@@ -95,7 +101,8 @@ def _create_input_model(self, x_train, y_train, x_val, y_val, params):
                                                y_val)
 
         model.add(Dense(last_neuron,
-                        activation=activation))
+                        activation=activation,
+                        kernel_initializer=params['kernel_initializer']))
 
         # bundle the optimizer with learning rate changes
         from talos.model.normalizers import lr_normalizer
@@ -112,6 +119,7 @@ def _create_input_model(self, x_train, y_train, x_val, y_val, params):
                         batch_size=params['batch_size'],
                         epochs=params['epochs'],
                         verbose=0,
+                        callbacks=[self.callback(self.experiment_name, params)],
                         validation_data=[x_val, y_val])
 
         # pass the output to Talos

talos/autom8/autoparams.py

@@ -46,9 +46,9 @@ def __init__(self,
             self.params = {}
         else:
             self.params = params
-
         if auto:
             self._automated()
+            self.resample_params(4)
 
     def _automated(self, shapes='fixed'):
 
@@ -77,7 +77,7 @@ def _automated(self, shapes='fixed'):
         if self._network:
             self.networks()
         else:
-            self.params['network'] = 'dense'
+            self.params['network'] = ['dense']
         self.last_activations()
 
     def shapes(self, shapes='auto'):
@@ -186,7 +186,9 @@ def kernel_initializers(self, kernel_inits='auto'):
 
         if kernel_inits == 'auto':
             self._append_params('kernel_initializer',
-                                ['glorot_uniform', 'glorot_normal',
+                                ['uniform', 'normal', 'he_normal',
+                                 'he_uniform', 'lecun_normal',
+                                 'glorot_uniform', 'glorot_normal',
                                  'random_uniform', 'random_normal'])
         else:
             self._append_params('kernel_initializer', kernel_inits)