Transition to v2.0 (#37)

* don't use unnecessary .keys() calls

* Fixed Tuples to be length enforced unlike lists. This allows one to define a set length for an iterative argument

* Fixed Tuples to be length enforced unlike lists. This allows one to define a set length for an iterative argument

* Removed legacy backend and API (dataclasses and custom typed interface). Updated markdown save call to support advanced types so that saved configurations are now valid spock config input files. Changed tuples to support length restrictions.

* updated versioneer

* updated versioneer pt.2

* updating GitPython calls to get the correct info and to check parent directories that was causing git errors

* added extra info to check for docker or k8s

* removed block dump notation

* added extra info write as comments to TOML. Fall back to no extra info for JSON and warn as comments are not allowed

README.md

@@ -9,24 +9,41 @@
 ## About
 
 `spock` is a framework that helps manage complex parameter configurations during research and development of Python 
-applications. `spock` let's you focus on the code you need to write instead of re-implementing boilerplate code like 
+applications. `spock` lets you focus on the code you need to write instead of re-implementing boilerplate code like 
 creating ArgParsers, reading configuration files, implementing traceability etc.
 
 In short, `spock` configurations are defined by simple and familiar class-based structures. This allows `spock` to 
 support inheritance, read from multiple markdown formats, and allow hierarchical configuration by composition.
 
 ## Quick Install
 
-Supports Python 3.6+
+Requires Python 3.6+
 
 ```bash
 pip install spock-config
 ```
 
-## What's New
+## Version(s)
+
+All prior versions are available on PyPi. If legacy API and backend support is needed please install a pre v2.0.0+ 
+version. We recommend refactoring your code to the new API and backend instead as legacy versions will be missing 
+recent features, bugfixes, and hotfixes.
+
+* v2.0.0+: Dropped support for legacy backend and API semantics
+* v1.1.0-v1.2.1: New API with support for legacy backend and legacy API semantics
+* v1.0.0: Legacy API and backend 
+
+## News
 
 See [Releases](https://github.com/fidelity/spock/releases) for more information.
 
+#### March 1st, 2021
+
+* Removed legacy backend and API (dataclasses and custom typed interface)
+* Updated markdown save call to support advanced types so that saved configurations are now valid `spock` config 
+  input files
+* Changed tuples to support length restrictions
+
 #### November 25th, 2020
 
 * Addition of [Advanced Types](docs/advanced_features/Advanced-Types.md)
@@ -47,11 +64,14 @@ and automatic defaults.
 * Easily Managed Parameter Groups: Each class automatically generates its own object within a single namespace.
 * Parameter Inheritance: Classes support inheritance allowing for complex configurations derived from a common base 
 set of parameters.
+* Complex Types: Nested Lists/Tuples, List/Tuples of Enum of `@spock` classes, List of repeated `@spock` classes
 * Multiple Configuration File Types: Configurations are specified from YAML, TOML, or JSON files.
-* Hierarchical Configuration: Composed from multiple configuration files via simple include statements.
+* Hierarchical Configuration: Compose from multiple configuration files via simple include statements.
 * Command-Line Overrides: Quickly experiment by overriding a value with automatically generated command line arguments.
 * Immutable: All classes are *frozen* preventing any misuse or accidental overwrites.
-* Tractability and Reproducibility: Save currently running parameter configuration with a single chained command. 
+* Tractability and Reproducibility: Save runtime parameter configuration to YAML, TOML, or JSON with a single chained 
+  command (with extra runtime info such as Git info, Python version, machine FQDN, etc). The saved markdown file can be
+  used as the configuration input to reproduce prior runtime configurations.
 
 #### Main Contributors
 

docs/Quick-Start.md

@@ -10,7 +10,8 @@ This is a quick and dirty guide to getting up and running with `spock`. Read the
 
 All examples can be found [here](https://github.com/fidelity/spock/blob/master/examples).
 
-Legacy documentation for the old API can be found [here](https://github.com/fidelity/spock/blob/master/docs/legacy)
+Legacy documentation for the old API (pre v2.0) can be 
+found [here](https://github.com/fidelity/spock/blob/master/docs/legacy)
 
 ### TL;DR
 1. Import the necessary components from `spock`

docs/basic_tutorial/Saving.md

@@ -2,7 +2,8 @@
 
 The current configuration of running python code can be saved to file by chaining the `save()` method before 
 the `generate()` call to the `ConfigArgBuilder` class. `spock` supports two ways to specify the path to write and the
-output file can be either YAML, TOML, or JSON (via the `file_extension` keyword argument).
+output file can be either YAML, TOML, or JSON (via the `file_extension` keyword argument). The saved markdown file can 
+be used as the configuration input to reproduce prior runtime configurations.
 
 ### Specify spock Special Parameter Type
 

setup.cfg

@@ -3,4 +3,4 @@ VCS = git
 style = pep440
 versionfile_source = spock/_version.py
 versionfile_build = spock/_version.py
-tag_prefix =
+tag_prefix=

spock/__init__.py

@@ -14,4 +14,4 @@
 __all__ = ["args", "builder", "config"]
 
 __version__ = get_versions()['version']
-del get_versions
+del get_versions

spock/_version.py

@@ -1,7 +1,3 @@
-# -*- coding: utf-8 -*-
-
-# Copyright 2019 FMR LLC <[email protected]>
-# SPDX-License-Identifier: Apache-2.0
 
 # This file helps to compute a version number in source trees obtained from
 # git-archive tarball (such as those provided by githubs download-from-tag
@@ -10,7 +6,7 @@
 # that just contains the computed version number.
 
 # This file is released into the public domain. Generated by
-# versioneer-0.18 (https://github.com/warner/python-versioneer)
+# versioneer-0.19 (https://github.com/python-versioneer/python-versioneer)
 
 """Git implementation of _version.py."""
 
@@ -45,7 +41,7 @@ def get_config():
     cfg = VersioneerConfig()
     cfg.VCS = "git"
     cfg.style = "pep440"
-    cfg.tag_prefix = "None"
+    cfg.tag_prefix = ""
     cfg.parentdir_prefix = "None"
     cfg.versionfile_source = "spock/_version.py"
     cfg.verbose = False
@@ -61,7 +57,7 @@ class NotThisMethod(Exception):
 
 
 def register_vcs_handler(vcs, method):  # decorator
-    """Decorator to mark a method as the handler for a particular VCS."""
+    """Create decorator to mark a method as the handler of a VCS."""
     def decorate(f):
         """Store f in HANDLERS[vcs][method]."""
         if vcs not in HANDLERS:
@@ -97,9 +93,7 @@ def run_command(commands, args, cwd=None, verbose=False, hide_stderr=False,
         if verbose:
             print("unable to find command, tried %s" % (commands,))
         return None, None
-    stdout = p.communicate()[0].strip()
-    if sys.version_info[0] >= 3:
-        stdout = stdout.decode()
+    stdout = p.communicate()[0].strip().decode()
     if p.returncode != 0:
         if verbose:
             print("unable to run %s (error)" % dispcmd)
@@ -169,6 +163,10 @@ def git_versions_from_keywords(keywords, tag_prefix, verbose):
         raise NotThisMethod("no keywords at all, weird")
     date = keywords.get("date")
     if date is not None:
+        # Use only the last line.  Previous lines may contain GPG signature
+        # information.
+        date = date.splitlines()[-1]
+
         # git-2.2.0 added "%cI", which expands to an ISO-8601 -compliant
         # datestamp. However we prefer "%ci" (which expands to an "ISO-8601
         # -like" string, which we must then edit to make compliant), because
@@ -304,6 +302,9 @@ def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command):
     # commit date: see ISO-8601 comment in git_versions_from_keywords()
     date = run_command(GITS, ["show", "-s", "--format=%ci", "HEAD"],
                        cwd=root)[0].strip()
+    # Use only the last line.  Previous lines may contain GPG signature
+    # information.
+    date = date.splitlines()[-1]
     pieces["date"] = date.strip().replace(" ", "T", 1).replace(" ", "", 1)
 
     return pieces
@@ -342,18 +343,18 @@ def render_pep440(pieces):
 
 
 def render_pep440_pre(pieces):
-    """TAG[.post.devDISTANCE] -- No -dirty.
+    """TAG[.post0.devDISTANCE] -- No -dirty.
 
     Exceptions:
-    1: no tags. 0.post.devDISTANCE
+    1: no tags. 0.post0.devDISTANCE
     """
     if pieces["closest-tag"]:
         rendered = pieces["closest-tag"]
         if pieces["distance"]:
-            rendered += ".post.dev%d" % pieces["distance"]
+            rendered += ".post0.dev%d" % pieces["distance"]
     else:
         # exception #1
-        rendered = "0.post.dev%d" % pieces["distance"]
+        rendered = "0.post0.dev%d" % pieces["distance"]
     return rendered
 
 
@@ -389,7 +390,7 @@ def render_pep440_old(pieces):
 
     The ".dev0" means dirty.
 
-    Eexceptions:
+    Exceptions:
     1: no tags. 0.postDISTANCE[.dev0]
     """
     if pieces["closest-tag"]:

spock/args.py

@@ -5,5 +5,5 @@
 
 """Handles import aliases to allow backwards compat with backends"""
 
-from spock.backend.dataclass.args import *
+# from spock.backend.dataclass.args import *
 from spock.backend.attr.typed import SavePath

spock/backend/__init__.py

@@ -8,4 +8,4 @@
 
 Please refer to the documentation provided in the README.md
 """
-__all__ = ["attr", "base", "dataclass"]
+__all__ = ["attr", "base"]

spock/backend/attr/saver.py

@@ -26,23 +26,68 @@ def __init__(self):
     def __call__(self, *args, **kwargs):
         return AttrSaver()
 
-    def _clean_up_values(self, payload, extra_info, file_extension):
+    def _clean_up_values(self, payload, file_extension):
+        # Dictionary to recursively write to
         out_dict = {}
-        for key, val in vars(payload).items():
-            # Skip comment append in JSON as it doesn't allow comments
-            if file_extension == '.json':
-                if isinstance(val, list):
-                    val = [attr.asdict(inner_val) for inner_val in val]
-                    out_dict.update({(key): val})
-                else:
-                    out_dict.update({key: attr.asdict(val)})
-            # Append comment tag to the base class and convert the spock class to a dict
-            else:
-                if isinstance(val, list):
-                    val = [attr.asdict(inner_val) for inner_val in val]
-                    out_dict.update({('# ' + key): val})
-                else:
-                    out_dict.update({('# ' + key): attr.asdict(val)})
+        # All of the classes are defined at the top level
+        all_spock_cls = set(vars(payload).keys())
+        out_dict = self._recursively_handle_clean(payload, out_dict, all_cls=all_spock_cls)
         # Convert values
-        clean_dict = self._clean_output(out_dict, extra_info)
+        clean_dict = self._clean_output(out_dict)
         return clean_dict
+
+    def _recursively_handle_clean(self, payload, out_dict, parent_name=None, all_cls=None):
+        """Recursively works through spock classes and adds clean data to a dictionary
+
+        Given a payload (Spockspace) work recursively through items that don't have parents to catch all
+        parameter definitions while correctly mapping nested class definitions to their base level class thus
+        allowing the output markdown to be a valid input file
+
+        *Args*:
+
+            payload: current payload (namespace)
+            out_dict: output dictionary
+            parent_name: name of the parent spock class if nested
+            all_cls: all top level spock class definitions
+
+        *Returns*:
+
+            out_dict: modified dictionary with the cleaned data
+
+        """
+        for key, val in vars(payload).items():
+            val_name = type(val).__name__
+            # This catches basic lists and list of classes
+            if isinstance(val, list):
+                # Check if each entry is a spock class
+                clean_val = []
+                repeat_flag = False
+                for l_val in val:
+                    cls_name = type(l_val).__name__
+                    # For those that are a spock class and are repeated (cls_name == key) simply convert to dict
+                    if (cls_name in all_cls) and (cls_name == key):
+                        clean_val.append(attr.asdict(l_val))
+                    # For those whose cls is different than the key just append the cls name
+                    elif cls_name in all_cls:
+                        # Change the flag as this is a repeated class -- which needs to be compressed into a single
+                        # k:v pair
+                        repeat_flag = True
+                        clean_val.append(cls_name)
+                    # Fall back to the passed in values
+                    else:
+                        clean_val.append(l_val)
+                # Handle repeated classes
+                if repeat_flag:
+                    clean_val = list(set(clean_val))[-1]
+                out_dict.update({key: clean_val})
+            # If it's a spock class but has a parent then just use the class name to reference the values
+            elif(val_name in all_cls) and parent_name is not None:
+                out_dict.update({key: val_name})
+            # Check if it's a spock class without a parent -- iterate the values and recurse to catch more lists
+            elif val_name in all_cls:
+                new_dict = self._recursively_handle_clean(val, {}, parent_name=key, all_cls=all_cls)
+                out_dict.update({key: new_dict})
+            # Either base type or no nested values that could be Spock classes
+            else:
+                out_dict.update({key: val})
+        return out_dict

spock/backend/attr/utils.py

@@ -135,6 +135,10 @@ def _recursive_list_to_tuple(value, typed, class_names):
     # from a composed payload
     if hasattr(typed, '__args__') and not isinstance(value, tuple) and not (isinstance(value, str)
                                                                             and value in class_names):
+        # Force those with origin tuple types to be of the defined length
+        if (typed.__origin__.__name__.lower() == 'tuple') and len(value) != len(typed.__args__):
+            raise ValueError(f'Tuple(s) use a fixed/defined length -- Length of the provided argument ({len(value)}) '
+                             f'does not match the length of the defined argument ({len(typed.__args__)})')
         # need to recurse before casting as we can't set values in a tuple with idx
         # Since it's generic it should be iterable to recurse and check it's children
         for idx, val in enumerate(value):