Merge pull request #1909 from nuagenetworks/dev

Release 5.0.0

.gitignore

@@ -17,6 +17,8 @@ deployments/*
 src/inventory/*
 hosts
 .vscode
+.ssh
+images/
 venv
 docker/*
 .pytest_cache

Documentation/AWS.md

@@ -97,7 +97,7 @@ The CIDRs for the VPC, WAN interface, LAN interface and private subnet must be s
 ## 6. Deploy Components
 After you have set up the environment and configured your components, you can use MetroAE to deploy your components with a single command.
 
-    metroae install everything
+    metroae-container install everything
 
 Alternatively, you can deploy individual components or perform individual tasks such as predeploy, deploy and postdeploy. See [DEPLOY.md](DEPLOY.md) for details.
 ## Questions, Feedback, and Contributing  

Documentation/BACKUP_RESTORE.md

@@ -13,11 +13,11 @@ Ensure that all components you wish to have backed up are specified under your d
 
 To perform a backup of all supported components issue:
 
-    metroae backup
+    metroae-container backup
 
 To perform a backup of a specific component issue:
 
-    metroae backup vsds
+    metroae-container backup vsds
 
 Substitute the component name to be backed up if different than `vsds`.
 
@@ -31,19 +31,19 @@ Ensure that all components you wish to have restored are specified under your de
 
 To restore all supported components issue:
 
-    metroae restore
+    metroae-container restore
 
 To restore a specific component issue:
 
-    metroae restore vsds
+    metroae-container restore vsds
 
 Substitute the component name to be backed up if different than `vsds`.
 
 Restore can alternatively be performed with each step separately using:
 
-    metroae restore vsds predeploy
-    metroae restore vsds deploy
-    metroae restore vsds postdeploy
+    metroae-container restore vsds predeploy
+    metroae-container restore vsds deploy
+    metroae-container restore vsds postdeploy
 
 ## TLS Configuration on VSC During Restore
 

Documentation/CHANGES_IN_V5.md

@@ -0,0 +1,44 @@
+# Changes in V5
+We have Made the following changes to MetroAE in v5 as compared to previous versions of MetroAE. All changes are summarised in this document.
+
+## MetroAE operation changes
+In v4 and before, MetroAE shipped with 2 modes of operation. The container and git clone. With the new release, we have made things a bit easier for the users. There is a single mode of operation - a new MetroAE container. This will replace the old MetroAE container, which will no longer be supported.
+
+This new container will be dynamically built on the users machine. Users will need to get the latest MetroAE repository on their machine. The only other pre-requisite for the user is to have docker installed on their MetroAE Host Machine. The changes for moving from v4 to v5 are as below depending on type of installation.
+
+### Moving from MetroAE git clone/download to MetroAE v5
+1. Download or git pull the latest MetroAE code
+2. Make sure docker is installed and running. The `docker ps` command can verify this.
+3. Use `./metroae-container` instead of `./metroae`. All other arguments remain the same.
+   e.g. Instead of this initial command
+   `./metroae install vsds -vvv` use `./metroae-container install vsds -vvv`
+   Another example for deployments other than defaults
+   `./metroae install vsds specialdeployment -vvv` use `./metroae-container install vsds specialdeployment -vvv`
+4. For all image paths, make sure they start with `/metroae`. Here `/metroae` refers to the present working directory for the user.
+   e.g.
+   `nuage_unzipped_files_dir: ./images/20.10.R4` changes to `nuage_unzipped_files_dir: /metroae/images/20.10.R4`
+5. All the specified paths for licenses, unzipped files, backup directories should be inside the MetroAE repository. e.g. you cannot specify `/opt` or `/tmp` for the MetroAE host. If your mount directory for images is outside the MetroAE folder, you can use a mount bind to put them inside the MetroAE directory.
+   e.g.
+   `sudo mount --bind -o ro /mnt/nfs-data /<your-repo-location>/images`
+6. Users do not need to run setup at all, all dependencies will be automatically taken care of with the new container in the background.
+7. For vcenter users only, MetroAE should be cloned or downloaded in a directory where ovftool is present. The entire vmware-ovftool folder must be present. In short, the path to ovftool should be somewhere inside the MetroAE top level folder. You can mount bind the ovftool to the metro repo location.
+   `sudo mount --bind -o ro /usr/lib/vmware-ovftool /<your-repo-location>/ovftool`
+
+### Moving from MetroAE v4 container/download to MetroAE v5
+1. Download or git pull the latest MetroAE code
+2. Destroy the old container using `./metroae-container destroy` command
+3. Use `./metroae-container` instead of `/metroae`. All other arguments remain the same.
+   e.g. Instead of this initial command
+   `metroae install vsds -vvv` use `./metroae-container install vsds -vvv`
+   Another example for deployments other than defaults
+   `metroae install vsds specialdeployment -vvv` use `./metroae-container install vsds specialdeployment -vvv`
+4. For all image paths, make sure they start with `/metroae` instead of `/metroae_data`. Here `/metroae` refers to the present working directory for the user.
+   e.g.
+   `nuage_unzipped_files_dir: /metroae_data/images/20.10.R4` changes to `nuage_unzipped_files_dir: /metroae/images/20.10.R4`
+5. You can create an `images` folder in `nuage-metroae` and move the `/metroae_data` mount folder under `/<your-repo-location>/images` athat way you can replace `/metroae_data` with `/metroae_data/images` in the deployment files.
+
+## Ansible and Python Changes
+MetroAE is now supported with Ansible version 3.4.0 and higher. Python3 is now required. Do not worry, the container that gets dynamically created should take care of the python, ansible and any other dependencies that are needed. This will not affect the user environment as all dependencies will be installed in the MetroAE container.
+
+## MetroAE Config
+MetroaAE Config is no longer bundled with MetroAE. Please refer to https://github.com/nuagenetworks/nuage-metroae-config to get information on how use MetroAE Config.

Documentation/CONFIG.md

@@ -1,78 +1,3 @@
 # MetroAE Config
 
-MetroAE Config is a template-driven VSD configuration tool. It utilizes the VSD API along with a set of common Feature Templates to create configuration in the VSD. The user will create a yaml or json data file with the necessary configuration parameters for the particular feature and execute simple CLI commands to push the configuration into VSD.
-
-MetroAE Config is available via the MetroAE container. Once pulled and setup, additional documentation will be available.
-
-## Configuration Engine Installation
-
-MetroAE Configuration engine is provided as one of the capabilities of the MetroAE Docker container. The installation of the container is handled via the metroae script. Along with the configuration container we also require some additional data.
-
-On a host where the configuration engine will be installed the following artifacts will be installed:
-
-* Docker container for configuration
-* Collection of configuration Templates
-* VSD API Specification
-
-### System Requirements
-
-* Operating System: RHEL/Centos 7.4+
-* Docker: Docker Engine 1.13.1+
-* Network Access: Internal and Public
-* Storage: At least 800MB
-
-#### Operating system
-
-The primary requirement for the configuration container is Docker Engine, however the installation, container operation and functionality is only tested on RHEL/Centos 7.4. Many other operating systems will support Docker Engine, however support for these is not currently provided and would be considered experimental only. A manual set of installation steps is provided for these cases.  
-
-#### Docker Engine
-
-The configuration engine is packaged into the MetroAE Docker container. This ensures that all package and library requirements are self-contained with no other host dependencies. To support this, Docker Engine must be installed on the host. The configuration container requirements, however, are quite minimal. Primarily, the Docker container mounts a local path on the host as a volume while ensuring that any templates and user data are maintained only on the host. The user never needs to interact directly with the container.  
-
-#### Network Access
-
-Currently the configuration container is hosted in an internal Docker registry and the public network as a secondary option, while the Templates and API Spec are hosted only publicly. The install script manages the location of these resources. The user does not need any further information. However, without public network access the installation will fail to complete.
-
-#### Storage
-
-The configuration container along with the templates requires 800MB of local disk space. Note that the container itself is ~750MB, thus it is recommended that during install a good network connection is available.
-
-#### User Privileges
-
-The user must have elevated privileges on the host system.
-
-### Installation
-
-Execute the following operations on the Docker host:
-
-1. Install Docker Engine
-
-    Various instructions for installing and enabling Docker are available on the Internet. One reliable source of information for Docker CE is hosted here:
-
-    [https://docs.docker.com/engine/install/centos/](https://docs.docker.com/engine/install/centos/)
-
-2. Add the user to the wheel and docker groups on the Docker host.
-
-3. Move or Copy the "metroae" script from the nuage-metroae repo to /usr/bin and set permissions correctly to make the script executable.
-
-4. Switch to the user that will operate "metroae config".
-
-5. Setup the container using the metroae script.
-
-    We are going to pull the image and setup the metro container in one command below. During the install we will be prompted for a location for the container data directory. This is the location where our user data, templates and VSD API specs will be installed and created and a volume mount for the container will be created. However this can occur orthogonally via "pull" and "setup" running at separate times which can be useful depending on available network bandwidth.
-
-    `[caso@metroae-host ~]$ metroae container setup`
-
-    The MetroAE container needs access to your user data. It gets access by internally mounting a directory from the host. We refer to this as the 'data directory'. The data directory is where you will have deployments, templates, documentation, and other useful files. You will be prompted to provide the path to a local directory on the host that will be mounted inside the container. You will be prompted during setup for this path.
-
-    The MetroAE container can be setup in one of the following configurations:
-
-    * Config only
-    * Deploy only
-    * Both config and deploy
-
-    During setup, you will be prompted to choose one of these options.
-
-6. Follow additional insturctions found in the documentaion that is copied to the Docker host during setup.
-
-Complete documentation will be made available in the data directory you specify during setup. The complete documentation includes how to configure your environment, usage information for the tool, and more.
+MetroaAE Config is no longer bundled with MetroAE. Please refer to https://github.com/nuagenetworks/nuage-metroae-config to get information on how use MetroAE Config.

Documentation/CONTRIBUTING.md

@@ -0,0 +1,81 @@
+# Contributing to MetroAE
+
+MetroAE is built on a community model. The code has been architected to make contribution simple and straightforward. You are welcome to join in the community to help us continuously improve MetroAE.
+
+## We Use [Github Flow](https://guides.github.com/introduction/flow/index.html), So All Code Changes Happen Through Pull Requests
+Pull requests are the best way to propose changes to the codebase (we use [Github Flow](https://guides.github.com/introduction/flow/index.html)). We actively welcome your pull requests:
+
+1. Fork the repo and create your branch from `dev`.
+2. If you've added code that should be tested, add tests.
+3. Make sure your code passes flake8.
+4. Issue that pull request!
+
+## Prerequisites / Requirements
+
+  All contributions must be consistent with the design of the existing workflows.
+
+  All contrinbutions must be submitted as pull requests to the _dev_ branch, reviewed, updated, and merged into the nuage-metroae repo.
+
+  You must have a github.com account and have been added as a collaborator to the nuage-metroae repo.
+
+## Contributing your code
+
+1. Developing your code.
+
+    The manner in which you develop the code contribution depends on the extent of the changes. Are you enhancing an existing playbook or role, or are you adding one or more new roles? Making changes to what already exists is simple. Just make your changes to the files that are already there.
+
+    Adding a new component or feature is a bit more involved. For example, if you are adding support for the installation of a new component, the following elements would be expected unless otherwise agreed upon by the repo owners:
+
+    1. A new user-input schema for the component must be created. See the exitsing files in the `schemas` directory.
+    2. A new deployment template must be created. See the existing files in the `src/deployment_templates` directory.
+    3. Add to the example data. All deployment templates and examples are auto-generated. The data in `src/raw_example_data` is used by the automatic generation to populate the examples properly. Also see the examples that have been auto-generated in the `examples/` directory.
+    4. Add your component and associated file references to `src/workflows.yml`.
+    5. Add your schema to `src/roles/common/vars/main.yml`.
+    6. Execute  `src/generate_all_from_schemas.sh` to create all the required files for your component.
+    7. Create the proper roles. The following roles are required unless otherwise agreed to by the repo owners: _newcomponent-predeploy_, _newcomponent-deploy_, _newcomponent-postdeploy_, _newcomponent-health_, and _newcomponent-destroy_ should be created under `src/roles/`
+    8. Create the proper playbooks to execute the roles: _newcomponent_predeploy.yml_, _newcomponent_deploy.yml_, _newcomponent_postdeploy.yml_, _newcomponent_health.yml_, and _newcomponent_destroy.yml_ should be created under `src/playbooks/with_build`
+    9. Test, modify, and retest until your code is working perfectly.
+
+2. Test all proposed contributions on the appropriate hypervisors in the `metro-fork` directory. If you choose not to provide support for one or more supported hypervisors, you must provide graceful error handling for those types.
+
+3. All python files modified or submitted must successfully pass a `flake8 --ignore=E501` test.
+
+4. Add a brief description of your bug fix or enhancement to `RELEASE_NOTES.md`.
+
+## Any contributions you make will be under the APACHE 2.0 Software License
+  In short, when you submit code changes, your submissions are understood to be under the same [APACHE License 2.0](https://www.apache.org/licenses/LICENSE-2.0) that covers the project. Feel free to contact the maintainers if that's a concern.
+
+## Report bugs using Github's [issues](https://github.com/nuagenetworks/nuage-metroae/issues)
+  We use GitHub issues to track public bugs.
+
+## Write bug reports with detail, background, and sample code
+
+  **Great Bug Reports** tend to have:
+
+  - A quick summary and/or background
+  - Steps to reproduce
+    - Be specific!
+    - Give sample code if you can.
+  - What you expected would happen
+  - What actually happens
+  - Notes (possibly including why you think this might be happening, or stuff you tried that didn't work)
+
+## Use a Consistent Coding Style
+
+  * 4 spaces for indentation rather than tabs
+  * 80 character line length
+  * TODO
+
+## License
+  By contributing, you agree that your contributions will be licensed under its APACHE 2.0 License.
+
+
+## Questions and Feedback
+
+Ask questions and get support on the [MetroAE site](https://devops.nuagenetworks.net/).  
+You may also contact us directly.  
+  Outside Nokia: [[email protected]](mailto:[email protected] "send email to nuage-metro project").  
+  Internal Nokia: [[email protected]](mailto:[email protected] "send email to nuage-metro project").
+
+## References
+  This document was adapted from the open-source contribution guidelines for [Facebook's Draft](https://github.com/facebook/draft-js/blob/a9316a723f9e918afde44dea68b5f9f39b7d9b00/CONTRIBUTING.md)