diff --git a/JSON.md b/JSON.md index da6e565c..51d92c27 100644 --- a/JSON.md +++ b/JSON.md @@ -2,9 +2,9 @@ The CFEngine Build website and tooling relies on JSON files for different purposes; -- The project file (`cfbs.json`) generated by running `cfbs init` when you start a new project -- The index of all available modules which is available on GitHub -- A `cfbs.json` file with necessary metadata for adding a module using URL (not from index) +* The project file (`cfbs.json`) generated by running `cfbs init` when you start a new project +* The index of all available modules which is available on GitHub +* A `cfbs.json` file with necessary metadata for adding a module using URL (not from index) All these 3 share 1 standard format, commonly called `cfbs.json`. @@ -194,7 +194,7 @@ The modules inside `build`, `provides`, and `index` use these fields: See the section below on [modules with input](#modules-with-input) for keys inside `input`, explanations of how this works and examples. Optional. -### Step folders +## Step folders As a project is built, `cfbs` creates intermediate folders for each module, for example: @@ -205,57 +205,49 @@ out/steps/001_masterfiles_5c7dc5b43088e259a94de4e5a9f17c0ce9781a0f/ These are copies of the module directories, where it's more "safe" to do things like run scripts or delete files. `cfbs build` should not edit files in your project / git repository, only the generated / temporary files inside the `out/` directory. -### All available build steps +## All available build steps The build steps below manipulate the temporary files in the steps directories and write results to the output policy set, in `out/masterfiles`. Unless otherwise noted, all steps are run inside the module's folder (`out/steps/...`) with sources / file paths relative to that folder, and targets / destinations mentioned below are relative to the output policy set (`out/masterfiles`, which in the end will be deployed as `/var/cfengine/masterfiles`) -#### `copy ` -- Copy a single file or a directory recursively. - -#### `json ` -- Merge the source json file into the destination json file. - -#### `append ` -- Append the source file to the end of destination file. - -#### `run ` -- Run a shell command / script. -- Usually used to prepare the module directory, delete files, etc. before a copy step. -- Running scripts should be avoided if possible. -- Script is run inside the module directory (the step folder). -- Additional space separated arguments are passed as arguments. - -#### `delete ` -- Delete multiple files or paths recursively. -- Files are deleted from the step folder. -- Typically used before copying files to the output policy set with the `copy` step. - -#### `directory ` -- Copy any .cf policy files recursively and add their paths to `def.json`'s `inputs`. -- Enable `services_autorun_bundles` class in `def.json`. -- Merge any `def.json` recursively into `out/masterfiles/def.json`. -- Copy any other files with their existing directory structure to destination. - -#### `bundles ` -- Ensure bundles are evaluated by adding them to the bundle sequence, using `def.json`. - - Note that this relies on using the default policy set from the CFEngine team, the Masterfiles Policy Framework, commonly added as the first module (`masterfiles`). +* `copy ` + * Copy a single file or a directory recursively. +* `json ` + * Merge the source json file into the destination json file. +* `append ` + * Append the source file to the end of destination file. +* `run ` + * Run a shell command / script. + * Usually used to prepare the module directory, delete files, etc. before a copy step. + * Running scripts should be avoided if possible. + * Script is run inside the module directory (the step folder). + * Additional space separated arguments are passed as arguments. +* `delete ` + * Delete multiple files or paths recursively. + * Files are deleted from the step folder. + * Typically used before copying files to the output policy set with the `copy` step. +* `directory ` + * Copy any .cf policy files recursively and add their paths to `def.json`'s `inputs`. + * Enable `services_autorun_bundles` class in `def.json`. + * Merge any `def.json` recursively into `out/masterfiles/def.json`. + * Copy any other files with their existing directory structure to destination. +* `bundles ` + * Ensure bundles are evaluated by adding them to the bundle sequence, using `def.json`. + * Note that this relies on using the default policy set from the CFEngine team, the Masterfiles Policy Framework, commonly added as the first module (`masterfiles`). Specifically, this build step adds the bundles to the variable `default:def.control_common_bundlesequence_end`, which the MPF looks for. -- Only manipulates the bundle sequence, to ensure policy files are copied and parsed, use other build steps, for example `copy` and `policy_files`. - -#### `policy_files ` -- Add policy (`.cf`) files to `inputs` key in `def.json`, ensuring they are parsed. - - Note that this relies on using the default policy set from the CFEngine team, the Masterfiles Policy Framework, commonly added as the first module (`masterfiles`). - - Only edits `def.json`, does not copy files. Should be used after a `copy` or `directory` build step. - - Does not add any bundles to the bundle sequence, to ensure a bundle is evaluated, use the `bundles` build step or the autorun mechanism. -- All paths are relative to `out/masterfiles`. -- If any of the paths are directories (end with `/`), the folder(s) are recursively searched and all `.cf` files are added. - - **Note:** Directories should be module-specific, otherwise this build step can find policy files from other modules (when they are mixed in the same directory). - -#### `input ` -- Converts the input data for a module into the augments format and merges it with the target augments file. -- Source is relative to module directory and target is relative to `out/masterfiles`. - - In most cases, the build step should be: `input ./input.json def.json` + * Only manipulates the bundle sequence, to ensure policy files are copied and parsed, use other build steps, for example `copy` and `policy_files`. +* `policy_files ` + * Add policy (`.cf`) files to `inputs` key in `def.json`, ensuring they are parsed. + * Note that this relies on using the default policy set from the CFEngine team, the Masterfiles Policy Framework, commonly added as the first module (`masterfiles`). + * Only edits `def.json`, does not copy files. Should be used after a `copy` or `directory` build step. + * Does not add any bundles to the bundle sequence, to ensure a bundle is evaluated, use the `bundles` build step or the autorun mechanism. + * All paths are relative to `out/masterfiles`. + * If any of the paths are directories (end with `/`), the folder(s) are recursively searched and all `.cf` files are added. + * **Note:** Directories should be module-specific, otherwise this build step can find policy files from other modules (when they are mixed in the same directory). +* `input ` + * Converts the input data for a module into the augments format and merges it with the target augments file. + * Source is relative to module directory and target is relative to `out/masterfiles`. + * In most cases, the build step should be: `input ./input.json def.json` ## Examples @@ -416,13 +408,15 @@ cfbs init && cfbs add https://github.com/cfengine/some-repo ## Modules with input + Some modules allow for users to add module input by responding to questions expressed under the `"input"` attribute in `cfbs.json`. User input can be added using the `cfbs input ` command, which stores responses in `.//input.json`. These responses are translated into augments which will be added to `./out/masterfiles/def.json` during `cfbs build`. -### Create single file example: +### Create single file example + The `"input"` attribute takes a list of input definitions as illustrated below. ```json @@ -538,7 +532,8 @@ would produce the following augment; } ``` -### Create a single file with content example: +### Create a single file with content example + A module that creates empty files is not too impressive on its own. Let us instead try to extend our previous example by having the module also ask for file contents. @@ -613,7 +608,8 @@ $ cat ./out/masterfiles/def.json } ``` -### Create multiple files example: +### Create multiple files example + Sometimes we would like a module to support taking an arbritary number of inputs. This can be done using a variable definition of type list. Let's extend our first example from creating a single to multiple files. @@ -716,7 +712,8 @@ $ cat ./out/masterfiles/def.json } ``` -### Create multiple files with content example: +### Create multiple files with content example + As a final example, let's see how we can build a module that takes an arbritary number of filename and content pairs as input.