Add NnfLustreMGT documentation #191
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Document how to create the NnfLustreMGT and ConfigMap for MGTs outside of NNF's control. Signed-off-by: Matt Richerson <[email protected]>
bdevcich
reviewed
Aug 2, 2024
| combinedMgtMdt: false | ||
| standaloneMgtPoolName: "" | ||
| [...] | ||
| ``` | ||
|
|
||
| ### NnfLustreMGT |
There was a problem hiding this comment.
Add a line after the subheading
bdevcich
reviewed
Aug 2, 2024
docs/guides/external-mgs/readme.md
Outdated
| combinedMgtMdt: false | ||
| standaloneMgtPoolName: "" | ||
| [...] | ||
| ``` | ||
|
|
||
| ### NnfLustreMGT | ||
| A `NnfLustreMGT` resource tracks which fsnames have been used on the MGT to prevent fsname re-use. Any Lustre file systems that are created through the NNF software will request an fsname to use from a `NnfLustreMGT` resource. Every MGT must have a corresponding `NnfLustreMGT` resource. For MGTs that are hosted on NNF hardware, the `NnfLustreMGT` resources are created automatically. The NNF software also erases any no longer used fsnames from disk for any internally hosted MGTs. For an MGT hosted on an external node, an admin must create an `NnfLustreMGT`. This resource ensures that fsnames will be created in a sequential order without any fsname re-use. However, after an fsname is no longer in use by a file system, it will not be erased from the MGT disk. An admin may decide to periodically run the `lctl erase_lcfg [fsname]` command to remove fsnames that are no longer in use. |
There was a problem hiding this comment.
Suggested change
| A `NnfLustreMGT` resource tracks which fsnames have been used on the MGT to prevent fsname re-use. Any Lustre file systems that are created through the NNF software will request an fsname to use from a `NnfLustreMGT` resource. Every MGT must have a corresponding `NnfLustreMGT` resource. For MGTs that are hosted on NNF hardware, the `NnfLustreMGT` resources are created automatically. The NNF software also erases any no longer used fsnames from disk for any internally hosted MGTs. For an MGT hosted on an external node, an admin must create an `NnfLustreMGT`. This resource ensures that fsnames will be created in a sequential order without any fsname re-use. However, after an fsname is no longer in use by a file system, it will not be erased from the MGT disk. An admin may decide to periodically run the `lctl erase_lcfg [fsname]` command to remove fsnames that are no longer in use. | |
| A `NnfLustreMGT` resource tracks which fsnames have been used on the MGT to prevent fsname re-use. Any Lustre file systems that are created through the NNF software will request an fsname to use from a `NnfLustreMGT` resource. Every MGT must have a corresponding `NnfLustreMGT` resource. For MGTs that are hosted on NNF hardware, the `NnfLustreMGT` resources are created automatically. The NNF software also erases any unused fsnames from disk for any internally hosted MGTs. For an MGT hosted on an external node, an admin must create an `NnfLustreMGT` resource. This resource ensures that fsnames will be created in a sequential order without any fsname re-use. However, after an fsname is no longer in use by a file system, it will not be erased from the MGT disk. An admin may decide to periodically run the `lctl erase_lcfg [fsname]` command to remove fsnames that are no longer in use. |
bdevcich
reviewed
Aug 2, 2024
docs/guides/external-mgs/readme.md
Outdated
| combinedMgtMdt: false | ||
| standaloneMgtPoolName: "" | ||
| [...] | ||
| ``` | ||
|
|
||
| ### NnfLustreMGT | ||
| A `NnfLustreMGT` resource tracks which fsnames have been used on the MGT to prevent fsname re-use. Any Lustre file systems that are created through the NNF software will request an fsname to use from a `NnfLustreMGT` resource. Every MGT must have a corresponding `NnfLustreMGT` resource. For MGTs that are hosted on NNF hardware, the `NnfLustreMGT` resources are created automatically. The NNF software also erases any no longer used fsnames from disk for any internally hosted MGTs. For an MGT hosted on an external node, an admin must create an `NnfLustreMGT`. This resource ensures that fsnames will be created in a sequential order without any fsname re-use. However, after an fsname is no longer in use by a file system, it will not be erased from the MGT disk. An admin may decide to periodically run the `lctl erase_lcfg [fsname]` command to remove fsnames that are no longer in use. |
There was a problem hiding this comment.
erases any ... fsnames from disk
which disk?
bdevcich
reviewed
Aug 2, 2024
docs/guides/external-mgs/readme.md
Outdated
| ``` | ||
|
|
||
| * `addresses` - This is a list of LNet addresses that could be used for this MGT. This should match any values that are used in the `externalMgs` field in the `NnfStorageProfiles`. | ||
| * `fsNameStart` - The first fsname to use. Subsequent fsnames will be incremented based on this starting fsname (e.g, `aaaaaaaa`, `aaaaaaab`, `aaaaaaac`). fsnames use lowercase letters `'a'`-`'z'`. |
There was a problem hiding this comment.
does this need to be a certain length?
bdevcich
reviewed
Aug 2, 2024
docs/guides/external-mgs/readme.md
Outdated
| * `addresses` - This is a list of LNet addresses that could be used for this MGT. This should match any values that are used in the `externalMgs` field in the `NnfStorageProfiles`. | ||
| * `fsNameStart` - The first fsname to use. Subsequent fsnames will be incremented based on this starting fsname (e.g, `aaaaaaaa`, `aaaaaaab`, `aaaaaaac`). fsnames use lowercase letters `'a'`-`'z'`. | ||
| * `fsNameBlackList` - This is a list of fsnames that should not be given to any NNF Lustre file systems. If the MGT is hosting any non-NNF Lustre file systems, their fsnames should be included in this blacklist. | ||
| * `fsNameStartReference` - This is an optional ObjectReference to a `ConfigMap` that holds a starting fsname. If this field is specified, it takes precedence over the `fsNameStart` field in the spec. The `ConfigMap` will be updated to the next available fsname everytime an fsname is assigned to a new Lustre file system. |
There was a problem hiding this comment.
Suggested change
| * `fsNameStartReference` - This is an optional ObjectReference to a `ConfigMap` that holds a starting fsname. If this field is specified, it takes precedence over the `fsNameStart` field in the spec. The `ConfigMap` will be updated to the next available fsname everytime an fsname is assigned to a new Lustre file system. | |
| * `fsNameStartReference` - This is an optional `ObjectReference` to a `ConfigMap` that holds a starting fsname. If this field is specified, it takes precedence over the `fsNameStart` field in the spec. The `ConfigMap` will be updated to the next available fsname every time an fsname is assigned to a new Lustre file system. |
bdevcich
approved these changes
Aug 2, 2024
Signed-off-by: Matt Richerson <[email protected]>
roehrich-hpe
reviewed
Aug 5, 2024
docs/guides/external-mgs/readme.md
Outdated
| combinedMgtMdt: false | ||
| standaloneMgtPoolName: "" | ||
| [...] | ||
| ``` | ||
|
|
||
| ### NnfLustreMGT | ||
|
|
||
| A `NnfLustreMGT` resource tracks which fsnames have been used on the MGT to prevent fsname re-use. Any Lustre file systems that are created through the NNF software will request an fsname to use from a `NnfLustreMGT` resource. Every MGT must have a corresponding `NnfLustreMGT` resource. For MGTs that are hosted on NNF hardware, the `NnfLustreMGT` resources are created automatically. The NNF software also erases any unused fsnames from the MGT disk for any internally hosted MGTs. For a MGT hosted on an external node, an admin must create an `NnfLustreMGT` resource. This resource ensures that fsnames will be created in a sequential order without any fsname re-use. However, after an fsname is no longer in use by a file system, it will not be erased from the MGT disk. An admin may decide to periodically run the `lctl erase_lcfg [fsname]` command to remove fsnames that are no longer in use. |
There was a problem hiding this comment.
Suggested change
| A `NnfLustreMGT` resource tracks which fsnames have been used on the MGT to prevent fsname re-use. Any Lustre file systems that are created through the NNF software will request an fsname to use from a `NnfLustreMGT` resource. Every MGT must have a corresponding `NnfLustreMGT` resource. For MGTs that are hosted on NNF hardware, the `NnfLustreMGT` resources are created automatically. The NNF software also erases any unused fsnames from the MGT disk for any internally hosted MGTs. For a MGT hosted on an external node, an admin must create an `NnfLustreMGT` resource. This resource ensures that fsnames will be created in a sequential order without any fsname re-use. However, after an fsname is no longer in use by a file system, it will not be erased from the MGT disk. An admin may decide to periodically run the `lctl erase_lcfg [fsname]` command to remove fsnames that are no longer in use. | |
| A `NnfLustreMGT` resource tracks which fsnames have been used on the MGT to prevent fsname re-use. Any Lustre file systems that are created through the NNF software will request an fsname to use from a `NnfLustreMGT` resource. Every MGT must have a corresponding `NnfLustreMGT` resource. For MGTs that are hosted on NNF hardware, the `NnfLustreMGT` resources are created automatically. The NNF software also erases any unused fsnames from the MGT disk for any internally hosted MGTs. | |
| For a MGT hosted on an external node, an admin must create an `NnfLustreMGT` resource. This resource ensures that fsnames will be created in a sequential order without any fsname re-use. However, after an fsname is no longer in use by a file system, it will not be erased from the MGT disk. An admin may decide to periodically run the `lctl erase_lcfg [fsname]` command to remove fsnames that are no longer in use. |
docs/guides/external-mgs/readme.md
Outdated
|
|
||
| A `NnfLustreMGT` resource tracks which fsnames have been used on the MGT to prevent fsname re-use. Any Lustre file systems that are created through the NNF software will request an fsname to use from a `NnfLustreMGT` resource. Every MGT must have a corresponding `NnfLustreMGT` resource. For MGTs that are hosted on NNF hardware, the `NnfLustreMGT` resources are created automatically. The NNF software also erases any unused fsnames from the MGT disk for any internally hosted MGTs. For a MGT hosted on an external node, an admin must create an `NnfLustreMGT` resource. This resource ensures that fsnames will be created in a sequential order without any fsname re-use. However, after an fsname is no longer in use by a file system, it will not be erased from the MGT disk. An admin may decide to periodically run the `lctl erase_lcfg [fsname]` command to remove fsnames that are no longer in use. | ||
|
|
||
| Below is an example `NnfLustreMGT` resource. The `NnfLustreMGT` resource for external MGSs should be created in the `nnf-system` namespace. |
There was a problem hiding this comment.
Suggested change
| Below is an example `NnfLustreMGT` resource. The `NnfLustreMGT` resource for external MGSs should be created in the `nnf-system` namespace. | |
| Below is an example `NnfLustreMGT` resource. The `NnfLustreMGT` resource for external MGSs must be created in the `nnf-system` namespace. |
| ``` | ||
|
|
||
| * `addresses` - This is a list of LNet addresses that could be used for this MGT. This should match any values that are used in the `externalMgs` field in the `NnfStorageProfiles`. | ||
| * `fsNameStart` - The first fsname to use. Subsequent fsnames will be incremented based on this starting fsname (e.g, `aaaaaaaa`, `aaaaaaab`, `aaaaaaac`). fsnames use lowercase letters `'a'`-`'z'`. `fsNameStart` should be exactly 8 characters long. |
There was a problem hiding this comment.
Mention the Lustre length limit. Is it 8 or 6?
docs/guides/external-mgs/readme.md
Outdated
|
|
||
| ### ConfigMap | ||
|
|
||
| For external MGTs, the `fsNameStartReference` should be used to point to a `ConfigMap` in the default namespace. The `ConfigMap` should not be removed during an argocd undeploy/deploy. This allows the nnf-sos sofware to be undeployed (including any `NnfLustreMGT` resources), without having the fsname reset back to the `fsNameStart` value on a redeploy. The Configmap that is created should be left empty initially. |
There was a problem hiding this comment.
Suggested change
| For external MGTs, the `fsNameStartReference` should be used to point to a `ConfigMap` in the default namespace. The `ConfigMap` should not be removed during an argocd undeploy/deploy. This allows the nnf-sos sofware to be undeployed (including any `NnfLustreMGT` resources), without having the fsname reset back to the `fsNameStart` value on a redeploy. The Configmap that is created should be left empty initially. | |
| For external MGTs, the `fsNameStartReference` should be used to point to a `ConfigMap` in the `default` namespace. The `ConfigMap` should not be removed during an argocd undeploy/deploy. This allows the nnf-sos sofware to be undeployed (including any `NnfLustreMGT` resources), without having the fsname reset back to the `fsNameStart` value on a redeploy. The Configmap that is created should be left empty initially. |
docs/guides/external-mgs/readme.md
Outdated
|
|
||
| ### ConfigMap | ||
|
|
||
| For external MGTs, the `fsNameStartReference` should be used to point to a `ConfigMap` in the default namespace. The `ConfigMap` should not be removed during an argocd undeploy/deploy. This allows the nnf-sos sofware to be undeployed (including any `NnfLustreMGT` resources), without having the fsname reset back to the `fsNameStart` value on a redeploy. The Configmap that is created should be left empty initially. |
There was a problem hiding this comment.
"argocd undeploy/deploy" => "undeploy/deploy".
I don't think we want to weave argocd so tightly into our docs.
docs/guides/external-mgs/readme.md
Outdated
|
|
||
| ### Argocd | ||
|
|
||
| * An empty ConfigMap should be deployed with the `0-early-config` application. It should be created in the `default` namespace, and it can have any name. |
There was a problem hiding this comment.
Oof.
Ok, "application" should be Application, since that's the name of a resource Kind. Now we have to explain what 0-early-config is, and we have no other argocd references anywhere in the NearNodeFlash docs, as far as I recall.
Can this be stripped-down and rewritten without reference to ArgoCD?
Signed-off-by: Matt Richerson <[email protected]>
ajfloeder
approved these changes
Aug 20, 2024
roehrich-hpe
approved these changes
Aug 21, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Document how to create the NnfLustreMGT and ConfigMap for MGTs outside of NNF's control.