Feat/add gpu vram (#780)

* Add GPU table and VRAM into specification. To be done: Adjust flavor name parser, pretty printer, generator. * Version 3.2, adjust examples. * Fix empty line and extra space. * Minor addition of information for AMD and intel. * Typo. * Note about 1/7 uncertainties. Nvidia spelling. Also mention older generations ... * Appease markdownlint. * More appeasement for markdownlint. One true fix (broken link) And tweak the double-space test for tolerating two spaces after a | in a table. * One more fix against double spaces. * add vram and vramperf to GPU (retrofitting v1 and v2) * bugfix: use correct variable * appease flake8 * GPU VRAM always comes with CU spec: Use "and" wording. Just for a bit better readability. * More specific meaning of GPU h modifiers. h on the SMs/CUs/EUs: High frequency h on the VRAM: High bandwidth Again, this is really only to differentiate if a vendor has several otherwise similar models that have a material difference in frequencyor bandwidth, such as e.g. a GDDR6 vs an HBM2e veriant ... or a low-power, low-frequency variant. Signed-off-by: Kurt Garloff <[email protected]> Signed-off-by: Matthias Büchse <[email protected]> Co-authored-by: Matthias Büchse <[email protected]>
SovereignCloudStack · Oct 17, 2024 · 1e24ed4 · 1e24ed4
1 parent ec6b4c2
commit 1e24ed4
Show file tree

Hide file tree

Showing 5 changed files with 155 additions and 28 deletions.
diff --git a/.markdownlint-cli2.jsonc b/.markdownlint-cli2.jsonc
@@ -43,9 +43,10 @@
         {
           "name": "double-spaces",
           "message": "Avoid double spaces",
-          "searchPattern": "/([^\\s>])  ([^\\s|])/g",
+          "searchPattern": "/([^\\s>|])  ([^\\s|])/g",
           "replace": "$1 $2",
-          "skipCode": true
+          "skipCode": true,
+	  "tables": false
         }
       ]
     }

diff --git a/Standards/scs-0100-v3-flavor-naming.md b/Standards/scs-0100-v3-flavor-naming.md
@@ -14,7 +14,7 @@ description: |
 
 ## Introduction
 
-This is the standard v3.1 for SCS Release 5.
+This is the standard v3.2 for SCS Release 8.
 Note that we intend to only extend it (so it's always backwards compatible),
 but try to avoid changing in incompatible ways.
 (See at the end for the v1 to v2 transition where we have not met that
@@ -417,15 +417,17 @@ is more significant.
 
 ### [OPTIONAL] GPU support
 
-Format: `_`\[`G/g`\]X\[N\]\[`-`M\]\[`h`\]
+Format: `_`\[`G/g`\]X\[N\[`-`M\[`h`\]\[`-`V\[`h`\]\]\]\]
 
 This extension provides more details on the specific GPU:
 
 - pass-through (`G`) vs. virtual GPU (`g`)
 - vendor (X)
 - generation (N)
 - number (M) of processing units that are exposed (for pass-through) or assigned; see table below for vendor-specific terminology
-- high-performance indicator (`h`)
+- high-frequency indicator (`h`) for compute units
+- amount of video memory (V) in GiB
+- an indicator for high-bandwidth memory
 
 Note that the vendor letter X is mandatory, generation and processing units are optional.
 
@@ -440,13 +442,29 @@ for AMD GCN-x=0.x, RDNA1=1, C/RDNA2=2, C/RDNA3=3, C/RDNA3.5=3.5, C/RDNA4=4, ...
 for Intel Gen9=0.9, Xe(12.1/DG1)=1, Xe(12.2)=2, Arc(12.7/DG2)=3 ...
 (Note: This may need further work to properly reflect what's out there.)
 
-The optional `h` suffix to the compute unit count indicates high-performance (e.g. high freq or special
-high bandwidth gfx memory such as HBM);
-`h` can be duplicated for even higher performance.
+The optional `h` suffix to the compute unit count indicates high-frequency GPU compute units.
+It is not normally recommended to use it except if there are several variants of cards within
+a generation of GPUs and with similar number of SMs/CUs/EUs.
+In case there are even more than two variants, the letter `h` can be duplicated for even
+higher frquencies.
 
-Example: `SCS-16V-64-500s_GNa-14h`
-This flavor has a pass-through GPU nVidia Ampere with 14 SMs and either high-bandwidth memory or specially high frequencies.
-Looking through GPU specs you could guess it's 1/4 of an A30.
+Please note that there are GPUs from one generation and vendor that have vastly different sizes
+(or different fractions are being passed to an instance with multi-instance-GPUs). The number
+M allows to differentiate between them and have an indicator of the compute capability and
+parallelism. M can not at all be compared between different generations let alone different
+vendors.
+
+The amount of video memory dedicated to the instance can be indicated by V (in binary
+Gigabytes). This number needs to be an integer - fractional memory sizes must be rounded
+down. An optional `h` can be used to indicate high bandwidth memory (such as HBM2+) with
+bandwidths well above 1GiB/s.
+
+Example: `SCS-16V-64-500s_GNa-14-6h`
+This flavor has a pass-through GPU nVidia Ampere with 14 SMs and 6 GiB of high-bandwidth video
+memory. Looking through GPU specs you could guess it's 1/4 of an A30.
+
+We have a table with common GPUs in the
+[implementation hints for this standard](scs-0100-w1-flavor-naming-implementation-testing.md)
 
 ### [OPTIONAL] Infiniband
 
@@ -490,14 +508,14 @@ an image is considered broken by the SCS team.
 
 ## Proposal Examples
 
-| Example                   | Decoding                                                                                       |
-| ------------------------- | ---------------------------------------------------------------------------------------------- |
-| SCS-2C-4-10n              | 2 dedicated cores (x86-64), 4GiB RAM, 10GB network disk                                        |
-| SCS-8Ti-32-50p_i1         | 8 dedicated hyperthreads (insecure), Skylake, 32GiB RAM, 50GB local NVMe                       |
-| SCS-1L-1u-5               | 1 vCPU (heavily oversubscribed), 1GiB Ram (no ECC), 5GB disk (unspecific)                      |
-| SCS-16T-64-200s_GNa-64_ib | 16 dedicated threads, 64GiB RAM, 200GB local SSD, Infiniband, 64 Passthrough nVidia Ampere SMs |
-| SCS-4C-16-2x200p_a1       | 4 dedicated Arm64 cores (A76 class), 16GiB RAM, 2x200GB local NVMe drives                      |
-| SCS-1V-0.5                | 1 vCPU, 0.5GiB RAM, no disk (boot from cinder volume)                                          |
+| Example                        | Decoding                                                                                       |
+| ------------------------------ | ---------------------------------------------------------------------------------------------- |
+| `SCS-2C-4-10n`                 | 2 dedicated cores (x86-64), 4GiB RAM, 10GB network disk                                        |
+| `SCS-8Ti-32-50p_i1`            | 8 dedicated hyperthreads (insecure), Skylake, 32GiB RAM, 50GB local NVMe                       |
+| `SCS-1L-1u-5`                  | 1 vCPU (heavily oversubscribed), 1GiB Ram (no ECC), 5GB disk (unspecific)                      |
+| `SCS-16T-64-200s_GNa-72-24_ib` | 16 dedicated threads, 64GiB RAM, 200GB local SSD, Infiniband, 72 Passthrough nVidia Ampere SMs |
+| `SCS-4C-16-2x200p_a1`          | 4 dedicated Arm64 cores (A76 class), 16GiB RAM, 2x200GB local NVMe drives                      |
+| `SCS-1V-0.5`                   | 1 vCPU, 0.5GiB RAM, no disk (boot from cinder volume)                                          |
 
 ## Previous standard versions
 

diff --git a/Standards/scs-0100-w1-flavor-naming-implementation-testing.md b/Standards/scs-0100-w1-flavor-naming-implementation-testing.md
@@ -32,7 +32,8 @@ See the [README](https://github.com/SovereignCloudStack/standards/tree/main/Test
 for more details.
 
 The functionality of this script is also (partially) exposed via the web page
-[https://flavors.scs.community/](https://flavors.scs.community/).
+[https://flavors.scs.community/](https://flavors.scs.community/), which can both
+parse SCS flavors names as well as generate them.
 
 With the OpenStack tooling (`python3-openstackclient`, `OS_CLOUD`) in place, you can call
 `cli.py -v parse v3 $(openstack flavor list -f value -c Name)` to get a report
@@ -45,6 +46,107 @@ will create a whole set of flavors in one go.
 To that end, it provides different options: either the standard mandatory and
 possibly recommended flavors can be created, or the user can set a file containing his flavors.
 
+### GPU table
+
+The most commonly used datacenter GPUs are listed here, showing what GPUs (or partitions
+of a GPU) result in what GPU part of the flavor name.
+
+#### Nvidia (`N`)
+
+We show the most popular recent generations here. older one are of course possible as well.
+
+##### Ampere (`a`)
+
+One Streaming Multiprocessor on Ampere has 64 (A30, A100) or 128 Cuda Cores (A10, A40).
+
+GPUs without MIG (one SM has 128 Cude Cores and 4 Tensor Cores):
+
+| Nvidia GPU | Tensor C | Cuda Cores | SMs | VRAM      | SCS name piece |
+|------------|----------|------------|-----|-----------|----------------|
+|  A10       |  288     |  9216      |  72 | 24G GDDR6 | `GNa-72-24`    |
+|  A40       |  336     | 10752      |  84 | 48G GDDR6 | `GNa-84-48`    |
+
+GPUs with Multi-Instance-GPU (MIG), where GPUs can be partitioned and the partitions handed
+out as as pass-through PCIe devices to instances. One SM corresponds to 64 Cuda Cores and
+4 Tensor Cores.
+
+| Nvidia GPU | Fraction | Tensor C | Cuda Cores | SMs | VRAM      | SCS GPU name   |
+|------------|----------|----------|------------|-----|-----------|----------------|
+|  A30       | 1/1      | 224      | 3584       |  56 | 24G HBM2  | `GNa-56-24`    |
+|  A30       | 1/2      | 112      | 1792       |  28 | 12G HBM2  | `GNa-28-12`    |
+|  A30       | 1/4      |  56      |  896       |  14 |  6G HBM2  | `GNa-14-6`     |
+|  A30X      | 1/1      | 224      | 3584       |  56 | 24G HBM2e | `GNa-56h-24h`  |
+| A100       | 1/1      | 432      | 6912       | 108 | 80G HBM2e | `GNa-108h-80h` |
+| A100       | 1/2      | 216      | 3456       |  54 | 40G HBM2e | `GNa-54h-40h`  |
+| A100       | 1/4      | 108      | 1728       |  27 | 20G HBM2e | `GNa-27h-20h`  |
+| A100       | 1/7      |  60+     |  960+      |  15+| 10G HBM2e | `GNa-15h-10h`+ |
+| A100X      | 1/1      | 432      | 6912       | 108 | 80G HBM2e | `GNa-108-80h`  |
+
+[+] The precise numbers for the 1/7 MIG configurations are not known by the author of
+this document and need validation.
+
+##### Ada Lovelave (`l`)
+
+No MIG support, 128 Cuda Cores and 4 Tensor Cores per SM.
+
+| Nvidia GPU | Tensor C | Cuda Cores | SMs | VRAM      | SCS name piece |
+|------------|----------|------------|-----|-----------|----------------|
+|  L4        | 232      |  7424      |  58 | 24G GDDR6 | `GNl-58-24`    |
+| L40        | 568      | 18176      | 142 | 48G GDDR6 | `GNl-142-48`   |
+| L40G       | 568      | 18176      | 142 | 48G GDDR6 | `GNl-142h-48`  |
+| L40S       | 568      | 18176      | 142 | 48G GDDR6 | `GNl-142hh-48` |
+
+##### Grace Hopper (`g`)
+
+These have MIG support and 128 Cuda Cores and 4 Tensor Cores per SM.
+
+| Nvidia GPU | Fraction | Tensor C | Cuda Cores | SMs | VRAM       | SCS GPU name   |
+|------------|----------|----------|------------|-----|------------|----------------|
+| H100       | 1/1      | 528      | 16896      | 132 |  80G HBM3  | `GNg-132-80h`  |
+| H100       | 1/2      | 264      |  8448      |  66 |  40G HBM3  | `GNg-66-40h`   |
+| H100       | 1/4      | 132      |  4224      |  33 |  20G HBM3  | `GNg-33-20h`   |
+| H100       | 1/7      |  72+     |  2304+     |  18+|  10G HBM3  | `GNg-18-10h`+  |
+| H200       | 1/1      | 528      | 16896      | 132 | 141G HBM3e | `GNg-132-141h` |
+| H200       | 1/2      | 264      | 16896      |  66 |  70G HBM3e | `GNg-66-70h`   |
+| ... |
+
+[+] The precise numbers for the 1/7 MIG configurations are not known by the author of
+this document and need validation.
+
+#### AMD Radeon (`A`)
+
+##### CDNA 2 (`2`)
+
+One CU contains 64 Stream Processors.
+
+| AMD Instinct| Stream Proc | CUs | VRAM       | SCS name piece |
+|-------------|-------------|-----|------------|----------------|
+| Inst MI210  |     6656    | 104 |  64G HBM2e | `GA2-104-64h`  |
+| Inst MI250  |    13312    | 208 | 128G HBM2e | `GA2-208-128h` |
+| Inst MI250X |    14080    | 229 | 128G HBM2e | `GA2-220-128h` |
+
+##### CDNA 3 (`3`)
+
+SRIOV partitioning is possible, resulting in pass-through for
+up to 8 partitions, somewhat similar to Nvidia MIG. 4 Tensor
+Cores and 64 Stream Processors per CU.
+
+| AMD GPU     | Tensor C | Stream Proc | CUs | VRAM       | SCS name piece |
+|-------------|----------|-------------|-----|------------|----------------|
+| Inst MI300X | 1216     | 19456       | 304 | 192G HBM3  | `GA3-304-192h` |
+| Inst MI325X | 1216     | 19456       | 304 | 288G HBM3  | `GA3-304-288h` |
+
+#### intel Xe (`I`)
+
+##### Xe-HPC (Ponte Vecchio) (`12.7`)
+
+1 EU corresponds to one Tensor Core and contains 128 Shading Units.
+
+| intel DC GPU | Tensor C | Shading U | EUs | VRAM       | SCS name piece    |
+|--------------|----------|-----------|-----|------------|-------------------|
+| Max 1100     |  56      |  7168     |  56 |  48G HBM2e | `GI12.7-56-48h`   |
+| Max 1550     | 128      | 16384     | 128 | 128G HBM2e | `GI12.7-128-128h` |
+
 ## Automated tests
 
 ### Errors

diff --git a/Tests/iaas/flavor-naming/cli.py b/Tests/iaas/flavor-naming/cli.py
@@ -72,7 +72,7 @@ def parse(cfg, version, name, output='none'):
             if flavorname is None:
                 print(f"NOT an SCS flavor: {namestr}")
             elif output == 'prose':
-                printv(name, end=': ')
+                printv(namestr, end=': ')
                 print(f"{prettyname(flavorname)}")
             elif output == 'yaml':
                 print(yaml.dump(flavorname_to_dict(flavorname), explicit_start=True))

diff --git a/Tests/iaas/flavor-naming/flavor_names.py b/Tests/iaas/flavor-naming/flavor_names.py
@@ -212,7 +212,7 @@ class GPU:
     type = "GPU"
     component_name = "gpu"
     gputype = TblAttr("Type", {"g": "vGPU", "G": "Pass-Through GPU"})
-    brand = TblAttr("Brand", {"N": "nVidia", "A": "AMD", "I": "Intel"})
+    brand = TblAttr("Brand", {"N": "Nvidia", "A": "AMD", "I": "Intel"})
     gen = DepTblAttr("Gen", brand, {
         "N": {'': '(unspecified)', "f": "Fermi", "k": "Kepler", "m": "Maxwell", "p": "Pascal",
               "v": "Volta", "t": "Turing", "a": "Ampere", "l": "AdaLovelace", "g": "GraceHopper"},
@@ -222,7 +222,9 @@ class GPU:
               "3": "Arc/Gen12.7/DG2"},
     })
     cu = OptIntAttr("#.N:SMs/A:CUs/I:EUs")
-    perf = TblAttr("Performance", {"": "Std Perf", "h": "High Perf", "hh": "Very High Perf", "hhh": "Very Very High Perf"})
+    perf = TblAttr("Frequency", {"": "Std Freq", "h": "High Freq", "hh": "Very High Freq"})
+    vram = OptIntAttr("#.V:GiB VRAM")
+    vramperf = TblAttr("Bandwidth", {"": "Std BW {<~1GiB/s)", "h": "High BW", "hh": "Very High BW"})
 
 
 class IB:
@@ -278,7 +280,7 @@ class Outputter:
     hype = "_%s"
     hwvirt = "_%?"
     cpubrand = "_%s%0%s"
-    gpu = "_%s%s%s%-%s"
+    gpu = "_%s%s%s%-%s%-%s"
     ib = "_%?"
 
     def output_component(self, pattern, component, parts):
@@ -341,7 +343,7 @@ class SyntaxV1:
     hwvirt = re.compile(r"\-(hwv)")
     # cpubrand needs final lookahead assertion to exclude confusion with _ib extension
     cpubrand = re.compile(r"\-([izar])([0-9]*)(h*)(?=$|\-)")
-    gpu = re.compile(r"\-([gG])([NAI])([^:h]*)(?::([0-9]+)|)(h*)")
+    gpu = re.compile(r"\-([gG])([NAI])([^:h]*)(?::([0-9]+)|)(h*)(?::([0-9]+)|)(h*)")
     ib = re.compile(r"\-(ib)")
 
     @staticmethod
@@ -366,7 +368,7 @@ class SyntaxV2:
     hwvirt = re.compile(r"_(hwv)")
     # cpubrand needs final lookahead assertion to exclude confusion with _ib extension
     cpubrand = re.compile(r"_([izar])([0-9]*)(h*)(?=$|_)")
-    gpu = re.compile(r"_([gG])([NAI])([^\-h]*)(?:\-([0-9]+)|)(h*)")
+    gpu = re.compile(r"_([gG])([NAI])([^\-h]*)(?:\-([0-9]+)|)(h*)(?:\-([0-9]+)|)(h*)")
     ib = re.compile(r"_(ib)")
 
     @staticmethod
@@ -697,10 +699,14 @@ def prettyname(flavorname, prefix=""):
     if flavorname.gpu:
         stg += "and " + _tbl_out(flavorname.gpu, "gputype")
         stg += _tbl_out(flavorname.gpu, "brand")
-        stg += _tbl_out(flavorname.gpu, "perf", True)
         stg += _tbl_out(flavorname.gpu, "gen", True)
         if flavorname.gpu.cu is not None:
-            stg += f"(w/ {flavorname.gpu.cu} SMs/CUs/EUs) "
+            stg += f"(w/ {flavorname.gpu.cu} {_tbl_out(flavorname.gpu, 'perf', True)}SMs/CUs/EUs"
+            # Can not specify VRAM without CUs
+            if flavorname.gpu.vram:
+                stg += f" and {flavorname.gpu.vram} GiB {_tbl_out(flavorname.gpu, 'vramperf', True)}VRAM) "
+            else:
+                stg += ") "
     # IB
     if flavorname.ib:
         stg += "and Infiniband "