[RFC] Model annotation and config guidelines

[bergwolf]

There are two ways to describe a certain part of a model, annotations and structured config files. Right now, we use annotations to tell the purpose of a certain layer of a model artifact, as well as the creation time, author, name, title, etc. of the model. While it surely works, it also creates difficulties for model spec adopters to parse the spec. One possible improvement is to move some of them into a structured config file.

We use the guidelines below to decide if a model artifact description should be an annotation or in a structured config file:

• If a description belongs to a specific layer, it SHOULD be an annotation
• If a description describes a general property of the model, it SHOULD be in a config
• It is RECOMMENDED that a single layer of a model has no more than two annotations

Following the above guidelines, we created a list of existing annotations and whether each should be converted to a config file field.

Annotation	Meaninng	Annotation or Config
org.cnai.model.architecture	the model architecture, such as transformer, cnn, rnn, etc.	Config
org.cnai.model.family	the model family, such as llama3, gpt2, qwen2, etc.	Config
org.cnai.model.name	the model name, such as llama3-8b-instruct, gpt2-xl, qwen2-vl-72b-instruct, etc.	Config
org.cnai.model.format	the model format, such as onnx, tensorflow, pytorch, etc.	Config
org.cnai.model.param.size	the size of the model parameters	Config
org.cnai.model.precision	the model precision, such as bf16, fp16, int8, etc.	Config
org.cnai.model.quantization	the model quantization, such as awq, gptq, etc	Config
org.cnai.model.created	date and time on which the model was built	Config
org.cnai.model.authors	the contact details of the people or organization responsible for the model	Config
org.cnai.model.url	the URL to find more information on the model	Config
org.cnai.model.documentation	the URL to get documentation on the model	Config
org.cnai.model.source	the URL to get source code for building the model	Config
org.cnai.model.version	the version of the packaged software	Config
org.cnai.model.revision	the source control revision identifier for the packaged software	Config
org.cnai.model.vendor	the name of the distributing entity, organization or individual	Config
org.cnai.model.licenses	the license(s) under which contained software is distributed	Config
org.cnai.model.ref.name	the name of the reference for a target, valid only for descriptors on index.json within the model layout.	Config
org.cnai.model.title	the human-readable title of the model	Config
org.cnai.model.description	the human-readable description of the software packaged in the model	Config
org.cnai.model.readme	the layer is a README.md file (boolean)	Annotation
org.cnai.model.license	the layer is a license file (boolean)	Annotation
org.cnai.model.config	the layer is a configuration file (boolean)	Annotation
org.cnai.model.model	the layer is a model file (boolean)	Annotation
org.cnai.model.filepath	the file path of the layer	Annotation

[tarilabs]

Hi, thank you for starting this!

It is RECOMMENDED that a single layer of a model has no more than two annotations

Can you provide an example for this? I'm not sure for instance why I could not use org.cnai.model.filepath for a value of "model-00001-of-00002.safetensors" and org.cnai.model.model for a value of "true" for a given layer

.

Also, I assume this config is not meant to be used as Manifest.config as that would conflict with using OCI image compliancy, but please confirm/correct my understanding?

[bergwolf]

Can you provide an example for this? I'm not sure for instance why I could not use org.cnai.model.filepath for a value of "model-00001-of-00002.safetensors" and org.cnai.model.model for a value of "true" for a given layer

You can. As a guideline, let's try not to put too many annotations to a layer to avoid complexity. I put two as a recommended limit but we can surely discuss how many is "too many" :)

Also, I assume this config is not meant to be used as Manifest.config as that would conflict with using OCI image compliancy, but please confirm/correct my understanding?

There are two options, and we should discuss them:

1. Save it in a separate layer. It is supposed to be transparent to OCI runtimes but we need to double-check for compatibility:

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "artifactType": "application/vnd.cnai.model.manifest.v1+json",
  "config": {
    "mediaType": "application/vnd.oci.empty.v1+json",
    "digest": "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a",
    "size": 2
  },
  "layers": [
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar",
      "digest": "sha256:e258d248fda94c63753607f7c4494ee0fcbe92f1a76bfdac795c9d84101eb317",
      "size": 1234
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar",
      "digest": "sha256:e258d248fda94c63753607f7c4494ee0fcbe92f1a76bfdac795c9d84101eb317",
      "size": 2345
    }
  ]
}

2. Save it in the manifest.config field, as guided by https://github.com/opencontainers/image-spec/blob/e7f7c0ca69b21688c3cea7c87a04e4503e6099e2/manifest.md#guidelines-for-artifact-usage. It requires OCI runtimes to recognize the artifact type.

For artifacts with a config blob, specify the artifactType to a common value for your artifact tooling, specify the config with the metadata for this artifact, and include the artifact in the layers

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "artifactType": "application/vnd.cnai.model.manifest.v1+json",
  "config": {
    "mediaType": "application/vnd.cnai.model.config.v1.tar+gzip",
    "digest": "sha256:5891b5b522d5df086d0ff0b110fbd9d21bb4fc7163af34d08286a2e846f6be03",
    "size": 1234
  },
  "layers": [
    {
      "mediaType": "application/vnd.cnai.model.layer.v1.tar",
      "digest": "sha256:e258d248fda94c63753607f7c4494ee0fcbe92f1a76bfdac795c9d84101eb317",
      "size": 2345
    }
  ]
}

[tarilabs]

1. It is supposed to be transparent to OCI runtimes but we need to double-check for compatibility

I am afraid the option described in the example would not be accepted either in OCI registry, as it's not a valid OCI Image afaik? While I like the idea, I think we need to test against OCI registry and Container Runtimes, my2c

[bergwolf]

@tarilabs You are right! I've updated the example to use the media type application/vnd.oci.image.layer.v1.tar rather than application/vnd.cnai.model.layer.v1.tar so that it can be compatible with existing OCI registry and container runtimes.

One more thing I'm not 100% sure is that while application/vnd.oci.empty.v1+json is defined as an Empty Descriptor by the OCI image spec, I'm not sure if it is widely implemented. Maybe we should just use application/vnd.oci.image.config.v1+json for the manifest config media type?

[sajayantony]

The media type application/vnd.oci.image.config.v1+json is typically used only when the artifact is actually a container image. For this case having a structured config might be appropriate.
You could refer Helm's config media type - iana.org/assignments/media-types/application/vnd.cncf.helm.config.v1+json

For e.g.

{
  "schemaVersion": 2,
  "config": {
    "mediaType": "application/vnd.cncf.helm.config.v1+json",
    "digest": "sha256:34bf03806938a59ee7dc3e2c33e314d0eaef573313ff9dcc677113502d568523",
    "size": 145
  },
  "layers": [
    {
      "mediaType": "application/vnd.cncf.helm.chart.content.v1.tar+gzip",
      "digest": "sha256:87a12bc6175d031e6b0077f73d6db045967a2fc0cb29d98cb384f937953e3843",
      "size": 4172
    }
  ],
  "annotations": {
    "org.opencontainers.image.created": "2024-12-17T20:10:50Z",
    "org.opencontainers.image.description": "A Helm chart for Kubernetes",
    "org.opencontainers.image.title": "hello-world",
    "org.opencontainers.image.version": "0.1.0"
  }
}

And the associated config

oras manifest fetch-config localhost:5000/helm/hello-world/hello-world:0.1.0 --pretty
{
  "name": "hello-world",
  "version": "0.1.0",
  "description": "A Helm chart for Kubernetes",
  "apiVersion": "v2",
  "appVersion": "1.16.0",
  "type": "application"
}

[bergwolf]

@sajayantony Yeah, in the end we should use something like application/vnd.cnai.model.config.v1+json for the MediaType. It is a known issue and tracked in #17.

[gorkem]

As discussed on the call.

We can remove org.cnai.model.config, cnai.model.model as config and model are layer mediaTypes on the specification. We have also decided to remove org.cnai.model.readme, org.cnai.model.license in favour of org.cnai.model.filepath which lists the files included in a layer and we will use on the convention that the filenames for these start with README and LICENSE respectively.

[gorkem]

Another thought should we define the Config as a JSON schema? Right now, it looks too much like key/value pairs.

[gaius-qi]

If use application/vnd.oci.image.config.v1+json as config media type now, Model Spec should consider how to define the generation logic of diffIDs to prevent containerd from generating different snapshots.

#22

[bergwolf]

update: org.cnai.model.ref.name is only valid for index.json. Let's hold on adding support for manifest index at the moment. So plan to drop it.