Add schema.org sdLicense

[mkuchnik]

Add a field for the metadata's license

See #544

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[mkuchnik]

https://schema.org/sdLicense is in the spec now. I set the sdLicenses to Apache 2.0, since the repository is Apache 2.0. sdLicense is either None or string for now.

Note: migration for bigcode-the-stack/metadata.json seems to have lost some information. I'm not sure if this is intended or not.

[marcenacp]

Thanks! Just wrote a few comments.

Also, do you want to use apache-2.0 (Hugging Face way) or a URL (Croissant way described in the specs)?

Croissant recommends using the URL of a known license, e.g., one of the licenses listed at https://spdx.org/licenses/.

[mkuchnik]

@marcenacp The migration tool applies these changes even to v0.8, so I manually merged the diffs. Also, I think the migration tool may reorder the alphabetical order differently than a JSON formatter/linter, but this seems minor.

[mkuchnik]

@marcenacp Is there something that would block the merge if I rebased onto the current main?

[Zack-83]

I think that documentation is required to explain in what the fields license and sdLicense differ. Do they accept labels/strings or URIs?

[marcenacp]

@Zack-83 They are schema.org fields so you can find the documentation on respectively https://schema.org/license and https://schema.org/sdLicense. Does this make sense to you? We could explicitly point to those URLs.

@mkuchnik Nothing blocks the merge on my side. Do you have all the rights for merge?

[Zack-83]

Suggested change

	self.add_error(f"License should be a list of str. Got: {license}")
	self.add_error(f"The object license should be a list of strings. Got: {license}")

[Zack-83]

Suggested change

	self.add_error(f"sdLicense should be a str. Got: {sd_license}")
	self.add_error(f"The license for the metadata (schema:sdLicense) should be a string. Got: {sd_license}")

[Zack-83]

@Zack-83 They are schema.org fields so you can find the documentation on respectively https://schema.org/license and https://schema.org/sdLicense. Does this make sense to you? We could explicitly point to those URLs.

@mkuchnik Nothing blocks the merge on my side. Do you have all the rights for merge?

@marcenacp thanks for considering my suggestion. I suppose that it would be useful to reference the URLs or disambiguate the two terms in another way at whatever position where the user inputs the license manually or where he/she gets some output, e.g. in the error messages of metadata.py.