Dataset Migration API documentation improvements #11192
Conversation
|
FWIW: The geospatial and social blocks are given local context URIs because those blocks do not have a blockURI defined. See the docs or the citation block for how that can be done - dataverse/scripts/api/data/metadatablocks/citation.tsv Lines 1 to 2 in 7283dbd Without that, a block is considered to be locally defined with terms different (w.r.t. URI) from those at other instances. That's possibly useful for experimental blocks, or modified ones, but leads to the problem with the migration API (and I assume semantic API as well) unless the receiving Dataverse instance has blocks that set the blockURI to be the same as the auto-generated one in the source Dataverse. |
| @@ -5,7 +5,7 @@ The Dataverse software includes several ways to add Datasets originally created | |||
|
|
|||
| This experimental migration API offers an additional option with some potential advantages: | |||
|
|
|||
| * metadata can be specified using the json-ld format used in the OAI-ORE metadata export | |||
| * metadata can be specified using the json-ld format used in the OAI-ORE metadata export (please note that the json-ld generated by OAI-ORE metadata export is not directly compatible with the Migration API, check example file below for reference) | |||
There was a problem hiding this comment.
Can we say a little more about how the generated OAI-ORE format is not compatible? I'm a bit confused and not sure what I should be taking away from a look at the example file. 🤔
| @@ -31,7 +31,7 @@ To import a dataset with an existing persistent identifier (PID), the provided j | |||
| curl -H X-Dataverse-key:$API_TOKEN -X POST $SERVER_URL/api/dataverses/$DATAVERSE_ID/datasets/:startmigration --upload-file dataset-migrate.jsonld | |||
| An example jsonld file is available at :download:`dataset-migrate.jsonld <../_static/api/dataset-migrate.jsonld>` . Note that you would need to replace the PID in the sample file with one supported in your Dataverse instance. | |||
| An example jsonld file is available at :download:`dataset-migrate.jsonld <../_static/api/dataset-migrate.jsonld>` . Note that you would need to replace the PID in the sample file with one supported in your Dataverse instance. You also need to replace the dataverse.siteUrl in jsonld @context with your current Dataverse site URL. | |||
|
|
|||
There was a problem hiding this comment.
I'm just putting this comment at the bottom.
@mjlassila from our conversation I was sort of hoping you'd include your BaseX and XQuery script at https://gist.github.com/mjlassila/ecdbd11447ccdf87995db20bfc5e686c 😄 . It it worth writing up and including? (I don't even know how to run it but I'm happy to learn!)
There was a problem hiding this comment.
@pdurbin I added details about OAI-ORE format used in the export and included simple XQuery script for conversion. XQuery is a rather esoteric language and not very well known, but it is highly powerful in many use cases, such as format conversion.
@qqmyers : Thanks for clarification! If out of the box geospatial and social blocks do not have blockURI defined, should I keep the example file as it is or modify it somehow? If I leave these out from the example file @context, citation metadata gets imported via Dataset Migration API but geospatial and social blocks will drop out silently. |
|
@mjlassila - that's hard to answer clearly. I just created #11195 as an issue to get those blocks changed. Once a change is merged there, we probably want the docs to talk about the current situation and probably still note that migrating from earlier versions has the problem you're documenting. To not hold up your PR, I'd suggest that you just go ahead and document how it is now (add additional explanation if you want) and we'll treat updating the docs as part of completing #11195. |
…uggest using XQuery to do json-ld transformation and add simple script for reference.
cmbz
added
FY25 Sprint 15
@qqmyers - I added a short explanation of why defining a local URI for community metadata blocks is currently necessary. |
What this PR does / why we need it:
This PR is related to topic Metadata format for Dataset Migration API in 6.5 in the chat.
I added a mention to the docs that the OAI-ORE format used in the export is not directly compatible with the Dataset Migration API and made the example file more comprehensive by including few additional fields from Geospatial and Social Science and Humanities Metadata blocks.
Preview at https://dataverse-guide--11192.org.readthedocs.build/en/11192/developers/dataset-migration-api.html