Delete datasets

[ddeboer]

The crawler notices when a registration URL (dataset description or catalog) was last seen as valid (schema:validUntil) but doesn’t yet act on this. It’s still an open question when we should delete a dataset from the Register.

Some options include:

• keep removed descriptions for future reference, just mark them as no longer available
• delete the dataset graph x days after the registration URL for it has been crawled successfully; or
• immediately delete the dataset graph if the catalog no longer contains the dataset.