Protocol for changes

[stephenwf]

This is a companion to This Madoc discussion where the process of integrating with external tools is being planned. Typically in Manifest Editing tools you will bring a Manifest to the tool, make your changes and then get back the Manifest (as json) with the changes you have made. Here I want to discuss other ways we could approach this - and how at a fundamental level we could make consuming changes from a Manifest Editor easier but also more flexible.

Simple use-case: Manifest label editor

For simplicity, let's strip the idea of this editor back to the absolute minimum. We have a UI that will take in a IIIF Manifest, and let you edit the label for that Manifest.

This is my editor. Now when I hit "save", what do we have? What could we have?

We have the new JSON, but we don't know what we did.

{ "label": {"none": ["Forth Bridge illustrations"] }}

We could also have a more verbose change:

{
  "type": "change-value",
  "property": "label",
  "value": {"none": ["Forth Bridge illustrations"] },
  "previous_value": {"none": ["Forth Bridge illustrations 1886-1887"] },
}

This is along the lines of what Madoc is proposing to do. Being able to take a Manifest, hand it off to an external tool, get the JSON back and then creating a list of "changes" by diffing them.

Some pseudo code to describe

function diffManifest(manifest, newManifest) {
  const changes = [];

  if (!fieldsMatch(manifest.label, newManifest.label)) {
    changes.push({
      type: 'change-value',
      property: 'label',      
      value: newManifest.label,
      previous_value: manifest.label,
    });
  }

  return changes;
}

The problem is that every tool may end up doing the same thing. Implementing some diff-ing and processing to update an external system - or take the changes from a Manifest Editor as a whole and replacing an entire JSON.

So if the Manifest Editor - in addition to providing a new Manifest as JSON - provided a clean well-defined list of changes then external system could consume these and apply changes on their own collection management systems.

Challenges

• How this would tie into ID generation for new entities (annotations / canvases)
• How more complex interactions like re-ordering or translation may work
• Could some things be treated as immutable (such as an annotation) and force the diff to include a replace instead of an edit
• What sort of UI could be generated from this changelog, could this be displayed in a manifest editor to summarise what has been changed
• How would this data be transmitted back to a containing API or client
• Could this "summary" be applied within Manifest editor as a way to store changes to a Manifest without saving the full manifest?
• Could a manifest editor have features turned on/off based on what the containing application supports?
• Could other specialty tools be built on this foundation, such as a "sorting room" style application with the same protocol?

[tomcrane]

For structural changes this may get really difficult to determine what the change actually is - whether it's an edit or a deletion.

Have I moved a range? Or created a different range?

Even a metadata value - any diff would have to apply to the whole metadata block and could only be change.
E.g., if there were 9 metadata entries before, and now when I save there are 10 - you don't really know what the change was. Did I remove two and add one?

Second Q

How dependent is this process on the manifest editor itself?

E.g., can the manifest editor be ignorant of how the diff is deduced?

[consuming app] - [diff bridge] - [manifest editor]

The "bridge" makes a note of the manifest when it's handed off... it then spends an unknown amount of time being edited.
At some point the user saves (or special-saves - save-and-hand-back) and the manifest editor sends the manifest to the bridge and it's this bridge that provides the "clean well-defined list of changes" back to the original app?

(like GitHub doesn't know what I'm up to when I'm editing code, it has to generate the diff on commit).

I guess you could package the "bridge" with the editor, but it's separate, not intertwined.
So the manifest editor could be developed independently of the change/diff functionality.

[stephenwf]

Have I moved a range? Or created a different range?

I think this is where stepping through each of the types of mutations would be valuable. Take each on a case-by-case basis. We would definitely need to store the changes as IIIF in the Vault, but the question is do we need to store additional information too. Ranges might be a case where the answer is no, we could differentiate that example with identifier changes. But with metadata it might be the case that we store the output as IIIF, but then also store the changes made more granularly.

For metadata in particular, we might want to store some extra information when editing that pass to a diff-bridge, to make it easier to update and resolve potential conflicts.

E.g., can the manifest editor be ignorant of how the diff is deduced?

I think it can be, yes. If the "editor" part that handled the mutations -> vault interaction was a library (probably extending: https://github.com/digirati-labs/hyperion/tree/master/packages/editor) then we could create an API for metadata editing:

const {
  metadata,
  setMetadataValue,
  setMetadataLabel,
  reorderMetadata,
  // ... etc
} = useMetadataEditor();

Which could improve diffing transparently. So long as the API used in the editor itself is well planned. For example a single function: updateAllMetadata() would limit what we could diff-wise.