I want to create new manifests from one or more very large manifests

[tomcrane]

This is the Sorting Room scenario. It's a distinct App that lives in the Manifest Editor shell.

I have a large set of images, and I want to produce multiple smaller manifests from that large set. The large set might itself be a manifest, but in other scenarios if might not yet exist as IIIF at all.

Detailed user story to follow, we won't do this in this phase of the project.

[tomcrane]

Additional Canadiana comments:

For the manifest splitter, the Heritage team would love a flow where they break apart the manifest into smaller manifests. Then, they would sequentially edit each new manifest and save them one by one. Maybe having arrows to navigate between the smaller manifests. A scenario we would have is a batch of newspaper issues that are all one manifest. They need to be split up to smaller manifests and then have individual work done to each one.

Notes

I think this goes against what I just said on #133 (comment)

One (and only one) of these is always the target; the Manifest being created/edited.

...which is interesting and perhaps argues against the idea that they could be the same App.

In fact it might define the difference between these two apps:
The sorting room has one source, many targets
The "Blender" app in #133 has many sources, one target.

The Blender fits more neatly into the App framework, I think, because the persistence service of the Shell is no different from the regular Manifest Editor - there's one Manifest being worked on and its pushed out to a Publish target.

This model of the sorting room:

Then, they would sequentially edit each new manifest and save them one by one.

... is like how the original sorting room app worked, but that was designed for a specific back end and workflow (for IDA).

We'd need to think about a Publish target that can accept multiple save destinations within the same Project (as defined in #146) which needs some further thinking.

This needs a https://github.com/digirati-co-uk/iiif-manifest-editor/labels/narrative written for it - which I can do!

[tomcrane]

One way of doing this is for Sorting Room to be a kind of Range editor - you mark out ranges on the big manifest, an external process can be responsible for creating new manifests from those ranges.

[tomcrane]

Is our basic MVP here a canvas-level range editor (not going down into regions).
Then people can externally process manifests that have had ranges added - here creating new manifests from the ranges.

[tomcrane]

Canadiana notes:

1. Manifest Splitting App

Because they might be splitting a reel into 1000 manifests, staff envision a workflow like:

1. From the app menu, the user opens the split manifest app.
2. A view appears showing all the canvases in the manifest in a grid on the left.
3. On the right, they see a table. Each row on the table will represent a manifest the user wants to create. The table allows users to enter: Label, summary, start canvas, end canvas, viewing direction, and behaviour for the sub manifests (ignore table titles in the image below.)
4. When the user clicks a manifest in the table, the canvases shown on the left side of the screen are highlighted between its start and end position.
5. When the user initiates the splitting, they can watch the progress of each manifest being created.
6. After the manifest has been split, the user can download a CSV of the results, which includes a link to the manifest that can be opened in the Digirati editor.
7. A button also appears for each manifest in the table that allows the user to open the manifest in the Digirati editor.

[tomcrane]

Is this interface actually a Range editor?
You're marking up the manifest into sections - these could be modelled as ranges. They have labels, summaries, whatever.

The big problem with this is saving - how do you get all your new manifests out of the editor? Where are they going?
By saving the original reel but now with additional Ranges, you give the task of creating the new manifests to something else.

[RussellMcOrmond]

My personal thinking is that this could be an external app in the shorter term, and wouldn't be part of an MVP. We are wanting to adopt new applications, but this also means new workflows for staff and new ways of thinking. I'm a fan of having a larger number of incremental upgrades rather than large deployments and documentation/training/etc.

I know we make assumptions about the compute capabilities of clients, but as a systems person I would prefer we didn't create massive manifests with hundreds of ranges referencing several thousands of images (IE: a scanned reel). A "book" that happened to have been filmed and then scanned as part of a reel should, in my mind, become a manifest of that book.

[brittnylapierre]

Is this interface actually a Range editor?
...

• That makes sense to me. Each row indicates a range. The table view came out of reels having potentially 1000 ranges in them, the team felt like the only reasonable way to split up the reel would be via rows in a table

The big problem with this is saving - how do you get all your new manifests out of the editor? Where are they going?
...

• Cool! I like this idea. So, our DIP manifests in our IIIF API for DIPs would be saved with the ranges specified. Then a service of our own making would go off and create IIIF manifests for each document for our access platform IIIF API. @RussellMcOrmond

I like the range idea because then there is a record for the staff so that they know what portions of the originating DIP/AIP manifest has been processed into a document level manifest. Some day it would be cool to be able to point them directly to that range in the DIP manifest from the document level manifest.

As I was modeling workflows the recording of DIP processing was one sticky point, which this helps solve! 😄

[RussellMcOrmond]

• Cool! I like this idea. So, our DIP manifests in our IIIF API for DIPs would be saved with the ranges specified. Then a service of our own making would go off and create IIIF manifests for each document for our access platform IIIF API.

Possibly a level of detail of our future processes that isn't relevant to the editor.

We will have a service which takes images stored in OAIS AIPs, generate canvases from them, and also generate manifests with the same structure as the AIPs. These manifests would be the source used to then create other manifests that reference those same canvases. This would be a read-only Presentation service.

Generally, the sources used to bring canvases into a manifest being edited will quite regularly be different servers. Sometimes it won't even be from servers we are running ourselves.

[tomcrane]

To summarise

As was the case with our original sorting room, the Manifests to be cut up are never intended for public consumption. A 5000-image microfilm roll could be worked on in many ways, we choose to turn it into a manifest because that makes it available to multiple visual tools. We understand that a huge manifest like that is not going to be a good user experience, just as a 5000 page book would not be. The reel manifest is a useful means to the end of sensible individual document manifests.

Much earlier I said:

We'd need to think about a Publish target that can accept multiple save destinations within the same Project (#43 (comment))

...but we don't, if as @RussellMcOrmond suggests this is entirely external to the Editor.

We will have a service which takes images stored in OAIS AIPs, generate canvases from them, and also generate manifests with the same structure as the AIPs. These manifests would be the source used to then create other manifests that reference those same canvases. This would be a read-only Presentation service.

• External processes generate large manifest
• A large manifest is ready at some URL, editor opens it
• Staff member marks up runs of canvases within this large manifest by creating Ranges with whatever labels and metadata are required
• The large manifest, now with its Ranges, is saved to (some location)
• External process picks up this manifest and generates n new manifests from the Ranges
• External process saves those manifests to (...)
• If necessary staff can edit/enrich those new manifests in the editor, but at this point they are just normal manifests

The ME doesn't need to worry about saving to multiple targets.

[tomcrane]

(Discovery: review the conclusion of the above thread that this is actually a Range Editor)

[RussellMcOrmond]

My personal commentary for several of these tickets:

Cut/copy/paste of multi-selected items feels to me to be a generic solution to many of the use cases that were presented.

While requests from users may be specific to a scenario (Creating a new manifest/collection from another manifest/collection, merging multiple manifests/collections, splitting manifests/collections, etc), I believe the process can be made very generic. The specific scenarios can be explained in documentation. I'm a very big fan of keeping the software as simple as possible, and help clarify complex operations in documentation.

It feels to me that by the time staff have set a new identifier and other metadata in a range (so the manifests created by the external process will have known identifiers/labels/metadata/etc), they could have (IMHO more easily) opened a new blank manifest and used cut/copy/paste from a source manifest window to fill in the required images.

I believe staff should have the right hardware to do the job, and for manifest editing that will include a multi-screen setup so they have the screen space to have the appropriate browser windows open to handle all of these scenarios easily.

[tomcrane]

We may not need a Range Editor yet. So doing this as suggested in the comment above would be simpler.