Initial Phase implementation

[weiweishi]

The initial implementation of CWRC Preservation

In Scope

Step 1: Develop the script to:

• fetch a list of objects in a JSON manifest that need to be preserved from CWRC, in the format of {pid:"******",timestamp:"********"}, where the pid is the object's persistent identifier, and timestamp is the lastModified timestamp from Fedora 3 in Islandora.
• Use this list to

  • fetch the header of each AIP as below

    CWRC-MODIFIED_DATE: "2016-12-21T20:14:23.888Z"
    CWRC-CHECKSUM: "66fafd8b3c023637ad10ebbc899c0bb6"
    CWRC-PID: "orlando:1bdcd4f4-11f5-4bb0-b6a4-5caaaca2079a"
    

  • Use the information in the header and in the JSON manifest to check if the object has already been ingested:

    • same pid
    • same LastModified timestamp (if CWRC-MODIFIED_DATE is newer than timestamp in JSON manifest, use CWRC-MODIFIED_DATE. Error if CWRC-MODIFIED_DATE is older - they might not be the same if the object has been changed between the JSON manifest is generated and when library script fetches the list)

  • If the object has yet to be ingest, download the bagged AIP

  • check the checksum in the header, with the checksum of the AIP downloaded

  • verify the bag with bagit gem

  • ingest the bag with swift-ingest gem

Step 2, before production deployment of this process,
DB logging for tracking preservation ingest in swift_ingest ualbertalib/swift_ingest#2

out of scope for initial phase

• check duplicates inside swift_ingest gem
• queue for ingest
• handle failed ingests