IBM Cloud Object Storage Backup

A script to run in your Jenkins pipeline to back up IBM Cloud Object Storage instances.

Motivation • Key Features • Usage • Extra • Stack • Resources

Motivation

My DevOps team needed to meet Business Continuity and Disaster Recovery (BCDR) requirements.

Key Features

• Backup COS data from data tier accounts to backup account
• Dry run option for rclone
• Rclone profile with private endpoint option
• Get COS service instances
• Get COS service credentials
• Get buckets by COS service instance
• Create rclone profiles
• Create service credentials w/ HMAC keys for COS service instances

Usage

# Clone this repository
$ git clone https://github.com/chunwahchung/ibm-cos-backup.git

# Install ibmcloud cli (macOS)
curl -fsSL https://clis.cloud.ibm.com/install/osx | sh

# Install ibmcloud cos plugin
ibmcloud plugin install cloud-object-storage -r "IBM Cloud"

# Go into the repository
$ cd ibm-cos-backup

# Run script
$ ./ibm_cos_backup.sh -d your-backup-account-api-key -s your-data-tier-account-api-key 2>&1 | tee ibm_cos_backup.$(date +%m-%d-%Y-%T-%Z).stdout.txt

Note: The version of bash I'm running is GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin20)

Extra

RCLONE configuration

You should have a rclone profile for your source and destination COS service instances.

[your_profile_name]
type = s3
provider = IBMCOS
env_auth = true
access_key_id = the_access_key_id_for_your_COS_instance
secret_access_key = the_secret_access_key_for_your_COS_instance
endpoint = s3.your-region.cloud-object-storage.appdomain.cloud

IBM COS Endpoints

The following is mentioned in Bucket Configuration under the Endpoints section:

Whenever possible use a private endpoint. It requires your client service to:

• a) Not be Cloud Foundry
• b) Be hosted on the IBM cloud Get the best performance free of charge for all traffic."

Stack

• IBM Cloud Platform
• IBM Cloud Object Storage
• rclone
• Bash
• Jenkins
• Groovy

Resources

• IBM Cloud CLI
• IBM Cloud Object Storage
• rclone
• Creating an API key
• Install ibmcloud cli
• Install ibmcloud cos plugin

---

My Website chunwahchung.me  ·  LinkedIn chunwahchung