Script to create Oracle slim images for all versions

[marcelo-ochoa]

This PR is a super-set of #1534
The idea is to provide an slim version of Oracle Docker image image only with binaries and the ability to recreate a fresh Oracle instance from golden images in RMAN compressed full backup format.
Once of mainly motivation of smaller image size and faster creation time is to have a Docker image ready to be used in CI/CD pipelines which requires a fresh RDBMS for unit testing, also an slim image will reduce the network traffic between the CI/CD testing server and the private/public registry hosting the slim image.
Comparison results using this new image:
Image size around 30% of the original:

19.3.0-ee-slim 2.56GB
19.3.0-ee 6.51GB
18.3.0-ee-slim 2.51GB
18.3.0-ee 8.4GB
18.4.0-xe-slim 2.48GB
18.4.0-xe 5.87GB
12.2.0.1-ee-slim 2.3GB
12.2.0.1-ee 5.98GB
12.1.0.2-ee-slim 1.84GB
12.1.0.2-ee 5GB
11.2.0.2-xe-slim 729MB
11.2.0.2-xe 1.13GB

DB Creation time, 3x faster, sample using XE version:

dbca : 8".8s (traditional way)
init-db.tar.gz : 3".5s
rman comp. : 2".59s

Docker image creation process uses multi-stage build and a modified script shared by Gerald Venzl and Jacek Gebal to cleanup non necessary stuff.
To get a golden image file either in format tar, tar.gz an RMAN directory with a setup script could be used in conjunction with oracle:18.4.0 image as:

$ docker run -ti --rm --name freshdb -v /tmp/setup:/opt/oracle/scripts/setup -v /run/shm:/dev/shm oracle/database:18.4.0-xe

directory /tmp/setup having:

$ ll /tmp/setup/
total 20
drwxr-xr-x 2 oracle oinstall  4096 feb 26 16:29 ./
drwxr-xr-x 8 mochoa mochoa   12288 feb 26 16:29 ../
-rwxr-xr-x 1 oracle oinstall   330 feb 26 16:29 zz-backup-full.sh*
$ cat /tmp/setup/zz-backup-full.sh
#!/bin/bash
[[ $UID = "54321" ]] && rman target /<<EOF
shutdown immediate;
startup mount;
backup as compressed backupset database format '$ORACLE_BASE/scripts/setup/%U';
backup current controlfile format '$ORACLE_BASE/scripts/setup/control.bks';
backup spfile format '$ORACLE_BASE/scripts/setup/spfile.bks';
alter database open;
exit;
EOF
[[ $UID = "0" ]] && su -p oracle -c "rman target /<<EOF
shutdown immediate;
startup mount;
backup as compressed backupset database format '$ORACLE_BASE/scripts/setup/%U';
backup current controlfile format '$ORACLE_BASE/scripts/setup/control.bks';
backup spfile format '$ORACLE_BASE/scripts/setup/spfile.bks';
alter database open;
exit;
EOF"

once a log output shows DATABASE IS READY TO USE! setup directory will have a golden image DB as RMAN format which will be passed as setup directory for oracle/database:18.4.0-xe-slim image or packaged as .tar or .tar.gz file format, here the content of full backup directory:

$ ll /tmp/setup/
total 699436
drwxr-xr-x 2 oracle oinstall      4096 feb 26 16:53 ./
drwxr-xr-x 8 mochoa mochoa       12288 feb 26 16:29 ../
-rw-r----- 1 oracle oinstall 347537408 feb 26 16:52 01uphaqd_1_1
-rw-r----- 1 oracle oinstall 175022080 feb 26 16:52 02upharq_1_1
-rw-r----- 1 oracle oinstall 174718976 feb 26 16:53 03uphasj_1_1
-rw-r----- 1 oracle oinstall  18808832 feb 26 16:53 control.bks
-rw-r----- 1 oracle oinstall    114688 feb 26 16:53 spfile.bks
-rwxr-xr-x 1 oracle oinstall       330 feb 26 16:29 zz-backup-full.sh*

Make sure that /tmp/setup is owned by Oracle (54321:54321) in order to get RMAN working.
To pack RMAN content as init-db.tar.gz just use:

$ sudo tar cvfz /tmp/init-db.tar.gz control.bks spfile.bks 0*
control.bks
spfile.bks
01uphaqd_1_1
02upharq_1_1
03uphasj_1_1

To start a new container using a golden RDBMS image use:
$ docker run -ti --rm --name testdb -v /tmp/setup:/opt/oracle/scripts/setup -v /run/shm:/dev/shm oracle/database:18.4.0-xe-slim

Each slim images requires a full Docker image built with tags -e or -x, for example:

$ mv /tmp/Downloads/LINUX.X64_193000_db_home.zip 19.3.0/
$ ./buildDockerImage.sh -i -v 19.3.0 -e
$ mv 19.3.0/LINUX.X64_193000_db_home.zip /tmp/Downloads/
$ ./buildDockerImage.sh -i -v 19.3.0 -e -l
$ docker image ls|grep 19.3.0
oracle/database                                       19.3.0-ee-slim        39d1395797f3        3 hours ago         2.56GB
oracle/database                                       19.3.0-ee             dc77d5310ce7        3 hours ago         6.51GB

Note that .zip file with install binaries are not required to build slim images, it uses ee image to extract the layer of binary files only, also I added a flag -l into buildImages.sh script to facilitate build process.

[marcelo-ochoa]

Yes #1534 is included in this new PR.

[Djelibeybi]

@gvenzl please go ahead and review this PR. @marcelo-ochoa has signed an OCA.

[gvenzl]

@yunus-qureshi, @rishabh20, @latakannan, can you please look at this PR and see whether it's feasible to add to the scripts?

[srderson]

@gvenzl @Djelibeybi - Any updates as to whether this may be merged at some point in the future?

[AnEmortalKid]

I gave a shot to this approach and ran into a couple of snags particularly around the permissions on the directory, might be related to mac/docker or not setting -u but just doing a chown -R 54321:54321 /tmp/setup would result in failures during the RMAN step. This was solved by just 😡 setting the permissions to 777 in the interim (if anyone trying to test this out runs into it and ends up frustrated), might also be related to not having an oracle user. Thank you so much for this and the steps @marcelo-ochoa !

To re-iterate the steps for anyone else following, they're something like this:

PreRequisite

• Have both the versions of the images (build them with buildDockerImage.Sh

$ mv /tmp/Downloads/LINUX.X64_193000_db_home.zip 19.3.0/
$ ./buildDockerImage.sh -i -v 19.3.0 -e
$ mv 19.3.0/LINUX.X64_193000_db_home.zip /tmp/Downloads/
$ ./buildDockerImage.sh -i -v 19.3.0 -e -l

Build your RMAN base

1. Setup the directory to mount to /opt/oracle/scripts/setup with the right permissions and the zz-backup-full.sh
2. Build your container with the mounted directory off or the non slim image
3. Once that's done, you have your RMAN base

Build your Container with SLIM

1. Mount a directory with the RMAN files (or a tar / tar.gz) version of them on to /opt/oracle/scripts/setup
2. Start the container with the slim image

I first read the 'slim' name and thought I could just use this for everything, but there's some steps needed prior to that (didn't read due to excitement) 🤦

[yunus-qureshi]

@marcelo-ochoa can we please resolve conflicts and repush

[marcelo-ochoa]

rename build script

[ripleymj]

I'm not sure why I'm seeing this when no one else is, but I tried borrowing this cleanup script for a different 19.3 EE container build and the DB failed to start after applying it. The error was ORA-12777, and MOS doc id 2741007.1 seems to have correctly identified that this missing install directory was the culprit. With this line and next removed, everything appears to be fine now.

[yunus-qureshi]

I am not fully convinced with this 2 step manual process.

@marcelo-ochoa @gvenzl how about converting this into an extension called slimming. Along with this new pre-create-database extension when the base image is extended with both we would get a slimmed down image with a pre-created DB for use in CI/CD pipelines.

[felipecrs]

I can't wait to put my hands on a slimmed + pre-started Oracle image. :-)

[milanvdm]

Any progress on this? Ive been building this PR manually for quite some time now and it would be great to get this merged to have a better CI/CD experience :)

[66Ton99]

@yunus-qureshi @gvenzl @rishabh20 @latakannan are you going to merge it?

[yunus-qureshi]

@marcelo-ochoa please resolve conflicts

[Djelibeybi]

The pull request doesn't need to be closed for the conflicts to be resolved. Once they are, this PR can be merged.

[yunus-qureshi]

@Djelibeybi thanks for reopening. Was closed by mistake