diff --git a/ADVANCED_USAGE.md b/ADVANCED_USAGE.md new file mode 100644 index 000000000..ec66f8b5e --- /dev/null +++ b/ADVANCED_USAGE.md @@ -0,0 +1,46 @@ +Configuration files +------------------------------- + +The layering and s2i build flows noted above for updating the list of plugins can also be used to update the configuration injected into the Jenkins deployment. However, don't forget the note about copying of config data and Persistent Volumes in the [startup notes](#startup-notes-for-the-jenkins-core-image). + +A typical scenario employed by our users has been extending the Jenkins image to add groovy init scripts to customize your Jenkins installation. + +A quick recipe of how to do that via layering would be: + +* mkdir -p contrib/openshift/configuration/init.groovy.d +* create a contrib/openshift/configuration/init.groovy.d/foo.groovy file with whatever groovy init steps you desire +* create a Dockerfile with (adjusting the image ref as you see fit) + +``` +FROM registry.access.redhat.com/ocp-tools-4/jenkins-rhel8:latest +COPY ./contrib/openshift /opt/openshift +``` + +And then update your Jenkins deployment to use the resulting image directly, or update the ImageStreamTag reference you Jenkins deployment is employing, with our new image. During startup, +the existing run script your new image inherits from this repositories Jenkins image will copy the groovy init script to the appropriate spot under the Jenkins home directory. + + +Usage +--------------------------------- + +For this, we will assume that you are using an `quay.io/openshift/origin-jenkins` for v4.x. + +If you want to set only the mandatory environment variables and store the database +in the `/tmp/jenkins` directory on the host filesystem, execute the following command: + +``` +$ docker run -d -e JENKINS_PASSWORD= -v /tmp/jenkins:/var/lib/jenkins openshift/jenkins-2-centos7 +``` + + +Jenkins admin user +--------------------------------- + +Authenticating into a Jenkins server running within the OpenShift Jenkins image is controlled by the [OpenShift Login plugin](https://github.com/openshift/jenkins-openshift-login-plugin), taking into account: + +* Whether or not the container is running in an OpenShift Pod +* How the [environment variables](https://github.com/openshift/jenkins#environment-variables) recognized by the image are set + +See the [OpenShift Login plugin documentation](https://github.com/openshift/jenkins-openshift-login-plugin) for details on how it manages authentication. + +However, when the default authentication mechanism for Jenkins is used, if you are using the OpenShift Jenkins image, you log in with the user name `admin`, supplying the password specified by the `JENKINS_PASSWORD` environment variable set on the container. If you do not override `JENKINS_PASSWORD`, the default password for `admin` is `password`. diff --git a/BASIC_USAGE.md b/BASIC_USAGE.md new file mode 100644 index 000000000..2eea2beac --- /dev/null +++ b/BASIC_USAGE.md @@ -0,0 +1,56 @@ +# Basic Uage of Jenkins Images + +When you run you startup the main Jenkins image in an OpenShift pod for the first time, it performs various set up actions, including: +* Setting the JVM parameters for the actual start of the Jenkins JVM +* Updating the /etc/passwd so that the random, non-root user ID employed works +* Copies all the default configuration from the image to the appropriate locations under the Jenkins home directory (which maps to the image's volume mount) +* Copies all the plugins to the appropriate locations under the Jenkins home directory + +By default, all copies to the Jenkins home directory are only done on the initial startup if a Persistent Volume is employed for the Jenkins deployment. There are ways to override that behavior by environment variables (see the next section below). But you can also recycle the PVCs during restarts of your Jenkins deployment if you update the image being used and want to reset the system that way. + + +Environment variables +--------------------------------- + +The image recognizes the following environment variables that you can set during +initialization by passing `-e VAR=VALUE` to the Docker run command. + +| Variable name | Description | +| :------------------------ | ----------------------------------------- | +| `JENKINS_PASSWORD` | Password for the 'admin' account when using default Jenkins authentication. | +| `OPENSHIFT_ENABLE_OAUTH` | Determines whether the OpenShift Login plugin manages authentication when logging into Jenkins. | +| `OPENSHIFT_PERMISSIONS_POLL_INTERVAL` | Specifies in milliseconds how often the OpenShift Login plugin polls OpenShift for the permissions associated with each user defined in Jenkins. | +| `INSTALL_PLUGINS` | Comma-separated list of additional plugins to install on startup. The format of each plugin spec is `plugin-id:version` (as in plugins.txt) | +| `OVERRIDE_PV_CONFIG_WITH_IMAGE_CONFIG` | When running this image with an OpenShift persistent volume for the Jenkins config directory, the transfer of configuration from the image to the persistent volume is only done the first startup of the image as the persistent volume is assigned by the persistent volume claim creation. If you create a custom image that extends this image and updates configuration in the custom image after the initial startup, by default it will not be copied over, unless you set this environment variable to some non-empty value other than 'false'. | +| `OVERRIDE_PV_PLUGINS_WITH_IMAGE_PLUGINS` | When running this image with an OpenShift persistent volume for the Jenkins config directory, the transfer of plugins from the image to the persistent volume is only done the first startup of the image as the persistent volume is assigned by the persistent volume claim creation. If you create a custom image that extends this image and updates plugins in the custom image after the initial startup, by default they will not be copied over, unless you set this environment variable to some non-empty value other than 'false'. | +| `OVERRIDE_RELEASE_MIGRATION_OVERWRITE` | When running this image with an OpenShift persistent volume for the Jenkins config directory, and this image is starting in an existing deployment created with an earlier version of this image, unless the environment variable is set to some non-empty value other than 'false', the plugins from the image will replace any versions of those plugins currently residing in the Jenkins plugin directory. | +| `SKIP_NO_PROXY_DEFAULT` | This environment variable applies to the agent images produced by this repository. By default, the agent images will create/update the 'no_proxy' environment variable with the hostnames for the Jenkins server endpoint and Jenkins JNLP endpoint, as communication flows to endpoints typically should *NOT* go through a HTTP Proxy. However, if your use case dictates those flows should not be exempt from the proxy, set this environment variable to any non-empty value other than 'false'. | +| `ENABLE_FATAL_ERROR_LOG_FILE` | When running this image with an OpenShift persistent volume claim for the Jenkins config directory, this environment variable allows the fatal error log file to persist if a fatal error occurs. The fatal error file will be located at `/var/lib/jenkins/logs`. | +| `NODEJS_AGENT_IMAGE` | Setting this value will override the image used for the default NodeJS agent pod configuration. For 3.x, the default NodeJS agent pod uses `docker.io/openshift/jenkins-agent-nodejs-8-centos7` or `registry.redhat.io/openshift3/jenkins-agent-nodejs-8-rhel7` depending whether you are running the centos or rhel version of the Jenkins image. This variable must be set before Jenkins starts the first time for it to have an effect. For 4.x, the image is included in the 4.0 payload via an imagestream in the openshift namespace, and the image spec points to the internal image registry. If you are running this image outside of OpenShift, you must either set this environment variable or manually update the setting to an accessible image spec. | +| `MAVEN_AGENT_IMAGE` | Setting this value overrides the image used for the default maven agent pod configuration. For 3.x, the default maven agent pod uses `docker.io/openshift/jenkins-agent-maven-35-centos7` or `registry.redhat.io/openshift3/jenkins-agent-maven-35-rhel7` depending whether you are running the centos or rhel version of the Jenkins image. For 4.x, the image is included in the 4.0 payload via an imagestream in the openshift namespace, and the image spec points to the internal image registry. If you are running this image outside of OpenShift, you must either set this environment variable or manually update the setting to an accessible image spec. This variable must be set before Jenkins starts the first time for it to have an effect. | +| `JENKINS_UC_INSECURE` | When your Jenkins Update Center repository is using a self-signed certificate with an unknown Certificate Authority, this variable allows one to bypass the repository's SSL certificate check. The variable applies to download of the plugin which may occur during Jenkins image build, if you build an extension of the jenkins image or if you run the Jenkins image and leverage one of the options to download additional plugins (use of s2i whith plugins.txt or use of `INSTALL_PLUGINS` environment variable. | +| `MAVEN_MIRROR_URL` | Allows you to specify a [Maven mirror repository](https://maven.apache.org/guides/mini/guide-mirror-settings.html) in the form of `MAVEN_MIRROR_URL=https://nexus.mycompany.com/repository/maven-public`. The mirror repository is used by maven as an additional location for artifacts downloads. For more details on [how maven mirrors works](https://maven.apache.org/guides/mini/guide-mirror-settings.html), you can refer to maven documentation. +| `USE_JAVA_VERSION` | Allows you to set the Java version used by the JNLP Client for the agent images. By default the value is set to `java-11`. To use Java 8 for the value for this env var to `java-8`. | +| `AGENT_BASE_IMAGE` | Setting this value overrides the image used for the 'jnlp' container in the sample kubernetes plug-in PodTemplates provided with this image. Otherwise, the image from the 'jenkins-agent-base:latest' ImageStreamTag in the 'openshift' namespace is used. | +| `JAVA_BUILDER_IMAGE` | Setting this value overrides the image used for the 'java-builder' container in the sample kubernetes plug-in PodTemplates provided with this image. Otherwise, the image from the 'java:latest' ImageStreamTag in the 'openshift' namespace is used. | +| `NODEJS_BUILDER_IMAGE` | Setting this value overrides the image used for the 'nodejs-builder' container in the sample kubernetes plug-in PodTemplates provided with this image. Otherwise, the image from the 'nodejs:latest' ImageStreamTag in the 'openshift' namespace is used. | +| `JAVA_FIPS_OPTIONS` | Per this [OpenJDK support article](https://access.redhat.com/documentation/en-us/openjdk/11/html-single/configuring_openjdk_11_on_rhel_with_fips/index#config-fips-in-openjdk) control how the JVM operates when running on a FIPS node. | + +You can also set the following mount points by passing the `-v /host:/container` flag to Docker. + +| Volume mount point | Description | +| :-------------------------- | ------------------------ | +| `/var/lib/jenkins` | Jenkins config directory | + +**Notice: When mounting a directory from the host into the container, ensure that the mounted +directory has the appropriate permissions and that the owner and group of the directory +matches the user UID or name which is running inside the container.** + +Inclusion of the `oc` binary +--------------------------------- + +To assist in interacting with the OpenShift API server while using this image, the `oc` binary, the CLI command for OpenShift, has been +installed in the master and agent images defined in this repository. + +**NOTE:** Backward compatibility is not guaranteed between differeing versions of `oc` and the +API Server. It is recommended that you align versions of this image present in the nodes of your cluster with your OpenShift API server. diff --git a/HACKING.md b/BUILDING.md similarity index 87% rename from HACKING.md rename to BUILDING.md index e0a9f0f1c..9860dc155 100644 --- a/HACKING.md +++ b/BUILDING.md @@ -1,15 +1,15 @@ -# Hacking +# Building Jenkins Images The basics to build this project and test it locally -## Locally +## Locally ## Building locally Jenkins images can be built using `podman`, `buildah` or `docker`. For historical reasons, the default build runtime is `docker` To build on podman: ``` -make VERSIONS="2 maven-3.5" BUILD_COMMAND="podman build --no-cache" +make VERSIONS="2 slave-base" BUILD_COMMAND="podman build --no-cache" ``` @@ -31,6 +31,3 @@ oc new-build https://github.com/origin/jenkins.git#your-branch-name --context-di ``` oc new-app jenkins-persistent -p NAMESPACE=$(oc project -q) -p JENKINS_IMAGE_STREAM_TAG=jenkins:latest ``` - - - diff --git a/CHANGELOG.md b/CHANGELOG.md deleted file mode 100644 index 13cc6d19f..000000000 --- a/CHANGELOG.md +++ /dev/null @@ -1,26 +0,0 @@ -# Changelog -All notable changes to this project will be documented in this file. - -## [Unreleased] - -## [4.2.001] - 2019-08-12 -### Added -- Use of OpenJDK 11 - -### Changed -- OpenShift Client Plugin 1.0.41 -- OpenShift Sync Plugin 1.0.41 [see CHANGELOG](https://github.com/openshift/jenkins-sync-plugin/blob/master/CHANGELOG.md) -- OpenShift Login Plugin [see CHANGELOG](https://github.com/openshift/jenkins-login-plugin/blob/master/CHANGELOG.md) -- Bump workflow-cps version to 2.73 -- Bump script-security to 1.62 -- Bump workflow-cps-global-lib version to 2.15 - -### Fixed -- Concurrency issue related to OpenShift Sync Plugin - -### Removed - -### Security -- https://jenkins.io/security/advisory/2019-07-31/ - - diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index dd01006ab..e81dd0c79 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,19 +1,19 @@ -# Development and release process for the OpenShift Jenkins plugins +# Contributing -The OpenShift organization currently maintains several plugins related to integration between V3 OpenShift and V2 Jenkins. +The OpenShift organization maintains several Github repositories that contain all of the source code for our Jenkins images and plugins. | | | | | | -------------- | -------------------------- | ------------------------ | -------------- | -| `OpenShift Client Plugin` | [OpenShift Repo](https://github.com/openshift/jenkins-client-plugin) | [Jenkins Repo](https://github.com/jenkinsci/openshift-client-plugin) | [Wiki](https://wiki.jenkins.io/display/JENKINS/OpenShift+Client+Plugin) | -| `OpenShift Sync Plugin` | [OpenShift Repo](https://github.com/openshift/jenkins-sync-plugin) | [Jenkins Repo](https://github.com/jenkinsci/openshift-sync-plugin) | [Wiki](https://wiki.jenkins.io/display/JENKINS/OpenShift+Sync+Plugin) | -| `OpenShift Login Plugin` | [OpenShift Repo](https://github.com/openshift/jenkins-openshift-login-plugin) | [Jenkins Repo](https://github.com/jenkinsci/openshift-login-plugin) | [Wiki](https://wiki.jenkins.io/display/JENKINS/OpenShift+Login+Plugin) | -| `OpenShift Pipeline Plugin` | [OpenShift Repo](https://github.com/openshift/jenkins-plugin) | [Jenkins Repo](https://github.com/jenkinsci/openshift-pipeline-plugin) | [Wiki](https://wiki.jenkins.io/display/JENKINS/OpenShift+Pipeline+Plugin) | +|`Jenkins Server Image`|[OpenShift Repo](https://github.com/openshift/jenkins)||| +|`Jenkins Agent Base Image`|[OpenShift Repo](https://github.com/openshift/jenkins)||| +| `OpenShift Client Plugin` | [OpenShift Repo](https://github.com/openshift/jenkins-client-plugin) | [Jenkins Repo](https://github.com/jenkinsci/openshift-client-plugin) | [Release Website](https://plugins.jenkins.io/openshift-client/) | +| `OpenShift Sync Plugin` | [OpenShift Repo](https://github.com/openshift/jenkins-sync-plugin) | [Jenkins Repo](https://github.com/jenkinsci/openshift-sync-plugin) | [Release Website](https://plugins.jenkins.io/openshift-sync/) | +| `OpenShift Login Plugin` | [OpenShift Repo](https://github.com/openshift/jenkins-openshift-login-plugin) | [Jenkins Repo](https://github.com/jenkinsci/openshift-login-plugin) | [Release Website](https://plugins.jenkins.io/openshift-login/) | -The *OpenShift Pipeline Plugin* is only in maintenance for the 3.x stream and is not supported in 4.x. The message to use the client plugin instead seems to have been received, and it has been quite a while since we have seen any github or bugzilla -activity with it. -As the development process for each of these plugins are similar in many ways, we are documenting the process here under -the repository for the OpenShift Jenkins images, where each of the plugins will reference this document in their respective + +As the development process for each of these plugins are similar in many ways, we are documenting the process here under +the repository for the OpenShift Jenkins images, where each of the plugins will reference this document in their respective READMEs. ## Constructing your development sandbox @@ -22,40 +22,26 @@ You will need an environment with the following tools installed: * Maven (the `mvn` command) * Git -* Java (need v8) +* Java 11 * an IDE or editor of your choosing -The Jenkins open source project already has a bunch of useful links on setting up your development environment, including -launching Jenkins from the IDE (though there are pros and cons for developing this way vs. developing against a Jenkins -server running in an OpenShift pod). - -Here are a few of those links for reference: - -* [https://wiki.jenkins.io/display/JENKINS/Extend+Jenkins](https://wiki.jenkins.io/display/JENKINS/Extend+Jenkins) (note, there are many useful child pages under this wiki page) -* [https://wiki.jenkins.io/display/JENKINS/Plugin+tutorial](https://wiki.jenkins.io/display/JENKINS/Plugin+tutorial) (one of the key child pages under Extend Jenkins) -* [https://wiki.jenkins.io/display/JENKINS/Setting+up+Eclipse+to+build+Jenkins](https://wiki.jenkins.io/display/JENKINS/Setting+up+Eclipse+to+build+Jenkins) (has concepts common to other IDEs undoubtedly) - ### Basic code development flow -Our plugins are constructed such that if they are running in an OpenShift pod, they can determine how to connect to the +Our plugins are constructed such that if they are running in an OpenShift pod, they can determine how to connect to the associated OpenShift master automatically, and no configuration of the plugin from the Jenkins console is needed. If you choose to run an external Jenkins server and you would like to test interaction with an OpenShift master, you will need to manually configure the plugin. See each plugin's README or the OpenShift documentation for the specifics. An example flow when running in an OpenShift pod: -1. Clone this git repository: +1. Clone the git repository: ``` - git clone https://github.com/openshift/-plugin.git + git clone https://github.com/openshift/-plugin.git ``` -1. Enable the provided git hooks: see https://github.com/openshift/jenkins/blob/master/README.md#plugin-installation-for-centos7-v4.10+ - ``` - git config core.hooksPath .githooks/ - ``` -1. In the root of the local repository, run maven +1. In the root of the local repository, run maven ``` - cd -plugin - mvn + cd -plugin + mvn clean package ``` 1. Maven will build target/.hpi (the Jenkins plugin binary) 1. Open Jenkins in your browser, log in as an administrator, and navigate as follows: @@ -78,214 +64,20 @@ Or you can ... #### ... mimic the PR testing done in OpenShift CI/CD implementation -What follows in ["Actual PR Testing"](#actual-pr-testing) is a complete description of the build and test flows for PRs against the [OpenShift/Jenkins Images repo](https://github.com/openshift/jenkins) and +What follows in ["Actual PR Testing"](#actual-pr-testing) is a complete description of the build and test flows for PRs against the [OpenShift/Jenkins Images repo](https://github.com/openshift/jenkins) and the three plugins we provide to facilitate OpenShift/Jenkins integration. In the end of that chapter will detail the tests run in the PR and how you can run them against your local clusters. ## Actual PR Testing -Each plugin repository, and the images repository, under [https://github.com/openshift](https://github.com/openshift) is under the umbrella of the Prow based OpenShift CI/CD infrastructure. As such, there is +Each plugin repository, and the images repository, under [https://github.com/openshift](https://github.com/openshift) is under the umbrella of the Prow based OpenShift CI/CD infrastructure. As such, there is 1. an `openshift-ci-robot` bot that accepts [these commands](https://go.k8s.io/bot-commands?repo=openshift%2Forigin) from within a PR 1. a [code review process](https://github.com/kubernetes/community/blob/master/contributors/guide/owners.md#the-code-review-process) that leverages the aforementioned commands 1. OpenShift CI/CD's [specific Prow configuration](https://github.com/openshift/release) for the various repositories of the project, detailed below -### Prow configuration details - -If you look at the [CI workflow configuration summary](https://github.com/openshift/release#ci-workflow-configuration), the `ci-operator/config`, `ci-operator/jobs`, and `ci-operator/templates` bullets there pertain in some fashion to the Jenkins related repositories. Specifics within that context: - -* For the plugins, we've managed to only need a `master` branch to date, and only `ci-operator/config` and `ci-operator/jobs` artifacts have had to be created for the three plugins -* For master and 4.x branches of the jenkins images out of [the OpenShift/Jenkins repo](https://github.com/openshift/jenkins), we now only need `ci-operator/config` and `ci-operator/jobs` artifacts. -* For 3.11, we had to also compose a special template under `ci-operator/templates` -* For branches prior to 3.11, the only tests performed are some basic image startup and `s2i` invocations that were the equivalent of running `make test` from the local copy of the repository on your workstation. They are still Jenkins based and not Prow based. No tests were executed in an actual OpenShift environment. At this point, only changes to address support cases are going into branches older than 3.11. - -#### Jenkins images - -Quick preamble ... some cool facts about OpenShift CI: -* it runs on OpenShift -* it leverages development features like Builds and ImageStreams extensively - -##### 3.11 specifics - -The ci-operator based configuration, the `ci-operator/config`, for PR testing in 3.11 is at [https://github.com/openshift/release/blob/master/ci-operator/config/openshift/jenkins/openshift-jenkins-openshift-3.11.yaml](https://github.com/openshift/release/blob/master/ci-operator/config/openshift/jenkins/openshift-jenkins-openshift-3.11.yaml). - -The parts at the beginning and end of that file are more generic setup needs for [ci-operator](https://github.com/openshift/ci-operator). - -What makes 3.11 a bit more complicated than 4.0 is -* During the 3.11 timeframe, `ci-operator` was pretty brand new. In particular, updating image streams on the test cluster was not baked into the system... enhancements came during 4.0 that helped there -* 3.x did not create image streams for the slave/agent example images we ship - -With that background, let's examine the relevant parts of [https://github.com/openshift/release/blob/master/ci-operator/config/openshift/jenkins/openshift-jenkins-openshift-3.11.yaml](https://github.com/openshift/release/blob/master/ci-operator/config/openshift/jenkins/openshift-jenkins-openshift-3.11.yaml). - -First, we make a copy of the test container images and store it as ImagStreamTag `cluster-tests`: - -``` - cluster-tests: - cluster: https://api.ci.openshift.org - name: origin-v3.11 - namespace: openshift - tag: tests -``` - -Then, we are going to build a new test container image with: - -``` -- from: cluster-tests - inputs: - src: - paths: - - destination_dir: . - source_path: /go/src/github.com/openshift/jenkins/test-e2e/. - to: tests - -``` - -which employs an OpenShift Docker Strategy Build, using this [Dockerfile](https://github.com/openshift/jenkins/blob/master/test-e2e/Dockerfile) for the docker build. And the buid is adding a [script](https://github.com/openshift/jenkins/blob/master/test-e2e/tag-in-image.sh) that will tag the Jenkins images created during the PRs build into the test cluster's jenkins imagestream in the openshift namespace. The `to: tests` line updates the existing test image. It is literally storing the output image from the OpenShift Docker Strategy Build into the `tests` ImageStreamTag in the CI systems's internal ImageStreams. - -Next, this stanza is for the main jenkins image: - -``` -- dockerfile_path: Dockerfile - from: base - inputs: - src: - paths: - - destination_dir: . - source_path: /go/src/github.com/openshift/jenkins/2/. - to: 2-centos - -``` - -Let's dive into this stanza: -* As with updating the test image, this yaml is short hand for defining an OpenShift Docker Build Strategy build, and it does a Docker build with: - * `from: base` is an imagestreamtag to the latest 3.11.x OpenShift CLI image ... that gets substituted into the Dockerfile's FROM clause for the Docker build - * `Dockerfile` means we will literally using the `Dockerfile` at the `source_path`, where the `/go/src/github.com/openshift/jenkins/2/.` corresponds to the `git checkout` for the git branch we are testing. - * the `to: 2-centos` tells `ci-operator` to set the output of the Docker Build Strategy build to an imagestreamtag named `2-centos` in the test artifacts. Then, `ci-operator` takes all such imagestremtag names, converts them to upper case, and then sets `IMAGE_` environment variables into the test system -* There are similar stanzas for the `slave-base`, `agent-maven-3.5`, and `agent-nodejs-8` images. - -But again, since we do not have imagestreams defined for those slave/agent images in 3.x, we have to do some more manipulation of the Prow setup. With that, let's move onto the `ci-operator/jobs` definitions for 3.11. - -The key two files are the [presubmit definition](https://github.com/openshift/release/blob/master/ci-operator/jobs/openshift/jenkins/openshift-jenkins-openshift-3.11-presubmits.yaml) and the [postsubmit definition](https://github.com/openshift/release/blob/master/ci-operator/jobs/openshift/jenkins/openshift-jenkins-openshift-3.11-postsubmits.yaml). - -The most relevant pieces of the presubmit (aside from a bunch of Prow and ci-operator gorp you can dismiss): -* Whether a PR related job is Prow based or Jenkins based is indicated by the `agent` setting. In the presubmits, they are all Prow based: - -``` - - agent: kubernetes -``` - -* There are unique Prow definitions for each branch a given repo. So this stanza signifies that this handles PRs for the `openshift-3.11` branch of the jenkins repo: - -``` - branches: - - openshift-3.11 -``` - -* If the building of the image fails for some reason within a PR (`yum mirror` flake during RPM install, Jenkins update center flake during plugin download, a bug in your PR), you can re-run the OpenShift Docker Strategy build defined in the config via a `/test image` comment made to the PR, which maps to the ci-operator definition with the stanza: - -``` - rerun_command: /test images -``` - -* To kick off a another run of the PR tests, you can type in `/test e2e-gcp` as a PR comment because of: - -``` - rerun_command: /test e2e-gcp -``` - -* To trigger the tagging of the PRs newly build Jenkins image into the test cluster, this stanza sets up an environment variable that allows code down the line to call that script we mounted into the test container earlier: - -``` - - name: PREPARE_COMMAND - value: tag-in-image.sh -``` - - -* As an optimization, we only run the Jenkins extended tests defined in https://github.com/openshift/origin. This is achieved via: - -``` - - name: TEST_COMMAND - value: TEST_FOCUS='openshift pipeline' TEST_PARALLELISM=3 run-tests -``` - -OpenShift extended tests are based on the Golang Ginkgo test framework. We leverage the focus feature of Ginkgo to limit the tests executed to the ones pertaining the our OpenShift/Jenkins integrations. The `run-tests` executable actually launches the Ginkgo tests against the test cluster, and will leverage `TEST_FOCUS`. The `TEST_PARALLELISM=3` is also worth noting, as it is unique to Jenkins e2e. We discovered that the amount of memory Jenkins needed to run Pipelines was such that we could overload our 3.11 based GCP clusters used for CI. Setting this environment variable restricted the amount of Jenkins based tests that would be run concurrently. - -The most relevant pieces of the postsubmit: -* There is 1 Prow based and 1 Jenkins based task in the postsubmits. - -* The Prow based one gathers artifacts from the various image and test jobs for analysis. Aside from debugging failures, the artifacts will prove helpful when we are updating the versions of any of the plugins the image depends on (more on that below). - -* The Jenkins based job is a legacy from our pre-Prow, pre-4.x days, that still runs on the OpenShift CI Jenkins server at https://ci.openshift.redhat.com/jenkins. It pushes new versions of the 3.11 images to docker.io. With 4.0, we are only pushing the community versions of the images to quay.io. - - -OK, now with `ci-operator/config` and `ci-operator/jobs` covered for 3.11, `ci-operator/templates` is the next item to cover. Reminder, the need for jenkins to define a template is unique to 3.11. - -The template is at [https://github.com/openshift/release/blob/master/ci-operator/templates/openshift/openshift-ansible/cluster-launch-e2e-openshift-jenkins.yaml](https://github.com/openshift/release/blob/master/ci-operator/templates/openshift/openshift-ansible/cluster-launch-e2e-openshift-jenkins.yaml). And by template, yes, we mean an OpenShift template used to create API objects, where parameters are defined to vary the specific settings of those API objects. The template to a large degree is a clone of https://github.com/openshift/release/blob/master/ci-operator/templates/openshift/openshift-ansible/cluster-launch-e2e.yaml, but with additions to: -* Tag in the jenkins image from the PR's build into the test cluster's jenkins imagestream -* Change the image pull spec of our example maven and nodejs agent images used to create our default k8s plugin pod templates configuration - -To tag in the Jenkins image, we added a `prepare` container into the e2e pod. It leverages the `PREPARE_COMMAND` and `IMAGE_2_CENTOS` environment variables noted above to call that script from the Jenkins repo that was added to the base test container from the CI system. - -To update the agent images used, the `IMAGE_MAVEN_AGENT` and `IMAGE_NODEJS_AGENT` environment variables, which the `ci-operator/config` noted above sets to the pull spec of the newly built images from the PR, are read in by the extended tests in https://github.com/openshift/origin. Those values in turn are used to set the [environment variables](https://github.com/openshift/jenkins#environment-variables) that the Jenkins image recognizes for overriding the default images used for the k8s plugin configuration. - -##### 4.x specifics - -So, by comparison, the `ci-operator/config` for the master branch is [here](https://github.com/openshift/release/blob/master/ci-operator/config/openshift/jenkins/openshift-jenkins-master.yaml). As versions of 4.x accumulate, you'll see more `openshift-jenkins-.yaml` files in that directory. As of this writing, the initial GA of 4.x has not occurred. - -Key differences from 3.11: -* The Prow based infra evolved in many ways....like leveraging AWS vs. GCP for example. But it got to the point also, where the mapping to imagestreams running in the test cluster was more direct. So in a stanza like: - -``` -- dockerfile_path: Dockerfile.rhel7 - from: base - inputs: - src: - paths: - - destination_dir: . - source_path: /go/src/github.com/openshift/jenkins/2/. -to: jenkins -``` - -that still executed an OpenShift Docker Strategy build against the `git checkout` from the PR, but now the `to: jenkins` line meant the system would tag our resulting image directly into the test cluster's `jenkins` imagestream. - -So no more need for the adding in [the tagging script](https://github.com/openshift/jenkins/blob/master/test-e2e/tag-in-image.sh) into the test container and doing special test cluster setup in the template. - -Also, with the agent images now having imagestreams in a 4.x cluster as part of being part of the installed payload, we can leverage the same pattern for those as well, simply tagging in the PR's versions of those images into the test cluster. Hence, no need for the `IMAGE_MAVEN_AGENT` and `IMAGE_NODEJS_AGENT` environment variables. - -With both those removed, we no longer needed a special template under `ci-operator/templates` for jenkins. We now use the default one in 4.x. - -You will also notice the use of `Dockerfile.rhel7` vs. `Dockerfile` for the `dockerfile_path`. This stems from the [move in 4.x to the UBI](https://github.com/openshift/jenkins#installation-openshift-v4) and the end of CentOS based content for OpenShift. - -The extended tests defined in [OpenShift origin](https://github.com/openshift/origin) were also reworked in 4.x. The use of Ginkgo focuses were moved to only within the test executables, and can no longer be specified from the command line. "Test suites" were defined for the most prominent focuses, including one for Jenkins called `openshift/jenkins-e2e`. This simplification, along with other rework, makes it easier to run the extended tests against existing clusters (including ones you stand up for your development ... more on that later). -The tests defined for jenkins are as of 4.10 a "meets min" validation that JenkinsPipelineStrategy builds work. And they are only invoked if you run `e2e-aws-jenkins` in openshift/origin PRs or openshift/cluster-samples-operator PRs. The more precise jenkins e2e's centering on the plugins are now defined in the sync and client plugins respectively (and are "controller like" vs. ginkgo). -The generic OpenShift conformance regression bucket is also still included: - -``` -tests: -- as: e2e-aws-jenkins-sync-plugin - skip_if_only_changed: ^docs/|\.md$|^(?:.*/)?(?:\.gitignore|OWNERS|PROJECT|LICENSE)$ - steps: - cluster_profile: aws - test: - - ref: jenkins-sync-plugin-e2e - workflow: ipi-aws -- as: e2e-aws-jenkins-client-plugin - skip_if_only_changed: ^docs/|\.md$|^(?:.*/)?(?:\.gitignore|OWNERS|PROJECT|LICENSE)$ - steps: - cluster_profile: aws - test: - - ref: jenkins-client-plugin-tests - workflow: ipi-aws -``` - -Those jenkins plugin e2e's are also used in their respective repos' PRs. The are defined in the [CI Step Registry](https://github.com/openshift/release/tree/master/ci-operator/step-registry/jenkins) - -Moving on to the `ci-operator/jobs` data, not as different from 3.11 as the `ci-operator/config`, [the presubmits](https://github.com/openshift/release/blob/master/ci-operator/jobs/openshift/jenkins/openshift-jenkins-master-presubmits.yaml) are the ci-operator and Prow related definitions for the `e2e-aws` and `e2e-aws-jenkins` test jobs noted above, as well as the job to build the images. Each can be re-run via `/test e2e-aws`, `/test e2e-aws-jenkins`, or `/test images`. - -Likewise, the [postsubmit](https://github.com/openshift/release/blob/master/ci-operator/jobs/openshift/jenkins/openshift-jenkins-master-postsubmits.yaml) has the Prow job defined to collect the test artifacts as with 3.11. The Jenkins based job to push the resulting images has been removed as part of the move off of docker.io and onto quay.io. The CI system in general will push the updates to quay.io for all relevant images, and we no longer need special jobs for our Jenkins images on this front. - #### Jenkins plugins -For the plugins, we've managed to only need a `master` branch to date in providing support against 3.x and 4.x. The OpenShift features needed to support the various OpenShift/Jenkins integrations landed in 3.4, and we are only actively supporting OpenShift/Jenkins images back to 3.6. An experimental branch `v3.6-fixes` was created in https://github.com/jenkinsci/openshift-sync-plugin a while back to confirm we *could* backport specific fixes to older versions if need be and craft versions like `1.0.24.1` if need be. But for simplicity that will be a last resort sort of thing. +For the plugins, we've managed to only need a `master` branch to date in providing support against 3.x and 4.x. The OpenShift features needed to support the various OpenShift/Jenkins integrations landed in 3.4, and we are only actively supporting OpenShift/Jenkins images back to 3.6. An experimental branch `v3.6-fixes` was created in https://github.com/jenkinsci/openshift-sync-plugin a while back to confirm we *could* backport specific fixes to older versions if need be and craft versions like `1.0.24.1` if need be. But for simplicity that will be a last resort sort of thing. Also note, we have not been updating older versions of the OpenShift/Jenkins image with newer versions of our 3 plugins to pull in fixes unless a support case comes in through bugzilla that dictates so. @@ -293,7 +85,7 @@ With that background as context: * During the 4.0 time frame PR testing for the supported plugins has migrated fully to Prow (and away from the OpenShift CI Jenkins server) * Only `ci-operator/config` and `ci-operator/jobs` artifacts are needed for the three plugins, similar to what exists for the [jenkins repo](https://github.com/openshift/jenkins). * The [deprecated plugin](https://github.com/openshift/jenkins-plugin) remains on jobs defined at [https://github.com/openshift/aos-cd-jobs](https://github.com/openshift/aos-cd-jobs) that run on the OpenShift CI Jenkins server ... note since its EOL notice on Aug 3 2018 there have been no further changes needed to that plugin from a support perspective. - + The `ci-operator/config` files for each plugin are very similar. For reference, here are the locations for the [sync plugin](https://github.com/openshift/release/blob/master/ci-operator/config/openshift/jenkins-sync-plugin/openshift-jenkins-sync-plugin-master.yaml), the [client plugin](https://github.com/openshift/release/blob/master/ci-operator/config/openshift/jenkins-client-plugin/openshift-jenkins-client-plugin-master.yaml), and [login plugin](https://github.com/openshift/release/blob/master/ci-operator/config/openshift/jenkins-openshift-login-plugin/openshift-jenkins-openshift-login-plugin-master.yaml). The 3 key elements are: @@ -359,37 +151,6 @@ The `ci-operator/jobs` files are very similar to the ones for the jenkins image ### Extended tests -##### 4.11 and later - -- [meets min jenkins pipeline strategy verification in openshift/origin, only for openshift/origin and openshift/cluster-samples-operator jobs](https://github.com/openshift/origin/blob/master/test/extended/builds/pipeline_origin_bld.go) -- [client plugin tests](https://github.com/openshift/jenkins-client-plugin/tree/master/test/e2e) -- [sync plugin tests](https://github.com/openshift/jenkins-sync-plugin/tree/master/test/e2e) -- login plugin uses the sync plugin tests, as it always fetches logs using the login plugin's command line with oauth token flow - -##### 4.19 and earlier - -First, the extended tests in [OpenShift Origin](https://github.com/openshift/origin) that we've made some references to ... where are the Jenkins ones specifically? There are two golang files: -* The [reduced jenkins e2e suite](https://github.com/openshift/origin/blob/master/test/extended/builds/pipeline_origin_bld.go) run in the OpenShift Build's regression bucket -* In addition to the minimal suite, [these tests](https://github.com/openshift/origin/blob/master/test/extended/builds/pipeline_jenkins_e2e.go) get run in the jenkins e2e for the image and each plugin - -The structure of those golang files are a bit unique from a Ginkgo perspective, and as compared to the other tests in OpenShift origin. Given the memory demands of Jenkins, as well as intermittent issues with Jenkins trying to contact the Update Center, we've gone through pains: -* To minimize the number of Jenkins instance running concurrently during the e2e. -* To minimize the number of times we have to bring Jenkins up - -So you'll see less use of `g.Context(..)` and `g.It(...)`, as well as cleaning up of resources between logical tests. Currently the divisions in the tests that result in concurrent test runs are between: -* The client plugin and sync plugin -* The ephemeral storage template and persistent storage template - -The `openshift-tests` binary in the 4.x branches of [OpenShift Origin](https://github.com/openshift/origin) includes those tests (and as it turns out, can be run against both 3.11 and 4.x clusters). Once you have a cluster up, and the `openshift-tests` binary built (run `hack/build-go.sh cmd/openshift-tests` from you clone of origin), you can: -* set and export KUBECONFIG to the location of the admin.kubeconfig for the cluster -* run `openshift-tests run openshift/jenkins-e2e --include-success` against the cluster ... the `openshift/jenkins-e2e` is considered a "suite" in `openshift-tests` and under the covers it leverages Ginkgo focuses to pick up the tests from those two golang files. - -To run extended tests against one of your clusters using a set of local changes of a plugin, from one of the plugin repo's top dirs, you can: -* run `docker build -f ./Dockerfile -t ` -* run `docker push ` -* run `oc tag --source=docker openshift/jenkins:2` -* run `openshift-tests run openshift/jenkins-e2e --include-success` ... the imagestream controller in OpenShift will pull the publicly accessible docker registry spec, like docker.io/gmontero/plugin-tests:latest, when the standard jenkins template is provisioned. - ### the "PR-Testing" folders in the plugins The Dockerfiles and scripts in these folders were used in the pre-Prow days, where the Jenkins jobs would build Docker images with the updated plugin local to the test nodes, and then via environment variables, the extended tests would provision Jenkins either using the standards templates, or specialized ones that would leverage the local test image. @@ -398,9 +159,9 @@ We can most likely remove these, but are holding on in case we have to create no ## Process for cutting a release of a plugin -Once we've merged changes into one of the OpenShift org GitHub repositories for a given plugin, we need to transfer the associated commit to the corresponding JenkinsCI org GitHub repository and follow [the upstream Jenkins project release process](https://wiki.jenkins.io/display/JENKINS/Hosting+Plugins) when we have deemed changes suitable for inclusion into the non-subscription OpenShift Jenkins image (the CentOS7 based one hosted on docker.io for 3.x, the UBI based one hosted on quay.io for 4.x). +Once we've merged changes into one of the OpenShift org GitHub repositories for a given plugin, we need to transfer the associated commit to the corresponding JenkinsCI org GitHub repository and follow [the upstream Jenkins project release process](https://wiki.jenkins.io/display/JENKINS/Hosting+Plugins) when we have deemed changes suitable for inclusion into the non-subscription OpenShift Jenkins image. -### Accounts and configuration (both local and upstream in various remote, Jenkins related resources) +### Accounts and configuration (both local and upstream in various remote, Jenkins related resources) Some key specifics from [the upstream Jenkins project release process](https://wiki.jenkins.io/display/JENKINS/Hosting+Plugins): @@ -417,7 +178,7 @@ one has to be a member of the github teams: ### Where the officially cut versions of each plugin are hosted For our Jenkins image repository to include particular versions of our plugins in the image, the plugin versions in question -need to be available at these locations, depending on the particular plugin of course. These are the official landing spots +need to be available at these locations, depending on the particular plugin of course. These are the official landing spots for a newly minted version of a particular plugin. * [https://updates.jenkins.io/download/plugins/openshift-client](https://updates.jenkins.io/download/plugins/openshift-client) @@ -447,11 +208,11 @@ In your clone of `https://github.com/jenkinsci/`, set up your git re 1. Then pull and rebase the latest changes from [https://github.com/openshift/jenkins-client-plugin](https://github.com/openshift/jenkins-client-plugin) with the following: ``` - $ git checkout master - $ git fetch upstream - $ git fetch upstream --tags - $ git cherry-pick # for each commit that needs to be migrated - $ git push origin master + $ git checkout master + $ git fetch upstream + $ git fetch upstream --tags + $ git cherry-pick # for each commit that needs to be migrated + $ git push origin master $ git push origin --tags ``` @@ -462,7 +223,7 @@ After pushing the desired commits to the [https://github.com/jenkinsci](https:// Prerequisite: your Git ID should have push access to the [https://github.com/jenkinsci](https://github.com/jenkinsci) repositories for this plugin; your Jenkins ID (again see [https://wiki.jenkins-ci.org/display/JENKINS/User+Account+on+Jenkins](https://wiki.jenkins-ci.org/display/JENKINS/User+Account+on+Jenkins)) is listed in the permission file for the plugin, like [https://github.com/jenkins-infra/repository-permissions-updater/blob/master/permissions/plugin-openshift-pipeline.yml](https://github.com/jenkins-infra/repository-permissions-updater/blob/master/permissions/plugin-openshift-pipeline.yml). Given these assumptions: 1. Then run `mvn release:prepare release:perform` -1. You'll minimally be prompted for the `release version`, `release tag`, and the `new development version`. Default choices will be provided for each, and the defaults are acceptable, so you can just hit the enter key for all three prompts. As an example, if we are currently at v1.0.36, it will provide 1.0.37 for the new `release version` and `release tag`. For the `new development version` it will provide 1.0.38-SNAPSHOT, which is again acceptable. +1. You'll minimally be prompted for the `release version`, `release tag`, and the `new development version`. Default choices will be provided for each, and the defaults are acceptable, so you can just hit the enter key for all three prompts. As an example, if we are currently at v1.0.36, it will provide 1.0.37 for the new `release version` and `release tag`. For the `new development version` it will provide 1.0.38-SNAPSHOT, which is again acceptable. 1. The `mvn release:prepare release:perform` command will take a few minutes to build the plugin and go through various verifications, followed by a push of the built artifacts up to the Jenkins Artifactory server. This typically works without further involvement but has failed for various reasons in the past. If so, to retry with the same release version, you will need to call `git reset --hard HEAD~2` to back off the two commits created as part of publishing the release (the "release commits", where the pom.xml is updated to reflect the new version and the next snapshot version), as well as use `git tag` to delete both the local and remote version of the corresponding tag. After deleting the commits and tags, use `git push -f` to update the commits at the Jenkinsci GitHub Org repo in question. Address whatever issues you have (you might have to solicit help on the Jenkins developer group: [https://groups.google.com/forum/#!forum/jenkinsci-dev](https://groups.google.com/forum/#!forum/jenkinsci-dev)) or on the #jenkins channel on freenode (Daniel Beck from CloudBees has been helpful and returned message on #jenkins), then try again. 1. If `mvn release:prepare release:perform` completes successfully, those "release commits" will look something like this if you ran `git log -2`: @@ -514,16 +275,16 @@ how they landed in [https://github.com/jenkinsci](https://github.com/jenkinsci) As referred to previously, the new plugin version will land at `https://updates.jenkins.io/download/plugins/`. Monitor that page for the existence of the new version of the plugin. Warning: the link for the new version can show up, but does not mean the file is available yet. Click the link to confirm you can download the new version of the plugin. When you can download the new version file, the new release is available. -At this point, we are back to the steps articulated in [the base plugin installation section of this repository's README](https://github.com/openshift/jenkins/blob/master/README.md#base-set-of-plugins). You'll +At this point, we are back to the steps articulated in [the base plugin installation section of this repository's README](https://github.com/openshift/jenkins/blob/master/README.md#base-set-of-plugins). You'll modify the [text file](https://github.com/openshift/jenkins/blob/master/2/contrib/openshift/base-plugins.txt) with the new version for whatever OpenShift plugin you have cut a new version for, and create a new PR. The image build and e2e extended test runs we detailed in ["Actual PR Testing"](#actual-pr-testing) will commence. -If the PR passes all tests and merges, the api.ci system will promote the jenkins images to quay.io for 4.x, and we have the separate jenkins job in the openshift ci jenkins server to push the 3.11 images to docker.io. +If the PR passes all tests and merges, the api.ci system will promote the jenkins images to quay.io. The image build from the PR in particular is of interest when it comes to plugin versions within our openshift/jenkins image, and what we have to do in creating the RPM based images hosted on registy.redhat.io/registry.access.redhat.com that we provide subscription level support for. The PR will have a link to the `ci/prow/images` job. If you click that link, then the `artifacts` link, then the next `artifacts` link, then `build-logs`, you'll see gzipped output from each of the image builds. Click the one for the main jenkins image. If you search for the string `Installed plugins:` you'll find the complete list of every plugin that was installed. Copy that output to clipboard and paste it into the PR that just merged. See [https://github.com/openshift/jenkins/pull/829#issuecomment-477637521](https://github.com/openshift/jenkins/pull/829#issuecomment-477637521) as an example. -### Step 2 (4.11 and later) +### Step 2 -The jenkins images got removed from the OCP install payload in 4.11, and we now use the CPaaS Images Akram created for us. A net of all this is that the OCP samples operator no longer modifies the jenkins related ImageStreams with +The jenkins images got removed from the OCP install payload in 4.11, and we now use the CPaaS Images Akram created for us. A net of all this is that the OCP samples operator no longer modifies the jenkins related ImageStreams with the image ref from the install payload. Jenkins is now treated like any other sample. With that though: - we no longer have to submit ART requests after our openshift/jenkins PRs merge - we do need to update our (ImageStreams)[https://github.com/openshift/jenkins/tree/master/openshift/imagestreams] with new image refs @@ -531,43 +292,10 @@ the image ref from the install payload. Jenkins is now treated like any other s Moving out of the payload, we also were able to create more than one ImageStreamTag, so we can now support some of the upgrade options customers have asked for in the past: - still upgrade when OCP upgrades (we use a precise SHA image ref for this) - use the OCP ImageStream scheduled import feature, so the ImageStream controllers periodically access registry.redhat.io if updates have been made -- use a constant Image ref for the ImageStreamTag, so that unless we change that ref in someway, the ImageStream update made by samples operator on an upgrade should no change the ImageStreamTag's image ref, and hence the Image Change Controller should +- use a constant Image ref for the ImageStreamTag, so that unless we change that ref in someway, the ImageStream update made by samples operator on an upgrade should no change the ImageStreamTag's image ref, and hence the Image Change Controller should not rollout a new Jenkins Deployment. The customer than can control when Jenkins is redeployed by doing an 'oc import-image' for the jenkins ImageStream Once the Jenkins ImageStreams in the repo are updated, wait for the "Automatic importer job update" commit in [OpenShift Library](https://github.com/openshift/library/commits/master) that pulls in our changes. After that, open a [samples operator](https://github.com/openshift/cluster-samples-operator) PR that follows [this process](https://github.com/openshift/cluster-samples-operator#update-the-content-in-this-repository-from-httpsgithubcomopenshiftlibrary) for updating the samples content for jenkins. NOTE: that script will pull in all the samples updates. If you only want to pull in jenkins, edit the git commits to undo those changes. NOTE: be sure the run the optional `e2e-aws-jenkins` test job as a sanity check. - -### Step 2 (4.10 and earlier): updating OSBS/Brew for generating the officially supported images available with Red Hat subscriptions - -First, some background: for all Red Hat officially supported content, to ensure protection from outside hackers, all content is built in a quarantined system with no access to the external Internet. As such, we have to inject all content into OpenShift's Brew server (see links like [https://pagure.io/koji/](https://pagure.io/koji/) and [https://osbs.readthedocs.io/en/latest/](https://osbs.readthedocs.io/en/latest/) if you are interested in the details/histories of this infrastructure), which is then scrubbed before official builds are run with it. The injection is specifically the creation of an RPM which contains all the plugin binaries. - -The team responsible for all this build infrastructure for OpenShift, the Automated Response Team or ART, not surprisingly has there own, separate, Jenkins instance that helps manage some of their dev flows. - -They have provided us a Jenkins pipeline (fitting, I know) that facilitates the building of the plugin RPM and its injection into the Brew pipeline that ultimate results in an image getting built. They have also provided a pipeline for updating the version of Jenkins core we have installed. The pipeline for updating the version of the Jenkins core in our image is at [https://github.com/openshift/aos-cd-jobs/tree/master/jobs/devex/jenkins-bump-version](https://github.com/openshift/aos-cd-jobs/tree/master/jobs/devex/jenkins-bump-version) and the pipeline for updating the set of plugins installed is at [https://github.com/openshift/aos-cd-jobs/tree/master/jobs/devex/jenkins-plugins](https://github.com/openshift/aos-cd-jobs/tree/master/jobs/devex/jenkins-plugins). - -Now, we *used* to be able to log onto their Jenkins server and initiate runs of those 2 pipelines, but towards the end of the 4.1 cycle, corporate processes and guidelines changed such that only members of the ART are allowed to access it. - -So now, you need to open a Jira bug on the ART board at [https://jira.coreos.com/secure/RapidBoard.jspa?rapidView=85](https://jira.coreos.com/secure/RapidBoard.jspa?rapidView=85) to inform them of what is needed. Supply these parameters: -* The jenkins core version to base off of (just supply what we are shipping with 3.11 or 4.x) -* The "OCP_RELEASE" is the OpenShift release (OCP == OpenShift Container Platform) ... so either 3.11, 4.0, 4.1, etc. -* The plugin list is the list you saved from the image build in the PR. Remove the "Installed plugins" header, but include the `:` lines - -An example of such a request is at [https://jira.coreos.com/browse/ART-673](https://jira.coreos.com/browse/ART-673). - -The job typically takes 10 to 15 minutes to succeed. Flakes with Jenkins upstream when downloading plugins is the #1 barrier to success. Just need to retry again until the Jenkins update center stabilizes. Once in a while there is a dist git hiccup (dist git is the git server used by brew). Again, just try again until it settles down. - -When the job succeeds, an email is sent to the folks in the mailing lists. Add yours to the list when submitting the job if it is not listed there. Submit a PR to update [https://github.com/openshift/aos-cd-jobs/blob/master/jobs/devex/jenkins-plugins/Jenkinsfile](https://github.com/openshift/aos-cd-jobs/blob/master/jobs/devex/jenkins-plugins/Jenkinsfile) to add your email if you'll be doing this long term. - -The email sent will contain links that will point you to the dist git commit for the new RPM. - -Store the job link and dist git link in the original PR, like [this comment](https://github.com/openshift/jenkins/pull/829#issuecomment-477686871). - -AND THE LAST PIECE !!!!! .... the openshift/jenkins images produced by OpenShift's Brew are listed at [https://brewweb.engineering.redhat.com/brew/packageinfo?packageID=67183](https://brewweb.engineering.redhat.com/brew/packageinfo?packageID=67183). Make sure the brew registry in in your docker config's insecure registry list (it is `brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888`). If you are on the Red Hat network or VPN, you can download the images and try them out. Also, QA/QE goes to this registry to verify bugs/features. - -NOTE: when they build those images in brew, they actually modify the `Dockerfile.rhel7` files to facilitate: -* some **magic** with respect to the yum repositories used that is particular to running in this quarantined environment -* switch the `INSTALL_JENKINS_VIA_RPMS` so our build scripts do not attempt to download plugins, but rather install the RPMs - -The file which triggers these things for the main image is at [https://github.com/openshift/ocp-build-data/blob/openshift-4.0/images/openshift-jenkins-2.yml](https://github.com/openshift/ocp-build-data/blob/openshift-4.0/images/openshift-jenkins-2.yml) for the 4.0 release. There are branches for the other releases. For the agent images you have [https://github.com/openshift/ocp-build-data/blob/openshift-4.0/images/jenkins-slave-base-rhel7.yml](https://github.com/openshift/ocp-build-data/blob/openshift-4.0/images/jenkins-slave-base-rhel7.yml), [https://github.com/openshift/ocp-build-data/blob/openshift-4.0/images/jenkins-agent-maven-35-rhel7.yml](https://github.com/openshift/ocp-build-data/blob/openshift-4.0/images/jenkins-agent-maven-35-rhel7.yml), and [https://github.com/openshift/ocp-build-data/blob/openshift-4.0/images/jenkins-agent-nodejs-8-rhel7.yml](https://github.com/openshift/ocp-build-data/blob/openshift-4.0/images/jenkins-agent-nodejs-8-rhel7.yml). diff --git a/PLUGINS.md b/PLUGINS.md new file mode 100644 index 000000000..6f521e13d --- /dev/null +++ b/PLUGINS.md @@ -0,0 +1,150 @@ +# Plugins + +### Base set of plugins + +An initial set of Jenkins plugins are included in the OpenShift Jenkins images. The general methodology +is that the CentOS7 image if first updated with any changes to the list of plugins. After some level +of verification with that image, the RHEL7 image is updated. +``` + +#### Plugin installation for CentOS7 V4 +In v4, that call has been moved to [this script](2/contrib/jenkins/install-jenkins-core-plugins.sh), which is called from +both `Dockerfile.localdev`, `Dockerfile.rhel7` and `Dockerfile.rhel8`. + +Where both [base-plugins.txt](2/contrib/openshift/base-plugins.txt) and [install-plugins.sh](2/contrib/jenkins/install-plugins.sh) +are copied into the image prior to that invocation. + +The running of `install-plugins.sh` will download the files listed in `base-plugins.txt`, and then open each plugin's manifest +and download any needed dependencies listed, including upgrading any previously installed dependencies as needed. + +#### Plugin installation for CentOS7 V4.10+ +Starting from `release-4.10`, the `base-plugins.txt` file is instead used to generate `bundle-plugins.txt` which is the comprehensive +list of plugins used by the Jenkins image. To generate this list, developers must run `make plugins-list` prior to commit `base-plugins.txt`. +A git hook is provided to enforce that `bundle-plugins.txt` is always newer than `base-plugins.txt` on every commit attempt. And `openshift-ci` +also runs the `make plugins-list` to be sure that the locally generated list of plugins does not change between the developer commit and the +ci run. +The `bundle-plugins.txt` becomes then the source of truth for the ran, tested and verified plugins list. This file is intended to be used by +anyone who wants to build a Jenkins Image with the exact same set of plugins. Hence, this file is used by the Red Hat internal release +team (ART) and does not alter the existing release process, except that instead of getting the list of plugins from a succesful build, we now get it +from a predefined, pre-test and historized file written to the code repository. + + + +To update the version of a plugin or add a new plugin, construct a PR for this repository that updates `base-plugins.txt` appropriately. +Administrators for this repository will make sure necessary tests are run and merge the PR when things are ready. + +When PRs for this repository's `openshift-3*` branches are merged, they kick off associated builds in the [`push_jenkins_images` job on OpenShift's public +Jenkins CI/CD server](https://ci.openshift.redhat.com/jenkins/view/All/job/push_jenkins_images/). When those builds complete, +new versions of the CentOS7 based versions of the images produced by this repository are pushed to Docker Hub. See the top of the README for the precise list. + +For v4.0, the job definitions for this repository in https://github.com/openshif/release result in our Prow based infrastructure to eventually +mirror the image content on quay.io. + +#### Plugin installation for RHEL7 V3 and V4 + +Only OpenShift developers working for Red Hat can update the list of plugins for the RHEL7 image. For those developers, visit this +[internal Jenkins server](https://buildvm.openshift.eng.bos.redhat.com:8443/job/devex/job/devex%252Fjenkins-plugins/) and log in (contact our CD team for permissions to this job). Click the `Build with parameters` link, update the `PLUGIN_LIST` field, and submit the build. The format of the data for the `PLUGIN_LIST` field is the same as `base-plugins.txt`. + +The complete list of plugins (i.e. including dependencies) needs to be provided though. The most straight forward approach is to mine the output of the CentOS7 build which passed verification for the complete list. Just search for `Installed plugins:` and leverage copy/paste to compile what is needed. + +Although this document will refrain on detailing the precise details, once the build on the internal Jenkins server is complete, +the processes will be set in motion to build the `jenkins-2-plugins` RPM that is installed by the [RHEL7 Dockerfile](2/Dockerfile.rhel7) when the next version of the RHEL7 based OpenShift Jenkins image is built. When new versions of OpenShift are released, associated versions of the RHEL7 based versions of the images produced by this repository are pushed to the Docker registry provided to RHEL7 subscribers. + +Some reference links for the OpenShift Jenkins developers and where things cross over with the CD/CL/Atomic/RHEL teams: +* http://pkgs.devel.redhat.com/cgit/rpms/?q=jenkins +* https://brewweb.engineering.redhat.com/brew/search?match=glob&type=package&terms=*jenkins* + +### Adding plugins or updating existing plugins + +A combination of the contents of this repository and the capabilities of OpenShift allow for a variety of ways to modify +the list of plugins either for the images directly produced from this repository, or by creating images which build +from the images directly produced from this repository. + +The specifics for each approach are detailed below. + +#### Installing using layering + +In order to install additional Jenkins plugins, the OpenShift Jenkins image provides a way similar to how +the [initial set of plugins are added](#plugin-installation-for-centos7-v3-only) to this image that will allow one +to add or update by layering on top of this image. The derived image will provide the same functionality +as described in this documentation, in addition it will also include all plugins you list in the `plugins.txt` file. + +To create a derived image in this fashion, create the following `Dockerfile`: + +``` +FROM openshift/jenkins-2-centos7 +COPY plugins.txt /opt/openshift/configuration/plugins.txt +RUN /usr/local/bin/install-plugins.sh /opt/openshift/configuration/plugins.txt +``` + +The format of `plugins.txt` file is: + +``` +pluginId:pluginVersion +``` + +For example, to install the github Jenkins plugin, you specify following to `plugins.txt`: + +``` +github:1.11.3 +``` + +After this, just run `docker build -t my_jenkins_image -f Dockerfile`. + +#### Installing using S2I build + +The [s2i](https://github.com/openshift/source-to-image) tool allows you to do additional modifications of this Jenkins image. +For example, you can use S2I to copy custom Jenkins Jobs definitions, additional +plugins or replace the default `config.xml` file with your own configuration. + +To do that, you can either use the standalone `s2i` tool, that will produce the +customized Docker image or you can use OpenShift Source build strategy. + +In order to include your modifications in Jenkins image, you need to have a Git +repository with following directory structure: + +* `./plugins` folder that contains binary Jenkins plugins you want to copy into Jenkins +* `./plugins.txt` file that list the plugins you want to install (see the section above) +* `./configuration/jobs` folder that contains the Jenkins job definitions +* `./configuration/config.xml` file that contains your custom Jenkins configuration + +Note that the `./configuration` folder will be copied into `/var/lib/jenkins` +folder, so you can also include additional files (like `credentials.xml`, etc.). + +To build your customized Jenkins image, you can then execute following command: + +```console +$ s2i build https://github.com/your/repository openshift/jenkins-2-centos7 your_image_name +``` +NOTE: if instead of adding a plugin you want to replace an existing plugin via dropping the binary plugin in the `./plugins` directory, +make sure the filename ends in `.jpi`. + +#### Installing on Startup + +The `INSTALL_PLUGINS` environment variable may be used to install a set of plugins on startup. When using a +persistent volume for `/var/lib/jenkins`, plugin installation will only happen on the initial run of the image. + +In the following example, the Groovy and Pull Request Builder plugins are installed + +``` +INSTALL_PLUGINS=groovy:1.30,ghprb:1.35.0 +``` + +### Plugins focused on integration with OpenShift + +A subset of the plugins included by the images of this repository play a direct part in integrating between Jenkins and OpenShift. + +* **OpenShift Client Plugin** +Visit [the upstream repository](https://github.com/openshift/jenkins-client-plugin) as well as the [Jenkins plugin wiki](https://wiki.jenkins-ci.org/display/JENKINS/OpenShift+Client+Plugin). With the lessons learned from OpenShift Pipeline Plugin, as well as adjustments to the rapid evolutions of both Jenkins and OpenShift, this plugin, with its fluent styled syntax and use of the `oc` binary (exposing all the capabilities of that command), is the preferred choice for interacting with OpenShift via either Jenkins Pipeline or Freestyle jobs. + +* **OpenShift Sync Plugin** +Visit [the upstream repository](https://github.com/openshift/jenkins-sync-plugin) as well as the [Jenkins plugin wiki](https://wiki.jenkins-ci.org/display/JENKINS/OpenShift+Sync+Plugin). This plugin facilitates the integration between the OpenShift Jenkinsfile Build Strategy and Jenkins Pipelines. It also facilitates auto-configuration of the agent pod templates for the Kubernetes Plugin. See the [OpenShift documentation](https://docs.openshift.com) for more details. + +* **OpenShift Login Plugin** +Visit [the upstream repository](https://github.com/openshift/jenkins-openshift-login-plugin) as well as the [Jenkins plugin wiki](https://wiki.jenkins-ci.org/display/JENKINS/OpenShift+Login+Plugin). This plugin integrates the authentication and authorization of your Jenkins instance with you OpenShift cluster, providing a single sign-on look and feel. You'll sign into the Jenkins server using the same credentials that you use to sign into the OpenShift Web Console or interact with OpenShift via the `oc` CLI. See the [OpenShift documentation](https://docs.openshift.com) for more details. + +For the above OpenShift Jenkins plugins, each of their READMEs have specifics unique to each of them on how to use and if so desired contribute to their development. That said, there is a good deal of commonality and shared infrastructure +related to developing, creating new versions, and ultimately updating the images of this repository with those new versions. If you would like to understand the specifics of that process, please visit our [contribution guide](CONTRIBUTING_TO_OPENSHIFT_JENKINS_IMAGE_AND_PLUGINS.md). + +* **Kubernetes Plugin** +Though not originated out of the OpenShift organization, this plugin is invaluable in that it allows agents to be dynamically provisioned on multiple Docker hosts using [Kubernetes](https://github.com/kubernetes/kubernetes). To learn how to use this plugin, see the [example](https://github.com/openshift/origin/tree/master/examples/jenkins/master-agent) available in the OpenShift Origin repository. For more details about this plugin, visit the [plugin](https://wiki.jenkins-ci.org/display/JENKINS/Kubernetes+Plugin) web site. diff --git a/README.md b/README.md index 15a097d9d..b41999b8d 100644 --- a/README.md +++ b/README.md @@ -1,488 +1,55 @@ -Jenkins Docker Image -==================== +# OpenShift Jenkins Images -This repository contains Dockerfiles for Jenkins Docker images intended for -use with [OpenShift v3 and OpenShift v4](https://github.com/openshift/origin) +## Introduction +This repository contains Dockerfiles for building Jenkins Master and Agent images intended for use with [OKD 4](https://www.okd.io/) and [Red Hat OpenShift 4](https://www.redhat.com/en/technologies/cloud-computing/openshift). -For an example of how to use it, [see this sample.](https://github.com/openshift/origin/blob/master/examples/jenkins/README.md) +## Hosted Images +All OpenShift 4 images (including the ones from this repository) are based +off of the [Red Hat Universal Base Image 8](https://catalog.redhat.com/software/containers/ubi8/5c647760bed8bd28d0e38f9f). -The images are pushed to DockerHub as openshift/jenkins-2-centos7, openshift/jenkins-slave-base-centos7, openshift/jenkins-agent-maven-35-centos7, and openshift/jenkins-agent-nodejs-8-centos7. +**NOTE:** Only the 64-bit JVM is available in all images. +### Community +These images are available via [quay.io](https://quay.io) and are community supported. +* [quay.io/openshift/origin-jenkins](https://quay.io/openshift/origin-jenkins) +* [quay.io/openshift/origin-jenkins-agent-nodejs](https://quay.io/openshift/origin-jenkins-agent-nodejs) +* [quay.io/openshift/origin-jenkins-agent-maven](https://quay.io/openshift/origin-jenkins-agent-maven) +* [quay.io/openshift/origin-jenkins-agent-base](https://quay.io/openshift/origin-jenkins-agent-base) -The slave-maven and slave-nodejs for both centos7 and rhel7 are being deprecated as part of v3.10 of OpenShift. -Additionally, development of these images will cease as of v3.10. And they are removed from this repository as -part of the v4.0 development cycle. +**NOTE:** The jenkins-agent-maven and jenkins-agent-nodejs image are no longer maintained as of version 4.11 and no longer published as of version 4.14. -Support for the [OpenShift Pipeline Plugin](https://github.com/openshift/jenkins-plugin) stopped with v3.11 of OpenShift. -The plugin itself and any samples around it are removed as part of v4.0. +### Red Hat OpenShift +These images are available via the [Red Hat Catalog](https://catalog.redhat.com) for customers with subscriptions. +#### 4.10 and lower +These images are intended for OpenShift 4.10 and lower. +* [openshift4/ose-jenkins](https://catalog.redhat.com/software/containers/openshift4/ose-jenkins/5cdd918ad70cc57c44b2d279) +* [openshift4/ose-jenkins-agent-base](https://catalog.redhat.com/software/containers/openshift4/ose-jenkins-agent-base/5cdd8e2fbed8bd5717d66e77) +* [openshift4/ose-jenkins-agent-maven](https://catalog.redhat.com/software/containers/openshift4/ose-jenkins-agent-maven/5cdd8fe55a13467289f615e7) +* [openshift4/ose-jenkins-agent-nodejs-12-rhel8](https://catalog.redhat.com/software/containers/openshift4/ose-jenkins-agent-nodejs-12-rhel8/5f6c39da1fa29796579cdff7) -For more information about using these images with OpenShift, please see the -official [OpenShift Documentation](https://docs.okd.io/latest/using_images/other_images/jenkins.html). +#### 4.11 and higher +These images are intended for OpenShift 4.11 and higher. +* [ocp-tools-4/jenkins-rhel8](https://catalog.redhat.com/software/containers/ocp-tools-4/jenkins-rhel8/5fe1f38288e9c2f788526306) +* [ocp-tools-4/jenkins-agent-base-rhel8](https://catalog.redhat.com/software/containers/ocp-tools-4/jenkins-agent-base-rhel8/6241e3457847116cf8577aea) -Versions ---------------------------------- -Jenkins versions currently provided are: -* [jenkins-2.x](../master/2) +**NOTE:** The jenkins-agent-maven and jenkins-agent-nodejs image are no longer maintained or published as of version 4.11. -For OpenShift v3, the options for the images` operating system are as follows: +## Building +Please see [BUILDING.md](https://github.com/openshift/jenkins/blob/master/BUILDING.md). -RHEL versions currently supported are: -* RHEL7 +## Basic Usage +Please see [BASIC_USAGE.md](https://github.com/openshift/jenkins/blob/master/BASIC_USAGE.md). -CentOS versions currently supported are: -* CentOS7 +## Advanced Usage +Please see [ADVANCED_USAGE.md](https://github.com/openshift/jenkins/blob/master/ADVANCED_USAGE.md). -For OpenShift v4, the operating system choice is reduced. All OpenShift v4 images (including the ones from this repository) are based -off of the ["Universal Based Image" or "UBI"](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux_atomic_host/7/html/getting_started_with_containers/using_red_hat_base_container_images_standard_and_minimal#using_rhel_atomic_base_images_minimal). +## Plugins +Please see [PLUGINS.md](https://github.com/openshift/jenkins/blob/master/PLUGINS.md). +## Security +Please see [SECURITY.md](https://github.com/openshift/jenkins/blob/master/SECURITY.md). -Installation (OpenShift V3) ---------------------------------- -Choose either the CentOS7 or RHEL7 based image: +## Testing +Please see [TESTING.md](https://github.com/openshift/jenkins/blob/master/TESTING.md). -* **RHEL7 based image** - -You can access these images from the Red Hat Container Catalog. For OpenShift v3 see: -* https://access.redhat.com/containers/#/registry.access.redhat.com/openshift3/jenkins-2-rhel7 -* https://access.redhat.com/containers/#/registry.access.redhat.com/openshift3/jenkins-slave-base-rhel7 -* https://access.redhat.com/containers/#/registry.access.redhat.com/openshift3/jenkins-agent-maven-35-rhel7 -* https://access.redhat.com/containers/#/registry.access.redhat.com/openshift3/jenkins-agent-nodejs-8-rhel7 - -To build a RHEL7 based image, you need to run Docker build on a properly -subscribed RHEL machine. - - ``` - $ git clone https://github.com/openshift/jenkins.git - $ cd jenkins - $ make build TARGET=rhel7 VERSION=2 - ``` - -You can also run the build using Podman. However, e2e tests does not run -with Podman (which is the default). - ``` - $ make build TARGET=rhel7 VERSION=2 BUILD_COMMAND="podman build" - ``` - -Also note, as of 3.11, the RHEL images are hosted at registry.redhat.io as well. This is the terms based -registry and requires credentials for access. See [Transitioning the Red Hat container registry](https://www.redhat.com/en/blog/transitioning-red-hat-container-registry) for details: -* registry.redhat.io/openshift3/jenkins-2-rhel7:v3.11 -* registry.redhat.io/openshift3/jenkins-agent-nodejs-8-rhel7:v3.11 -* registry.redhat.io/openshift3/jenkins-agent-maven-35-rhel7:v3.11 -* registry.redhat.io/openshift3/jenkins-slave-base-rhel7:v3.11 - -The openshift cluster install for 3.11 will ensure that credentials are provided and subsequently available on the nodes -in the cluster to facilitate image pulling. - - -* **CentOS7 based image** - -The v3.x images are available on DockerHub. An example download command is: - - ``` - $ docker pull openshift/jenkins-2-centos7 - ``` - -To build a Jenkins image from scratch run: - - ``` - $ git clone https://github.com/openshift/jenkins.git - $ cd jenkins - $ make build VERSION=2 - ``` - -**Notice: By omitting the `VERSION` parameter, the build/test action will be performed -on all provided versions of Jenkins.** - -If you are curious about the precise level of Jenkins for either `jenkins-2-centos7` or `jenkins-2-rhel7`, then -you can execute: - - - $ docker run -it /etc/alternatives/java -jar /usr/lib/jenkins/jenkins.war --version - - -For example: - - - $ docker run -it docker.io/openshift/jenkins-2-centos7:latest /etc/alternatives/java -jar /usr/lib/jenkins/jenkins.war --version - -Installation (OpenShift V4) ---------------------------------- - -Starting with v4.0, the images are only available on quay.io for public community support. Their pull specs are: -* quay.io/openshift/origin-jenkins: -* quay.io/openshift/origin-jenkins-agent-nodejs: -* quay.io/openshift/origin-jenkins-agent-maven: -* quay.io/openshift/origin-jenkins-agent-base: - -Visit quay.io to discover the set of tags for each image. For example, for the core jenkins image, the tags are [here](https://quay.io/repository/openshift/origin-jenkins?tab=tags) - -The images are also still available at the Red Hat Container Catalog for customers with subscriptions, -though with some changes in the naming. - -As with the initial introduction in 3.11, given the [transitioning of the Red Hat container registry](https://www.redhat.com/en/blog/transitioning-red-hat-container-registry), the RHEL based images are available at both registry.access.redhat.com and registry.redhat.io. -The terms based registry, registry.redhat.io, which requires credentials for access, is the strategic direction, and -will be the only location for RHEL8 based content when that is available. The pull secret you obtain from try.openshift.com includes -access to registry.redhat.io. The image pull specs are: -* registry.redhat.io/openshift4/ose-jenkins: -* registry.redhat.io/openshift4/ose-jenkins-agent-nodejs: -* registry.redhat.io/openshift4/ose-jenkins-agent-maven: -* registry.redhat.io/openshift4/ose-jenkins-agent-base: - -OpenShift v4 also removes the 32 bit JVM option. Only 64 bit will be provided for all images. - -The `Dockerfile.rhel7` variants still exists, but as part of the `CentOS` vs. `RHEL` distinction no longer existing, the various `Dockerfile` files have been renamed to `Dockerfile.localdev` to more clearly denote that they are for builds on developers' local machines that most likely do not have a Red Hat subscription / entitlement. The `Dockerfile.localdev` variants are structured to allow building of the images on machines without `RHEL` subscriptions, even though the base images are no longer based on `CentOS`. Subscriptions are still required for use of `Dockerfile.rhel7`. - -With any local builds, if for example you plan on submitting a PR to this repository, you still build the same way as with OpenShift v3 with respect to the `make` invocations. - -Be aware, no support in any way is provided for running images created from any of the `Dockerfile.localdev` files. And in fact the images hosted on both quay.io and the Red Hat Container Catalog are based off the `Dockerfile.rhel7` files. - -And lastly, as part of 4.x cluster installs, the OpenShift Jenkins image version corresponding to the cluster version is part of the image payload for the install. So the `jenkins` ImageStream in the `openshift` namespace will have image references that point to the image registry associated with your install instead of these public registries noted above. There is also an ImageStream for each of the agent images in the `openshift` namespace in 4.x installs. - - -Startup notes for the Jenkins core image ---------------------------------- - -When you run you startup the main Jenkins image in an OpenShift pod for the first time, it performs various set up actions, including: -* Setting the JVM parameters for the actual start of the Jenkins JVM -* Updating the /etc/passwd so that the random, non-root user ID employed works -* Copies all the default configuration from the image to the appropriate locations under the Jenkins home directory (which maps to the image's volume mount) -* Copies all the plugins to the appropriate locations under the Jenkins home directory - -By default, all copies to the Jenkins home directory are only done on the initial startup if a Persistent Volume is employed for the Jenkins deployment. There are ways to override that behavior by environment variables (see the next section below). But you can also recycle the PVCs during restarts of your Jenkins deployment if you update the image being used and want to reset the system that way. - - -Environment variables ---------------------------------- - -The image recognizes the following environment variables that you can set during -initialization by passing `-e VAR=VALUE` to the Docker run command. - -| Variable name | Description | -| :------------------------ | ----------------------------------------- | -| `JENKINS_PASSWORD` | Password for the 'admin' account when using default Jenkins authentication. | -| `OPENSHIFT_ENABLE_OAUTH` | Determines whether the OpenShift Login plugin manages authentication when logging into Jenkins. | -| `OPENSHIFT_PERMISSIONS_POLL_INTERVAL` | Specifies in milliseconds how often the OpenShift Login plugin polls OpenShift for the permissions associated with each user defined in Jenkins. | -| `INSTALL_PLUGINS` | Comma-separated list of additional plugins to install on startup. The format of each plugin spec is `plugin-id:version` (as in plugins.txt) | -| `OVERRIDE_PV_CONFIG_WITH_IMAGE_CONFIG` | When running this image with an OpenShift persistent volume for the Jenkins config directory, the transfer of configuration from the image to the persistent volume is only done the first startup of the image as the persistent volume is assigned by the persistent volume claim creation. If you create a custom image that extends this image and updates configuration in the custom image after the initial startup, by default it will not be copied over, unless you set this environment variable to some non-empty value other than 'false'. | -| `OVERRIDE_PV_PLUGINS_WITH_IMAGE_PLUGINS` | When running this image with an OpenShift persistent volume for the Jenkins config directory, the transfer of plugins from the image to the persistent volume is only done the first startup of the image as the persistent volume is assigned by the persistent volume claim creation. If you create a custom image that extends this image and updates plugins in the custom image after the initial startup, by default they will not be copied over, unless you set this environment variable to some non-empty value other than 'false'. | -| `OVERRIDE_RELEASE_MIGRATION_OVERWRITE` | When running this image with an OpenShift persistent volume for the Jenkins config directory, and this image is starting in an existing deployment created with an earlier version of this image, if the environment variable is not set or set to any value other than 'false', the plugins from the image will replace any versions of those plugins currently residing in the Jenkins plugin directory. | -| `SKIP_NO_PROXY_DEFAULT` | This environment variable applies to the agent/slave images produced by this repository. By default, the agent/slave images will create/update the 'no_proxy' environment variable with the hostnames for the Jenkins server endpoint and Jenkins JNLP endpoint, as communication flows to endpoints typically should *NOT* go through a HTTP Proxy. However, if your use case dictates those flows should not be exempt from the proxy, set this environment variable to any non-empty value other than 'false'. | -| `ENABLE_FATAL_ERROR_LOG_FILE` | When running this image with an OpenShift persistent volume claim for the Jenkins config directory, this environment variable allows the fatal error log file to persist if a fatal error occurs. The fatal error file will be located at `/var/lib/jenkins/logs`. | -| `NODEJS_SLAVE_IMAGE` | Setting this value will override the image used for the default NodeJS agent pod configuration. For 3.x, the default NodeJS agent pod uses `docker.io/openshift/jenkins-agent-nodejs-8-centos7` or `registry.redhat.io/openshift3/jenkins-agent-nodejs-8-rhel7` depending whether you are running the centos or rhel version of the Jenkins image. This variable must be set before Jenkins starts the first time for it to have an effect. For 4.x, the image is included in the 4.0 payload via an imagestream in the openshift namespace, and the image spec points to the internal image registry. If you are running this image outside of OpenShift, you must either set this environment variable or manually update the setting to an accessible image spec. | -| `MAVEN_SLAVE_IMAGE` | Setting this value overrides the image used for the default maven agent pod configuration. For 3.x, the default maven agent pod uses `docker.io/openshift/jenkins-agent-maven-35-centos7` or `registry.redhat.io/openshift3/jenkins-agent-maven-35-rhel7` depending whether you are running the centos or rhel version of the Jenkins image. For 4.x, the image is included in the 4.0 payload via an imagestream in the openshift namespace, and the image spec points to the internal image registry. If you are running this image outside of OpenShift, you must either set this environment variable or manually update the setting to an accessible image spec. This variable must be set before Jenkins starts the first time for it to have an effect. | -| `JENKINS_UC_INSECURE` | When your Jenkins Update Center repository is using a self-signed certificate with an unknown Certificate Authority, this variable allows one to bypass the repository's SSL certificate check. The variable applies to download of the plugin which may occur during Jenkins image build, if you build an extension of the jenkins image or if you run the Jenkins image and leverage one of the options to download additional plugins (use of s2i whith plugins.txt or use of `INSTALL_PLUGINS` environment variable. | -| `MAVEN_MIRROR_URL` | Allows you to specify a [Maven mirror repository](https://maven.apache.org/guides/mini/guide-mirror-settings.html) in the form of `MAVEN_MIRROR_URL=https://nexus.mycompany.com/repository/maven-public`. The mirror repository is used by maven as an additional location for artifacts downloads. For more details on [how maven mirrors works](https://maven.apache.org/guides/mini/guide-mirror-settings.html), you can refer to maven documentation. -| `USE_JAVA_VERSION` | Allows you to set the Java version used by the JNLP Client for the agent images. By default the value is set to `java-11`. To use Java 8 for the value for this env var to `java-8`. | -| `AGENT_BASE_IMAGE` | Setting this value overrides the image used for the 'jnlp' container in the sample kubernetes plug-in PodTemplates provided with this image. Otherwise, the image from the 'jenkins-agent-base:latest' ImageStreamTag in the 'openshift' namespace is used. | -| `JAVA_BUILDER_IMAGE` | Setting this value overrides the image used for the 'java-builder' container in the sample kubernetes plug-in PodTemplates provided with this image. Otherwise, the image from the 'java:latest' ImageStreamTag in the 'openshift' namespace is used. | -| `NODEJS_BUILDER_IMAGE` | Setting this value overrides the image used for the 'nodejs-builder' container in the sample kubernetes plug-in PodTemplates provided with this image. Otherwise, the image from the 'nodejs:latest' ImageStreamTag in the 'openshift' namespace is used. | -| `JAVA_FIPS_OPTIONS` | Per this [OpenJDK support article](https://access.redhat.com/documentation/en-us/openjdk/11/html-single/configuring_openjdk_11_on_rhel_with_fips/index#config-fips-in-openjdk) control how the JVM operates when running on a FIPS node. | - -You can also set the following mount points by passing the `-v /host:/container` flag to Docker. - -| Volume mount point | Description | -| :-------------------------- | ------------------------ | -| `/var/lib/jenkins` | Jenkins config directory | - -**Notice: When mounting a directory from the host into the container, ensure that the mounted -directory has the appropriate permissions and that the owner and group of the directory -matches the user UID or name which is running inside the container.** - -Inclusion of the `oc` binary ---------------------------------- - -To assist in interacting with the OpenShift API server while using this image, the `oc` binary, the CLI command for OpenShift, has been -installed in the master and slave images defined in this repository. - - -However, it needs to be noted that backward compatibility is not guaranteed between different versions of `oc` and the OpenShift -API Server. As such, it is recommended that you align versions of this image present in the nodes of your cluster with your -OpenShift API server. In other words, you should use the version specific tag instead of the `latest` tag. - -| Jenkins image version | `oc` client version | -| :-------------------------- | ------------------------ | -| `jenkins-*-centos7:v3.7` | 3.7 `oc` binary | -| `jenkins-*-centos7:v3.6` | 3.6 `oc` binary | -| `jenkins-*-centos7:latest` | `oc` binary from `docker.io/openshift/origin:latest` image | -| `jenkins-*-rhel7:v3.7` | 3.7 `oc` binary | -| `jenkins-*-rhel7:v3.6` | 3.6 `oc` binary | -| `jenkins-*-rhel7:latest` | 3.6 `oc` binary \*\* | - - -**Notice: the `latest` tag for the RHEL7 images will point to 3.6 indefinitely in order to support users on older clusters with older slave -configurations that point to the "latest" tag. This way, they will have an older `oc` client which should be able to communicate with both 3.6 -and newer versions of OpenShift API Servers. As the support policy is less stringent for the CentOS7 image, the `latest` tag there will -make the more obvious correlation to the latest built version of OpenShift (which can include pre-GA versions). - -**Notice: There is an additional consideration with the pod configurations for the Kubernetes Plugin; earlier versions of this image -did not specify the "pull always" policy for the default agents/slaves configured. As a result, users may have older/different images on -your nodes depending when the images were pulled. Starting with the 3.7 release, the default changed to "pull always" to avoid this problem -in the future. But if you started using this image prior to 3.7, verification of your Kubernetes plugin configurations for the image pull -policy used is warranted to guarantee consistency around what image is being used on each of your nodes. - -The `oc` binary is still included in the v4 images as well. And the same recommendations around client/server version synchronization still apply. - -Jenkins security advisories, the "master" image from this repository, and the `oc` binary ---------------------------------- - -Any security advisory related updates to Jenkins core or the plugins we include in the OpenShift Jenkins master image will only occur in the v3.11 and v4.x -branches of this repository. - -We do support running the v3.11 version of the master image against older v3.x (as far back as v3.4) OpenShift clusters if you want to pick up Jenkins security advisory -updates. Per the prior section, we advise that you import a version of `oc` into your Jenkins installation that matches your OpenShift -cluster via the "Global Tool Configuration" option in Jenkins either via the UI, CLI, or groovy init scripts. - -Our OpenShift Client Plugin has some documentation on doing this [here](https://github.com/openshift/jenkins-client-plugin#setting-up-jenkins-nodes). - -Also note for the RHEL image, the v3.11 image examines whether it is running in an OpenShift Pod and what version the cluster is at. If the cluster is at a version prior to v3.11, the Maven and NodeJS agent example configuration for the kubernetes plugin will point to registry.access.redhat.com for -the image setting. If the cluster is at v3.11, the image setting will point to the terms based registry at registry.access.io. - - -Plugins ---------------------------------- - -### Base set of plugins - -An initial set of Jenkins plugins are included in the OpenShift Jenkins images. The general methodology -is that the CentOS7 image if first updated with any changes to the list of plugins. After some level -of verification with that image, the RHEL7 image is updated. - -#### Plugin installation for CentOS7 V3 Only - -The top level list of plugins to install is located [here](2/contrib/openshift/base-plugins.txt). The -format of the file is: - -``` -pluginId:pluginVersion -``` - -For v3, the file is processed by the following call in the [CentOS7 Dockerfile](2/Dockerfile): - -``` -/usr/local/bin/install-plugins.sh /opt/openshift/base-plugins.txt -``` - -#### Plugin installation for CentOS7 V4 -In v4, that call has been moved to [this script](2/contrib/jenkins/install-jenkins-core-plugins.sh), which is called from -both `Dockerfile.localdev`, `Dockerfile.rhel7` and `Dockerfile.rhel8`. - -Where both [base-plugins.txt](2/contrib/openshift/base-plugins.txt) and [install-plugins.sh](2/contrib/jenkins/install-plugins.sh) -are copied into the image prior to that invocation. - -The running of `install-plugins.sh` will download the files listed in `base-plugins.txt`, and then open each plugin's manifest -and download any needed dependencies listed, including upgrading any previously installed dependencies as needed. - -#### Plugin installation for CentOS7 V4.10+ -Starting from `release-4.10`, the `base-plugins.txt` file is instead used to generate `bundle-plugins.txt` which is the comprehensive -list of plugins used by the Jenkins image. To generate this list, developers must run `make plugins-list` prior to commit `base-plugins.txt`. -A git hook is provided to enforce that `bundle-plugins.txt` is always newer than `base-plugins.txt` on every commit attempt. And `openshift-ci` -also runs the `make plugins-list` to be sure that the locally generated list of plugins does not change between the developer commit and the -ci run. -The `bundle-plugins.txt` becomes then the source of truth for the ran, tested and verified plugins list. This file is intended to be used by -anyone who wants to build a Jenkins Image with the exact same set of plugins. Hence, this file is used by the Red Hat internal release -team (ART) and does not alter the existing release process, except that instead of getting the list of plugins from a succesful build, we now get it -from a predefined, pre-test and historized file written to the code repository. - - - -To update the version of a plugin or add a new plugin, construct a PR for this repository that updates `base-plugins.txt` appropriately. -Administrators for this repository will make sure necessary tests are run and merge the PR when things are ready. - -When PRs for this repository's `openshift-3*` branches are merged, they kick off associated builds in the [`push_jenkins_images` job on OpenShift's public -Jenkins CI/CD server](https://ci.openshift.redhat.com/jenkins/view/All/job/push_jenkins_images/). When those builds complete, -new versions of the CentOS7 based versions of the images produced by this repository are pushed to Docker Hub. See the top of the README for the precise list. - -For v4.0, the job definitions for this repository in https://github.com/openshif/release result in our Prow based infrastructure to eventually -mirror the image content on quay.io. - -#### Plugin installation for RHEL7 V3 and V4 - -Only OpenShift developers working for Red Hat can update the list of plugins for the RHEL7 image. For those developers, visit this -[internal Jenkins server](https://buildvm.openshift.eng.bos.redhat.com:8443/job/devex/job/devex%252Fjenkins-plugins/) and log in (contact our CD team for permissions to this job). Click the `Build with parameters` link, update the `PLUGIN_LIST` field, and submit the build. The format of the data for the `PLUGIN_LIST` field is the same as `base-plugins.txt`. - -The complete list of plugins (i.e. including dependencies) needs to be provided though. The most straight forward approach is to mine the output of the CentOS7 build which passed verification for the complete list. Just search for `Installed plugins:` and leverage copy/paste to compile what is needed. - -Although this document will refrain on detailing the precise details, once the build on the internal Jenkins server is complete, -the processes will be set in motion to build the `jenkins-2-plugins` RPM that is installed by the [RHEL7 Dockerfile](2/Dockerfile.rhel7) when the next version of the RHEL7 based OpenShift Jenkins image is built. When new versions of OpenShift are released, associated versions of the RHEL7 based versions of the images produced by this repository are pushed to the Docker registry provided to RHEL7 subscribers. - -Some reference links for the OpenShift Jenkins developers and where things cross over with the CD/CL/Atomic/RHEL teams: -* http://pkgs.devel.redhat.com/cgit/rpms/?q=jenkins -* https://brewweb.engineering.redhat.com/brew/search?match=glob&type=package&terms=*jenkins* - -### Adding plugins or updating existing plugins - -A combination of the contents of this repository and the capabilities of OpenShift allow for a variety of ways to modify -the list of plugins either for the images directly produced from this repository, or by creating images which build -from the images directly produced from this repository. - -The specifics for each approach are detailed below. - -#### Installing using layering - -In order to install additional Jenkins plugins, the OpenShift Jenkins image provides a way similar to how -the [initial set of plugins are added](#plugin-installation-for-centos7-v3-only) to this image that will allow one -to add or update by layering on top of this image. The derived image will provide the same functionality -as described in this documentation, in addition it will also include all plugins you list in the `plugins.txt` file. - -To create a derived image in this fashion, create the following `Dockerfile`: - -``` -FROM openshift/jenkins-2-centos7 -COPY plugins.txt /opt/openshift/configuration/plugins.txt -RUN /usr/local/bin/install-plugins.sh /opt/openshift/configuration/plugins.txt -``` - -The format of `plugins.txt` file is: - -``` -pluginId:pluginVersion -``` - -For example, to install the github Jenkins plugin, you specify following to `plugins.txt`: - -``` -github:1.11.3 -``` - -After this, just run `docker build -t my_jenkins_image -f Dockerfile`. - -#### Installing using S2I build - -The [s2i](https://github.com/openshift/source-to-image) tool allows you to do additional modifications of this Jenkins image. -For example, you can use S2I to copy custom Jenkins Jobs definitions, additional -plugins or replace the default `config.xml` file with your own configuration. - -To do that, you can either use the standalone `s2i` tool, that will produce the -customized Docker image or you can use OpenShift Source build strategy. - -In order to include your modifications in Jenkins image, you need to have a Git -repository with following directory structure: - -* `./plugins` folder that contains binary Jenkins plugins you want to copy into Jenkins -* `./plugins.txt` file that list the plugins you want to install (see the section above) -* `./configuration/jobs` folder that contains the Jenkins job definitions -* `./configuration/config.xml` file that contains your custom Jenkins configuration - -Note that the `./configuration` folder will be copied into `/var/lib/jenkins` -folder, so you can also include additional files (like `credentials.xml`, etc.). - -To build your customized Jenkins image, you can then execute following command: - -```console -$ s2i build https://github.com/your/repository openshift/jenkins-2-centos7 your_image_name -``` -NOTE: if instead of adding a plugin you want to replace an existing plugin via dropping the binary plugin in the `./plugins` directory, -make sure the filename ends in `.jpi`. - -#### Installing on Startup - -The `INSTALL_PLUGINS` environment variable may be used to install a set of plugins on startup. When using a -persistent volume for `/var/lib/jenkins`, plugin installation will only happen on the initial run of the image. - -In the following example, the Groovy and Pull Request Builder plugins are installed - -``` -INSTALL_PLUGINS=groovy:1.30,ghprb:1.35.0 -``` - -### Plugins focused on integration with OpenShift - -A subset of the plugins included by the images of this repository play a direct part in integrating between Jenkins and OpenShift. - -* **OpenShift Client Plugin** -Visit [the upstream repository](https://github.com/openshift/jenkins-client-plugin) as well as the [Jenkins plugin wiki](https://wiki.jenkins-ci.org/display/JENKINS/OpenShift+Client+Plugin). With the lessons learned from OpenShift Pipeline Plugin, as well as adjustments to the rapid evolutions of both Jenkins and OpenShift, this plugin, with its fluent styled syntax and use of the `oc` binary (exposing all the capabilities of that command), is the preferred choice for interacting with OpenShift via either Jenkins Pipeline or Freestyle jobs. - -* **OpenShift Sync Plugin** -Visit [the upstream repository](https://github.com/openshift/jenkins-sync-plugin) as well as the [Jenkins plugin wiki](https://wiki.jenkins-ci.org/display/JENKINS/OpenShift+Sync+Plugin). This plugin facilitates the integration between the OpenShift Jenkinsfile Build Strategy and Jenkins Pipelines. It also facilitates auto-configuration of the slave pod templates for the Kubernetes Plugin. See the [OpenShift documentation](https://docs.openshift.com) for more details. - -* **OpenShift Login Plugin** -Visit [the upstream repository](https://github.com/openshift/jenkins-openshift-login-plugin) as well as the [Jenkins plugin wiki](https://wiki.jenkins-ci.org/display/JENKINS/OpenShift+Login+Plugin). This plugin integrates the authentication and authorization of your Jenkins instance with you OpenShift cluster, providing a single sign-on look and feel. You'll sign into the Jenkins server using the same credentials that you use to sign into the OpenShift Web Console or interact with OpenShift via the `oc` CLI. See the [OpenShift documentation](https://docs.openshift.com) for more details. - -For the above OpenShift Jenkins plugins, each of their READMEs have specifics unique to each of them on how to use and if so desired contribute to their development. That said, there is a good deal of commonality and shared infrastructure -related to developing, creating new versions, and ultimately updating the images of this repository with those new versions. If you would like to understand the specifics of that process, please visit our [contribution guide](CONTRIBUTING_TO_OPENSHIFT_JENKINS_IMAGE_AND_PLUGINS.md). - -* **Kubernetes Plugin** -Though not originated out of the OpenShift organization, this plugin is invaluable in that it allows slaves to be dynamically provisioned on multiple Docker hosts using [Kubernetes](https://github.com/kubernetes/kubernetes). To learn how to use this plugin, see the [example](https://github.com/openshift/origin/tree/master/examples/jenkins/master-slave) available in the OpenShift Origin repository. For more details about this plugin, visit the [plugin](https://wiki.jenkins-ci.org/display/JENKINS/Kubernetes+Plugin) web site. - -Configuration files -------------------------------- - -The layering and s2i build flows noted above for updating the list of plugins can also be used to update the configuration injected into the Jenkins deployment. However, don't forget the note about copying of config data and Persistent Volumes in the [startup notes](#startup-notes-for-the-jenkins-core-image). - -A typical scenario employed by our users has been extending the Jenkins image to add groovy init scripts to customize your Jenkins installation. - -A quick recipe of how to do that via layering would be: - -* mkdir -p contrib/openshift/configuration/init.groovy.d -* create a contrib/openshift/configuration/init.groovy.d/foo.groovy file with whatever groovy init steps you desire -* create a Dockerfile with (adjusting the image ref as you see fit) - -``` -FROM registry.access.redhat.com/openshift3/jenkins-2-rhel7:v3.11 -COPY ./contrib/openshift /opt/openshift -``` - -And then update your Jenkins deployment to use the resulting image directly, or update the ImageStreamTag reference you Jenkins deployment is employing, with our new image. During startup, -the existing run script your new image inherits from this repositories Jenkins image will copy the groovy init script to the appropriate spot under the Jenkins home directory. - - -Usage ---------------------------------- - -For this, we will assume that you are using an `openshift/jenkins-2-centos7` image for v3.x, or -`quay.io/openshift/origin-jenkins` for v4.x. - -If you want to set only the mandatory environment variables and store the database -in the `/tmp/jenkins` directory on the host filesystem, execute the following command: - -``` -$ docker run -d -e JENKINS_PASSWORD= -v /tmp/jenkins:/var/lib/jenkins openshift/jenkins-2-centos7 -``` - - -Jenkins admin user ---------------------------------- - -Authenticating into a Jenkins server running within the OpenShift Jenkins image is controlled by the [OpenShift Login plugin](https://github.com/openshift/jenkins-openshift-login-plugin), taking into account: - -* Whether or not the container is running in an OpenShift Pod -* How the [environment variables](https://github.com/openshift/jenkins#environment-variables) recognized by the image are set - -See the [OpenShift Login plugin documentation](https://github.com/openshift/jenkins-openshift-login-plugin) for details on how it manages authentication. - -However, when the default authentication mechanism for Jenkins is used, if you are using the OpenShift Jenkins image, you log in with the user name `admin`, supplying the password specified by the `JENKINS_PASSWORD` environment variable set on the container. If you do not override `JENKINS_PASSWORD`, the default password for `admin` is `password`. - - -Test ---------------------------------- - -This repository also provides a test framework which checks basic functionality -of the Jenkins image. - -With v3, users can choose between testing Jenkins based on a RHEL (where you are running on a platform with subscriptions) or CentOS image. -With v4, there is not CentOS vs. RHEL distinction, but we still use TARGET to control whether subscriptions are required when building the test image, -and we reuse the v3 values (i.e. `rhel7`) for that purpose. - -* **RHEL based image** - - To test a RHEL7 based Jenkins image, you need to run the test on a properly - subscribed RHEL machine. - - ``` - $ cd jenkins - $ make test TARGET=rhel7 VERSION=2 - ``` - -* **CentOS based image** - - ``` - $ cd jenkins - $ make test VERSION=2 - ``` - -**Notice: By omitting the `VERSION` parameter, the build/test action will be performed -on all provided versions of Jenkins. Since we are currently providing only version `2`, -you can omit this parameter.** - -## PR testing for this repository - -As with the plugins focused on OpenShift integration, see [the contribution guide](CONTRIBUTING_TO_OPENSHIFT_JENKINS_IMAGE_AND_PLUGINS.md). +## Contributing +Please see [CONTRIBUTING.md](https://github.com/openshift/jenkins/blob/master/CONTRIBUTING.md). diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 000000000..e69de29bb diff --git a/TESTING.md b/TESTING.md new file mode 100644 index 000000000..52083b28f --- /dev/null +++ b/TESTING.md @@ -0,0 +1,34 @@ +Test +--------------------------------- + +This repository also provides a test framework which checks basic functionality +of the Jenkins image. + +With v3, users can choose between testing Jenkins based on a RHEL (where you are running on a platform with subscriptions) or CentOS image. +With v4, there is not CentOS vs. RHEL distinction, but we still use TARGET to control whether subscriptions are required when building the test image, +and we reuse the v3 values (i.e. `rhel7`) for that purpose. + +* **RHEL based image** + + To test a RHEL7 based Jenkins image, you need to run the test on a properly + subscribed RHEL machine. + + ``` + $ cd jenkins + $ make test TARGET=rhel8 VERSION=2 + ``` + +* **CentOS based image** + + ``` + $ cd jenkins + $ make test VERSION=2 + ``` + +**Notice: By omitting the `VERSION` parameter, the build/test action will be performed +on all provided versions of Jenkins. Since we are currently providing only version `2`, +you can omit this parameter.** + +## PR testing for this repository + +As with the plugins focused on OpenShift integration, see [the contribution guide](CONTRIBUTING_TO_OPENSHIFT_JENKINS_IMAGE_AND_PLUGINS.md).