Overview
I want to demo a CI/CD pipeline within OpenShift and Jenkins. I have two projects: “dev” and “prod”. Some SW sources reside in github. The Jenkinsfile, which defines the pipeline is stored in github as well.
This demo focuses on the Jenkinspipeline, disabling any (auto)magic, which OpenShift might bring. Each build and deployment step is defined with Jenkins.
Note 1:
This blog adresses mostly infrastructure people, who ave no clue from Jenkins or app development. So there is much room for more sophisticated solutions.
Note 2:
There are smarter (aka more automated) ways to do this. E.g. let a new commit in the git-repo trigger a built in dev or let a new image, which is labeled “ready for prod” be deployed in prod through OpenShift triggers. But i wanted to learn how to utilise Jenkins.
Doing
Prerequisites
I’m running OpenShift Version 3.9. I’ve added two users, named alice and bob, as well as the user admin. The later has full cluster-admin rights, while alice and bob should only be able to administer the projects dev and prod.
Creating my own SW-repository
I’m utilising some existing application [1]. But as i need to test how changes arrive in my installation i decided to fork:
1) login to github.com (with my own account: mschreie) 2) browse to https://github.com/wkulhanek/openshift-tasks 3) and fork this to your own account 4) i now have the repo: https://github.com/mschreie/openshift-tasks
To make some changes:
This can be cloned to a local clone, edited there, committed and pushed back
Or:
This can be edited in the web UI of github and committed there.
Both ways need to assure that a change will end up within our repository https://github.com/mschreie/openshift-tasks. (in my case)
Installing Jenkins
(1) oc login -u system:admin (2) oc new-project jenkins (3) oc adm policy add-role-to-user edit alice (4) oc adm policy add-role-to-user edit bob (5) oc new-app jenkins-persistent --param ENABLE_OAUTH=true --param MEMORY_LIMIT=2Gi --param VOLUME_CAPACITY=4G (6) oc adm pod-network make-projects-global jenkins
First i make sure i’m logged in as admin (1). I create a new project, in which jenkins should reside (2). I enabled alice and bob to edit the jenkins project (3) and (4).
Line (5) does instal jenkins. It claims a persistent volume (via persistent volume claim). As i do not have auto provisioning in place, i prepare a couple of persistent volumes before hand.
As i want jenkins to be able to test deployed services by connecting to the service, i need to define the network of the jenkins project as global (6). This enables all pods from within this project to connect to all pods in any other project. It also enables all pods from other projects to connect to the pods within the jenkins network. The later is not needed, but setting up network policy management seems to be an overkill for now.
Preparing dev project
(1) oc new-project dev (2) oc policy add-role-to-user edit system:serviceaccount:jenkins:jenkins -n dev (3) oc adm policy add-role-to-user edit alice (4) oc adm policy add-role-to-user edit bob (5) oc new-app jboss-eap70-openshift:1.6~https://github.com/mschreie/openshift-tasks (6) oc expose svc openshift-tasks (7) oc set triggers dc openshift-tasks --manual
I created the project “dev” as i mentioned before (1) and added alice and bob to be able to edit the project (3), (4).
I also need to allow jenkins (out of the jenkins project) to built and deploy within the dev project (2).
Creating the new application is done in line (5). To be able to connect from outside of OpenShift i added a route (6).
As i do not want anything deployed or built automatically but only through jenkins, i set all triggers ion the deployment config to manual (7).
Preparing prod project
This is a two step approach. I first create the application within the project the same way as i did with dev. I alter some things to
a) be sure the image gets pulled from dev
b) get rid of everthing which is needed for building (as building is done in dev).
I export this into an template which can then be used to directly create all objects the way they are needed.
First step:
(1) oc new-project prod
(2) oc policy add-role-to-user edit system:serviceaccount:jenkins:jenkins -n prod
(3) oc adm policy add-role-to-user edit alice
(4) oc adm policy add-role-to-user edit bob
(5) oc new-app jboss-eap70-openshift:1.6~https://github.com/mschreie/openshift-tasks
(6) oc expose svc openshift-tasks
(7) oc set triggers dc openshift-tasks --manual
(8) oc edit dc openshift-tasks -n prod
I create the project and the application as in dev (line (1) to (7). I then need to change the deployment config to get the image from the dev project (8). Within triggers->imageChangeParams->from i change the namespace (where the image is found) to dev (see line 8 of the extract):
... (1) triggers: (2) - imageChangeParams: (3) containerNames: (4) - openshift-tasks (5) from: (6) kind: ImageStreamTag (7) name: openshift-tasks:latest (8) namespace: dev (9) lastTriggeredImage: docker-registry.default.svc:5000/dev/openshift-tasks@sha256:80776f8976481d712a878e322426964064dc96857176a542e82c60b769f7d10a (10) type: ImageChange ...
I also get rid of all artifacts which are not needed in prod project:
(9) oc delete is/openshift-tasks -n prod (10) oc delete bc/openshift-tasks -n prod (11) oc delete builds/openshift-tasks-1 -n prod
Now the prod project is fine. I also need to allow prod that we can pull images from dev
oc policy add-role-to-user system:image-puller system:serviceaccount:prod:default -n dev
We could stop here and move on to the Jenkins pipeline. But as these are to many manual tasks i prefer to define a template wich would role out prod easier…
Second Step:
I create the template (1) and delete (2) the prod project with all objects inside thereafter:
(1) oc export dc,route,svc --as-template=openshift-tasks-prod > openshift-tasks-prod.xml (2) oc delete project prod
I wait till the project is realy gone (and not in terminating state)
(1) oc get projects
And then i create the project prod once again:
(1) oc new-project prod (2) oc policy add-role-to-user edit system:serviceaccount:jenkins:jenkins -n prod (3) oc adm policy add-role-to-user edit alice (4) oc adm policy add-role-to-user edit bob (5) oc process -f openshift-tasks-prod.xml | oc create -f - (6) oc set triggers dc openshift-tasks --manual
I do not need to expose the service as the route is part of the template. I’m not sure whether i need to switch the triggers to manual once again (6).
The Prod project is still allowed to pull from dev, as this definition is part of the dev- project.
Note: The template is very strictly tied to this setup. There are no parameters or variables which make this template also work in other environments. But as this template is only to be used here., this is fine to me.
Creating a Jenkins pipeline:
creating the Jenkinsfile
I created the following Jenkinsfile and uploaded it into a git repository:
cat Jenkinsfile ( 1) pipeline { ( 2) agent any ( 3) stages { ( 4) stage('Dev: Build Tasks') { ( 5) steps { ( 6) openshiftBuild bldCfg: 'openshift-tasks', namespace: 'dev', showBuildLogs: 'true', verbose: 'false', waitTime: '', waitUnit: 'sec' ( 7) openshiftVerifyBuild bldCfg: 'openshift-tasks', checkForTriggeredDeployments: 'false', namespace: 'dev', verbose: 'false', waitTime: '' ( 8) } ( 9) } (10) stage('Dev: Tag Image') { (11) steps { (12) openshiftTag alias: 'false', destStream: 'openshift-tasks', destTag: '${BUILD_NUMBER}', destinationAuthToken: '', destinationNamespace: 'dev', namespace: 'dev', srcStream: 'openshift-tasks', srcTag: 'latest', verbose: 'false' (13) } (14) } (15) stage('Dev: Deploy new image') { (16) steps { (17) openshiftDeploy depCfg: 'openshift-tasks', namespace: 'dev', verbose: 'false', waitTime: '', waitUnit: 'sec' (18) openshiftVerifyDeployment depCfg: 'openshift-tasks', namespace: 'dev', replicaCount: '1', verbose: 'false', verifyReplicaCount: 'true', waitTime: '', waitUnit: 'sec' (19) openshiftVerifyService namespace: 'dev', svcName: 'openshift-tasks', verbose: 'false', retryCount: '5' (20) } (21) } (22) stage('acknowledge prod') { (23) steps { (24) timeout(time: 3, unit: 'MINUTES') { (25) input 'Ready for Prod?' (26) } (27) } (28) } (29) stage('Prod: Tag Image') { (30) steps { (31) openshiftTag alias: 'false', destStream: 'openshift-tasks', destTag: '${BUILD_NUMBER}', destinationAuthToken: '', destinationNamespace: 'prod', namespace: 'dev', srcStream: 'openshift-tasks', srcTag: 'latest', verbose: 'false' (32) } (33) } (34) stage('Prod: Deploy new image') { (35) steps { (36) openshiftDeploy depCfg: 'openshift-tasks', namespace: 'prod', verbose: 'false', waitTime: '', waitUnit: 'sec' (37) openshiftVerifyDeployment depCfg: 'openshift-tasks', namespace: 'prod', replicaCount: '1', verbose: 'false', verifyReplicaCount: 'true', waitTime: '', waitUnit: 'sec' (38) openshiftVerifyService namespace: 'prod', svcName: 'openshift-tasks', verbose: 'false', retryCount: '5' (39) } (40) } (41) } (42)}
This file makes use of the OpenShift Pipeline Jenkins Plugin. You can see that most steps made, are verified thereafter (e.g line 6: openshiftBuild and line 7: openshiftVerifyBuild).
The stages “Dev: Build Tasks” (line 4), “Dev: Tag Image” (line 10), and “Dev: Deploy new image” (line 15) work on the dev project as defined by namespace: ‘dev’. The openshiftVerifyService i line 19 only works because we made the network of the jenkins project a global network. Otherwise pods from one project would not be able to reach a service provided by a different project.
Stage “acknowledge prod” stating line 22 needs a manual interaction. If “proceed” is pressed the pipeline continues and rolls out into prod. If “abort” is pressed, this deployment will not be prompted into prod.
As you can see in prod we only trigger a new deployment (stage starting line 35). Build has been done in dev.
create the jenkins pipe
I do not know how to do these steps by cmd-line, so i
browse to my jenkins url and login as alice or bob: https://jenkins-jenkins.apps.<openshift_master_default_subdomain>/ In the navigator on the left, click New Item. I type some meaningful Name like "openshift-tasks-application" for the item name, select Pipeline for the job type, and click OK.
In the next form i switch underneath the header “pipeline” i switch to
Definition; "Pipeline script from SCM" SCM: "Git"
Repository URL points to the git-repository in which i uploaded the Jenkinsfile. This could be the same repository in which i forked the SW itself (e.g. https://github.com/mschreie/openshift-tasks). If the Jenkinsfile resides in the main directory of this repository, you define:
Script Path: "Jenkinsfile"
and leave all other parameters as they are. Press “save”.
Testing
Browse to the openshift-task application of
prod (http://openshift-tasks-dev.<openshift_master_default_subdomain>) as well as of
dev (http://openshift-tasks-dev.<openshift_master_default_subdomain>) and memorise the main page.
In the repository of the SW change the file
src/main/webapp/index.jsp
by changing the line
OpenShift Tasks Demo (Wolfgang's Baseline Version 1.0)
to something else. Assure that this is committed and pushed into the github repository
I then navigate to the Jenkins UI and click “Build Now” within my pipeline. The stages Declarative: Checkout SCM
Dev: Build Tasks
Dev: Tag Image
Dev: Deploy new image
should run through.
The stage “acknowledge prod” should wait for your acknowledgement or time out after three minutes.
Before pressing continue, please verify that the Website of dev changed, while the website of prod did not appear (or did not change) jet. When pressing continue, the web site of prod should also show your change.
Conclusion
There are many ways to automate a build pipeline within OpenShift. The outlined approach utilises Jenkins, which is a powerful tool to for CI/CD, but stays very simple (not using different methods / triggers for different stages), and is explained so that non-dev people, like me, understand.
Markus Schreier 