Setting Up Jenkins Pipeline on Ubuntu

Introduction

This comprehensive tutorial will guide you through the process of setting up a Jenkins pipeline on an Ubuntu system. You will learn how to install Jenkins, configure it for pipelines, create a pipeline project, define pipeline stages and steps, integrate with version control systems, and troubleshoot your pipeline. By the end of this tutorial, you will have a solid understanding of how to leverage Jenkins pipelines for your software development and deployment needs.

Skills Graph

Introduction to Jenkins and Pipelines

Jenkins is a popular open-source automation server that helps developers and teams streamline their software development and deployment processes. It provides a powerful platform for building, testing, and deploying applications, making it an essential tool in the DevOps ecosystem.

One of the key features of Jenkins is its support for pipelines, which are a series of automated steps that define the entire application delivery process. Pipelines allow developers to define, manage, and visualize their build, test, and deployment workflows as code, making them more maintainable, reproducible, and scalable.

Jenkins pipelines are defined using a domain-specific language (DSL) that is based on the Groovy programming language. This language allows developers to create complex, multi-stage pipelines that can handle a wide range of tasks, such as:

• Compiling and building the application
• Running unit, integration, and end-to-end tests
• Deploying the application to various environments (e.g., development, staging, production)
• Integrating with version control systems (e.g., Git, SVN)
• Notifying stakeholders of the pipeline's status

By using Jenkins pipelines, teams can achieve a high degree of automation, consistency, and traceability in their software delivery process, which can lead to faster release cycles, improved quality, and better collaboration between development and operations teams.

In the following sections, we'll explore how to set up and configure Jenkins on Ubuntu, create and manage Jenkins pipelines, and leverage advanced features to streamline your software delivery process.

Installing Jenkins on Ubuntu

Prerequisites

Before installing Jenkins on your Ubuntu 22.04 system, ensure that you have the following prerequisites:

• Java Development Kit (JDK) version 11 or later installed
• Wget or Curl installed for downloading the Jenkins package

Installing Jenkins

1. Update the package index:

sudo apt-get update

2. Install the necessary dependencies:

sudo apt-get install -y openjdk-11-jdk

3. Add the Jenkins repository to your system's sources list:

wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -
sudo sh -c 'echo deb https://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'

4. Install Jenkins:

sudo apt-get update
sudo apt-get install -y jenkins

5. Start the Jenkins service:

sudo systemctl start jenkins

6. Verify the installation by accessing the Jenkins web interface at http://your_server_ip:8080. You should see the Jenkins setup wizard.

Completing the Initial Setup

1. Unlock Jenkins by copying the initial administrator password from the file /var/lib/jenkins/secrets/initialAdminPassword.

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

2. Follow the on-screen instructions to complete the setup, including installing recommended plugins and creating your first admin user.

3. Once the setup is complete, you can access the Jenkins dashboard at http://your_server_ip:8080.

Congratulations! You have successfully installed Jenkins on your Ubuntu 22.04 system. In the next section, we'll explore how to configure Jenkins for pipelines.

Configuring Jenkins for Pipelines

Enabling the Pipeline Plugin

1. Log in to the Jenkins web interface.
2. Click on "Manage Jenkins" in the left-hand menu.
3. Click on "Manage Plugins".
4. Search for "Pipeline" and select the "Pipeline" and "Pipeline: Groovy" plugins.
5. Click "Install without restart" to install the plugins.

Configuring Global Pipeline Libraries

Jenkins supports the use of global pipeline libraries, which allow you to share common pipeline code across multiple projects. To configure global pipeline libraries:

1. Click on "Manage Jenkins" and then "Configure System".
2. Scroll down to the "Global Pipeline Libraries" section.
3. Click "Add" to create a new library.
4. Provide a name for the library and select the appropriate retrieval method (e.g., Git, SVN).
5. Configure the library's source code location and other settings as needed.
6. Click "Save" to apply the changes.

Configuring Credentials

Jenkins uses credentials to securely store sensitive information, such as version control system passwords, cloud service credentials, and more. To configure credentials:

1. Click on "Manage Jenkins" and then "Manage Credentials".
2. Click "Add Credentials" to create a new credential.
3. Select the appropriate credential type (e.g., Username with password, SSH Username with private key) and provide the necessary information.
4. Click "OK" to save the credential.

Configuring Nodes and Agents

Jenkins can run build and deployment tasks on remote systems, known as agents. To configure nodes and agents:

1. Click on "Manage Jenkins" and then "Manage Nodes and Clouds".
2. Click "New Node" to create a new node.
3. Provide a name for the node and select the "Permanent Agent" option.
4. Configure the node's details, such as the remote root directory, labels, and launch method.
5. Click "Save" to create the node.

By configuring Jenkins for pipelines, you've laid the groundwork for creating and managing your application's build, test, and deployment workflows. In the next section, we'll explore how to create a Jenkins pipeline project.

Creating a Jenkins Pipeline Project

Creating a New Pipeline Project

1. Log in to the Jenkins web interface.
2. Click on "New Item" in the left-hand menu.
3. Enter a name for your pipeline project and select "Pipeline" as the project type.
4. Click "OK" to create the new pipeline project.

Defining the Pipeline Script

Jenkins pipelines are defined using a Groovy-based domain-specific language (DSL). You can define your pipeline script in the web interface or store it in a file within your project's source code repository.

Here's an example pipeline script that demonstrates the basic structure:

pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                // Perform build-related tasks
                sh 'mvn clean install'
            }
        }

        stage('Test') {
            steps {
                // Run unit and integration tests
                sh 'mvn test'
            }
        }

        stage('Deploy') {
            steps {
                // Deploy the application to a target environment
                sh 'ansible-playbook deploy.yml'
            }
        }
    }

    post {
        always {
            // Perform cleanup or notification tasks
            junit 'target/surefire-reports/*.xml'
        }
    }
}

In this example, the pipeline consists of three stages: "Build", "Test", and "Deploy". Each stage contains one or more steps that perform specific tasks, such as running Maven commands or executing an Ansible playbook.

The post section defines actions to be performed after the pipeline completes, regardless of the outcome.

Saving and Running the Pipeline

1. Click "Save" to save the pipeline script.
2. Click "Build Now" to trigger the pipeline.

You can monitor the progress of the pipeline run in the Jenkins web interface, and view the output of each stage and step.

By creating a Jenkins pipeline project, you've set the foundation for automating your application's build, test, and deployment processes. In the next section, we'll explore how to define more complex pipeline stages and steps.

Defining Pipeline Stages and Steps

Pipeline Stages

Stages are the high-level sections of a Jenkins pipeline, representing the major steps in your application's build, test, and deployment process. Each stage can contain one or more steps that perform specific tasks.

Here are some common pipeline stages:

• Checkout: Retrieves the source code from a version control system.
• Build: Compiles the application and generates the necessary artifacts.
• Test: Runs unit, integration, and end-to-end tests to ensure the application's quality.
• Deploy: Deploys the application to a target environment, such as development, staging, or production.
• Notify: Sends notifications to stakeholders about the pipeline's status.

Pipeline Steps

Steps are the individual tasks that make up a stage. Jenkins provides a wide range of built-in steps, such as running shell commands, publishing test results, and deploying to cloud platforms.

Here are some examples of pipeline steps:

steps {
    // Checkout source code from a Git repository
    git 'https://github.com/example/my-app.git'

    // Run Maven commands to build the application
    sh 'mvn clean install'

    // Run unit tests using JUnit
    junit 'target/surefire-reports/*.xml'

    // Deploy the application to a Docker container
    docker.image('my-app:latest').deploy()

    // Send a notification to Slack
    slackSend channel: '#my-channel', message: 'Pipeline completed successfully'
}

Conditional Execution and Parallel Stages

Jenkins pipelines also support conditional execution and parallel stages, allowing you to create more complex and dynamic workflows.

stage('Deploy') {
    when {
        branch 'main'
    }
    steps {
        // Deploy to production
        sh 'ansible-playbook prod-deploy.yml'
    }
}

stage('Parallel Tests') {
    parallel {
        stage('Unit Tests') {
            steps {
                // Run unit tests
                sh 'mvn test'
            }
        }
        stage('Integration Tests') {
            steps {
                // Run integration tests
                sh 'mvn verify -Pintegration'
            }
        }
    }
}

In this example, the "Deploy" stage is only executed when the pipeline is running on the "main" branch. The "Parallel Tests" stage runs unit and integration tests concurrently, improving the overall pipeline execution time.

By mastering the definition of pipeline stages and steps, you can create highly customized and efficient build, test, and deployment workflows for your applications.

Integrating Jenkins with Version Control Systems

Jenkins can seamlessly integrate with various version control systems (VCS), allowing you to automatically trigger pipeline builds when code changes are detected. This integration ensures that your pipeline is always up-to-date with the latest codebase and helps maintain the traceability of your software delivery process.

Integrating with Git

Jenkins provides built-in support for integrating with Git, the most popular distributed version control system. To configure Jenkins to work with a Git repository:

1. In the Jenkins web interface, go to "Manage Jenkins" > "Configure System".
2. Scroll down to the "Git" section and configure the path to the Git executable on your system.
3. In your pipeline project, add a "Checkout" stage that retrieves the source code from your Git repository:

stage('Checkout') {
    steps {
        git 'https://github.com/example/my-app.git'
    }
}

4. Optionally, you can configure Jenkins to automatically trigger a pipeline build when a new commit is pushed to the repository. This can be done by adding a "GitHub webhook" or a "GitLab webhook" in the pipeline project's configuration.

Integrating with Subversion (SVN)

Jenkins also supports integration with Subversion (SVN), another popular version control system. The configuration process is similar to the Git integration:

1. In the Jenkins web interface, go to "Manage Jenkins" > "Configure System".
2. Scroll down to the "Subversion" section and configure the path to the SVN executable on your system.
3. In your pipeline project, add a "Checkout" stage that retrieves the source code from your SVN repository:

stage('Checkout') {
    steps {
        checkout([$class: 'SubversionSCM', locations: [[credentialsId: 'svn-credentials', remote: 'https://svn.example.com/my-app']]])
    }
}

4. As with Git, you can configure Jenkins to automatically trigger a pipeline build when changes are detected in the SVN repository.

By integrating Jenkins with your version control system, you can ensure that your pipeline is always up-to-date with the latest codebase and maintain a clear history of your software delivery process.

Summary

In this "Setting Up Jenkins Pipeline on Ubuntu" tutorial, you have learned how to install and configure Jenkins on an Ubuntu system, create a Jenkins pipeline project, define pipeline stages and steps, integrate with version control systems, and monitor and troubleshoot your Jenkins pipelines. With this knowledge, you can now effectively leverage Jenkins pipelines to streamline your software delivery process and improve your overall CI/CD workflow.