Jenkins & Enterprise CI/CD

Scripted Pipelines & Groovy

18 min Lesson 4 of 28

Scripted Pipelines & Groovy

Jenkins has two pipeline syntaxes. You met Declarative in Lesson 3 — a structured, opinionated DSL that fits 80-90 % of CI/CD work. Scripted Pipeline is the other syntax: raw Groovy running inside the Jenkins CPS (Continuation-Passing Style) sandbox, giving you a full Turing-complete language at the cost of less guardrails. At Google-scale and at every major bank running Jenkins, you will encounter both — sometimes in the same file.

What Makes Scripted Different

A Scripted Pipeline is wrapped in a node block, not a pipeline block. Everything inside is Groovy code that runs on the agent you name. There is no enforced stages structure, no automatic post DSL key — you write try/catch/finally yourself.

// Minimal Scripted Pipeline
node('linux && docker') {
    try {
        stage('Checkout') {
            checkout scm
        }

        stage('Build') {
            sh 'make build'
        }

        stage('Test') {
            sh 'make test'
            junit 'build/reports/**/*.xml'
        }

        stage('Push') {
            withCredentials([usernamePassword(
                credentialsId: 'dockerhub-creds',
                usernameVariable: 'DOCKER_USER',
                passwordVariable: 'DOCKER_PASS'
            )]) {
                sh '''
                    echo "$DOCKER_PASS" | docker login -u "$DOCKER_USER" --password-stdin
                    docker push myorg/myapp:${GIT_COMMIT[0..7]}
                '''
            }
        }
    } catch (err) {
        currentBuild.result = 'FAILURE'
        throw err
    } finally {
        cleanWs()
        if (currentBuild.result == 'FAILURE') {
            slackSend channel: '#alerts', message: "Build FAILED: ${env.BUILD_URL}"
        }
    }
}

Key observations: node('linux && docker') selects an agent by label expression. stage() is a function call, not a key. Error handling is plain Groovy — you must set currentBuild.result explicitly, then re-throw so Jenkins marks the run as failed.

When You Actually Need Scripted

The answer is: more rarely than you think, but when you need it you really need it. Here are the legitimate production use cases:

Dynamic stage generation — you need to create stages from a list that is only known at runtime (e.g., one stage per microservice in a monorepo). Declarative stages is static; Scripted lets you for (svc in services) { stage("Deploy ${svc}") { … } }.
Complex conditional branching — when the when DSL in Declarative is too limiting. E.g., multi-level environment + tag + source branch logic with shared-library helper calls in conditions.
Parallel with dynamic fan-out — building a Map of parallel branches at runtime and passing it to parallel(branches).
Legacy migration — inherited pipelines from before Declarative existed (Jenkins < 2.5 era) that are too risky to rewrite wholesale.

Declarative is the default choice at every major company. The Jenkins documentation and the Google/Netflix Jenkins patterns all recommend Declarative unless you hit a concrete wall. Scripted gives power; Declarative gives safety and readability for the 10 engineers who will maintain the file after you.

Dynamic Parallel Stages — The Real Power

The canonical reason to reach for Scripted is runtime-computed parallelism. Suppose you have a microservices repo and a helper that returns which services changed in the current commit. You cannot express this in Declarative.

// Dynamic parallel fan-out — Scripted Pipeline
@Library('myorg-shared') _

def changedServices = []

node('controller') {
    stage('Detect Changes') {
        checkout scm
        changedServices = detectChangedServices()   // shared-library helper
    }
}

// Build a map of parallel branches
def branches = [:]
for (String svc in changedServices) {
    def s = svc   // capture loop variable — classic Groovy closure bug!
    branches["Build ${s}"] = {
        node('linux && docker') {
            checkout scm
            dir("services/${s}") {
                sh "docker build -t myorg/${s}:${env.GIT_COMMIT[0..7]} ."
                sh "docker push myorg/${s}:${env.GIT_COMMIT[0..7]}"
            }
        }
    }
}

stage('Build Services') {
    parallel branches
}

Groovy closure variable capture bug: in a for loop, the loop variable svc is shared across all closures. By the time the closure runs, the loop has finished and svc holds only the last value. Always copy it with def s = svc inside the loop. This is the single most common scripted-pipeline production bug.

Mixing Scripted Inside Declarative

You do not have to choose one syntax for the entire file. Jenkins lets you embed a script { } block anywhere inside a Declarative steps block — that block is raw Groovy Scripted code. This is the best of both worlds pattern used in most mature Jenkins installations.

pipeline {
    agent any
    stages {
        stage('Build Matrix') {
            steps {
                script {
                    // Full Groovy available here
                    def envs = ['dev', 'staging', 'prod']
                    def deployBranches = [:]

                    envs.each { e ->
                        def env = e
                        deployBranches["Deploy to ${env}"] = {
                            sh "helm upgrade --install myapp-${env} chart/ \
                                --namespace ${env} \
                                --set image.tag=${GIT_COMMIT[0..7]}"
                        }
                    }

                    if (env.BRANCH_NAME == 'main') {
                        parallel deployBranches
                    } else {
                        echo "Skipping multi-env deploy on feature branch"
                    }
                }
            }
        }
    }
    post {
        failure {
            slackSend channel: '#deploys', message: "Deploy failed: ${env.BUILD_URL}"
        }
    }
}

The script { } block gives you full Groovy. Outside it, you are in safe Declarative DSL with automatic post handling, options, parameters, and environment blocks. This hybrid is what Netflix and Airbnb pipeline templates look like in practice.

Decision tree: Declarative → Hybrid → Full Scripted, in order of preference.

CPS Sandbox — What You Cannot Do

Jenkins runs Scripted Pipelines in a CPS-transformed sandbox that serializes the build state to disk so it can survive a controller restart. This imposes real constraints that bite production teams:

No non-serializable objects on the stack — you cannot store an open FileInputStream or a Groovy Closure in a field that crosses a node boundary. The CPS transformer will fail to serialize them.
No @NonCPS shortcut for complex logic — methods annotated @NonCPS bypass CPS transformation and run as regular JVM code, but they cannot call CPS-transformed methods (e.g., sh, echo). Split your code: pure logic in @NonCPS helpers, Jenkins steps in regular CPS methods.
Groovy standard library restrictions — many classes are blocked by the script security sandbox. You will hit org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException. Either approve the signature in Manage Jenkins → In-process Script Approval or move the logic to a Shared Library (Lesson 6) which runs with broader trust.

Always put CPU-heavy or non-serializable Groovy logic into @NonCPS helper methods and keep the pipeline's top-level methods thin (just Jenkins step calls). This produces faster, safer, more maintainable pipelines and avoids the majority of CPS serialization errors.

Groovy Patterns Worth Knowing

You do not need to be a Groovy expert, but these patterns appear in every real Scripted Pipeline:

readFile('path') / writeFile file: 'path', text: content — read build artifacts into Groovy strings for conditional logic.
String GStrings: "Deploy to ${env.BRANCH_NAME}" — standard Groovy interpolation, but use single quotes inside sh steps when you want shell variable expansion, not Groovy expansion.
def result = sh(script: 'git rev-parse --short HEAD', returnStdout: true).trim() — capture command output into a Groovy variable.
error('message') — explicit pipeline failure; sets result and throws, cleaner than throwing raw exceptions.

The single vs double quote rule is one of the most-cited Jenkins production gotchas. Inside sh '…' single-quoted strings the dollar sign goes to the shell. Inside sh "…" double-quoted strings, Groovy interpolates first — ${MY_VAR} is expanded by Groovy before the shell sees the command, which means a missing Groovy variable silently injects nothing rather than the shell variable you expected.

Scripted Pipelines are a powerful tool in the Jenkins arsenal. Use them deliberately: default to Declarative, reach for script { } blocks when you need Groovy logic inside one stage, and escalate to full Scripted only when you need runtime-computed pipeline graphs. The next lesson covers Agents and Distributed Builds, which applies to both syntaxes.