DevOps Culture & Fundamentals

Project: Map a Delivery Pipeline

25 min Lesson 10 of 28

Project: Map a Delivery Pipeline

The first nine lessons gave you the vocabulary, the frameworks, and the mental models. This capstone project makes them concrete. You will analyze a realistic sample organization — a mid-size e-commerce company called ShopFast — and produce a fully annotated delivery pipeline map: the same artifact a senior DevOps engineer produces during an onboarding week or a DevEx audit.

A pipeline map is not a pretty slide. It is a living document that answers: where does value flow, where does it slow down, and where do failures hide? After this project you will be able to do the same analysis for any organization you join.

This is an active project lesson. Read the scenario, study the diagrams, then complete the mapping exercise at the end. The goal is not a single right answer — it is the habit of systematic analysis that separates senior engineers from juniors.

The ShopFast Scenario

ShopFast runs a monolithic Rails API, a React SPA, and five microservices (inventory, payments, notifications, search, recommendations). The team has 40 engineers in three squads. Their current pipeline, reconstructed from interviews, looks like this:

Developers push to feature branches and open PRs. Reviews take 1–3 days.
Merges to main trigger a Jenkins pipeline. Build takes 22 minutes.
A manual QA sign-off is required before staging promotion. QA owns one shared environment. Queue wait is typically 4 hours.
Staging runs for 48 hours minimum ("soak time", enforced by policy).
A release committee meets every Tuesday to approve production deployments.
Production deployments happen every two weeks on Friday afternoons. Rollbacks are manual and take 45 minutes.
Monitoring is Datadog. Alerts go to a shared Slack channel. On-call rotation exists but runbooks are out of date.

Before drawing a single box, a senior engineer computes the DORA snapshot from the data above. Deployment frequency: roughly 2/month. Lead time for changes: 2 weeks + 4-hour QA queue + 3-day review = ~17 days in the worst case. These numbers alone tell you the pipeline has a structural problem — they sit in DORA's Low band.

Step 1 — Identify the Stages and Actors

Every delivery pipeline has the same five macro-stages regardless of tooling. Map ShopFast onto them first, then overlay their actual tools:

ShopFast current-state pipeline map — stages, actors, and lead-time breakdown. Yellow = bottleneck; red = calendar-driven waste.

Step 2 — Label Every Transition

A raw stage diagram is not enough. A professional pipeline map labels every transition between stages: what triggers it, what gate must pass, and how long it typically waits. Use a simple table alongside the diagram:

# ShopFast pipeline transition inventory
# Format: FROM → TO | Trigger | Gate | Avg Wait

feature branch → main        | PR merged    | 2 approvals (no automation)  | ~2 days
main → Jenkins build         | git push     | automatic (webhook)          | <1 min (queue)
Jenkins → QA environment     | build passes | manual QA team action        | ~4 hours
QA environment → staging     | QA sign-off  | QA lead approves ticket      | same-day
staging → release committee  | 48h elapsed  | Tuesday meeting slot         | up to 6 days
release committee → prod     | vote passes  | ops manual deploy script     | same Friday

Immediately visible: five of the six transitions have a human gate. Only the webhook to Jenkins is automatic. This is the signature of a bureaucratic pipeline — high ceremony, low throughput. In CALMS terms (Lesson 2), it scores poorly on Lean and Sharing.

Step 3 — Apply Value-Stream Thinking

Value-stream mapping, borrowed from lean manufacturing, splits every activity into value-added time (the computer is actually working) versus wait time (humans, queues, policies). For ShopFast:

Value-added time: ~22 minutes (Jenkins build) + ~2 hours (actual QA testing) + ~5 minutes (deploy script). Total: ~2.5 hours.
Wait time: ~17 days worst case. The process cycle efficiency (value-added / total lead time) is roughly 0.7% — a textbook low-performing organization.

Elite DORA performers target process cycle efficiency above 10%. Big-tech teams routinely exceed 50%. The gap is almost never in the CI build time; it is always in the human handoff queues.

When mapping any pipeline for the first time, ask: "What would happen if we removed every human gate except security review?" The answer forces the team to articulate whether each gate is genuinely risk-reducing or merely organizational habit. You will find that most gates were installed to fix a specific past incident and were never revisited.

Step 4 — Identify Failure Injection Points

A complete pipeline map also marks where failures are most likely to escape to production. ShopFast's map has three high-risk zones:

No automated integration tests — Jenkins runs unit tests only. Integration bugs surface in QA, days after the code was written. The cost to fix a bug found in QA is 5-10x the cost to fix it at the PR stage (shift-left principle, Lesson 1).
Single shared QA environment — when two squads need QA simultaneously, one waits. This "environment contention" is a hidden bottleneck that does not show up in the nominal pipeline diagram.
45-minute manual rollback — if a production incident requires rollback, 45 minutes of MTTR is committed before anything starts. The DORA MTTR goal for elite performers is under 1 hour total; ShopFast spends 45 minutes just on the rollback mechanics.

Step 5 — Draw the Future-State Map

Every pipeline audit produces two artifacts: the current-state map (above) and a future-state map with concrete improvement proposals. For ShopFast, a realistic six-month target:

ShopFast future-state pipeline — each bottleneck replaced with an automated gate; DORA targets shown at the bottom.

Step 6 — Write the Pipeline-as-Code Definition

The future-state map is only aspirational until it exists as code. A senior DevOps engineer writes the skeleton pipeline definition immediately after the mapping session, so the team has something concrete to iterate on:

# .github/workflows/delivery.yml — ShopFast future-state skeleton
name: Delivery Pipeline

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  lint-and-unit:
    runs-on: ubuntu-24.04
    steps:
      - uses: actions/checkout@v4
      - name: Run linter
        run: bundle exec rubocop --parallel
      - name: Run unit tests
        run: bundle exec rspec spec/unit --format progress

  integration:
    needs: lint-and-unit
    runs-on: ubuntu-24.04
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: test
    steps:
      - uses: actions/checkout@v4
      - name: Run integration tests
        run: bundle exec rspec spec/integration

  build-image:
    needs: integration
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-24.04
    steps:
      - uses: actions/checkout@v4
      - name: Build and push Docker image
        uses: docker/build-push-action@v5
        with:
          push: true
          tags: ghcr.io/shopfast/api:${{ github.sha }}

  deploy-staging:
    needs: build-image
    runs-on: ubuntu-24.04
    environment: staging
    steps:
      - name: Deploy to staging via Helm
        run: |
          helm upgrade --install shopfast-api ./charts/api \
            --set image.tag=${{ github.sha }} \
            --namespace staging --wait --timeout 5m

  smoke-test-staging:
    needs: deploy-staging
    runs-on: ubuntu-24.04
    steps:
      - name: Run smoke tests against staging
        run: npx playwright test --grep @smoke --base-url https://staging.shopfast.io

  deploy-prod-canary:
    needs: smoke-test-staging
    runs-on: ubuntu-24.04
    environment: production        # requires human approval in GitHub Environments
    steps:
      - name: Canary deploy 5%
        run: |
          helm upgrade --install shopfast-api ./charts/api \
            --set image.tag=${{ github.sha }} \
            --set canary.weight=5 \
            --namespace production --wait

Notice the environment: production gate above. This is the one human approval gate that belongs in a modern pipeline — a lightweight, auditable sign-off recorded in GitHub, not a committee meeting. It satisfies change-management audit requirements without adding days of wait time. Never remove all human gates from a production deployment path for regulated industries — remove the unnecessary ones and keep the accountable ones.

The Mapping Deliverable: What You Submit

A professional pipeline map deliverable has five sections. Use this as a template for any audit:

Current-state diagram — stages, actors, tools, and lead times at each transition.
Value-stream analysis — total lead time, value-added time, process cycle efficiency.
Bottleneck inventory — ranked list of constraints (apply Theory of Constraints: fix the biggest bottleneck first).
Failure escape analysis — where can a bug survive to production? At what stage is it cheapest to catch it?
Future-state proposal — target DORA metrics, automation changes, and a pipeline-as-code skeleton.

You have now completed the full foundations stage of this course. Tutorials 2–50 build on everything covered here: CALMS, DORA, Twelve-Factor, the toolchain landscape, and — most importantly — the analytical habit this project demonstrates. When in doubt, come back to the pipeline map. It is the most important diagnostic tool in a DevOps engineer's toolkit.