Testing & TDD

Code Coverage Analysis

30 min Lesson 16 of 35

Understanding Code Coverage

Code coverage is a metric that measures the percentage of your codebase that is executed when your automated tests run. It helps you identify untested parts of your code and assess the thoroughness of your test suite. While 100% coverage doesn't guarantee bug-free code, it provides valuable insights into your testing efforts.

Why Code Coverage Matters

Code coverage serves several important purposes in software development:

Identifies Untested Code: Highlights areas of your codebase that lack test coverage
Measures Test Completeness: Provides quantitative data about test suite thoroughness
Guides Testing Efforts: Helps prioritize which areas need more testing
Prevents Regressions: Ensures critical paths are tested before deployment
Improves Code Quality: Encourages writing testable, modular code

Important: High code coverage is a good indicator but not a guarantee of quality. You can have 100% coverage with poor tests that don't actually validate behavior. Focus on meaningful assertions, not just line execution.

Types of Code Coverage

There are several types of coverage metrics, each measuring different aspects of code execution:

1. Line Coverage (Statement Coverage)

Measures the percentage of code lines executed during tests:

function calculateDiscount(price, quantity) {
    let discount = 0;          // Line 1 - covered
    if (quantity > 10) {        // Line 2 - covered
        discount = 0.1;         // Line 3 - NOT covered if quantity <= 10
    }
    return price * discount;    // Line 4 - covered
}

// Test only covers lines 1, 2, 4 (75% line coverage)
test('no discount for small orders', () => {
    expect(calculateDiscount(100, 5)).toBe(0);
});

2. Branch Coverage (Decision Coverage)

Measures whether each branch of conditional statements is executed:

function validateAge(age) {
    if (age > 18) {         // Branch point
        return 'adult';     // True branch
    } else {
        return 'minor';     // False branch
    }
}

// Need tests for both branches (100% branch coverage)
test('adult age', () => {
    expect(validateAge(25)).toBe('adult');
});

test('minor age', () => {
    expect(validateAge(15)).toBe('minor');
});

3. Function Coverage

Measures the percentage of defined functions that are called:

class Calculator {
    add(a, b) { return a + b; }      // Called in tests
    subtract(a, b) { return a - b; } // Called in tests
    multiply(a, b) { return a * b; } // NOT called
    divide(a, b) { return a / b; }   // NOT called
}

// Only 50% function coverage (2 out of 4 functions tested)

4. Condition Coverage

Measures whether each boolean sub-expression has been evaluated to both true and false:

function isEligible(age, hasLicense) {
    if (age >= 18 && hasLicense) {  // Two conditions
        return true;
    }
    return false;
}

// Full condition coverage requires testing:
// age >= 18 (true/false) AND hasLicense (true/false)
test('eligible with both conditions true', () => {
    expect(isEligible(20, true)).toBe(true);
});

test('not eligible - young with license', () => {
    expect(isEligible(16, true)).toBe(false);
});

test('not eligible - adult without license', () => {
    expect(isEligible(25, false)).toBe(false);
});

JavaScript Code Coverage with Istanbul/nyc

Istanbul (now called nyc) is the most popular code coverage tool for JavaScript. It integrates seamlessly with Jest, Mocha, and other testing frameworks:

Setting Up nyc with Mocha

# Install nyc
npm install --save-dev nyc

# package.json
{
  "scripts": {
    "test": "mocha",
    "test:coverage": "nyc mocha"
  }
}

# Run tests with coverage
npm run test:coverage

nyc Configuration (.nycrc.json)

{
  "all": true,
  "include": ["src/**/*.js"],
  "exclude": [
    "**/*.spec.js",
    "**/*.test.js",
    "**/node_modules/**",
    "**/tests/**"
  ],
  "reporter": ["text", "html", "lcov"],
  "check-coverage": true,
  "lines": 80,
  "functions": 80,
  "branches": 80,
  "statements": 80
}

Configuration Tips:

Use "all": true to include untested files in coverage report
Exclude test files, configuration files, and third-party code
Set realistic coverage thresholds that increase over time
Use multiple reporters: text for terminal, html for detailed viewing

Jest Built-in Coverage

Jest has Istanbul built-in, making coverage analysis extremely simple:

# Run Jest with coverage
npm test -- --coverage

# Or add to package.json
{
  "scripts": {
    "test": "jest",
    "test:coverage": "jest --coverage --watchAll=false"
  }
}

# jest.config.js
module.exports = {
  collectCoverageFrom: [
    'src/**/*.{js,jsx,ts,tsx}',
    '!src/**/*.d.ts',
    '!src/index.js',
    '!src/serviceWorker.js'
  ],
  coverageThreshold: {
    global: {
      branches: 75,
      functions: 75,
      lines: 75,
      statements: 75
    }
  }
};

PHP Code Coverage with PHPUnit

PHPUnit provides comprehensive code coverage analysis using Xdebug or phpdbg:

Prerequisites

# Install Xdebug (option 1)
pecl install xdebug

# Or use phpdbg (option 2, included with PHP)
phpdbg -qrr vendor/bin/phpunit --coverage-html coverage

# Verify Xdebug is enabled
php -v | grep Xdebug

PHPUnit Configuration (phpunit.xml)

<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
         bootstrap="vendor/autoload.php"
         colors="true">
    <testsuites>
        <testsuite name="Unit">
            <directory suffix="Test.php">./tests/Unit</directory>
        </testsuite>
    </testsuites>

    <coverage processUncoveredFiles="true">
        <include>
            <directory suffix=".php">./app</directory>
        </include>
        <exclude>
            <directory suffix=".php">./app/Console</directory>
            <file>./app/Http/Kernel.php</file>
        </exclude>
        <report>
            <html outputDirectory="./coverage/html"/>
            <clover outputFile="./coverage/clover.xml"/>
            <text outputFile="php://stdout" showUncoveredFiles="true"/>
        </report>
    </coverage>
</phpunit>

Running PHPUnit with Coverage

# Generate HTML coverage report
./vendor/bin/phpunit --coverage-html coverage

# Generate text coverage report
./vendor/bin/phpunit --coverage-text

# Generate multiple formats
./vendor/bin/phpunit --coverage-html coverage --coverage-clover coverage.xml

# Laravel specific
php artisan test --coverage
php artisan test --coverage --min=80

Understanding Coverage Reports

Coverage reports provide detailed insights into your test coverage. Here's how to interpret them:

Terminal Text Report

Code Coverage Report:
  2024-02-14 10:30:00

 Summary:
  Classes: 85.71% (6/7)
  Methods: 78.26% (18/23)
  Lines:   82.35% (140/170)

 App\Services\PaymentService
  Methods: 100.00% (5/5)
  Lines:   95.00% (19/20)

 App\Controllers\OrderController
  Methods: 66.67% (4/6)
  Lines:   75.00% (45/60)

HTML Coverage Report

HTML reports provide interactive, color-coded views of your code:

Green lines: Covered by tests
Red lines: Not covered by tests
Orange lines: Partially covered (some branches)
Gray lines: Non-executable (comments, declarations)

Best Practice: Keep HTML coverage reports out of version control. Add coverage/ to your .gitignore file. These reports are generated artifacts that should be created locally or in CI/CD pipelines.

Setting Coverage Thresholds

Coverage thresholds automatically fail builds if coverage drops below specified levels:

Jest Thresholds

// jest.config.js
module.exports = {
  coverageThreshold: {
    global: {
      branches: 80,
      functions: 85,
      lines: 85,
      statements: 85
    },
    './src/services/': {
      branches: 90,
      functions: 90,
      lines: 90,
      statements: 90
    },
    './src/utils/formatting.js': {
      branches: 100,
      functions: 100,
      lines: 100,
      statements: 100
    }
  }
};

PHPUnit Coverage Check

<!-- phpunit.xml -->
<coverage processUncoveredFiles="true">
    <include>
        <directory suffix=".php">./app</directory>
    </include>
    <report>
        <html outputDirectory="./coverage"/>
    </report>
</coverage>

<!-- Require minimum coverage (PHPUnit 9.3+) -->
<source>
    <include>
        <directory suffix=".php">./app</directory>
    </include>
</source>

# Command-line coverage check
phpunit --coverage-text --coverage-filter app/Services

# Fail build if coverage below threshold (CI/CD)
phpunit --coverage-text | grep -E "^\s+Lines:\s+([0-9]+\.[0-9]+)%" | \
  awk '{if ($2 < 80.00) exit 1}'

Coverage in CI/CD Pipelines

Integrate coverage checks into your continuous integration workflow:

GitHub Actions Example

name: Tests with Coverage

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '18'

      - name: Install dependencies
        run: npm ci

      - name: Run tests with coverage
        run: npm run test:coverage

      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v3
        with:
          files: ./coverage/lcov.info
          flags: unittests
          fail_ci_if_error: true

Best Practices for Code Coverage

1. Don't Chase 100% Coverage Blindly

Focus on covering critical business logic, not boilerplate. Some code (like simple getters/setters, configuration files) doesn't need tests.

2. Use Coverage to Find Gaps, Not as a Goal

Coverage is a tool to identify untested code, not a metric to game. Write tests for behavior, not coverage numbers.

3. Combine with Mutation Testing

Tools like Stryker (JavaScript) and Infection (PHP) verify that your tests actually catch bugs, not just execute lines.

4. Set Realistic, Incremental Thresholds

Start with current coverage and gradually increase thresholds. Don't set 90% coverage on a project with 40% coverage.

5. Exclude the Right Files

Don't include test files, migrations, configuration, or generated code in coverage calculations.

Common Pitfalls:

Coverage Theater: Writing tests that execute code without meaningful assertions
Ignoring Branch Coverage: Focusing only on line coverage misses untested conditional paths
Testing Implementation Details: High coverage on internal functions that could change
Neglecting Edge Cases: Covering happy paths but missing error handling
False Confidence: Assuming 100% coverage means bug-free code

Practical Exercise

Exercise 1: Analyze coverage for a shopping cart module

// shopping-cart.js
class ShoppingCart {
    constructor() {
        this.items = [];
    }

    addItem(product, quantity = 1) {
        if (quantity <= 0) {
            throw new Error('Quantity must be positive');
        }

        const existingItem = this.items.find(item => item.product.id === product.id);

        if (existingItem) {
            existingItem.quantity += quantity;
        } else {
            this.items.push({ product, quantity });
        }
    }

    removeItem(productId) {
        const index = this.items.findIndex(item => item.product.id === productId);
        if (index !== -1) {
            this.items.splice(index, 1);
        }
    }

    getTotal() {
        return this.items.reduce((sum, item) => {
            return sum + (item.product.price * item.quantity);
        }, 0);
    }

    applyDiscount(percentage) {
        if (percentage < 0 || percentage > 100) {
            throw new Error('Invalid discount percentage');
        }

        const total = this.getTotal();
        return total * (1 - percentage / 100);
    }
}

// Write tests to achieve:
// - 100% line coverage
// - 100% branch coverage
// - 100% function coverage
// Run: jest --coverage

Exercise 2: Improve coverage for a PHP validator class

// app/Services/EmailValidator.php
class EmailValidator
{
    public function validate(string $email): bool
    {
        if (empty($email)) {
            return false;
        }

        if (!filter_var($email, FILTER_VALIDATE_EMAIL)) {
            return false;
        }

        [$local, $domain] = explode('@', $email);

        if (strlen($local) > 64) {
            return false;
        }

        if (!checkdnsrr($domain, 'MX')) {
            return false;
        }

        return true;
    }
}

// Run: phpunit --coverage-html coverage
// Identify untested branches
// Write tests to reach 100% coverage

Exercise 3: Set up coverage thresholds for a project

Run coverage analysis on your current project
Identify current coverage percentages
Set thresholds 5-10% higher than current coverage
Configure CI/CD to fail on threshold violations
Write tests to meet new thresholds