Node.js & Express

Process Management & Clustering

18 min Lesson 28 of 40

Process Management & Clustering

Node.js runs on a single thread by default, which means it can only utilize one CPU core. To take full advantage of multi-core systems and ensure high availability, we need to understand process management and clustering.

Understanding the Single Thread Limitation

Node.js uses an event-driven, non-blocking I/O model that runs on a single thread. While this is efficient for I/O operations, CPU-intensive tasks can block the entire application:

const express = require('express');
const app = express();

app.get('/fast', (req, res) => {
  res.json({ message: 'This is fast!' });
});

app.get('/slow', (req, res) => {
  // CPU-intensive task blocks the entire server
  let result = 0;
  for (let i = 0; i < 5000000000; i++) {
    result += i;
  }
  res.json({ result });
});

app.listen(3000);

Warning: While the /slow endpoint is processing, ALL other requests (including /fast) will be blocked because Node.js runs on a single thread.

The Cluster Module

Node.js includes a built-in cluster module that allows you to create child processes (workers) that share the same server port:

const cluster = require('cluster');
const http = require('http');
const os = require('os');

const numCPUs = os.cpus().length;

if (cluster.isMaster) {
  console.log(`Master ${process.pid} is running`);

  // Fork workers for each CPU core
  for (let i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  cluster.on('exit', (worker, code, signal) => {
    console.log(`Worker ${worker.process.pid} died`);
    // Replace the dead worker
    cluster.fork();
  });
} else {
  // Workers share TCP connection
  const server = http.createServer((req, res) => {
    res.writeHead(200);
    res.end('Hello from worker ' + process.pid);
  });

  server.listen(3000);
  console.log(`Worker ${process.pid} started`);
}

Clustering with Express

Here's how to implement clustering with an Express application:

// server.js
const cluster = require('cluster');
const os = require('os');
const app = require('./app'); // Your Express app

const numCPUs = os.cpus().length;

if (cluster.isMaster) {
  console.log(`Master process ${process.pid} starting ${numCPUs} workers`);

  for (let i = 0; i < numCPUs; i++) {
    const worker = cluster.fork();
    console.log(`Worker ${worker.process.pid} spawned`);
  }

  cluster.on('online', (worker) => {
    console.log(`Worker ${worker.process.pid} is online`);
  });

  cluster.on('exit', (worker, code, signal) => {
    console.log(`Worker ${worker.process.pid} died with code ${code}`);
    
    if (!worker.exitedAfterDisconnect) {
      console.log('Worker crashed. Starting a new worker...');
      cluster.fork();
    }
  });

  // Graceful shutdown
  process.on('SIGTERM', () => {
    console.log('SIGTERM received. Shutting down gracefully...');
    
    for (const id in cluster.workers) {
      cluster.workers[id].disconnect();
    }
  });
} else {
  const PORT = process.env.PORT || 3000;
  
  app.listen(PORT, () => {
    console.log(`Worker ${process.pid} listening on port ${PORT}`);
  });
}

Note: The master process doesn't execute application code. It only manages worker processes. All application logic runs in worker processes.

Worker Communication

Workers can communicate with the master process and with each other:

if (cluster.isMaster) {
  const worker = cluster.fork();

  // Send message to worker
  worker.send({ cmd: 'notifyRequest' });

  // Receive message from worker
  worker.on('message', (msg) => {
    console.log('Master received:', msg);
  });
} else {
  // Worker receives message from master
  process.on('message', (msg) => {
    if (msg.cmd === 'notifyRequest') {
      console.log('Worker notified');
    }
  });

  // Worker sends message to master
  process.send({ status: 'ready' });
}

PM2 Process Manager

PM2 is a production-grade process manager for Node.js applications with built-in load balancing:

# Install PM2 globally
npm install -g pm2

# Start your app with clustering
pm2 start app.js -i max  # max = number of CPU cores

# Or specify number of instances
pm2 start app.js -i 4

# Start with a custom name
pm2 start app.js -i max --name "my-app"

PM2 Configuration File

Create an ecosystem.config.js file for advanced PM2 configuration:

module.exports = {
  apps: [{
    name: 'api-server',
    script: './server.js',
    instances: 'max',
    exec_mode: 'cluster',
    env: {
      NODE_ENV: 'development',
      PORT: 3000
    },
    env_production: {
      NODE_ENV: 'production',
      PORT: 8080
    },
    max_memory_restart: '500M',
    error_file: './logs/err.log',
    out_file: './logs/out.log',
    log_date_format: 'YYYY-MM-DD HH:mm:ss',
    autorestart: true,
    watch: false,
    max_restarts: 10,
    min_uptime: '10s'
  }]
};

# Use the config file
pm2 start ecosystem.config.js

# Start in production mode
pm2 start ecosystem.config.js --env production

PM2 Commands

# Monitor all processes
pm2 monit

# List all processes
pm2 list

# Show process details
pm2 show api-server

# View logs
pm2 logs
pm2 logs api-server

# Restart app
pm2 restart api-server

# Reload app (zero-downtime restart)
pm2 reload api-server

# Stop app
pm2 stop api-server

# Delete app from PM2
pm2 delete api-server

# Save process list
pm2 save

# Resurrect saved processes
pm2 resurrect

# Startup script (run on system boot)
pm2 startup

Tip: Use pm2 reload instead of pm2 restart for zero-downtime deployments in cluster mode.

Zero-Downtime Restarts

PM2's reload feature enables zero-downtime deployments:

// app.js - Graceful shutdown handling
const express = require('express');
const app = express();

app.get('/', (req, res) => {
  res.send('Hello World');
});

const server = app.listen(3000);

// Graceful shutdown
process.on('SIGINT', () => {
  console.log('SIGINT received. Closing server gracefully...');
  
  server.close(() => {
    console.log('Server closed. Exiting process.');
    process.exit(0);
  });

  // Force close after 10 seconds
  setTimeout(() => {
    console.error('Forcing shutdown after timeout');
    process.exit(1);
  }, 10000);
});

Load Balancing Strategy

PM2 uses a round-robin load balancing algorithm by default, but you can customize it:

module.exports = {
  apps: [{
    name: 'api',
    script: './server.js',
    instances: 4,
    exec_mode: 'cluster',
    instance_var: 'INSTANCE_ID',
    // Load balancing strategies
    listen_timeout: 3000,
    kill_timeout: 5000
  }]
};

Monitoring and Metrics

# Enable PM2 web monitoring
pm2 web

# Monitor CPU and Memory usage
pm2 monit

# Generate startup script for system boot
pm2 startup

# Save current process list
pm2 save

# Reset restart counter
pm2 reset api-server

# Show metrics
pm2 describe api-server

Handling Shared State

Since each worker is a separate process, they don't share memory. Use external storage for shared state:

const express = require('express');
const redis = require('redis');
const app = express();

// Use Redis for shared state
const client = redis.createClient();

app.get('/counter', async (req, res) => {
  // Increment counter in Redis (shared across all workers)
  const count = await client.incr('page:counter');
  res.json({
    count,
    worker: process.pid
  });
});

Warning: Never use in-memory variables for shared state in clustered applications. Use external stores like Redis, MongoDB, or PostgreSQL.

Best Practices

Number of Workers: Use os.cpus().length or slightly less to leave resources for the OS
Memory Limits: Set max_memory_restart in PM2 to prevent memory leaks
Graceful Shutdown: Always implement proper cleanup on SIGINT/SIGTERM signals
Health Checks: Implement /health endpoints for load balancers
Logging: Use centralized logging (e.g., Winston, Bunyan) for aggregating logs from all workers
Error Handling: Use cluster.on('exit') to restart crashed workers automatically

Practice Exercise:

Create a Node.js application with the following requirements:

Implement clustering to utilize all CPU cores
Add a /health endpoint that returns worker PID and uptime
Implement graceful shutdown handling
Create a PM2 ecosystem.config.js with production settings
Add a CPU-intensive endpoint (/calculate) and test load distribution across workers
Use Redis to track total request count across all workers

Summary

In this lesson, you learned:

Understanding Node.js single-thread limitations
Using the cluster module for multi-core utilization
Implementing PM2 for production process management
Zero-downtime deployments with PM2 reload
Worker communication and coordination
Load balancing strategies
Handling shared state in clustered environments
Best practices for process management