WebSockets & Real-Time Apps

Scaling WebSocket Applications

18 min Lesson 21 of 35

Introduction to Scaling WebSockets

As your real-time application grows, a single server won't handle all concurrent connections. This lesson explores how to scale WebSocket applications horizontally across multiple servers while maintaining real-time functionality.

Scaling Challenges

WebSocket applications face unique scaling challenges:

Stateful Connections: Each client maintains a persistent connection to a specific server
Message Broadcasting: Events need to reach clients connected to different servers
Session Affinity: Clients must reconnect to the same server instance
Shared State: Application state must be synchronized across instances

Key Concept: Unlike stateless HTTP applications, WebSocket servers maintain active connections, making horizontal scaling more complex.

Horizontal Scaling Architecture

A typical scaled WebSocket architecture includes:

┌─────────┐
│ Clients │
└────┬────┘
     │
┌────▼──────────┐
│ Load Balancer │ (Sticky Sessions)
└────┬──────────┘
     │
┌────▼────┬──────────┬──────────┐
│ Server1 │ Server2  │ Server3  │
└────┬────┴─────┬────┴─────┬────┘
     │          │          │
     └──────┬───┴──────┬───┘
            │          │
       ┌────▼──────────▼────┐
       │ Redis Pub/Sub       │ (Message Bus)
       └─────────────────────┘

Sticky Sessions (Session Affinity)

Sticky sessions ensure clients reconnect to the same server:

// Nginx configuration for sticky sessions
upstream websocket_servers {
    ip_hash;  # Route based on client IP
    server 192.168.1.10:3000;
    server 192.168.1.11:3000;
    server 192.168.1.12:3000;
}

server {
    location /socket.io/ {
        proxy_pass http://websocket_servers;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
    }
}

Important: IP-based sticky sessions may not work perfectly with clients behind NAT or using mobile networks with changing IPs.

Redis Adapter for Socket.io

The Socket.io Redis adapter enables message broadcasting across multiple servers:

// Install dependencies
// npm install @socket.io/redis-adapter redis

const { Server } = require('socket.io');
const { createAdapter } = require('@socket.io/redis-adapter');
const { createClient } = require('redis');

const io = new Server(3000);

// Create Redis clients (pub and sub)
const pubClient = createClient({
    host: 'localhost',
    port: 6379
});
const subClient = pubClient.duplicate();

// Connect Redis clients
Promise.all([pubClient.connect(), subClient.connect()]).then(() => {
    // Attach Redis adapter to Socket.io
    io.adapter(createAdapter(pubClient, subClient));
    console.log('Redis adapter connected');
});

// This broadcast will reach ALL servers
io.emit('message', 'Hello from Server 1');

How Redis Adapter Works

The adapter uses Redis Pub/Sub to synchronize messages:

Server 1 receives a message and calls io.emit()
The adapter publishes the message to Redis
All server instances (including Server 1) subscribe to Redis channels
Each server receives the message and broadcasts to its local clients
All clients across all servers receive the message

Best Practice: Use Redis for message synchronization and a separate database (like PostgreSQL or MongoDB) for persistent data storage.

Complete Multi-Server Example

// server.js - Run multiple instances on different ports
const express = require('express');
const { Server } = require('socket.io');
const { createAdapter } = require('@socket.io/redis-adapter');
const { createClient } = require('redis');

const app = express();
const PORT = process.env.PORT || 3000;
const server = app.listen(PORT);

const io = new Server(server, {
    cors: {
        origin: '*'
    }
});

// Redis setup
const pubClient = createClient({ url: 'redis://localhost:6379' });
const subClient = pubClient.duplicate();

Promise.all([pubClient.connect(), subClient.connect()]).then(() => {
    io.adapter(createAdapter(pubClient, subClient));
    console.log(`Server running on port ${PORT} with Redis adapter`);
});

// Track connected users across all servers
const connectedUsers = new Map();

io.on('connection', (socket) => {
    console.log(`Client connected to server on port ${PORT}`);

    socket.on('register', (username) => {
        socket.username = username;
        connectedUsers.set(socket.id, username);

        // Broadcast to ALL servers
        io.emit('user_joined', {
            username,
            serverPort: PORT,
            totalUsers: io.engine.clientsCount
        });
    });

    socket.on('message', (data) => {
        // This reaches clients on ALL servers
        io.emit('message', {
            username: socket.username,
            message: data.message,
            serverPort: PORT,
            timestamp: new Date()
        });
    });

    socket.on('disconnect', () => {
        const username = connectedUsers.get(socket.id);
        connectedUsers.delete(socket.id);

        io.emit('user_left', {
            username,
            serverPort: PORT
        });
    });
});

Running Multiple Server Instances

# Terminal 1
PORT=3000 node server.js

# Terminal 2
PORT=3001 node server.js

# Terminal 3
PORT=3002 node server.js

Shared State Management

For shared state across servers, use Redis for quick access:

const redis = require('redis');
const stateClient = redis.createClient({ url: 'redis://localhost:6379' });

await stateClient.connect();

// Store user online status
io.on('connection', async (socket) => {
    const userId = socket.handshake.auth.userId;

    // Set user as online in Redis
    await stateClient.hSet('users:online', userId, JSON.stringify({
        socketId: socket.id,
        serverPort: PORT,
        connectedAt: new Date()
    }));

    // Get all online users
    const onlineUsers = await stateClient.hGetAll('users:online');
    socket.emit('online_users', Object.values(onlineUsers).map(JSON.parse));

    socket.on('disconnect', async () => {
        await stateClient.hDel('users:online', userId);
    });
});

Load Balancing Considerations

Load Balancer Requirements:

Must support WebSocket protocol (HTTP/1.1 Upgrade)
Should implement sticky sessions (IP hash or cookie-based)
Health checks for WebSocket endpoints
Proper timeout configuration (WebSockets are long-lived)

PM2 Cluster Mode

Use PM2 to manage multiple Node.js instances:

// ecosystem.config.js
module.exports = {
    apps: [{
        name: 'websocket-app',
        script: './server.js',
        instances: 4,  // 4 instances
        exec_mode: 'cluster',
        env: {
            NODE_ENV: 'production'
        }
    }]
};

// Start with PM2
// pm2 start ecosystem.config.js

Caution: PM2 cluster mode alone doesn't handle WebSocket scaling. You still need Redis adapter for cross-instance communication.

Monitoring Scaled Applications

// Add monitoring to each server instance
const os = require('os');

setInterval(() => {
    const metrics = {
        serverPort: PORT,
        connectedClients: io.engine.clientsCount,
        memoryUsage: process.memoryUsage(),
        cpuUsage: os.loadavg(),
        uptime: process.uptime()
    };

    // Send to monitoring service or log
    console.log('Server Metrics:', metrics);

    // Optionally publish to Redis for centralized monitoring
    pubClient.publish('server:metrics', JSON.stringify(metrics));
}, 30000); // Every 30 seconds

Exercise:

Set up Redis on your local machine
Create a Socket.io server with Redis adapter
Run 3 instances of the server on different ports (3000, 3001, 3002)
Create a simple chat client that connects to any instance
Verify that messages sent to one server reach clients on other servers
Monitor which server each client connects to

Summary

Scaling WebSocket applications requires:

Sticky sessions to maintain client-server affinity
Redis adapter for cross-server message broadcasting
Shared state management using Redis or similar
Proper load balancer configuration
Process managers like PM2 for instance management
Monitoring and health checks