Security & Performance

Logging and Security Monitoring

16 min Lesson 14 of 35

Understanding Security Logging and Monitoring

You cannot protect what you cannot see. Security logging and monitoring enable you to detect attacks, investigate incidents, prove compliance, and respond to threats in real-time. Without comprehensive logging, you're flying blind.

Detection vs. Prevention: Average time to detect a breach: 207 days. Comprehensive logging and monitoring can reduce this to hours or even minutes, minimizing damage.

What to Log for Security

Log security-relevant events that help detect, investigate, and respond to security incidents:

// Node.js security event logging with Winston
const winston = require('winston');
const path = require('path');

// Create security logger
const securityLogger = winston.createLogger({
level: 'info',
format: winston.format.combine(
winston.format.timestamp(),
winston.format.json()
),
defaultMeta: { service: 'security-service' },
transports: [
// Separate file for security events
new winston.transports.File({
filename: 'logs/security.log',
maxsize: 10485760, // 10MB
maxFiles: 10
}),
// Critical alerts to separate file
new winston.transports.File({
filename: 'logs/security-alerts.log',
level: 'error',
maxsize: 10485760,
maxFiles: 5
})
]
});

// Log authentication events
function logAuthEvent(eventType, userId, ip, success, details = {}) {
securityLogger.info({
event: 'authentication',
type: eventType, // login, logout, password_reset, etc.
userId: userId,
ip: ip,
success: success,
userAgent: details.userAgent,
timestamp: new Date().toISOString(),
...details
});
}

// Log authorization failures
function logAuthzFailure(userId, resource, action, ip) {
securityLogger.warn({
event: 'authorization_failure',
userId: userId,
resource: resource,
action: action,
ip: ip,
timestamp: new Date().toISOString()
});
}

// Log suspicious activity
function logSuspiciousActivity(type, details) {
securityLogger.error({
event: 'suspicious_activity',
type: type, // brute_force, sql_injection, xss, etc.
details: details,
timestamp: new Date().toISOString()
});
}
Privacy Warning: Never log sensitive data: passwords, credit cards, SSNs, full session tokens. Log only what's necessary for security investigation. Comply with GDPR, CCPA data retention rules.

Security Events to Monitor

// Comprehensive security event monitoring

// 1. Authentication Events
- Successful logins (with timestamp, IP, user agent)
- Failed login attempts (track failed attempts per IP/user)
- Account lockouts
- Password changes
- Password reset requests
- MFA events (success/failure)

// 2. Authorization Events
- Access denied (403) events
- Privilege escalation attempts
- Role/permission changes
- Access to sensitive resources

// 3. Data Access Events
- Bulk data exports
- Access to PII/sensitive data
- Database query patterns (unusual queries)
- File access (especially sensitive files)

// 4. Configuration Changes
- User creation/deletion
- Role/permission modifications
- System configuration changes
- Security settings changes

// 5. Application Errors
- Unexpected exceptions
- Input validation failures
- API rate limit violations
- CSRF token failures

// 6. Network Events
- Unusual traffic patterns
- Geographic anomalies
- Multiple IPs per user
- Port scanning attempts

Structured Logging for Security Analysis

// Use structured logging for easier analysis
const { createLogger, format, transports } = require('winston');
const { combine, timestamp, json, errors } = format;

const logger = createLogger({
format: combine(
errors({ stack: true }),
timestamp({ format: 'YYYY-MM-DD HH:mm:ss' }),
json()
),
defaultMeta: {
service: 'my-app',
environment: process.env.NODE_ENV
},
transports: [
new transports.File({ filename: 'logs/app.log' })
]
});

// Example: Log failed login with context
logger.warn({
event_type: 'authentication_failure',
event_category: 'security',
user_id: req.body.email,
ip_address: req.ip,
user_agent: req.headers['user-agent'],
request_id: req.id,
failure_reason: 'invalid_password',
attempt_number: 3,
geo_location: await getGeoLocation(req.ip),
timestamp: new Date().toISOString()
});
Best Practice: Use structured logging (JSON) instead of plain text. Structured logs are easier to parse, search, and analyze with automated tools like ELK Stack, Splunk, or Datadog.

Log Correlation and Context

// Add correlation IDs to track requests across services
const { v4: uuidv4 } = require('uuid');

// Middleware to add request ID
app.use((req, res, next) => {
req.id = uuidv4();
res.setHeader('X-Request-ID', req.id);
next();
});

// Include request ID in all logs
logger.info({
request_id: req.id,
event: 'user_login',
user_id: user.id,
ip: req.ip
});

// Later, in another service/microservice
logger.info({
request_id: req.headers['x-request-id'],
event: 'data_access',
resource: 'user_profile',
user_id: user.id
});

// Now you can trace the entire request flow:
// Login → Profile Access → Data Export

Real-Time Security Alerting

// Alert on suspicious patterns
const nodemailer = require('nodemailer');
const redis = require('redis');
const client = redis.createClient();

// Track failed login attempts
async function trackFailedLogin(email, ip) {
const key = `failed_login:${email}:${ip}`;
const attempts = await client.incr(key);
await client.expire(key, 3600); // 1 hour window

// Alert after 5 failed attempts
if (attempts === 5) {
await sendSecurityAlert({
type: 'brute_force_attempt',
email: email,
ip: ip,
attempts: attempts,
severity: 'high'
});
}

// Block after 10 attempts
if (attempts >= 10) {
await blockIP(ip, 3600);
await sendSecurityAlert({
type: 'ip_blocked',
ip: ip,
reason: 'brute_force',
severity: 'critical'
});
}
}

// Send security alerts
async function sendSecurityAlert(alertData) {
const transporter = nodemailer.createTransporter({
host: process.env.SMTP_HOST,
port: 587,
secure: false,
auth: {
user: process.env.SMTP_USER,
pass: process.env.SMTP_PASS
}
});

await transporter.sendMail({
from: 'security@example.com',
to: 'security-team@example.com',
subject: `[${alertData.severity.toUpperCase()}] Security Alert: ${alertData.type}`,
text: JSON.stringify(alertData, null, 2)
});

// Also log to security logger
securityLogger.error({
event: 'security_alert',
...alertData
});
}

Anomaly Detection

// Detect unusual user behavior
class AnomalyDetector {
async detectLoginAnomaly(userId, ip, userAgent) {
// Get user's typical login patterns
const recentLogins = await getUserRecentLogins(userId, 30);

const anomalies = [];

// Check for geographic anomaly
const currentGeo = await getGeoLocation(ip);
const usualCountries = recentLogins.map(l => l.country);
if (!usualCountries.includes(currentGeo.country)) {
anomalies.push({
type: 'geographic_anomaly',
message: `Login from unusual country: ${currentGeo.country}`
});
}

// Check for impossible travel
const lastLogin = recentLogins[0];
if (lastLogin) {
const timeDiff = Date.now() - lastLogin.timestamp;
const distance = calculateDistance(lastLogin.geo, currentGeo);
const speedKmH = distance / (timeDiff / 3600000);

if (speedKmH > 900) { // Faster than airplane
anomalies.push({
type: 'impossible_travel',
message: `Login from ${distance}km away in ${timeDiff/60000} minutes`
});
}
}

// Check for unusual time
const hour = new Date().getHours();
const usualHours = recentLogins.map(l => new Date(l.timestamp).getHours());
const avgHour = usualHours.reduce((a, b) => a + b, 0) / usualHours.length;
if (Math.abs(hour - avgHour) > 6) {
anomalies.push({
type: 'unusual_time',
message: `Login at unusual hour: ${hour}:00`
});
}

return anomalies;
}
}
Machine Learning: Advanced anomaly detection uses ML models trained on normal behavior. Libraries like TensorFlow.js or cloud services (AWS GuardDuty, Azure Sentinel) can detect subtle patterns humans miss.

Audit Trail Best Practices

// Implement tamper-proof audit trail
const crypto = require('crypto');

class AuditLog {
constructor() {
this.logs = [];
this.lastHash = '0';
}

addEntry(event, userId, details) {
const entry = {
timestamp: Date.now(),
event: event,
userId: userId,
details: details,
previousHash: this.lastHash
};

// Create hash of this entry
const entryString = JSON.stringify(entry);
entry.hash = crypto
.createHash('sha256')
.update(entryString)
.digest('hex');

this.logs.push(entry);
this.lastHash = entry.hash;

// Persist to immutable storage
this.persistEntry(entry);
}

verifyIntegrity() {
let previousHash = '0';

for (let entry of this.logs) {
if (entry.previousHash !== previousHash) {
return { valid: false, tamperedEntry: entry };
}

const entryWithoutHash = { ...entry };
delete entryWithoutHash.hash;
const computedHash = crypto
.createHash('sha256')
.update(JSON.stringify(entryWithoutHash))
.digest('hex');

if (computedHash !== entry.hash) {
return { valid: false, tamperedEntry: entry };
}

previousHash = entry.hash;
}

return { valid: true };
}
}

Log Retention and Management

# Log rotation with logrotate (Linux)
sudo nano /etc/logrotate.d/myapp

# Add configuration:
/var/log/myapp/*.log {
daily # Rotate daily
rotate 90 # Keep 90 days
compress # Compress old logs
delaycompress # Compress after 1 day
notifempty # Don't rotate if empty
create 0640 www-data www-data
sharedscripts
postrotate
systemctl reload myapp
endscript
}

# Security logs - longer retention
/var/log/myapp/security.log {
weekly
rotate 52 # Keep 1 year
compress
notifempty
create 0600 root root # Stricter permissions
}
Exercise: Implement comprehensive security logging:
  1. Create a security logger with Winston or similar
  2. Log all authentication and authorization events
  3. Implement request correlation with unique IDs
  4. Create real-time alerting for brute force attempts
  5. Build anomaly detection for geographic/time patterns
  6. Implement tamper-proof audit trail with hashing
  7. Configure log rotation with 90-day retention