Security & Performance
Server Hardening
Understanding Server Hardening
Server hardening is the process of securing a server by reducing its attack surface. This involves removing unnecessary software, closing unused ports, configuring firewalls, and implementing security best practices.
Attack Surface: The sum of all possible entry points an attacker could use to gain unauthorized access to your system. Smaller attack surface = more secure system.
Operating System Security Basics
Start with a minimal, secure OS installation:
# Update system packages (Ubuntu/Debian)
sudo apt update
sudo apt upgrade -y
sudo apt dist-upgrade -y
sudo apt autoremove -y
# Enable automatic security updates
sudo apt install unattended-upgrades
sudo dpkg-reconfigure --priority=low unattended-upgrades
# Configure automatic updates
sudo nano /etc/apt/apt.conf.d/50unattended-upgrades
# Uncomment:
# "${distro_id}:${distro_codename}-security";
# Check for pending updates
/usr/lib/update-notifier/apt-check --human-readable
sudo apt update
sudo apt upgrade -y
sudo apt dist-upgrade -y
sudo apt autoremove -y
# Enable automatic security updates
sudo apt install unattended-upgrades
sudo dpkg-reconfigure --priority=low unattended-upgrades
# Configure automatic updates
sudo nano /etc/apt/apt.conf.d/50unattended-upgrades
# Uncomment:
# "${distro_id}:${distro_codename}-security";
# Check for pending updates
/usr/lib/update-notifier/apt-check --human-readable
Best Practice: Always start with a minimal OS installation. Don't install GUI components on production servers. Less installed software = fewer vulnerabilities.
Firewall Configuration (UFW)
Uncomplicated Firewall (UFW) provides an easy-to-use interface for iptables:
# Install and enable UFW
sudo apt install ufw
sudo ufw status
# Default policies: deny incoming, allow outgoing
sudo ufw default deny incoming
sudo ufw default allow outgoing
# Allow specific services
sudo ufw allow ssh # Port 22
sudo ufw allow 80/tcp # HTTP
sudo ufw allow 443/tcp # HTTPS
sudo ufw allow from 192.168.1.0/24 to any port 3306 # MySQL from private network
# Allow specific IP addresses
sudo ufw allow from 203.0.113.4
sudo ufw allow from 203.0.113.0/24
# Deny specific ports
sudo ufw deny 23 # Telnet
# Enable firewall
sudo ufw enable
# View rules with numbers
sudo ufw status numbered
# Delete rule by number
sudo ufw delete 3
# Reset firewall
sudo ufw reset
sudo apt install ufw
sudo ufw status
# Default policies: deny incoming, allow outgoing
sudo ufw default deny incoming
sudo ufw default allow outgoing
# Allow specific services
sudo ufw allow ssh # Port 22
sudo ufw allow 80/tcp # HTTP
sudo ufw allow 443/tcp # HTTPS
sudo ufw allow from 192.168.1.0/24 to any port 3306 # MySQL from private network
# Allow specific IP addresses
sudo ufw allow from 203.0.113.4
sudo ufw allow from 203.0.113.0/24
# Deny specific ports
sudo ufw deny 23 # Telnet
# Enable firewall
sudo ufw enable
# View rules with numbers
sudo ufw status numbered
# Delete rule by number
sudo ufw delete 3
# Reset firewall
sudo ufw reset
SSH Hardening
SSH is a common attack vector. Proper configuration is critical:
# Edit SSH configuration
sudo nano /etc/ssh/sshd_config
# Recommended settings:
Port 2222 # Change default port
PermitRootLogin no # Disable root login
PasswordAuthentication no # Use keys only
PubkeyAuthentication yes # Enable key authentication
PermitEmptyPasswords no # No empty passwords
X11Forwarding no # Disable X11
MaxAuthTries 3 # Limit login attempts
ClientAliveInterval 300 # Timeout idle sessions
ClientAliveCountMax 2
AllowUsers developer deployer # Whitelist users
Protocol 2 # SSH protocol 2 only
HostbasedAuthentication no
IgnoreRhosts yes
# Restart SSH service
sudo systemctl restart sshd
# Generate SSH key pair (on client)
ssh-keygen -t ed25519 -C "your_email@example.com"
# Copy public key to server
ssh-copy-id -i ~/.ssh/id_ed25519.pub user@server_ip
# Set correct permissions
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
sudo nano /etc/ssh/sshd_config
# Recommended settings:
Port 2222 # Change default port
PermitRootLogin no # Disable root login
PasswordAuthentication no # Use keys only
PubkeyAuthentication yes # Enable key authentication
PermitEmptyPasswords no # No empty passwords
X11Forwarding no # Disable X11
MaxAuthTries 3 # Limit login attempts
ClientAliveInterval 300 # Timeout idle sessions
ClientAliveCountMax 2
AllowUsers developer deployer # Whitelist users
Protocol 2 # SSH protocol 2 only
HostbasedAuthentication no
IgnoreRhosts yes
# Restart SSH service
sudo systemctl restart sshd
# Generate SSH key pair (on client)
ssh-keygen -t ed25519 -C "your_email@example.com"
# Copy public key to server
ssh-copy-id -i ~/.ssh/id_ed25519.pub user@server_ip
# Set correct permissions
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
Warning: Before disabling password authentication, ensure you can login with SSH keys. Always keep an active SSH session open when testing SSH configuration changes.
Fail2Ban - Intrusion Prevention
Fail2Ban monitors logs and bans IPs showing malicious behavior:
# Install Fail2Ban
sudo apt install fail2ban
# Create local configuration
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
sudo nano /etc/fail2ban/jail.local
# Basic configuration:
[DEFAULT]
bantime = 3600 # Ban for 1 hour
findtime = 600 # 10 minute window
maxretry = 5 # 5 failed attempts
destemail = admin@example.com
sendername = Fail2Ban
action = %(action_mwl)s # Ban and email with logs
[sshd]
enabled = true
port = 2222
logpath = /var/log/auth.log
maxretry = 3
[nginx-http-auth]
enabled = true
filter = nginx-http-auth
port = http,https
logpath = /var/log/nginx/error.log
# Start Fail2Ban
sudo systemctl start fail2ban
sudo systemctl enable fail2ban
# Check status
sudo fail2ban-client status
sudo fail2ban-client status sshd
# Unban an IP
sudo fail2ban-client set sshd unbanip 192.168.1.100
sudo apt install fail2ban
# Create local configuration
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
sudo nano /etc/fail2ban/jail.local
# Basic configuration:
[DEFAULT]
bantime = 3600 # Ban for 1 hour
findtime = 600 # 10 minute window
maxretry = 5 # 5 failed attempts
destemail = admin@example.com
sendername = Fail2Ban
action = %(action_mwl)s # Ban and email with logs
[sshd]
enabled = true
port = 2222
logpath = /var/log/auth.log
maxretry = 3
[nginx-http-auth]
enabled = true
filter = nginx-http-auth
port = http,https
logpath = /var/log/nginx/error.log
# Start Fail2Ban
sudo systemctl start fail2ban
sudo systemctl enable fail2ban
# Check status
sudo fail2ban-client status
sudo fail2ban-client status sshd
# Unban an IP
sudo fail2ban-client set sshd unbanip 192.168.1.100
Minimal Service Installation
Remove unnecessary services to reduce attack surface:
# List all running services
sudo systemctl list-units --type=service --state=running
# Disable unnecessary services
sudo systemctl stop cups # Printing service
sudo systemctl disable cups
sudo systemctl stop avahi-daemon # Service discovery
sudo systemctl disable avahi-daemon
# Remove unnecessary packages
sudo apt remove --purge package_name
sudo apt autoremove
# Check for listening ports
sudo netstat -tulpn
# or
sudo ss -tulpn
# Identify process on specific port
sudo lsof -i :80
sudo systemctl list-units --type=service --state=running
# Disable unnecessary services
sudo systemctl stop cups # Printing service
sudo systemctl disable cups
sudo systemctl stop avahi-daemon # Service discovery
sudo systemctl disable avahi-daemon
# Remove unnecessary packages
sudo apt remove --purge package_name
sudo apt autoremove
# Check for listening ports
sudo netstat -tulpn
# or
sudo ss -tulpn
# Identify process on specific port
sudo lsof -i :80
Principle of Least Privilege: Only install and run services that are absolutely necessary for your application. Every additional service is a potential vulnerability.
User and Permission Management
# Create service user without login
sudo useradd -r -s /bin/false nginx_user
# Add user to specific group
sudo usermod -aG www-data deploy_user
# Set password policies
sudo nano /etc/login.defs
# Set:
PASS_MAX_DAYS 90
PASS_MIN_DAYS 1
PASS_MIN_LEN 12
PASS_WARN_AGE 7
# Install password quality checking
sudo apt install libpam-pwquality
sudo nano /etc/security/pwquality.conf
# Set:
minlen = 12
dcredit = -1 # At least one digit
ucredit = -1 # At least one uppercase
lcredit = -1 # At least one lowercase
ocredit = -1 # At least one special char
# Audit user accounts
sudo lastlog
sudo last
sudo who
sudo w
sudo useradd -r -s /bin/false nginx_user
# Add user to specific group
sudo usermod -aG www-data deploy_user
# Set password policies
sudo nano /etc/login.defs
# Set:
PASS_MAX_DAYS 90
PASS_MIN_DAYS 1
PASS_MIN_LEN 12
PASS_WARN_AGE 7
# Install password quality checking
sudo apt install libpam-pwquality
sudo nano /etc/security/pwquality.conf
# Set:
minlen = 12
dcredit = -1 # At least one digit
ucredit = -1 # At least one uppercase
lcredit = -1 # At least one lowercase
ocredit = -1 # At least one special char
# Audit user accounts
sudo lastlog
sudo last
sudo who
sudo w
File System Security
# Set secure file permissions
sudo chmod 644 /var/www/html/*.html
sudo chmod 755 /var/www/html
sudo chown -R www-data:www-data /var/www/html
# Find files with SUID bit (potential security risk)
sudo find / -perm -4000 -type f 2>/dev/null
# Find world-writable files
sudo find / -xdev -type f -perm -0002 -ls 2>/dev/null
# Find files with no owner
sudo find / -xdev -nouser -o -nogroup 2>/dev/null
# Mount filesystems with security options
sudo nano /etc/fstab
# Add options:
/dev/sda1 /var noexec,nosuid 0 2
/tmp /tmp tmpfs defaults,noexec,nosuid,nodev 0 0
sudo chmod 644 /var/www/html/*.html
sudo chmod 755 /var/www/html
sudo chown -R www-data:www-data /var/www/html
# Find files with SUID bit (potential security risk)
sudo find / -perm -4000 -type f 2>/dev/null
# Find world-writable files
sudo find / -xdev -type f -perm -0002 -ls 2>/dev/null
# Find files with no owner
sudo find / -xdev -nouser -o -nogroup 2>/dev/null
# Mount filesystems with security options
sudo nano /etc/fstab
# Add options:
/dev/sda1 /var noexec,nosuid 0 2
/tmp /tmp tmpfs defaults,noexec,nosuid,nodev 0 0
Security Monitoring and Auditing
# Install audit daemon
sudo apt install auditd audispd-plugins
# Enable audit rules for sensitive files
sudo auditctl -w /etc/passwd -p wa -k passwd_changes
sudo auditctl -w /etc/shadow -p wa -k shadow_changes
sudo auditctl -w /etc/ssh/sshd_config -p wa -k sshd_config_changes
# Make rules persistent
sudo nano /etc/audit/rules.d/audit.rules
# Add:
-w /etc/passwd -p wa -k passwd_changes
-w /etc/shadow -p wa -k shadow_changes
-w /var/log/lastlog -p wa -k logins
# Search audit logs
sudo ausearch -k passwd_changes
sudo ausearch -ui 1000 # Search by user ID
# Generate audit report
sudo aureport --summary
sudo apt install auditd audispd-plugins
# Enable audit rules for sensitive files
sudo auditctl -w /etc/passwd -p wa -k passwd_changes
sudo auditctl -w /etc/shadow -p wa -k shadow_changes
sudo auditctl -w /etc/ssh/sshd_config -p wa -k sshd_config_changes
# Make rules persistent
sudo nano /etc/audit/rules.d/audit.rules
# Add:
-w /etc/passwd -p wa -k passwd_changes
-w /etc/shadow -p wa -k shadow_changes
-w /var/log/lastlog -p wa -k logins
# Search audit logs
sudo ausearch -k passwd_changes
sudo ausearch -ui 1000 # Search by user ID
# Generate audit report
sudo aureport --summary
Security Layers: Defense in depth - implement multiple layers of security. If one layer fails, others provide protection. Combine firewall, intrusion detection, auditing, and monitoring.
Automated Security Scanning
# Install Lynis security auditing tool
sudo apt install lynis
# Run security audit
sudo lynis audit system
# Review suggestions
sudo cat /var/log/lynis.log
# Install RKHunter (rootkit detection)
sudo apt install rkhunter
sudo rkhunter --update
sudo rkhunter --check
# Schedule daily scans
sudo crontab -e
# Add:
0 3 * * * /usr/bin/rkhunter --cronjob --update --quiet
sudo apt install lynis
# Run security audit
sudo lynis audit system
# Review suggestions
sudo cat /var/log/lynis.log
# Install RKHunter (rootkit detection)
sudo apt install rkhunter
sudo rkhunter --update
sudo rkhunter --check
# Schedule daily scans
sudo crontab -e
# Add:
0 3 * * * /usr/bin/rkhunter --cronjob --update --quiet
Exercise: Harden a test server:
- Install minimal Ubuntu Server
- Configure UFW with only necessary ports
- Harden SSH configuration and disable password auth
- Install and configure Fail2Ban
- Disable all unnecessary services
- Set up automatic security updates
- Run Lynis audit and address high-priority findings