Why SSH hardening matters on Linux servers
SSH is the primary remote access vector for most Linux servers, therefore improving SSH security reduces the attack surface for unauthorized access. A hardened SSH posture prevents brute force, credential theft, and privilege escalation attempts, while preserving legitimate administrator access.
Hardening covers multiple layers, from authentication methods to transport settings, rate limits at the network layer, and monitoring. This guide focuses on practical, repeatable steps you can apply on production servers without breaking remote access.
Prepare: inventory and backup sshd_config
Before making changes, inventory servers and record current SSH settings, enabled services, and open firewall rules. Confirm you have an alternate authenticated session or console access in case a configuration change locks you out.
Create a backup of sshd_config and home directory authorized keys for relevant accounts. Use versioned backups to allow quick rollback, for example copy ss hd_config to a safe location and snapshot the server if possible.
Implement key based authentication securely
Generate keys on administrator workstations with strong algorithms and no passphrase only when automated access is required. Prefer RSA with 3072 bits or Ed25519 keys for interactive access. Keep private keys off servers and protected by file system permissions.
Deploy public keys to user authorized_keys files under home directories, ensure permissions are strict, for example user home directories not writable by others, .ssh directory mode 700, authorized_keys mode 600. Remove unused or stale keys during rollout.
Disable password and root login
Edit /etc/ssh/sshd_config to enforce key based authentication, set PasswordAuthentication to no, and disable root login by setting PermitRootLogin to prohibit password or to no. Restart the SSH service after change, and avoid making this change on all admin sessions at once.
Test key based logins from a separate connection before terminating password access. Keep at least one emergency administrative account with console access or an out of band management path for recovery.
Harden sshd_config recommended settings
Adjust sshd_config for minimum necessary exposure. Useful options include Protocol set to 2 in older systems, AllowUsers or AllowGroups for access restriction, and ClientAliveInterval with a reasonable ClientAliveCountMax to reduce idle sessions. Avoid aggressive timeouts that disrupt automation.
- Use AllowUsers to restrict who can connect from which hosts where practical
- Set PasswordAuthentication no to force keys
- Set PermitRootLogin no to block direct root access
- Use MaxAuthTries to limit consecutive failures
Keep a copy of your working sshd_config and apply configuration changes in small increments to verify behavior before wide deployment.
Rate limiting: nftables or iptables basics
Rate limiting reduces credential stuffing and brute force impact by limiting new connections per source. On modern Linux use nftables to implement connection rate limits with matched counters, on older systems iptables can achieve similar blocking by tracking recent attempts.
- Use a per source limit for new connections to port 22 to prevent mass scanning
- Reject or drop excess connection attempts and log offending addresses for analysis
Apply rules in a staging environment, monitor legitimate traffic for false positives, and ensure your firewall rules do not block jump hosts or automation pipelines.
Use Fail2Ban and PAM controls
Fail2Ban provides dynamic blocking of offending IPs by parsing logs and applying temporary bans. Configure jails for sshd to ban after a small number of failures and tune ban duration based on your threat model and traffic patterns.
PAM can enforce account lockouts and complexity rules at authentication time. Combine Fail2Ban with PAM settings for layered defense, and ensure log paths referenced by Fail2Ban match your system layout.
Logging and auditing SSH access
Enable detailed SSH logging by setting LogLevel to VERBOSE to capture key fingerprint information at login. Forward logs to a central syslog or SIEM to detect distributed attacks and correlate events across hosts.
Regularly review logs for repeated failures, suspicious source addresses, and unexpected account activity. Retain logs based on your compliance needs and rotate them to avoid disk exhaustion.
Verify, test, and recover safely
After each change, verify access from a secondary session and simulate failure scenarios to confirm recovery steps. Use test accounts to validate restrictions, and schedule maintenance windows for broad changes across multiple servers.
Keep documented rollback procedures, including how to restore the sshd_config backup, how to remove a firewall rule set, and how to re enable password login temporarily from a trusted network if necessary.
Conclusion and FAQs
SSH hardening on Linux requires a combination of authentication policy, transport configuration, network rate limiting, and active monitoring. Focus first on enforcing key based authentication and removing direct root access, then add automated protections like Fail2Ban and network level rate limits. Logging and audits complete the picture by giving you visibility into attempted attacks so you can refine rules and respond to incidents. Always implement changes incrementally, verify from secondary access, and maintain backups and recovery paths to avoid lockouts that could impact operations.
Automating these steps with configuration management can improve consistency across fleets, but be careful to test templates before wide rollout. Keep keys and administrative credentials tightly controlled, and review keys and user accounts periodically to remove unnecessary access.
Q: How soon can I disable password login after deploying keys for all admins?
A: Only after you have validated key based login from an independent connection for each account and verified no automation relies on password authentication.
Q: Will rate limiting block legitimate cloud provider health checks?
A: Rate limits must be tuned and exceptions added for known monitoring or load balancer IPs to avoid false positives.
Q: Is Fail2Ban enough to stop persistent attackers?
A: Fail2Ban helps reduce noise and slow attackers, but combine it with network rate limiting, least privilege access, and strong keys for defense in depth.
Q: How do I audit keys and stale accounts at scale?
A: Use scripts to collect authorized_keys and last login data, or use configuration management tools to report and remediate stale entries across hosts.