Secure Your Ubuntu Server with SSH: Step-by-Step Guide

Did you know that over 90% of cyberattacks target weak remote access configurations? Many system administrators overlook basic security measures, leaving their servers vulnerable. Properly configuring SSH is the first line of defense against unauthorized access.

We’ll walk you through essential steps to harden your remote server using OpenSSH. From changing default ports to enforcing key-based authentication, these techniques help block brute-force attacks. Modern security practices go beyond basic setups—firewall adjustments and session monitoring add extra layers of protection.

This guide combines practical configurations with advanced strategies. Whether you’re managing a single machine or multiple systems, these steps ensure safer connections. For a deeper dive into initial setup, check our OpenSSH installation tutorial.

Key Takeaways

• Changing default ports reduces automated attack risks
• Key authentication is more secure than passwords
• Firewall rules must align with SSH configurations
• Session monitoring detects suspicious activity early
• Automated scripts simplify ongoing maintenance

Introduction

SSH remains the backbone of remote server management, yet many overlook its vulnerabilities. This protocol handles sensitive tasks, from file transfers to system updates. Without proper safeguards, attackers exploit weak points within minutes.

Default configurations are a major risk. OpenSSH installations often use port 22 and permit password logins. These settings attract automated bots scanning for easy targets. Over 70% of brute-force attacks focus on SSH ports.

A defense-in-depth strategy works best. Combine OpenSSH hardening with tools like fail2ban or firewall rules. This layered approach blocks threats at multiple stages.

Real-world breaches show the stakes. Unsecured implementations lead to data theft, ransomware, or worse. Proactive measures reduce these risks significantly.

Prerequisites for Securing SSH

A secure setup demands specific prerequisites—here’s what you’ll need. Without these foundations, even the best SSH server adjustments won’t guarantee safety. Let’s verify your system’s readiness.

Ubuntu Server with OpenSSH

Most Ubuntu Server installations include OpenSSH by default. Confirm it’s active with systemctl status ssh. Desktop users must install it manually. An internet connection ensures you fetch the latest patches during configuration.

Terminal Access and Sudo Privileges

Admin rights are non-negotiable. Check /etc/sudoers or your user group for sudo privileges. Losing access mid-configuration locks you out—keep a backup terminal session open.

Firewall and Port Configuration Knowledge

UFW simplifies firewall management, but missteps block legitimate traffic. Master basic commands like ufw allow and ufw deny. Understand how port configuration impacts remote access before changing defaults.

Pro tip: Document each change. Reversing errors becomes effortless with clear records.

Step 1: Update Your System Packages

Keeping your system updated is the foundation of server security. Outdated packages risk exploits, especially in OpenSSL libraries. Start by running the following command:

This fetches the latest patches and installs them. For kernel updates, a reboot may be required. Check pending upgrades with:

apt list --upgradable

Occasionally, held-back packages cause dependency issues. Resolve them with sudo apt --with-new-pkgs upgrade.

Production systems need careful scheduling. Use unattended-upgrades for automated security patches. Combine this with manual checks for full coverage.

Verified updates close security gaps faster. Pair this step with firewall adjustments for layered protection.

Step 2: Install OpenSSH Server

The OpenSSH package transforms any system into a secure remote access point. Begin with this essential command to install the openssh-server package:

sudo apt install openssh-server

Ubuntu automatically resolves dependencies like libc6 and openssl. The client package (ssh) often comes preinstalled, while the ssh server component requires manual setup.

Verify package integrity using GPG keys. Run apt-key list to check trusted signatures. Repository errors may occur if sources aren’t updated—always run sudo apt update first.

Post-installation, these directories become critical:

• /etc/ssh – Configuration files
• /var/log/auth.log – Authentication attempts
• /usr/lib/openssh – Binary locations

For advanced users, source installation offers version control. Download from openssh.com and compile with ./configure. This method suits custom security patches.

Step 3: Enable and Start the SSH Service

Modern Linux systems rely on systemd for service control. Proper initialization ensures persistent remote access without manual restarts. We’ll configure automatic startup and verify operational status.

Activating Persistent SSH Availability

Use this command to enable the ssh service during boot sequences:

sudo systemctl enable --now ssh

The --now flag starts the service immediately. Systemd creates symbolic links in /etc/systemd/system for automatic loading.

Verifying Operational Status

Check real-time service health with:

systemctl status ssh

Active services show “running” with recent timestamps. Watch for these common states:

• Active (running): Normal operation
• Inactive (dead): Service stopped
• Failed: Configuration errors present

For deeper diagnostics, use journalctl -u ssh. This displays timestamped logs with error details. Failed starts often reveal missing dependencies or permission issues.

Step 4: Adjust the UFW Firewall for SSH

Network security begins with precise firewall rules for remote connections. Ubuntu’s Uncomplicated Firewall (UFW) simplifies this process with intuitive commands. Start by allowing SSH port traffic:

sudo ufw allow ssh

Activate the firewall immediately with sudo ufw enable. This implements default deny policies while permitting explicit SSH access. Always verify rules using ufw status.

Service names like “ssh” map to standard ports (22). For custom ports, specify numbers directly:

sudo ufw allow 2222/tcp

Prevent accidental lockouts during configuration:

• Keep existing SSH sessions active
• Test new rules before disabling old ones
• Use ufw --dry-run for safe testing

Advanced users combine UFW with iptables for granular control. Rate limiting adds extra protection:

sudo ufw limit ssh

This automatically blocks repeated connection attempts. For production systems, consider connection tracking to monitor active sessions.

How to Secure SSH on Ubuntu Server – Step-by-Step Guide

The configuration file holds the keys to SSH security adjustments. Located at /etc/ssh/sshd_config, this text-based control center dictates authentication methods, port behavior, and access protocols. Proper editing prevents common attack vectors while maintaining functionality.

Essential Backup Protocol

Always create backups before modifying live settings. Execute this command to preserve your original configuration file:

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak

This safeguard allows instant rollback if changes cause issues. Store backups with strict permissions (600) to prevent unauthorized access.

Critical Security Settings

These parameters demand attention in your ssh setup:

• PermitRootLogin – Disable for non-root access
• PasswordAuthentication – Set to ‘no’ for key-only
• X11Forwarding – Disable unless required

After edits, test syntax with sshd -t. Reload changes using systemctl reload ssh for zero downtime. Version control systems like Git help track modifications over time.

Step 5: Change the Default SSH Port

Port 22 is a hacker’s favorite target—time to make their job harder. Automated bots constantly scan this default port, probing for weak credentials. Switching to a non-standard number reduces unwanted traffic instantly.

sudo nano /etc/ssh/sshd_config

Locate the line #Port 22, uncomment it, and replace 22 with a high-numbered port (e.g., 2111). Save changes and restart the service:

sudo systemctl restart ssh

Critical Follow-Up Steps

• Firewall updates: Adjust UFW or iptables to allow the new port
• SELinux/AppArmor: Modify policies if active
• Client configurations: Update ~/.ssh/config or specify ports manually

Test port availability with nc -zv your_server_ip 2111. Avoid conflicts with common services (e.g., 80, 443). While this isn’t foolproof, it’s a quick win against automated attacks.

For advanced users, port knocking adds stealth—only allowing access after a specific sequence. Tools like knockd automate this process.

Step 6: Disable Root Login

Direct root logins create unnecessary risks that most administrators can eliminate. The /etc/ssh/sshd_config file controls this critical setting through the PermitRootLogin directive.

PermitRootLogin no

This forces all users to authenticate with standard accounts first. Create alternative admin accounts with sudo privileges before applying this change.

Consider these alternatives when complete blocking isn’t feasible:

• prohibit-password allows key-based root access
• without-password (deprecated but still functional)
• Restricted console-only root access

Monitor login attempts with grep "root" /var/log/auth.log. Failed tries often indicate brute-force attacks targeting the root user.

Configure /etc/sudoers carefully when removing root access. The visudo command prevents syntax errors that could lock you out permanently.

Step 7: Set Up SSH Key-Based Authentication

Key-based authentication eliminates password vulnerabilities while streamlining remote access. Cryptographic ssh keys provide mathematical security that brute-force attacks cannot crack. This method pairs a private key (kept secret) with a public key installed on servers.

Generate SSH Key Pair

Modern systems support ED25519 algorithms for stronger protection. Run this command to create a new pair:

ssh-keygen -t ed25519

The tool saves keys in ~/.ssh/ by default. Consider these options during generation:

• Passphrase: Encrypts private keys with an extra password
• Key length: ED25519 uses fixed 256 bits vs RSA’s variable sizes
• File names: Custom paths help manage multiple keys

Copy Public Key to Server

Transfer the public key using this streamlined command:

ssh-copy-id user@your_server

This automatically appends keys to ~/.ssh/authorized_keys. For manual transfers, our SSH key configuration guide details advanced methods.

Enforce Key-Only Logins

Edit /etc/ssh/sshd_config to disable weaker methods:

PasswordAuthentication no
PubkeyAuthentication yes

Restart the service to apply changes. Always maintain backup access methods during transitions. For teams, consider centralized key management solutions to handle revocations efficiently.

Pro tip: Rotate keys quarterly using the same process. Compromised credentials lose value when replaced regularly.

Step 8: Limit Authentication Attempts

Brute-force attacks thrive on unlimited login attempts—time to shut them down. OpenSSH’s MaxAuthTries directive controls how many failed logins a user gets before disconnection. Set this to 2 in /etc/ssh/sshd_config:

MaxAuthTries 2

This stops attackers after two wrong guesses. Combine this with Fail2Ban for automated IP blocking. The tandem approach creates layered security against credential stuffing.

Monitoring Strategies

Effective defense requires watching attack patterns:

• Rate limiting: Count attempts per minute, not just total tries
• Journal integration: Pipe logs to systemd-journal for centralized tracking
• Geo-blocking: Restrict countries with high attack volumes

“One unprotected SSH port can become the entry point for enterprise-wide breaches.”

Legal teams should review logging policies. Some jurisdictions require attack notification if personally identifiable information is exposed. Always encrypt stored logs containing IP addresses.

Test changes with ssh -v before enforcing limits. Production systems need gradual rollout to avoid locking out legitimate users.

Step 9: Test SSH Access from Another Machine

Validating remote access ensures configurations work as intended before enforcing restrictions. Use this basic SSH command from a different device:

ssh -p 2222 user@hostname

Verbose Mode Debugging

Add -v to the command for real-time connection diagnostics:

ssh -v -p 2222 user@hostname

This reveals handshake details, authentication steps, and error causes. Three -v flags increase verbosity for deep troubleshooting.

Cross-Platform Testing

Verify compatibility across operating systems:

• Windows: Use PowerShell or PuTTY
• macOS/Linux: Native terminal clients
• Mobile: Apps like Termius or JuiceSSH

Network paths impact connectivity. Tools like traceroute map hops between devices. Firewalls or NATs might block custom ports—confirm with:

telnet hostname 2222

Client Configuration Adjustments

Edit ~/.ssh/config for persistent settings:

Host myserver
  HostName hostname
  Port 2222
  User username

Timeout and keepalive settings prevent drops:

For advanced SSH command syntax, reference our detailed guide.

Troubleshooting Common SSH Issues

Even with proper configuration, SSH connections sometimes fail. Understanding these errors helps resolve issues quickly. We’ll break down frequent problems and their solutions.

Connection Refused Errors

This error means the server isn’t accepting connections. First, check if the SSH service runs:

systemctl status sshd

Next, verify port availability:

• Use ss -plnt to see listening ports
• Check firewall rules with ufw status
• Confirm network routes aren’t blocked

For deeper analysis, our SSH connectivity guide covers advanced diagnostics.

Permission Denied (Publickey)

This frustrating error often stems from key file permissions. Set correct access rights:

chmod 600 ~/.ssh/private_key

Advanced Problem Solving

Some issues require specialized tools:

• SELinux/AppArmor: Check security contexts
• TCP Wrappers: Review /etc/hosts.allow
• Protocol mismatch: Use ssh -v for version checks

“90% of SSH problems stem from three causes: service status, permissions, or network paths.”

For X11 forwarding issues, verify both client and server settings. Always test changes in a staging environment first.

Conclusion

Security is never a one-time setup—it requires continuous attention. The layered approach we’ve implemented combines SSH hardening with network protections and monitoring.

Regular audits catch configuration drift. Schedule monthly checks for unauthorized changes and review access logs. Automation tools like Ansible simplify these repetitive tasks.

Complementary measures boost protection further. Consider intrusion detection systems alongside your security setup. Tools like Fail2Ban add active defense against brute-force attempts.

Bookmark this final checklist for reference:

• Rotate cryptographic keys quarterly
• Update firewall rules with any service changes
• Monitor authentication attempts daily

Proper SSH configuration creates resilient systems. Stay vigilant, automate maintenance, and your server will remain protected against evolving threats.