How to Set Up SSH Public Key Authentication to Connect to a Remote System

Karna Sodari

2 weeks ago

Today, I implemented the industry-standard method for securing remote system access: SSH (Secure Shell) public key authentication.

I did this because, frankly, password-based authentication isn’t secure enough. It’s too vulnerable to brute-force and dictionary attacks. By leveraging asymmetric cryptography, I’ve made my systems significantly more secure, scalable, and automatable.

This log entry documents my complete setup process, detailing everything from how I generated the keys to how I achieved my first verified remote login.

Background & Theory

What Is SSH?

SSH (Secure Shell Protocol) is a cryptographic network protocol that enables secure communication over an unsecured network. It operates on TCP port 22 by default and provides:

Encrypted data transmission
Remote command execution
Secure file transfer (SFTP/SCP)
Port forwarding and tunneling

How Public Key Authentication Works

Public key authentication is based on asymmetric (public-key) cryptography. It involves a mathematically linked key pair:

Key Type	Location	Visibility	Purpose
Private Key	Client machine	Secret	Used to sign the authentication request
Public Key	Remote server	Shareable	Used to verify the signature

Authentication Flow:

[Client]                              [Server]
   |                                      |
   |--- SSH Connection Request ---------->|
   |                                      |--- Checks ~/.ssh/authorized_keys
   |<-- Cryptographic Challenge ----------|
   |                                      |
   |--- Signs Challenge w/ Private Key -->|
   |                                      |--- Verifies w/ Public Key
   |<-- Access Granted ------------------|

The server never sees the private key. The client proves identity by successfully signing a challenge that only the holder of the private key can produce.

Prerequisites

Before proceeding, ensure the following:

Local machine: Linux, macOS, or Windows (with OpenSSH or PuTTY)
Remote server: Running SSH daemon (sshd) — Linux/Unix based
Network access: Ability to reach the remote server via TCP port 22
User account: An existing user account on the remote server
Permissions: Ability to write to ~/.ssh/ on the remote server

Step-by-Step Setup

Check for Existing SSH Keys

Before generating a new key pair, check if one already exists on your local machine:

ls -la ~/.ssh/

Expected Output (if keys exist):

-rw-------  1 user user 3381 Jan 10 2026 id_rsa
-rw-r--r--  1 user user  741 Jan 10 2026 id_rsa.pub
-rw-------  1 user user  411 Jan 10 2026 id_ed25519
-rw-r--r--  1 user user  101 Jan 10 2026 id_ed25519.pub

Common key file names:

id_rsa / id_rsa.pub — RSA keys
id_ed25519 / id_ed25519.pub Ed25519 keys (recommended)
id_ecdsa / id_ecdsa.pubECDSA keys

Research Note: If suitable keys already exist, skip to Step 3 to avoid overwriting existing configurations.

Generate an SSH Key Pair

Option A: Ed25519 (Recommended Modern & Secure)

ssh-keygen -t ed25519 -C "your_email@example.com"

Parameter Breakdown:

Flag	Value	Description
`-t`	`ed25519`	Key type: EdDSA using Curve25519
`-C`	`"comment"`	A label (usually email) to identify the key

Option B: RSA 4096-bit (Legacy Compatible)

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

Additional RSA Flags:

Flag	Value	Description
`-b`	`4096`	Key size in bits (minimum 2048 for RSA)

Interactive Prompts:

Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/user/.ssh/id_ed25519): [Press Enter or specify path]
Enter passphrase (empty for no passphrase): [Enter a strong passphrase]
Enter same passphrase again: [Confirm passphrase]

Your identification has been saved in /home/user/.ssh/id_ed25519
Your public key has been saved in /home/user/.ssh/id_ed25519.pub
The key fingerprint is:
SHA256:xXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxXxX your_email@example.com
The key's randomart image is:
+--[ED25519 256]--+
|        .o+o..   |
|         oo.o    |
|        + .+ .   |
|       o X .=    |
|      . S +.o.   |
|       . +..o.   |
|      . =.=+.    |
|       o.B+o.    |
|        +E+.     |
+----[SHA256]-----+

Security Note: Always set a passphrase. It encrypts the private key file on disk, providing a second layer of protection if your machine is compromised.

Copy the Public Key to the Remote Server

Using `ssh-copy-id` (Easiest)

ssh-copy-id -i ~/.ssh/id_ed25519.pub username@remote_host

Flags Explained:

Flag	Description
`-i`	Specifies the identity (public key) file to copy

Sample Output:

/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/user/.ssh/id_ed25519.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s)
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed
username@remote_host's password: [Enter password]

Number of key(s) added: 1

Now try logging into the machine, with: "ssh 'username@remote_host'"
and check to make sure that only the key(s) you wanted were added.

Manual Copy (Universal)

Use this method when ssh-copy-id is not available (e.g., Windows without OpenSSH tools):

Step 2a — Display the public key:

cat ~/.ssh/id_ed25519.pub

Output:

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIExamplePublicKeyStringHere your_email@example.com

Step 2b — Log into the remote server:

ssh username@remote_host

Step 2c — Create .ssh directory and set permissions:

mkdir -p ~/.ssh
chmod 700 ~/.ssh

Step 2d — Append the public key to authorized_keys:

echo "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIExamplePublicKeyStringHere your_email@example.com" >> ~/.ssh/authorized_keys

Step 2e — Set correct file permissions:

chmod 600 ~/.ssh/authorized_keys

Using `ssh` Pipe (Single Command)

cat ~/.ssh/id_ed25519.pub | ssh username@remote_host "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

Verify Permissions on the Remote Server

Incorrect permissions are the most common cause of SSH key authentication failures. SSH is strict about this.

# On the remote server:
ls -la ~/.ssh/

Required Permissions:

Path	Permission	Octal	Reason
`~/.ssh/` (directory)	`drwx------`	`700`	Only owner can read/write/execute
`~/.ssh/authorized_keys`	`-rw-------`	`600`	Only owner can read/write
`~` (home directory)	`drwx------` or `drwxr-xr-x`	`700`/`755`	Cannot be group/world-writable

Fix permissions if needed:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chown -R $USER:$USER ~/.ssh

Configure SSH Daemon on the Remote Server (Optional but Recommended)

Edit the SSH daemon configuration file to harden settings:

sudo nano /etc/ssh/sshd_config

Key directives to review/set:

# Enable public key authentication
PubkeyAuthentication yes

# Path to authorized keys file
AuthorizedKeysFile .ssh/authorized_keys

# Disable password authentication (after confirming key auth works!)
PasswordAuthentication no

# Disable root login
PermitRootLogin no

# Only allow specific users (optional)
AllowUsers yourusername

# Use strong ciphers only
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com

# Disable empty passwords
PermitEmptyPasswords no

Apply changes by restarting the SSH daemon:

# systemd (Ubuntu/Debian/CentOS 7+)
sudo systemctl restart sshd

# SysVinit (older systems)
sudo service ssh restart

Critical Warning: Do NOT disable PasswordAuthentication until you have successfully tested key-based login. Locking yourself out of a remote server can be catastrophic.

Test the Connection

ssh -i ~/.ssh/id_ed25519 username@remote_host

Verbose Mode (for debugging):

ssh -v -i ~/.ssh/id_ed25519 username@remote_host

Expected Output (successful):

OpenSSH_8.9p1, OpenSSL 3.0.2
debug1: Reading configuration data /home/user/.ssh/config
debug1: Connecting to remote_host [192.168.1.100] port 22.
debug1: Connection established.
debug1: identity file /home/user/.ssh/id_ed25519 type 3
debug1: Authenticating to remote_host:22 as 'username'
debug1: Trying private key: /home/user/.ssh/id_ed25519
debug1: Authentication succeeded (publickey).
Welcome to Ubuntu 22.04.3 LTS
username@remote_host:~$

Configure SSH Client Config (Optional Quality of Life)

Create or edit ~/.ssh/config on your local machine to simplify connections:

nano ~/.ssh/config

# Default settings for all hosts
Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
    AddKeysToAgent yes
    IdentityFile ~/.ssh/id_ed25519

# Remote server alias
Host myserver
    HostName 192.168.1.100
    User username
    Port 22
    IdentityFile ~/.ssh/id_ed25519
    ForwardAgent no

# Jump host (bastion) example
Host internal-server
    HostName 10.0.0.50
    User admin
    ProxyJump myserver

Set correct permissions on the config file:

chmod 600 ~/.ssh/config

Now connect with just:

ssh myserver

Using SSH Agent for Passphrase Management

If you set a passphrase, use ssh-agent to avoid re-entering it every session:

# Start the agent
eval "$(ssh-agent -s)"

# Output:
# Agent pid 12345

# Add your key to the agent
ssh-add ~/.ssh/id_ed25519

# Enter passphrase: [enter once]
# Identity added: /home/user/.ssh/id_ed25519

# List keys in agent
ssh-add -l

Auto-start ssh-agent on login (add to ~/.bashrc or ~/.zshrc):

if [ -z "$SSH_AUTH_SOCK" ]; then
   eval "$(ssh-agent -s)"
   ssh-add ~/.ssh/id_ed25519
fi

Windows-Specific Setup

Using OpenSSH (Windows 10/11 Built-in)

# Check if OpenSSH is installed
Get-WindowsCapability -Online | Where-Object Name -like 'OpenSSH*'

# Install OpenSSH Client
Add-WindowsCapability -Online -Name OpenSSH.Client~~~~0.0.1.0

# Generate key (in PowerShell or CMD)
ssh-keygen -t ed25519 -C "your_email@example.com"
# Keys saved to: C:\Users\YourName\.ssh\

# Copy public key to server
type $env:USERPROFILE\.ssh\id_ed25519.pub | ssh username@remote_host "cat >> ~/.ssh/authorized_keys"

Using PuTTY

1. Open PuTTYgen
2. Select Ed25519 or RSA 4096 from "Type of key to generate"
3. Click "Generate" and move mouse randomly
4. Set a Key passphrase
5. Save Private Key (.ppk format) — keep secure
6. Copy the text from "Public key for pasting into OpenSSH authorized_keys"
7. Paste it into ~/.ssh/authorized_keys on the remote server
8. In PuTTY: Connection > SSH > Auth > Browse to your .ppk file

Debug Commands

# Verbose SSH connection (levels: -v, -vv, -vvv)
ssh -vvv username@remote_host

# Check sshd logs on remote server
sudo journalctl -u sshd -f
sudo tail -f /var/log/auth.log      # Debian/Ubuntu
sudo tail -f /var/log/secure        # RHEL/CentOS

# Test sshd config syntax
sudo sshd -t

# Check if sshd is running
sudo systemctl status sshd

# Check authorized_keys content
cat ~/.ssh/authorized_keys

# Verify SELinux context (RHEL/CentOS)
ls -Z ~/.ssh/authorized_keys
restorecon -Rv ~/.ssh/

Security Best Practices

Key Management

# Rotate keys periodically — generate new, add to server, remove old
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_new -C "rotated-2026"
ssh-copy-id -i ~/.ssh/id_ed25519_new.pub username@remote_host
# Remove old key from authorized_keys after verifying new key works

# Check all authorized keys on server
cat ~/.ssh/authorized_keys | wc -l   # count
# Audit and remove keys that are no longer needed

Multi-Key & Advanced Scenarios

Managing Multiple Keys

# Generate separate keys for different servers/purposes
ssh-keygen -t ed25519 -f ~/.ssh/id_work -C "work-server"
ssh-keygen -t ed25519 -f ~/.ssh/id_github -C "github"
ssh-keygen -t ed25519 -f ~/.ssh/id_prod -C "production-server"

~/.ssh/config for multiple keys:

Host github.com
    IdentityFile ~/.ssh/id_github

Host work-server
    HostName 10.0.1.50
    User deploy
    IdentityFile ~/.ssh/id_work

Host prod-*
    User root
    IdentityFile ~/.ssh/id_prod
    StrictHostKeyChecking yes

Certificate-Based Authentication (Enterprise)

# Sign a public key with a CA (Certificate Authority) — enterprise use
ssh-keygen -s /path/to/ca_key -I "user_cert_id" -n username -V +52w ~/.ssh/id_ed25519.pub

# The resulting certificate: ~/.ssh/id_ed25519-cert.pub
# Valid for 52 weeks, for user "username"

Summary & Quick Reference

End-to-End Setup in 5 Commands

# 1. Generate key pair (local machine)
ssh-keygen -t ed25519 -C "your_email@example.com"

# 2. Copy public key to remote server
ssh-copy-id -i ~/.ssh/id_ed25519.pub username@remote_host

# 3. Verify permissions on remote server
ssh username@remote_host "chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys"

# 4. Test connection
ssh -i ~/.ssh/id_ed25519 username@remote_host

# 5. (Optional) Disable password auth after confirming key auth works
ssh username@remote_host "sudo sed -i 's/#PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config && sudo systemctl restart sshd"

References

OpenSSH Documentation:
NIST SP 800-57: Recommendation for Key Management
RFC 4252: The Secure Shell (SSH) Authentication Protocol
RFC 8709: Ed25519 and Ed448 Public Key Algorithms for the Secure Shell Protocol
CIS Benchmark for SSH: https://www.cisecurity.org/benchmark/distribution_independent_linux