By Diego Patrik

SSH, or Secure Socket Shell, is a network protocol that allows remote access to users, and especially administrators, securely on insecure networks. It was created to be a replacement for telnet, which unlike SSH, does not allow communication in an encrypted form. SSH follows the client-server model and is widely used for managing systems and applications remotely, allowing you to connect to another computer over the network, execute commands, and move files. Any Linux or MacOS user can connect to a remote server from a terminal screen, and even Windows users can easily establish communication through SSH clients such as Putty.

The basic way to use it is to connect via a terminal, with the most popular forms of authentication being user and password authentication and key authentication.

In an authentication using SSH keys, the client can successfully connect to the server when the server recognizes the client’s public key and the client proves to be the owner of the key.

Figure 1 – Illustration of a key-based SSH connection – Source: Aakash Yadav no Better Programming

Using SSH key-based authentication may be a bit more complex, but keeping security in mind, we can say that it’s superior to using password authentication, as it avoids, for example, the possibility of using weak passwords as the only access factor to the server and eavesdropping attacks while typing the password and sending the password to an unknown server due to typing error. Although it solves these problems, this famous copy-and-paste key method also has its drawbacks.

First, there is no check on the server’s identity. When connecting to a server for the first time, a message similar to the following is displayed:

The authenticity of host '111.111.11.111 (111.111.11.111)' can't be established.
ECDSA key fingerprint is fd:fd:d4:f9:77:fe:73:84:e1:55:00:ad:d6:6d:22:fe.
Are you sure you want to continue connecting (yes/no)? yes

The message tells us that the remote host is not recognized and, in practice, the user automatically accepts to continue without verifying that the host is in fact the server to which they wish to connect. Other negative points are:

  • Flexibility: there is no possibility to give time-based access
  • Transparency: analyzing the logs is not an easy task
  • Scalability: not very manageable when there are multiple servers and multiple users

As the amount of servers and users grows, and it becomes a headache to manage SSH keys, a central form of authentication like LDAP and/or Kerberos can be used for account management, but at the same time, we realize that a central authentication is a single point of failure.

Although using a PKI X.509 certificate system brings a solution to this issue with SSH key usage not being scalable, there is a costly submission and validation process involved to get the certificate signed. As an alternative, we found OpenSSH, which supports the creation of simple certificates with an associated CA infrastructure.

Benefits of Using Certificate Authentication

In the SSH world, certificate authentication can be considered as a next level to key-based authentication. This method works similarly to SSL, which is widely known and used across the Internet for HTTPS traffic.

In a certificate-based SSH authentication, by trusting the certificate authority (CA), the client trusts all the servers signed by that CA. And a server trusts all user keys if the key has a CA signature, and the server trusts the CA to sign new user keys.

Figure 2 – Illustration of a SSH connection with certificates – Source: Aakash Yadav no Better Programming

The main gains in using this form of authentication are:

  • Security: by issuing certificates for a short period of time one can decrease the risk of unauthorized access to servers
  • Flexibility: With a certificate you can configure several options, thus achieving flexibility
  • Transparency: IDs make logs easier to debug
  • Scalability: possibility to create a CA server (guaranteeing security) with a web application with authentication that requests 2FA to issue certificates
  • Host verification: authenticity verification of the host is done automatically

Certificates contain a public key, identity information and validity details. They are signed with a common SSH public key using the ssh-keygen tool. Using this tool, there are two types of certificates: user and host. The user certificate is for authenticating clients to servers, while the host certificate authenticates servers to the client. For both certificates, you must configure sshd to trust the public key of the CA.

How to Implement?

The first step of implementation is to create your own Certificate Authority or Certificate Authority (CA), which in this case is simply creating a normal key pair.

Generating a key pair:

    $   ssh-keygen -t rsa -f ca_key

The previous command, will generate two files: ca_key (private key) and ca_key.pub (public key). The public key will be shared with all the hosts that should have access.

Configuring the host to recognize the CA:

  1. Copy the CA public key (pub) to /etc/ssh/ on the server.
  2. Configure SSH servers to trust the CA by adding the following line to the end of the file /etc/ssh/sshd_config 
TrustedUserCAKeys /etc/ssh/ca_key.pub

3. Run the service sshd reload command to reload the settings.

Signing the host key (issuing the server certificate):

  1. Copy the server’s public key (/etc/ssh/ssh_host_rsa_key.pub) to the CA machine
  2. Sign using the command:
    $   ssh-keygen -s ca_key -I Host -V +52w -h ssh_host_rsa_key.pub

-s = private key (CA)

-I = identifier (Will appear in logs)

-V = validity

-h = flag to indicate that you are signing the host key

3. Copy the file generated by the previous command (ssh_host_rsa_key-cert.pub) back to the server in /etc/ssh/

4. Add the following line to the end of /etc/ssh/sshd_config on the server:

HostCertificate /etc/ssh/ssh_host_rsa_key-cert.pub

5. Run the service sshd reload command to reload the settings.

 

Configuring the Client to trust CA signatures:

To do this, you need to copy the content of the public key (ca_key.pub) and add it to the known_hosts file in the format:

@cert-authority * ssh-rsa <content of the public key>

The value * in this case is a default for matching, which represents the domain, or the list of domains for which this certificate should be trusted. It can be passed as *.exemplo.com or host1,host2,host3 for example. For example, by setting the value to *.exemplo.com, and initiating a connection for the first time with host.example.com, the connection occurs directly, without the need to accept trusting the host.

Signing the user key (Issuing the client certificate):

  1. Copy the client’s public key, which by default is located in ~/.ssh/id_rsa.pub (if any), to the CA machine
  2. Sign using the command:
    $   ssh-keygen -s ca_key -I Client -n root,host -V +52w id_rsa.pub

-s = private key (CA)

-I = Identifier (Will appear in logs)

-V = Validity

-n = list of users the client has access to on the server

3. Copy the file generated by the previous command (id_rsa-cert.pub) back to the client, in ~/.ssh/

At this point, it is already possible to establish a connection between client and server using SSH certificate. And to do this, the command for the connection does not change:

    $   ssh user@remote_host

As mentioned, earlier, in the example below we see that the SSH connection using certificate occurs similarly to the key-based way, but without displaying messages like “The authenticity of host <host> can’t be established”, when connecting to a host for the first time.

It’s only by enabling verbose mode and analyzing the output that we find information regarding certificate usage:

Configuration applied on the client:

Configuration applied on the server:

Best practices

To further improve security when using SSH certificates, we can list some recommendations as best practices:

  • Separate access to different hosts.
  • Have distinct CA keys for signing Client/Server certificates.
  • In a production environment, a recommendation would be to store the CA private key on an offline machine with an administrator account, using it only when it’s necessary to issue new certificates.
  • As an alternative to using a dedicated machine as the CA, it’s possible to use an API-based solution such as Vault. Still another possibility would be to leverage the use of Microsoft’s Active Directory.

Integration with Active Directory

As already mentioned, in a real-world usage scenario, besides taking good practices into consideration, something interesting would be to use Active Directory (AD) as the central authentication platform and certificate authority for a certificate-based SSH authentication.

One way to do this, is to use the same tool we use in Linux, OpenSSH, which is available from Windows Server 2019 and can be installed by following the steps on this page from Microsoft’s official documentation.

With OpenSSH installed, you need to start the sshd service with the following commands at an elevated PowerShell prompt on the server:

# Set the sshd service to be started automatically
Get-Service -Name sshd | Set-Service -StartupType Automatic

# Now start the sshd service
Start-Service sshd

To generate the CA, simply use the standard command at a PowerShell prompt

    $   ssh-keygen -t rsa -f ca_key

The OpenSSH client includes scp, which is a secure file transfer utility and can be used to follow along with the steps described above.

For a cloud scenario using Azure, Microsoft provides a guide on how to use Azure AD with SSH certificate-based authentication to log into Linux VMs on Azure.

Negative points

Just like password authentication and key-based authentication, we also find pros and cons in the certificate-based authentication method. The first negative point we can observe is the greater complexity of implementation and the fact that there is far less documentation about it. Another point to consider is that, so far, it’s not yet possible to initiate a connection by certificate using clients like Putty.

Conclusion

We understand how key-based SSH authentication works and how using certificate-based SSH authentication can solve most of your problems. Besides improving the security of the SSH infrastructure, as we have seen, this method also delivers scalability, flexibility, and transparency, which are enough reasons to face all the initial complexity involved.

References

BOTT, Amarnath. How to Use SSH Certificates for Scalable, Secure, and More Transparent Server Access. Better Programming, 2021. Available at: https://betterprogramming.pub/how-to-use-ssh-certificates-for-scalable-secure-and-more-transparent-server-access-720a87af6617/. Accessed on: 14 March 2023.

DIGITALOCEAN. Como configurar a autenticação baseada em chaves SSH em um servidor Linux. DigitalOcean, 2021. Available at: https://www.digitalocean.com/community/tutorials/how-to-configure-ssh-key-based-authentication-on-a-linux-server-pt/. Accessed on: 14 March 2023.

HOLM, Kevin. How to SSH Properly. Teleport, 2018. Available at: https://goteleport.com/blog/how-to-ssh-properly/. Accessed on: 14 March 2023.

TECHTARGET. Secure Shell (SSH). SearchSecurity, 2019. Available at: https://www.techtarget.com/searchsecurity/definition/Secure-Shell/. Accessed on: 14 March 2023.

THORNTECH. Passwords vs. SSH keys – what’s better for authentication? ThornTech, 2018. Available at: https://www.thorntech.com/passwords-vs-ssh/. Accessed on: 14 March 2023.

MENDHEKAR, Puneet et al. Scalable and secure access with SSH. Facebook Engineering, 2016. Available at: https://engineering.fb.com/2016/09/12/security/scalable-and-secure-access-with-ssh/. Accessed on: 14 March 2023.

MICROSOFT. OpenSSH key management. Microsoft Docs, 2021. Available at: https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_keymanagement/. Accessed on: 14 March 2023.

RED HAT. Using OpenSSH Certificate Authentication. Red Hat Enterprise Linux 6 Deployment Guide, 2011. Available at: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/deployment_guide/sec-using_openssh_certificate_authentication/. Accessed on: 14 March 2023.

WANG, Jun. Managing servers with OpenSSH Certificate Authority. iBug, 2019. Available at: https://ibug.io/blog/2019/12/manage-servers-with-ssh-ca/. Accessed on: 14 March 2023.