huh. well, I don’t agree with it, but it’s a well-structured argument.
I just… don’t find any of these proposed downsides to be particularly arduous, and when I look at the constant inbound traffic that’s always trying to brute-force something or other, I see no reason to take my chances. the article does a pretty good job of enumerating many of the ways I could be inadvertently exposed; I just disagree that those things aren’t a problem, in part because I don’t trust myself to be hypervigilant about every single one of them at all times.
I’ve had password-based ssh login turned off entirely on all my machines for something like twenty years now, for both internet and LAN traffic. it’s fine. you learn pretty quickly to not lose your ssh key - which you should be making sure to learn no matter what, because if you leave a copy of the file somewhere and forget it’s there, you have unexpected attack surface.
“somewhat” cursed: Disable password authentication. If you have password reasons, leverage that x25519 private keys can be (almost) any 32 byte random string and kPub generation cheap/fast. Sha256(password) -> your private key.
I’m genuinely curious: how is a private key generated from a password more secure than the password? Is it just because no one is brute-forcing ssh private keys in that way yet or is there some cryptographic reason?
It’s not more secure in a traditional sense, the point is the password convenience without having the server expose password authentication as a supported method so any password guessers will leave it alone. In the nontraditional sense there are services that enforce public key authentication even though you don’t want to bother to supply it with one for other reasons.
I can think of one very public source code host which enforce public key authentication that I have negative trust in, where public keys can be scraped so you can start scanning the internet and see if they reappear elsewhere.
I can think of one very public source code host which enforce public key authentication that I have negative trust in, where public keys can be scraped so you can start scanning the internet and see if they reappear elsewhere.
I believe this is all of them, no? GitHub, GitLab, Codeberg, and Gitea all seem to have public keys available at /username.keys.
You’ve just given anyone who is aware that people might be doing this free reign to attempt to brute force your password hashes haven’t you?
Worse, I don’t think you can salt this scheme? So they can probably build a rainbow table esque structure and brute force everyone using this scheme simultaneously.
This is only true if you use the same SSH keypair for those services as you use for regular logins. My personal view is that you definitely shouldn’t, because attackers can use those public keys to probe for various things against your hosts (eg, a SSH client can find out if a SSH server accepts a given public key for authentication for a particular account).
(I’m the author of the linked-to article, but the information disclosures of known public keys is a completely different topic. In some circles it’s well known; see eg https://words.filippo.io/dispatches/whoami-updated/ )
I should have expanded the explanation of my view: for Github push keys and other things like it, I don’t think you need to go to the ‘generate private key through hashing’ approach. In most environments you’re extremely unlikely to wind up needing to push from a random machine that has not already been set up for this, so you don’t need to be able to produce a keypair from a seed phrase you can memorize. The only keypairs you need to be able to produce that way are keypairs you’d use from not set up machines.
(You can imagine scenarios where you might need to push something to make an urgent production deployment from an extremely minimally set up machine, but it’s a question of trade offs and how likely things are. Even then you might be better off with some kind of ‘break glass’ SSH keypair that is pre-deployed in encrypted form and maybe has to be enabled on the forge side before you can use it.)
Everyone has different security and convenience models, and this seems reasonable. AllowUser and strong passwords can work.
One risk the post doesn’t address is that if a server you SSH into with a password is compromised, they can snoop your password and use that to escalate access to other machines that have the same password.
Perhaps the author has different passwords for each machine, in which case he has a great memory, or a password manager. The latter doesn’t seem to satisfy “you might need to log in to your account in a situation where you don’t have your keypair available” any more than SSH keys.
I’m fortunate enough to be able to only access from my personal machines, and so restrict access behind Wireguard, and use FIDO2 SSH keys (laptop/desktop/keyring) to avoid having to deal with passwords.
Perhaps the author has different passwords for each machine, in which case he has a great memory, or a password manager. The latter doesn’t seem to satisfy “you might need to log in to your account in a situation where you don’t have your keypair available” any more than SSH keys.
And if a password manager does satisfy that concern, you can put an ssh keypair into most of them.
I’m the author of the linked-to article. In my situation at work we effectively have a single security domain across all of our servers, with common passwords (it’s an old fashioned Unix login server style environment). My work desktop is deliberately not in this security domain and so uses a different password, but the net effect is that I only have a few passwords to memorize for work, not one per server.
(I also mostly use SSH keypair authentication, with encrypted keys that are loaded into the SSH agent on my desktop but purged when I lock my screen and step away from it, which is a bit complex to arrange but conveniently magical.)
The latter doesn’t seem to satisfy “you might need to log in to your account in a situation where you don’t have your keypair available” any more than SSH keys.
To be fair, you probably have your password manager on your phone and can easily read the password off of that and type it in directly, whereas this isn’t feasible with a private key. Of course, you could transfer the private key over USB or something, but that’s generally a hassle unfortunately.
OP has a somewhat uncommon use case these days (and he admits it): he’s a sysadmin in a large organisation. I imagine, he roams his domain and might need to access any machine within it from any other. Like we used to. Most people these days never touch a keybord that is not attached to their laptop. So they, obviously, always have access to their ssh keys. In this set up it’s only prudent to disallow password login just to reduce attack surface. OP correctly point out that username-password access involves a lot of machinery (including all 9 circles of PAM hell, and bonus circles of third-party PAM plugins). There’s simply no reason for most people to learn all that to make it secure when they don’t use passords at all.
From my own network, passwords are okay under limited circumstances. From the rest of the internet, is very strongly prefer public keys or yubikey one-time passwords. Once you go down the yubikey route, a great enhancement is to generate a keypair on your yubikey and, using gnupg for your ssh keychain, use that key’s public side for ssh. I’m sure that this works for other hardware tokens too.
What shouldn’t get lost here is that for those using OpenSSH, Match blocks in your /etc/ssh/sshd_config make doing this fairly simple:
...
PermitRootLogin no
PubKeyAuthentication yes
PasswordAuthentication no
KbdInteractiveAuthentication no
...
## my_user can use passwords from anywhere
Match user my_user
PasswordAuthentication yes
KbdInteractiveAuthentication yes
## Anyone can use passwords from the local network
Match Address 127.0.0.0/8
PasswordAuthentication yes
KbdInteractiveAuthentication yes
## Don't allow user: ansible except from protected networks.
Match Address *:!::1,!127.0.0.0/8
DenyUsers ansible
I can count on one hand the number of times I have joined a company that allows password authentication from the Internet that has /etc/login.defs set up to enforce any complexity requirements or rotation policy.
So sure, if everyone does everything right it’s fine. That approach scales terribly though. Requiring SSH keys reduces the concern that someone is being lazy.
Now allowing SSH over the internet directly to your servers is still a bad idea regardless of auth process because SSH doesn’t have a way to check whether the key has a passphrase set, so you still want something like a bastion where you can enforce 2FA.
There are a lot of ways you can enforce MFA through SSH. For example, there are PAM modules you can add to the stack for SSH that enforce additional challenge response steps.
You can also refuse to add an SSH key unless it’s been attested as having been generated inside a HSM like a Yubikey. It’s easy to configure the SSH server so that it gets authorised keys from a root-controlled file or lookup program instead of a file each user controls.
It’s not generally my preference, but you can also set up an SSH CA. The CA you select can use whatever authentication you like, when generating short lived certificates for users to use to log in.
Sure, I understand that. My point was more that you typically don’t want to set all of that up on every server in a fleet. So to the question of “should I be able to ssh into application servers from the internet” the answer is still probably no, even with SSH keys only.
One problem I have with SSH password authentication is:
I have to type in the password of my local account All. The. Time. Every time I want to install packages. Every time I want to edit a config file. Every time I want to clear out a Go module cache directory (because Go marks modules as read-only. Yes I could’ve done a recursive chmod before rm, but sudo rm -rf is easier). Every time I want to install software updates. Every time I want to write an image to an SD card or USB drive. Every time I want to mount something and don’t want to/can’t use the standard auto mount functionality. Every time I come back after having been away for a few minute. Every time I boot the machine (and I turn off e.g my desktop every night).
I also generally trust anyone with physical or remote access to those machines: they’re mostly just friends and family. The risk of them trying to find my password in a leaked passwords database or brute-force crack my password and then doing something nefarious is extremely small.
As a result, I use a weak, easily typed password on these machines. The cost/benefit of using a stronger one which I have to grab from a password manager every time I need to enter the local account password just isn’t worth it.
All of this would go out the window if I enabled SSH password authentication; I’d have to switch to a much more secure password that’s strong enough to withstand being exposed to the public internet, which would be a significant convenience loss.
You don’t need to authenticate with your password to do any of those things if you configure your machine appropriately. You can use biometrics, or a FIDO2 token, or a PIN (different from your password), or one of the various SSO solutions, or even all options at the same time.
Yes, it’s a PITA to configure these on Linux, on Solaris it was a breeze.
I am not going to spend time setting up different hardware tokens on all my machines (Touch ID on Macs, some Ubikey thing probably on non-Macs) and then figure out how to forward biometric authentication to like 5 different servers running different versions of different operating systems when I interact with them over SSH.
Roughly nobody else will do that either.
That means that, yes, I (and pretty much everyone else) do need to authenticate with passwords to do those things I mentioned.
In many of the cases you could instead ssh to root using public key authentication with ssh-agent. Then you only need to type your password on unlock or reboot.
So you mean making my local account have a complex and unique password but enabling SSH root login and using ssh root@localhost as a sort of faux sudo? I guess that would work for a lot of stuff, but not for the numerous scripts which expect to be run as some non-root user and use sudo to gain root as necessary… Seems like a “solution” to a non-problem that introduces more issues than it’s worth
The non-problem is that you were complaining at length about having to type your password a lot, and how that encouraged you to use an insecure password. (I should add that to my list of issues with sudo, it’s not one I remember seeing before.)
I was thinking the same thing. It does seem a bit awkward in $CURRENT_YEAR that we still have services that use your “local account password” to protect access to your machine over the internet.
If you really want to enable password authentication, maybe you can set up a new user with a secure password specifically for access over the internet, whitelist only that one for SSH, and then set up an SUID script that allows that specific user only to automatically switch to your local account in the ~/.profile?
Taking a step in the opposite direction, certificate-based authentication is also a thing for people or orgs whose threat models require it. More of a hassle to initially set up, but less of a hassle to actually use (as you don’t need to store the user’s certificates on the machines they’re connecting to). In my opinion, provides far superior security than SSH keys. Managing cert-based auth is made easier by using some sort of automation product or managed access control, ie Teleport or similar.
I don’t allow password login to my real accounts, but each of my machines has a secret backup user with a unique and complex password which I’ve configured sshd to allow password login to. The passwords are impossible to brute force, and cannot be leaked from elsewhere because every password in unique. The only way they could get the passwords is by pwning my password manager, in which case I’m already screwed and they probably have a few of my SSH keys too.
If you need your password manager to access the passwords of the backup accounts, why not just use ssh keys? Or are you memorizing those complex unique backup passwords somehow?
I should probably go update my 2048-bit RSA keys on all my systems to use those new-fangled elliptic curve keys. Some services are already refusing to support 2048-bit RSA..
I agree that a strong password mitigates most of the risks from password authentication. The ability to SSH in from anywhere is good (but you’ll want to rotate your password after that).
But once you have multiple hosts, you’re remembering multiple passwords, which strains the amount of entropy you can produce as a person. If you use key auth only, you could have one shared password and it could be 123456 and you’d be fine (just in terms of SSH, that is.)
I always disallow password auth on production servers (and any other machines in the same network). Furthermore, I shut down all ports by default, and open only the specific ports that each production server needs, so even if you’re on (or ssh’d into) a server, you can’t ssh from there to another server (with only a few exceptions). The first rule of real security is to disallow everything by default (including not even installing any software that is not required), and only install/enable that which is necessary. Generally, ssh access shouldn’t be coming through the front door, either; it should be coming through a separate ingress network, with security and monitoring (including auditability) designed for logging and monitoring superuser-style access, since that’s what ops often are.
huh. well, I don’t agree with it, but it’s a well-structured argument.
I just… don’t find any of these proposed downsides to be particularly arduous, and when I look at the constant inbound traffic that’s always trying to brute-force something or other, I see no reason to take my chances. the article does a pretty good job of enumerating many of the ways I could be inadvertently exposed; I just disagree that those things aren’t a problem, in part because I don’t trust myself to be hypervigilant about every single one of them at all times.
I’ve had password-based ssh login turned off entirely on all my machines for something like twenty years now, for both internet and LAN traffic. it’s fine. you learn pretty quickly to not lose your ssh key - which you should be making sure to learn no matter what, because if you leave a copy of the file somewhere and forget it’s there, you have unexpected attack surface.
“somewhat” cursed: Disable password authentication. If you have password reasons, leverage that x25519 private keys can be (almost) any 32 byte random string and kPub generation cheap/fast. Sha256(password) -> your private key.
I’m genuinely curious: how is a private key generated from a password more secure than the password? Is it just because no one is brute-forcing ssh private keys in that way yet or is there some cryptographic reason?
I make no comments on the overall soundness of the idea, but I think it addresses the password snooping issue I mentioned in https://lobste.rs/s/gvo8fy/thoughts_on_having_ssh_allow_password#c_nkj4ho .
Pubkey auth doesn’t exist the key to the server.
It’s not more secure in a traditional sense, the point is the password convenience without having the server expose password authentication as a supported method so any password guessers will leave it alone. In the nontraditional sense there are services that enforce public key authentication even though you don’t want to bother to supply it with one for other reasons.
I can think of one very public source code host which enforce public key authentication that I have negative trust in, where public keys can be scraped so you can start scanning the internet and see if they reappear elsewhere.
I believe this is all of them, no? GitHub, GitLab, Codeberg, and Gitea all seem to have public keys available at
/username.keys.If public keys can be scraped…
You’ve just given anyone who is aware that people might be doing this free reign to attempt to brute force your password hashes haven’t you?
Worse, I don’t think you can salt this scheme? So they can probably build a rainbow table esque structure and brute force everyone using this scheme simultaneously.
This is only true if you use the same SSH keypair for those services as you use for regular logins. My personal view is that you definitely shouldn’t, because attackers can use those public keys to probe for various things against your hosts (eg, a SSH client can find out if a SSH server accepts a given public key for authentication for a particular account).
(I’m the author of the linked-to article, but the information disclosures of known public keys is a completely different topic. In some circles it’s well known; see eg https://words.filippo.io/dispatches/whoami-updated/ )
Sure, you can reduce the damage by only using this keypair for certain things.
If someone can reverse my GitHub push private key though… that would be pretty bad all by itself.
I should have expanded the explanation of my view: for Github push keys and other things like it, I don’t think you need to go to the ‘generate private key through hashing’ approach. In most environments you’re extremely unlikely to wind up needing to push from a random machine that has not already been set up for this, so you don’t need to be able to produce a keypair from a seed phrase you can memorize. The only keypairs you need to be able to produce that way are keypairs you’d use from not set up machines.
(You can imagine scenarios where you might need to push something to make an urgent production deployment from an extremely minimally set up machine, but it’s a question of trade offs and how likely things are. Even then you might be better off with some kind of ‘break glass’ SSH keypair that is pre-deployed in encrypted form and maybe has to be enabled on the forge side before you can use it.)
There’s more than one hash function to chose from. You can run n number of rounds of it.
Everyone has different security and convenience models, and this seems reasonable. AllowUser and strong passwords can work.
One risk the post doesn’t address is that if a server you SSH into with a password is compromised, they can snoop your password and use that to escalate access to other machines that have the same password.
Perhaps the author has different passwords for each machine, in which case he has a great memory, or a password manager. The latter doesn’t seem to satisfy “you might need to log in to your account in a situation where you don’t have your keypair available” any more than SSH keys.
I’m fortunate enough to be able to only access from my personal machines, and so restrict access behind Wireguard, and use FIDO2 SSH keys (laptop/desktop/keyring) to avoid having to deal with passwords.
And if a password manager does satisfy that concern, you can put an ssh keypair into most of them.
I’m the author of the linked-to article. In my situation at work we effectively have a single security domain across all of our servers, with common passwords (it’s an old fashioned Unix login server style environment). My work desktop is deliberately not in this security domain and so uses a different password, but the net effect is that I only have a few passwords to memorize for work, not one per server.
(I also mostly use SSH keypair authentication, with encrypted keys that are loaded into the SSH agent on my desktop but purged when I lock my screen and step away from it, which is a bit complex to arrange but conveniently magical.)
To be fair, you probably have your password manager on your phone and can easily read the password off of that and type it in directly, whereas this isn’t feasible with a private key. Of course, you could transfer the private key over USB or something, but that’s generally a hassle unfortunately.
OP has a somewhat uncommon use case these days (and he admits it): he’s a sysadmin in a large organisation. I imagine, he roams his domain and might need to access any machine within it from any other. Like we used to. Most people these days never touch a keybord that is not attached to their laptop. So they, obviously, always have access to their ssh keys. In this set up it’s only prudent to disallow password login just to reduce attack surface. OP correctly point out that username-password access involves a lot of machinery (including all 9 circles of PAM hell, and bonus circles of third-party PAM plugins). There’s simply no reason for most people to learn all that to make it secure when they don’t use passords at all.
From my own network, passwords are okay under limited circumstances. From the rest of the internet, is very strongly prefer public keys or yubikey one-time passwords. Once you go down the yubikey route, a great enhancement is to generate a keypair on your yubikey and, using gnupg for your ssh keychain, use that key’s public side for ssh. I’m sure that this works for other hardware tokens too.
What shouldn’t get lost here is that for those using OpenSSH,
Matchblocks in your/etc/ssh/sshd_configmake doing this fairly simple:Match blocks are described here: https://man.openbsd.org/sshd_config.
I can count on one hand the number of times I have joined a company that allows password authentication from the Internet that has /etc/login.defs set up to enforce any complexity requirements or rotation policy.
So sure, if everyone does everything right it’s fine. That approach scales terribly though. Requiring SSH keys reduces the concern that someone is being lazy.
Now allowing SSH over the internet directly to your servers is still a bad idea regardless of auth process because SSH doesn’t have a way to check whether the key has a passphrase set, so you still want something like a bastion where you can enforce 2FA.
There are a lot of ways you can enforce MFA through SSH. For example, there are PAM modules you can add to the stack for SSH that enforce additional challenge response steps.
You can also refuse to add an SSH key unless it’s been attested as having been generated inside a HSM like a Yubikey. It’s easy to configure the SSH server so that it gets authorised keys from a root-controlled file or lookup program instead of a file each user controls.
It’s not generally my preference, but you can also set up an SSH CA. The CA you select can use whatever authentication you like, when generating short lived certificates for users to use to log in.
Sure, I understand that. My point was more that you typically don’t want to set all of that up on every server in a fleet. So to the question of “should I be able to ssh into application servers from the internet” the answer is still probably no, even with SSH keys only.
One problem I have with SSH password authentication is:
I have to type in the password of my local account All. The. Time. Every time I want to install packages. Every time I want to edit a config file. Every time I want to clear out a Go module cache directory (because Go marks modules as read-only. Yes I could’ve done a recursive chmod before rm, but
sudo rm -rfis easier). Every time I want to install software updates. Every time I want to write an image to an SD card or USB drive. Every time I want to mount something and don’t want to/can’t use the standard auto mount functionality. Every time I come back after having been away for a few minute. Every time I boot the machine (and I turn off e.g my desktop every night).I also generally trust anyone with physical or remote access to those machines: they’re mostly just friends and family. The risk of them trying to find my password in a leaked passwords database or brute-force crack my password and then doing something nefarious is extremely small.
As a result, I use a weak, easily typed password on these machines. The cost/benefit of using a stronger one which I have to grab from a password manager every time I need to enter the local account password just isn’t worth it.
All of this would go out the window if I enabled SSH password authentication; I’d have to switch to a much more secure password that’s strong enough to withstand being exposed to the public internet, which would be a significant convenience loss.
You don’t need to authenticate with your password to do any of those things if you configure your machine appropriately. You can use biometrics, or a FIDO2 token, or a PIN (different from your password), or one of the various SSO solutions, or even all options at the same time.
Yes, it’s a PITA to configure these on Linux, on Solaris it was a breeze.
I am not going to spend time setting up different hardware tokens on all my machines (Touch ID on Macs, some Ubikey thing probably on non-Macs) and then figure out how to forward biometric authentication to like 5 different servers running different versions of different operating systems when I interact with them over SSH.
Roughly nobody else will do that either.
That means that, yes, I (and pretty much everyone else) do need to authenticate with passwords to do those things I mentioned.
In many of the cases you could instead ssh to root using public key authentication with ssh-agent. Then you only need to type your password on unlock or reboot.
Doing everything in a root shell instead of doing things as a normal user and using sudo when necessary is a great tip for security, thanks 👍
I didn’t say “everything”. What do you think is the security problem? ssh has a much better track record than sudo.
Pretty sure they meant
ssh root@localhost echoinstead ofsudo echo.Being passive aggressive doesn’t help anything.
So you mean making my local account have a complex and unique password but enabling SSH root login and using ssh root@localhost as a sort of faux sudo? I guess that would work for a lot of stuff, but not for the numerous scripts which expect to be run as some non-root user and use sudo to gain root as necessary… Seems like a “solution” to a non-problem that introduces more issues than it’s worth
The non-problem is that you were complaining at length about having to type your password a lot, and how that encouraged you to use an insecure password. (I should add that to my list of issues with sudo, it’s not one I remember seeing before.)
No, the non-problem is that password auth is disabled. That’s ultimately what all this is a proposed solution to.
I was thinking the same thing. It does seem a bit awkward in $CURRENT_YEAR that we still have services that use your “local account password” to protect access to your machine over the internet.
If you really want to enable password authentication, maybe you can set up a new user with a secure password specifically for access over the internet, whitelist only that one for SSH, and then set up an SUID script that allows that specific user only to automatically switch to your local account in the ~/.profile?
Taking a step in the opposite direction, certificate-based authentication is also a thing for people or orgs whose threat models require it. More of a hassle to initially set up, but less of a hassle to actually use (as you don’t need to store the user’s certificates on the machines they’re connecting to). In my opinion, provides far superior security than SSH keys. Managing cert-based auth is made easier by using some sort of automation product or managed access control, ie Teleport or similar.
I don’t allow password login to my real accounts, but each of my machines has a secret backup user with a unique and complex password which I’ve configured sshd to allow password login to. The passwords are impossible to brute force, and cannot be leaked from elsewhere because every password in unique. The only way they could get the passwords is by pwning my password manager, in which case I’m already screwed and they probably have a few of my SSH keys too.
If you need your password manager to access the passwords of the backup accounts, why not just use ssh keys? Or are you memorizing those complex unique backup passwords somehow?
I would bet it’s more an escape hatch situation. “Oh no, ssh 10.5 disabled ssh-rsa keys of my type and now I have to log in manually somehow.”
… new paranoia unlocked.
I should probably go update my 2048-bit RSA keys on all my systems to use those new-fangled elliptic curve keys. Some services are already refusing to support 2048-bit RSA..
I agree that a strong password mitigates most of the risks from password authentication. The ability to SSH in from anywhere is good (but you’ll want to rotate your password after that).
But once you have multiple hosts, you’re remembering multiple passwords, which strains the amount of entropy you can produce as a person. If you use key auth only, you could have one shared password and it could be
123456and you’d be fine (just in terms of SSH, that is.)I always disallow password auth on production servers (and any other machines in the same network). Furthermore, I shut down all ports by default, and open only the specific ports that each production server needs, so even if you’re on (or ssh’d into) a server, you can’t ssh from there to another server (with only a few exceptions). The first rule of real security is to disallow everything by default (including not even installing any software that is not required), and only install/enable that which is necessary. Generally, ssh access shouldn’t be coming through the front door, either; it should be coming through a separate ingress network, with security and monitoring (including auditability) designed for logging and monitoring superuser-style access, since that’s what ops often are.