> [The attacker] placed a trap which waited for any user to SSH into the jenkins user, which would then hijack any available forwarded SSH keys to try to add the attacker’s SSH key to root@ on as many other hosts as possible.
Can the system you log into via ssh just dump your forwarded PRIVATE key? That easily?
Or this was about ssh client being patched on jenkins box to add malicious keys wherever the devops ssh'd from jenkins box?
When you log into a host with SSH agent forwarding turned on, the private key data itself isn't available to the host you're logging into. However, when you try to SSH onwards from that host, agent forwarding means that the authentication handshake is forwarded through to the agent running on your client, which of course has access to your private keys.
So, even though the private key data itself isn't directly available to the host, any code running which can inspect the SSH_AUTH_SOCK environment variable of the session that just logged in can use that var to silently authenticate with other remote systems on your behalf.
If you've already found a list of candidate hosts (e.g. by inspecting ~/.ssh/known_hosts) then your malware can simply loop over the list, trying to log in as root@ (or user@) and compromising them however you like. Which is what happened here, by copying a malicious authorized_keys2 file with a malicious key onto the target hosts. You don't need to patch the ssh client; it's just ssh agent forwarding doing its thing. :|
There's another scary implication. If the user has set up SSH on their machines and added their own key as a valid key for logging in (for instance if you have 2 machines and you use the same SSH key for logging into everything), the attacker can log into the users own machine and compromise it. This makes remediation even more difficult because even after you have secured all the servers, the attacker can get back in through an affected user's machine.
Right. I covered this specifically in the writeup, because it's a use case that we have too. Our proposal is:
> If you need to regularly copy stuff from server to another (or use SSH to GitHub to check out something from a private repo), it might be better to have a specific SSH ‘deploy key’ created for this, stored server-side and only able to perform limited actions.
And this is the approach we're taking going forwards.
If the problem is that you only ever want to read from git when an admin is logged into the machine, i guess the safest bet would be to use a temporary deploy key (or temporarily copy the deploy key onto the machine until you've finished admining).
Forwarding all the keys from your agent is a recipe to end up pwned like we did, however.
Can the system you log into via ssh just dump your forwarded PRIVATE key? That easily?
Or this was about ssh client being patched on jenkins box to add malicious keys wherever the devops ssh'd from jenkins box?