1. Hardcoded credentials are a plague. You should consider tagging all of your secrets so that they're easier to scan for. Github automatically scans for secrets, which is great.
2. Jenkins is particularly bad for security. I've seen it owned a million and one times.
3. Containers are overused as a security boundary and footguns like `--privileged` completely eliminate any boundary.
4. Environment variables are a dangerous place to store secrets - they're global to the process and therefor easy to leak. I've thought about this a lot lately, especially after log4j. I think one pattern that may help is clearing the variables after you've loaded them into memory.
Another I've considered is encrypting the variables. A lot of the time what you have is something like this:
Secret Store -> Control Plane Agent -> Container -> Process
Where secrets flow from left to right. The control plane agent and container have full access to the credentials and they're "plaintext" in the Process's environment.
In theory you should be able to pin the secrets to that process with a key. During your CD phase you would embed a private key into the process's binary (or a file on the container) and then tell your Secret Manager to use the associated public key to transmit the secrets. The process could decrypt those secrets with its private key but they're E2E encrypted across any hops between the Secret Store and Process and they can't be leaked without explicitly decrypting them first.
> Environment variables are a dangerous place to store secrets - they're global to the process and therefor easy to leak.
The two real problems with environment variables are:
1. Environment variables are traditionally readable by any other process in the system. There are settings you can do on modern kernels to turn this off, but how do you know that you will always run on such a system?
2. Environment variables are inherited to all subprocesses by default, unless you either unset them after you fork() (but before you exec()), or if you take special care to use execve() (or similar) function to provide your own custom-made environment for the new process.
A foreign process’s environment variables are only readable if the current UID is root or is the same as the foreign process’s ID. As user joe I can’t see user andrea’s process’s envvars.
> 1. Environment variables are traditionally readable by any other process in the system. There are settings you can do on modern kernels to turn this off, but how do you know that you will always run on such a system?
I think that this would require being the same user as the process you're trying to read. Access to proc/pid/environ should require that iirc. You can very easily go further by restricting procfs using hidepid.
And ptrace restrictions are pretty commonplace now I think? So the attacker has to be a parent process or root.
> 2. Environment variables are inherited to all subprocesses by default, unless you either unset them after you fork() (but before you exec()), or if you take special care to use execve() (or similar) function to provide your own custom-made environment for the new process.
Yeah, this goes to my "easy to leak" point.
Either way though you're talking about "attacker has remote code execution", which is definitely worth considering, but I don't think it matters with regards to env vs anything else.
Files suffer from (1), except generally worse. File handles suffer from (2) afaik.
Embedding the private key into the binary doesn't help too much if the attacker is executing with the ability to ptrace you, but it does make leaking much harder ie: you can't trick a process into dumping cleartext credentials from the env just by crashing it.
I think your agent idea is good. I'd want to add in a way for the agent to detect when a key is used twice (to catch other processes using the key) or when the code you wrote didn't get the key directly (to catch proxies), and then a way to kill or suspend the process for review. Would be pretty sweet.
Start doing security what way, exactly? I defined a threat model and a mitigation. And it's pretty straightforward - a single keypair that ties environment variables to their deployment.
The article you linked to is about signing. It doesn't solve "I need to put an AWS key into the environment of a process".
Some thoughts:
1. Hardcoded credentials are a plague. You should consider tagging all of your secrets so that they're easier to scan for. Github automatically scans for secrets, which is great.
2. Jenkins is particularly bad for security. I've seen it owned a million and one times.
3. Containers are overused as a security boundary and footguns like `--privileged` completely eliminate any boundary.
4. Environment variables are a dangerous place to store secrets - they're global to the process and therefor easy to leak. I've thought about this a lot lately, especially after log4j. I think one pattern that may help is clearing the variables after you've loaded them into memory.
Another I've considered is encrypting the variables. A lot of the time what you have is something like this:
Secret Store -> Control Plane Agent -> Container -> Process
Where secrets flow from left to right. The control plane agent and container have full access to the credentials and they're "plaintext" in the Process's environment.
In theory you should be able to pin the secrets to that process with a key. During your CD phase you would embed a private key into the process's binary (or a file on the container) and then tell your Secret Manager to use the associated public key to transmit the secrets. The process could decrypt those secrets with its private key but they're E2E encrypted across any hops between the Secret Store and Process and they can't be leaked without explicitly decrypting them first.