Terraform configurations often rely on secrets like API keys, passwords, and access credentials to automate infrastructure provisioning. These secrets are essential for interacting with cloud providers, databases, and other external systems. However, including secrets directly within Terraform code poses a significant security risk. Accidental exposure through code sharing or version control can compromise your infrastructure.

Managing secrets when provisioning infrastructure with terraform can be quite daunting. You have to think of ways to securely include your secret into your terraform configuration without necessarily exposing it externally.

Let's say you have a database to provision with terraform. How do you provide the credentials for your database without including them in source code and at the same time ensuring that the automated flow of terraform is not interrupted?

Ensuring the security and integrity of your infrastructure deployments requires careful management of secrets in Terraform. Secrets, like API keys and passwords, are essential for provisioning resources but should never be stored in plain text.

In reality, there are quite a number of options available to you to securely store sensitive data when writing terraform code.

They can include:

•  Using Environmental Variables

• Encrypting Secrets (KMS, SOPS)

•  Using Cloud Services (AWS secret manager)

1.  Environmental Variables

Environmental variables can be used to keep plain-text secrets out of your code by taking advantage of terraform's native support for environmental variables.
By simply using the TF_VAR_var feature of terraform, we can provide sensitive data without making them an intrinsic part of source code.

For example,to assign a password value of to our database using terraform:
 

# create RDS Postgres Instance
resource "aws_db_instance" "mydb" {
    db_name = "mydb"
    engine = "postgres"
    engine_version = "15"
    instance_class = "db.t4g.micro"
    allocated_storage = 10

    publicly_accessible = false
    skip_final_snapshot = true
    db_subnet_group_name = aws_db_subnet_group.private.name 

    username = "var.username"
    password = var.password

     depends_on = [ 
        aws_db_subnet_group.private
     ]
}

We can then provide the password value by executing the following Linux command
 

export TF_VAR_password="mypass123"
export TF_VAR_username="root"

This saves us the risk of having to include the value in source code or carelessly exposing this value.
What's more? you can prevent this command from showing up in the history command by properly configuring the Histcontrol environmental variable. When this is set correctly, commands that start with a space do not show up in the history. This way, we can execute the export command by preceding it with a space and thus prevent our secret from being exposed.

Now if you decide to use environmental variables, you must have a strategy for storing and managing your secrets. The most popular way of doing this is by using password managers.
Password managers like LastPass or  1Pass are efficient ways of storing and managing your secrets.

Advantages of this approach

• It is an easy-to-use solution.

• It integrates well with password managers (pass, lastpass 1pass etc).

• It keeps sensitive data out of your source code.

Disdvantages of this approach

• It defeats automation by inserting manual approaches into the terraform workflow.

• It is difficult to use in a collaborative environment.

• It is quite primitive and provides no security guarantees.

2.  Encrypting Secrets (KMS, SOPS)

This is a more advanced way of managing secrets in terraform.
In this approach, we create a file containing the secrets. This file is then encrypted and checked into version control along with our terraform configuration.

2a. KMS

To use AWS KMS to manage secrets:

Step 1. Create an AWS Customer Managed Key

NB: Make sure that your terraform user has sufficient IAM permissions to invoke this key.

Step 2. Use the AWS KMS encrypt command to encrypt your credentails file.
 

aws kms encrypt \
    --key-id <YOUR-CMK-KEY-ID> \
    --plaintext fileb://credentials.yml \
    --output text \
    --query CiphertextBlob | base64 \
    --decode > credentails.yml.encrypted

This will encrypt our create an encrypted file called credentials.yml.encrypted which will contain our credentials. We can now safely delete the credentials.yml file. The credentials.yml.encrypted file is secure enough to be checked into version control.

Step 3. Use the aws_kms_secrets terraform data source to decrypt this secret. Use terraform locals to refer to the decrypted secret. Finally, update your resource to make use of this value.
 

# Use AWS KMS to decrypt database credentials
data "aws_kms_secrets" "db_credentials" {
  secret {
    # ... potentially other configuration ...
    name    = "db_creds"
    payload = filebase64("./db_creds.yml.encrypted")
  }
}

# Use locals to grab the decrypted KMS key
locals {
    db_creds = yamldecode(data.aws_kms_secrets.db_credentials.plaintext["db_creds"])
}

# Create RDS Postgress instance
resource "aws_db_instance" "mydb" {
    db_name = "mydb"
    engine = "postgres"
    engine_version = "15"
    instance_class = "db.t4g.micro"
    allocated_storage = 10

    publicly_accessible = false
    skip_final_snapshot = true
    db_subnet_group_name = aws_db_subnet_group.private.name 

    username = local.db_creds.username
    password = local.db_creds.password

    depends_on = [ 
        aws_db_subnet_group.private
     ]
}

2b. SOPS
SOPS - short for Secret Operation s - is an open-source text file editor that encrypts/decrypts files automatically.
SOPS can easily integrate with AWS KMS, GCP KMS, hashicorp vault etc.
Here is a great resource to learn more about SOPS.

This approach is by far a more elegant, efficient and secure method of adding secrets and other sensitive data while provisioning infrastructure with terraform.

3.  Using Cloud Services (AWS secret manager)

AWS Secrets Manager is a cloud service offered by AWS which can help to securely manage, retrieve, and rotate database credentials, API keys, and other secrets throughout their lifecycles.

To use Secret Manager in Terraform:

• Create Your secret

• Reference this secret in your resource configuration:

# Use data source to reference secrets manager secret
data "aws_secretsmanager_secret_version" "credentials" {
    secret_id = "db_creds-v2"
}

# Use locals to grab the decrypted key from secret manager
locals {
    db_credentials = jsondecode(
        data.aws_secretsmanager_secret_version.credentials.secret_string
    )
}

# Create RDS Postgress instance
resource "aws_db_instance" "mydb" {
    db_name = "mydb"
    engine = "postgres"
    engine_version = "15"
    instance_class = "db.t4g.micro"
    allocated_storage = 10

    publicly_accessible = false
    skip_final_snapshot = true
    db_subnet_group_name = aws_db_subnet_group.private.name 

    username = local.db_credentials.username
    password = local.db_credentials.password

    depends_on = [ 
        aws_db_subnet_group.private
     ]
}

Again, make sure that your terraform user has the proper IAM permissions.

One major advantage of this method over AWS KMS is that you do not need to create a credentials file at all. This way, you can always modify your secrets without ever editing the source code!

Using AWS Secrets Manager to handle sensitive data in terraform is an elegant, secure and reliable strategy.

Key practices for managing secrets securely in Terraform:

• External Storage: Avoid storing secrets directly in your Terraform configuration files. Instead, use dedicated secret management services or encrypted external storage solutions.
• Secret Management Tools: Leverage tools like HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, or cloud provider-specific offerings. These services provide secure storage, access control, and auditability for your secrets.
• Environment Variables: Store secrets as environment variables during Terraform execution. This keeps them separate from your codebase and version control system.
• Terraform Backend Encryption: Utilize Terraform's backend configuration with encryption at rest. This safeguards your Terraform state, which might contain references to secrets.
• Least Privilege: Grant only the necessary permissions to Terraform processes and users to access secrets. This minimizes the potential damage if a breach occurs.
• Regular Rotation: Regularly rotate your secrets to reduce the window of vulnerability if compromised.

Benefits of Secure Secrets Management

Implementing robust secrets management practices in Terraform offers several advantages:

• Enhanced Security: By keeping secrets out of your code and version control, you significantly reduce the risk of unauthorized access and exposure.
  Improved Infrastructure Reliability: Secure secrets management ensures consistent and reliable infrastructure provisioning without compromising sensitive information.
• Simplified Collaboration: External storage enables teams to collaborate on infrastructure management without sharing sensitive secrets directly.
• Streamlined Workflows: Secret management tools can automate tasks like secret rotation and access control, streamlining your infrastructure automation processes.

By following these guidelines, you can effectively safeguard your secrets and strengthen the overall security of your Terraform-managed infrastructure.