Managing infrastructure as code (IaC) is a cornerstone of modern DevOps practices, and Terraform has become a go-to tool for this purpose. However, as infrastructure grows in complexity, so does the need for efficient management of multiple environments, such as development, staging, and production. This is where Terragrunt comes in. Terragrunt is a thin wrapper around Terraform that provides extra tools for keeping configurations DRY (Don't Repeat Yourself) and managing multiple environments more effectively.

In this article, we will dive deep into best practices for managing multiple environments with Terragrunt. Whether you are new to Terragrunt or looking to refine your usage, this guide will provide valuable insights and practical tips.

What is Terragrunt?

Terragrunt is an open-source tool that extends Terraform by offering features to manage complex infrastructure setups. It addresses some of the pain points associated with Terraform, such as the handling of multiple environments, by introducing a hierarchical configuration system and DRY principles. 

Before proceeding, you might also want to read this introduction to Terragrunt.

Key Features of Terragrunt

1. Keep your Terraform code DRY: Terragrunt allows you to define reusable Terraform modules, avoiding duplication.
2. Manage multiple environments: It simplifies the management of multiple environments through hierarchical configurations.
3. Automated dependency management: Terragrunt can automatically apply and destroy resources in the correct order.
4. Built-in locking and state management: It integrates with remote state backends to ensure safe and consistent state management.

Why Use Terragrunt for Multiple Environments?

Managing multiple environments in Terraform can become cumbersome due to the need to duplicate code and configurations. Terragrunt simplifies this by providing a structured way to define and manage these environments, ensuring consistency and reducing maintenance overhead.

Best Practices for Managing Multiple Environments with Terragrunt

1. Use a Hierarchical Directory Structure

A well-organized directory structure is crucial for managing multiple environments efficiently. Terragrunt recommends a hierarchical structure where each environment is a separate folder. Here is an example:

infrastructure-live
├── dev
│   ├── us-east-1
│   │   └── app
│   │       └── terragrunt.hcl
├── staging
│   ├── us-east-1
│   │   └── app
│   │       └── terragrunt.hcl
└── prod
    ├── us-east-1
        └── app
            └── terragrunt.hcl

In this structure:

• infrastructure-live is the root directory containing environment-specific configurations.
• dev, staging, and prod are directories for each environment.
• us-east-1 is a region-specific subdirectory.
• Each environment/region combination contains a terragrunt.hcl file with the specific configuration for that context.

2. DRY Principle with Terragrunt Configuration

Leverage Terragrunt’s ability to include configurations to avoid duplication. You can create a common configuration file and reference it in environment-specific configurations.

Common configuration (common.hcl):

inputs = {
  aws_region = "us-east-1"
  environment = "common"
}

Environment-specific configuration (terragrunt.hcl):

include {
  path = find_in_parent_folders("common.hcl")
}

inputs = {
  environment = "dev"
}

In this setup:

• common.hcl defines common variables such as aws_region.
• Each environment-specific terragrunt.hcl file includes the common configuration and can override or add additional inputs specific to that environment.

3. Use Terragrunt for Remote State Management

Managing state files is critical in Terraform. Terragrunt can help by automatically configuring remote state storage. Define a remote state configuration in a common file and include it in your environment-specific files.

Remote state configuration (remote_state.hcl):

remote_state {
  backend = "s3"
  config = {
    bucket = "my-terraform-state"
    key    = "${path_relative_to_include()}/terraform.tfstate"
    region = "us-east-1"
  }
}

Include remote state in environment-specific configuration (terragrunt.hcl):

include {
  path = find_in_parent_folders("remote_state.hcl")
}

In this example:

• remote_state.hcl specifies the S3 bucket, key, and region for the remote state.
• Each environment-specific terragrunt.hcl includes this remote state configuration, ensuring all environments use a consistent state backend.

4. Automate Workflows with Terragrunt

Terragrunt can automate common Terraform workflows, such as applying and destroying resources. Use hooks to run scripts before or after Terraform commands.

Example hook configuration:

terraform {
  before_hook "before_apply" {
    commands = ["apply"]
    execute  = ["bash", "-c", "echo Running pre-apply script"]
  }

  after_hook "after_apply" {
    commands = ["apply"]
    execute  = ["bash", "-c", "echo Running post-apply script"]
  }
}

In this example:

• before_hook runs a script before the apply command.
• after_hook runs a script after the apply command.

5. Manage Dependencies with Terragrunt

If your infrastructure has dependencies between modules (e.g., a network module needed by an application module), you can define these dependencies in Terragrunt to ensure they are applied in the correct order.

Example dependency configuration:

dependencies {
  paths = ["../network"]
}

In this configuration:

• The dependencies block specifies that the current module depends on the network module located in a sibling directory.

6. Secure Sensitive Data

Use environment variables or encrypted files to manage sensitive data such as API keys or database passwords. Avoid hardcoding sensitive information in your Terragrunt configurations.

Example using environment variables:

inputs = {
  db_password = get_env("DB_PASSWORD", "default_password")
}

In this example:

• The db_password input is retrieved from an environment variable. If the variable is not set, a default value is used.

7. Regularly Update Terragrunt and Terraform

Ensure you are using the latest versions of Terragrunt and Terraform to benefit from new features, improvements, and security patches. Regular updates can prevent compatibility issues and enhance the stability of your infrastructure.

Use Cases and Practical Scenarios

Scenario 1: Managing a Multi-Region Application

Suppose you have an application that needs to be deployed across multiple AWS regions. You can use Terragrunt to manage the infrastructure for each region while keeping your Terraform configurations DRY.

Directory structure:

infrastructure-live
├── dev
│   ├── us-east-1
│   │   └── app
│   │       └── terragrunt.hcl
│   ├── us-west-2
│   │   └── app
│   │       └── terragrunt.hcl
├── staging
│   ├── us-east-1
│   │   └── app
│   │       └── terragrunt.hcl
│   ├── us-west-2
│   │   └── app
│   │       └── terragrunt.hcl
└── prod
    ├── us-east-1
    │   └── app
    │       └── terragrunt.hcl
    ├── us-west-2
        └── app
            └── terragrunt.hcl

Common configuration (common.hcl):

inputs = {
  app_name = "my-app"
  aws_region = get_env("AWS_REGION")
}

Environment-specific configuration (dev/us-east-1/app/terragrunt.hcl):

include {
  path = find_in_parent_folders("common.hcl")
}

inputs = {
  environment = "dev"
}

In this scenario:

• You define a common configuration that includes inputs like app_name and aws_region.
• Each environment/region-specific terragrunt.hcl file includes the common configuration and sets the environment input.

Scenario 2: Managing Dependencies in a Microservices Architecture

In a microservices architecture, services often depend on shared resources such as a VPC or database. Terragrunt can help manage these dependencies.

Directory structure:

infrastructure-live
├── dev
│   ├── vpc
│   │   └── terragrunt.hcl
│   ├── database
│   │   └── terragrunt.hcl
│   └── app
│       └── terragrunt.hcl

VPC configuration (dev/vpc/terragrunt.hcl):

terraform {
  source = "git::ssh://git@github.com/my-org/terraform-modules.git//vpc"
}

Database configuration (dev/database/terragrunt.hcl):

dependencies {
  paths = ["../vpc"]
}

terraform {
  source = "git::ssh://git@github.com/my-org/terraform-modules.git//database"
}

App configuration (dev/app/terragrunt.hcl):

dependencies {
  paths = ["../vpc", "../database"]
}

terraform {
  source = "git::ssh://git@github.com/my-org/terraform-modules.git//app"
}

In this scenario:

• The vpc module is defined first and used as a dependency by both the database and app modules.
• The database module depends on the vpc module.
• The app module depends on both the vpc and database modules.

Scenario 3: Automating Infrastructure Provisioning

You want to automate the provisioning of your infrastructure to ensure consistency and reduce manual errors. Use Terragrunt hooks to run pre-apply and post-apply scripts.

Example hook configuration (dev/app/terragrunt.hcl):

terraform {
  before_hook "before_apply" {
    commands = ["apply"]
    execute  = ["bash", "-c", "echo Running pre-apply script"]
  }

  after_hook "after_apply" {
    commands = ["apply"]
    execute  = ["bash", "-c", "echo Running post-apply script"]
  }
}

In this example:

• The before_apply hook runs a script before the apply command, which can be used for tasks such as validating configurations or notifying stakeholders.
• The after_apply hook runs a script after the apply command, which can be used for tasks such as sending notifications or running integration tests.

Final Thoghts on Terragrunt

Terragrunt offers powerful features that can significantly streamline the management of multiple environments in Terraform. By following best practices such as maintaining a hierarchical directory structure, adhering to the DRY principle, automating workflows, and managing dependencies, you can achieve a more efficient and maintainable infrastructure setup.

Incorporating Terragrunt into your DevOps toolkit will not only enhance your ability to manage complex infrastructure but also ensure that your Terraform code remains clean, DRY, and easy to manage across different environments.

For more detailed information, you can refer to the official Terragrunt documentation.

By following these best practices and applying them to practical scenarios, you can leverage the full potential of Terragrunt, making your infrastructure management more robust and scalable.

Happy Clouding !!!