Introduction:
Hello everyone, Today we want to share some information about the new tool which we found. Nowadays we use this tool to manage and scale our Terraform projects. We also use this tool to generate Terraform code to build infrastructure on AWS.
If you've been on the lookout for a solution to streamline your infrastructure management tasks, you're in the right place. With no further ado, allow us to introduce you to this impressive tool - "Terramate".
What is Terramate?
Terramate is a ground-breaking tool that adds capabilities such as stacks, orchestration, code generation and data sharing to Terraform.
It also helps to keep your code DRY—that's 'Don't Repeat Yourself' for the uninitiated. It empowers you to share data and code across all stacks, effectively slashing code duplication. This is pretty helpful when you work with multiple Terraform state backends. But that's not all.
Terramate also orchestrates your stacks so that not all stacks are executed all the time, but only stacks that have changed within a specific pull request (PR) will be executed (planned/applied).
Use Case
"We know, we've introduced some technical terms like 'stacks' and 'code duplication'. You might be thinking, 'What are these, and why should I care when I can just perform my daily tasks on Terraform?'
Don't worry, we're here to help. Understanding these terms and concepts will save you time and effort every time you use Terraform in the future. We promise to keep things simple and focus only on the most important features of Terramate in this blog.
Let's start by understanding how Terraform works before even jumping on to Terraform (assuming you have a basic knowledge of Terraform). Terraform allows you to create infrastructure in any cloud provider like AWS, right? To connect with a cloud provider and create infrastructure on their servers (which we call the cloud), we add a 'provider block' in the Terraform configuration file. After that, we run 'terraform init' to initialize the backend. This step is simply asking for permission to create infrastructure on our behalf in the cloud.
When we want to create an infrastructure in the cloud, we write Terraform code and execute 'terraform plan' and 'terraform apply' to build our infrastructure. But what happens when the infrastructure code becomes complex and voluminous?
Executing the 'terraform apply' phase might consume a substantial amount of time, and even minor modifications to the infrastructure code can extend the duration significantly. This phenomenon occurs because Terraform initiates API requests for all resources governed within a given Terraform Stack. These requests are used to verify if your Terraform code remains consistent with the resources currently deployed in your selected cloud provider; it happens when you have a bigger infrastructure.
This is where Terramate comes in. One of its key features is the ability to create 'stacks'..." let's read about the stack.
What are Stacks in Terramate?
Imagine stacks as piles of bricks, each brick consisting of a specific code for building a certain kind of infrastructure, and the entire pile of bricks helps in building the entire wall which in this case is our entire infrastructure.
But how is this different from merely dividing your code into smaller parts?
The distinction lies in how Stacks function. It is different because Stacks allow us to run terraform commands for each stack and that could be different infra like network could be one stack, EC2 could be another, storage another. This simply means that if I want to make a small change in my Terraform code and if it is under a Stack it allows us to make changes as we can run the Terraform command for that particular block only. Through isolated and smaller units, the execution becomes really fast.
Also, it's worth mentioning that stacks help us break big Terraform projects into smaller pieces (a stack for AWS Lambda function, a stack for VM, a stack for S3- Bucket …). This lets us only work on what's changed and do it all at the same time. This is great because it makes our pipeline run much quicker, which probably means it costs less, especially if you use CI/CD tools like GitHub Actions that charge per minute. It also lets different developers easily work on different parts of the project at the same time without getting in each other's way.
Isn’t it gonna save you time and effort? Of course, it will.
Moreover, Stacks help reduce the risk of 'apply' failures. As your code is broken down into smaller units, and you're executing your code in these smaller units, the likelihood of entire code failure decreases.
Okay now we have talked a lot about stacks let us show how to create multiple stacks in a terraform project.
Step 1: Create a new repo in your GitHub account and clone it in your local and open in your favorite IDE.
Step 2: Create a folder in which you want to start your Terramate project.
Here we are creating a folder with the name “Infrasity”. You can choose any name according to your preference. You can also work with the root directory of your project for this.
Step 3: (for non git-repo) Create a file named “terramate.tm.hcl” and add the following code.
/* File name: terramate.tm.hcl */
terramate{
config{
}
}
For git-repo , you don’t need this file.
Step 4: Creating multiple stacks
To create multiple stacks just type this command “terramate create <stack name>”
The stack name can be of your choice(a stack is a directory with a stack.tm.hcl file inside).
You can take help from the below pictures:-
Here we have created a stack with the name env-1 and to create it we have used the simple command “terramate create env-1"
As above here we have created a stack with the name env-2.
Again as above here we have created a stack with the name env-3.
And that's it you have learned about how to create multiple stacks in any Terraform project.
For your help, we have also shared what your directory structure should look like after creating multiple stacks.
We hope we have tried to explain to you what stacks are and how to create multiple stacks in the most simple manner but you know what we are more excited about showing you how to make a stack and how to generate code in it.
So without wasting any time let us share with you how to make a stack and how to generate an S3 Bucket Terraform resource through Terramate.
Note: In this blog, we are only gonna generate code in a single environment, and in the next blog we will share how to generate Terraform code in multiple environments.
So let's start the step-by-step guide for creating a stack and generating S3 resources in it.
Step 1: Continuing with the previous repo, and opening your favorite IDE.
Step 2: Creating a stack
To create a stack just type this command “terramate create <stack name>”
The folder name can be of your choice.
Because we are generating an s3 code, I have given my folder name to aws-s3
# file: aws-s3/stack.tm.hcl
stack {
name = "aws-s3"
description = "aws-s3"
id = "f9dd5336-cda1-447b-b495-87e273591240"
}
id is always unique for each stack.
A folder aws-s3 and a file stack.tm.hcl will be generated.
In the stack.tm.hcl “name and description will be assigned to your stack (the value of this name and description will be of your folder name) and also a random ID will be assigned to your stack.
That's it you have generated your first stack.
Step 3: Generating code for the s3 bucket
To generate code for the s3 bucket in aws you have to modify your stack file by adding the HCL code which will generate a main.tf file in the folder aws-s3 which will contain the code for the s3 bucket.
So after you have added the HCL code in aws-s3/stack.tm.hcl should look like this
# file: aws-s3/stack.tm.hcl
stack {
name = "aws-s3"
description = "aws-s3"
id = "f9dd5336-cda1-447b-b495-87e273591240"
}
generate_hcl "main.tf" {
content {
resource "aws_s3_bucket" "infraSity" {
name = "infrasity-aws-s3"
acl = "private"
versioning {
enabled = true
}
}
}
}
Step 4: Now run Terramate to generate S3 bucket code with the command “terramate generate” in your terminal and see the magic of Terramate for you.
You should see a result like this, if not please check if you have made any mistake while following any of the above steps.
As a result the main.tf file will be generated in the stack folder and it should have our code for the s3 bucket.
// TERRAMATE: GENERATED AUTOMATICALLY DO NOT EDIT
resource "aws_s3_bucket" "infraSity" {
acl = "private"
name = "infrasity-aws-s3"
versioning {
enabled = true
}
}
In case you are finding errors while following the above steps we have shared the structure of my directory because Terramate uses a hierarchical structure approach and it is very important because without that it won't work.
Hope this one was informative:
In conclusion, discovering Terramate has transformed our experience with Terraform. It's not just about generating Terraform code anymore; it's about doing so efficiently and manageably.
The concept of Stacks has been a game-changer. It saves considerable time and effort during 'terraform apply', significantly reducing the risk of failures. By creating multiple stacks, we've managed to create different environments within one Terraform project, giving our code a more organized and manageable structure.
We understand that managing complex infrastructure can be tough, and making small changes without a lengthy process is important. That's why we recommend checking out Terramate and its examples.
Terramate has simplified our Terraform experience, and we believe it can do the same for you. So why wait? Give Terramate a try and improve your Terraform experience today!
Also join our Discord channel for all future Updates about Terramate.