Terraforming my path to AWS
I created my first AWS account on 2012 for training purposes but I closed it several years ago because I wasn’t using it at all. Now cloud computing is predominant and I want to learn more about it, and, maybe, get some certifications.
In Cloud Computing is something usual, and also a best practice, to use Infrastructure as Code. About three years ago I was in charge of deploying my company’s custom SIEM solution on Azure for a customer, and the customer requested it to be done using IaC. I didn’t know anything about IaC back then, and my first approach was to use powershell scripts created by copying directly from Microsoft’s Azure documentation and adapting them to my needs, in a couple of days, I had my scripts doing what I wanted them to do.
But that wasn’t what our customer was expecting, they asked me to use terraform, and then I spent my 2018 birthday’s weekend learning terraform, and it was worth it, because that project allowed me to learn what it was bleeding edge technology, at least in Spain, that project involved Azure, the Azure source code repositories, Circle CI, Hashicorp Vault and Terraform.
So I wanted to bring back some of that knowledge which was buried deep on my mind and mix it with what I’m learning about AWS.
The objective
My initial objective was to setup a t2.micro instance running an apache web server behind an Elastic Load Balancer and use an Availability Set to expand a minimum of two instances and be able to access them alternatively in a round-robin fashion.
Installing terraform
The version I had installed was three years old, so the first thing to do was updating it. As it’s a single binary, that could be done by simply downloading and replacing the binary.
But, since I last installed terraform, Hashicorp has shared an apt repository and I would rather use it in order to keep it updated along my system.
The commands I ran were not exactly the ones stated at Hashicorp’s documentation because I was sure I already had curl and gnupg installed, instead I ran:
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
sudo apt-get update && sudo apt-get install -y software-properties-common terraform
And then I had terraform available in my system:
terraform $ terraform --version
Terraform v1.0.9
on linux_amd64
I’m a command line man, so I installed terraform autocompletion features for my shell, don’t forget to reload your environment in order to get it to work on your current shell:
terraform -install-autocomplete
source ~/.bashrc
Defining the infrastructure as code environment
Terraform depends on having aws CLI installed and configured, as I already had configured aws cli, this wasn’t a step I had to perform, but is worth it to remind everybody about it.
One thing I don’t get used to is the terraform’s init step, because it has to be performed once you have your main.tf file written, or at least, it has to contain the providers section.
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.63"
}
}
required_version = ">= 1.0.9"
}
The init process downloaded the corresponding provider plugin and I was able to format and validate my first terraform source (copied from the examples).
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.63"
}
}
required_version = ">= 1.0.9"
}
provider "aws" {
profile = "default"
region = "eu-west-1"
}
resource "aws_launch_template" "terra1" {
name_prefix = "terra1_"
image_id = "ami-05cd35b907b4ffe77"
instance_type = "t2.micro"
key_name = "ec2-keypair"
vpc_security_group_ids = ["sg-04cbc9fc53450397e", "sg-0d2c02f4547301136"]
user_data = <<EOF
IyEvYmluL2Jhc2gKIyBVc2UgdGhpcyBmb3IgeW91ciB1c2VyIGRhdGEgKHNjcmlwdCBmcm9tIHRv
cCB0byBib3R0b20pCiMgaW5zdGFsbCBodHRwZCAoTGludXggMiB2ZXJzaW9uKQp5dW0gdXBkYXRl
IC15Cnl1bSBpbnN0YWxsIC15IGh0dHBkCnN5c3RlbWN0bCBzdGFydCBodHRwZApzeXN0ZW1jdGwg
ZW5hYmxlIGh0dHBkCmVjaG8gIjxoMT5IZWxsbyBXb3JsZCBmcm9tICQoaG9zdG5hbWUgLWYpPC9o
MT4iID4gL3Zhci93d3cvaHRtbC9pbmRleC5odG1s
EOF
}
resource "aws_autoscaling_group" "terra1" {
availability_zones = ["eu-west-1a"]
desired_capacity = 2
max_size = 3
min_size = 2
launch_template {
id = aws_launch_template.terra1.id
version = "$Latest"
}
load_balancers = [aws_elb.terra1.name]
}
resource "aws_elb" "terra1" {
security_groups = ["sg-04cbc9fc53450397e", "sg-0d2c02f4547301136"]
availability_zones = ["eu-west-1a"]
health_check {
healthy_threshold = 2
unhealthy_threshold = 2
timeout = 3
interval = 30
target = "HTTP:80/"
}
listener {
lb_port = 80
lb_protocol = "http"
instance_port = "80"
instance_protocol = "http"
}
}
Please note that the security groups id are related to the account and region, you have to use your own.
$ terraform fmt
main.tf
$ terraform validate
Success! The configuration is valid.
And then, I ran terraform plan to know what is going to happen
$ terraform apply
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# aws_autoscaling_group.terra1 will be created
+ resource "aws_autoscaling_group" "terra1" {
+ arn = (known after apply)
+ availability_zones = [
+ "eu-west-1",
]
+ default_cooldown = (known after apply)
+ desired_capacity = 2
+ force_delete = false
+ force_delete_warm_pool = false
+ health_check_grace_period = 300
+ health_check_type = (known after apply)
+ id = (known after apply)
+ load_balancers = (known after apply)
+ max_size = 3
+ metrics_granularity = "1Minute"
+ min_size = 2
+ name = (known after apply)
+ name_prefix = (known after apply)
+ protect_from_scale_in = false
+ service_linked_role_arn = (known after apply)
+ vpc_zone_identifier = (known after apply)
+ wait_for_capacity_timeout = "10m"
+ launch_template {
+ id = (known after apply)
+ name = (known after apply)
+ version = "$Latest"
}
}
# aws_elb.terra1 will be created
+ resource "aws_elb" "terra1" {
+ arn = (known after apply)
+ availability_zones = [
+ "eu-west-1",
]
+ connection_draining = false
+ connection_draining_timeout = 300
+ cross_zone_load_balancing = true
+ dns_name = (known after apply)
+ id = (known after apply)
+ idle_timeout = 60
+ instances = (known after apply)
+ internal = (known after apply)
+ name = (known after apply)
+ security_groups = [
+ "sg-04cbc9fc53450397e",
+ "sg-0d2c02f4547301136",
]
+ source_security_group = (known after apply)
+ source_security_group_id = (known after apply)
+ subnets = (known after apply)
+ tags_all = (known after apply)
+ zone_id = (known after apply)
+ health_check {
+ healthy_threshold = 2
+ interval = 30
+ target = "HTTP:80/"
+ timeout = 3
+ unhealthy_threshold = 2
}
+ listener {
+ instance_port = 80
+ instance_protocol = "http"
+ lb_port = 80
+ lb_protocol = "http"
}
}
# aws_instance.app_server will be created
+ resource "aws_instance" "app_server" {
+ ami = "ami-05cd35b907b4ffe77"
+ arn = (known after apply)
+ associate_public_ip_address = (known after apply)
+ availability_zone = (known after apply)
+ cpu_core_count = (known after apply)
+ cpu_threads_per_core = (known after apply)
+ disable_api_termination = (known after apply)
+ ebs_optimized = (known after apply)
+ get_password_data = false
+ host_id = (known after apply)
+ id = (known after apply)
+ instance_initiated_shutdown_behavior = (known after apply)
+ instance_state = (known after apply)
+ instance_type = "t2.micro"
+ ipv6_address_count = (known after apply)
+ ipv6_addresses = (known after apply)
+ key_name = (known after apply)
+ monitoring = (known after apply)
+ outpost_arn = (known after apply)
+ password_data = (known after apply)
+ placement_group = (known after apply)
+ placement_partition_number = (known after apply)
+ primary_network_interface_id = (known after apply)
+ private_dns = (known after apply)
+ private_ip = (known after apply)
+ public_dns = (known after apply)
+ public_ip = (known after apply)
+ secondary_private_ips = (known after apply)
+ security_groups = (known after apply)
+ source_dest_check = true
+ subnet_id = (known after apply)
+ tags = {
+ "Department" = "Automation"
+ "Name" = "TerraformedInstance"
}
+ tags_all = {
+ "Department" = "Automation"
+ "Name" = "TerraformedInstance"
}
+ tenancy = (known after apply)
+ user_data = (known after apply)
+ user_data_base64 = (known after apply)
+ vpc_security_group_ids = (known after apply)
+ capacity_reservation_specification {
+ capacity_reservation_preference = (known after apply)
+ capacity_reservation_target {
+ capacity_reservation_id = (known after apply)
}
}
+ ebs_block_device {
+ delete_on_termination = (known after apply)
+ device_name = (known after apply)
+ encrypted = (known after apply)
+ iops = (known after apply)
+ kms_key_id = (known after apply)
+ snapshot_id = (known after apply)
+ tags = (known after apply)
+ throughput = (known after apply)
+ volume_id = (known after apply)
+ volume_size = (known after apply)
+ volume_type = (known after apply)
}
+ enclave_options {
+ enabled = (known after apply)
}
+ ephemeral_block_device {
+ device_name = (known after apply)
+ no_device = (known after apply)
+ virtual_name = (known after apply)
}
+ metadata_options {
+ http_endpoint = (known after apply)
+ http_put_response_hop_limit = (known after apply)
+ http_tokens = (known after apply)
}
+ network_interface {
+ delete_on_termination = (known after apply)
+ device_index = (known after apply)
+ network_interface_id = (known after apply)
}
+ root_block_device {
+ delete_on_termination = (known after apply)
+ device_name = (known after apply)
+ encrypted = (known after apply)
+ iops = (known after apply)
+ kms_key_id = (known after apply)
+ tags = (known after apply)
+ throughput = (known after apply)
+ volume_id = (known after apply)
+ volume_size = (known after apply)
+ volume_type = (known after apply)
}
}
# aws_launch_template.terra1 will be created
+ resource "aws_launch_template" "terra1" {
+ arn = (known after apply)
+ default_version = (known after apply)
+ id = (known after apply)
+ image_id = "ami-05cd35b907b4ffe77"
+ instance_type = "t2.micro"
+ key_name = "ec2-keypair"
+ latest_version = (known after apply)
+ name = (known after apply)
+ name_prefix = "terra1_"
+ tags_all = (known after apply)
+ user_data = <<-EOT
IyEvYmluL2Jhc2gKIyBVc2UgdGhpcyBmb3IgeW91ciB1c2VyIGRhdGEgKHNjcmlwdCBmcm9tIHRv
cCB0byBib3R0b20pCiMgaW5zdGFsbCBodHRwZCAoTGludXggMiB2ZXJzaW9uKQp5dW0gdXBkYXRl
IC15Cnl1bSBpbnN0YWxsIC15IGh0dHBkCnN5c3RlbWN0bCBzdGFydCBodHRwZApzeXN0ZW1jdGwg
ZW5hYmxlIGh0dHBkCmVjaG8gIjxoMT5IZWxsbyBXb3JsZCBmcm9tICQoaG9zdG5hbWUgLWYpPC9o
MT4iID4gL3Zhci93d3cvaHRtbC9pbmRleC5odG1s
EOT
+ vpc_security_group_ids = [
+ "sg-04cbc9fc53450397e",
+ "sg-0d2c02f4547301136",
]
+ metadata_options {
+ http_endpoint = (known after apply)
+ http_protocol_ipv6 = (known after apply)
+ http_put_response_hop_limit = (known after apply)
+ http_tokens = (known after apply)
}
}
Plan: 4 to add, 0 to change, 0 to destroy.
At the first run of terraform apply it failed because I used regions were I should have used availability zones, but after fixing that little mistake, everything went up.
I checked if the instances were created, and they were:
I also checked for the ELB:
After visually checking all resources were in place, the moment of truth came:
terraform $ for i in {1..10}; do curl "tf-lb-20211016150838776700000001-1128121490.eu-west-1.elb.amazonaws.com"; done
<h1>Hello World from ip-172-31-3-156.eu-west-1.compute.internal</h1>
<h1>Hello World from ip-172-31-3-156.eu-west-1.compute.internal</h1>
<h1>Hello World from ip-172-31-2-214.eu-west-1.compute.internal</h1>
<h1>Hello World from ip-172-31-3-156.eu-west-1.compute.internal</h1>
<h1>Hello World from ip-172-31-2-214.eu-west-1.compute.internal</h1>
<h1>Hello World from ip-172-31-3-156.eu-west-1.compute.internal</h1>
<h1>Hello World from ip-172-31-2-214.eu-west-1.compute.internal</h1>
<h1>Hello World from ip-172-31-3-156.eu-west-1.compute.internal</h1>
<h1>Hello World from ip-172-31-2-214.eu-west-1.compute.internal</h1>
<h1>Hello World from ip-172-31-3-156.eu-west-1.compute.internal</h1>
Just what I expected, each connection served from a different instance on a round-robin fashion
And after finish the exercise was time to clean the house up, to not incur on unwanted expenses, as all resources were created using terraform, they were removed the same way:
terraform $ terraform destroy aws_launch_template.terra1: Refreshing state... [id=lt-0e7a1e32de292def5] aws_elb.terra1: Refreshing state... [id=tf-lb-20211016150838776700000001] aws_autoscaling_group.terra1: Refreshing state... [id=terraform-20211016150926410100000002] Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: - destroy Terraform will perform the following actions: # aws_autoscaling_group.terra1 will be destroyed ... # aws_elb.terra1 will be destroyed ... # aws_launch_template.terra1 will be destroyed ... Plan: 0 to add, 0 to change, 3 to destroy. Do you really want to destroy all resources? Terraform will destroy all your managed infrastructure, as shown above. There is no undo. Only 'yes' will be accepted to confirm. Enter a value: yes aws_autoscaling_group.terra1: Destroying... [id=terraform-20211016150926410100000002] aws_autoscaling_group.terra1: Still destroying... [id=terraform-20211016150926410100000002, 10s elapsed] aws_autoscaling_group.terra1: Still destroying... [id=terraform-20211016150926410100000002, 20s elapsed] aws_autoscaling_group.terra1: Still destroying... [id=terraform-20211016150926410100000002, 30s elapsed] aws_autoscaling_group.terra1: Still destroying... [id=terraform-20211016150926410100000002, 40s elapsed] aws_autoscaling_group.terra1: Still destroying... [id=terraform-20211016150926410100000002, 50s elapsed] aws_autoscaling_group.terra1: Still destroying... [id=terraform-20211016150926410100000002, 1m0s elapsed] aws_autoscaling_group.terra1: Still destroying... [id=terraform-20211016150926410100000002, 1m10s elapsed] aws_autoscaling_group.terra1: Destruction complete after 1m16s aws_launch_template.terra1: Destroying... [id=lt-0e7a1e32de292def5] aws_elb.terra1: Destroying... [id=tf-lb-20211016150838776700000001] aws_launch_template.terra1: Destruction complete after 0s aws_elb.terra1: Destruction complete after 1s Destroy complete! Resources: 3 destroyed.
Conclusions
It has been refreshing using again terraform scripts, it’s a bit tedious when you want to work with variables, but yet powerful.
This is a widely used tool for Infrastructure as Code, and it’s important to know how to use it. Terraform also allows integration with CI tools for validating the code and for approval workflows, which is great for simplifying the operators work and prevent human mistakes.
And, again, the relevant files are on my homelab repo
References
- Terraform’s instalaltion: https://learn.hashicorp.com/tutorials/terraform/install-cli
- Terraform’s AWS tutorial: https://learn.hashicorp.com/tutorials/terraform/aws-build
- Raul Basak’s Terraform ELB guide: https://medium.com/@ratulbasak93/aws-elb-and-autoscaling-using-terraform-9999e6266734
- AWS course by Stephane Maarek: https://www.udemy.com/course/aws-certified-solutions-architect-associate-saa-c02/