Infrastructure as a Code using Terraform Modules (Part-II)

10 min readMay 12, 2020

Hello Everyone, now we are here with the part-II of Terraform Modules. If you are reading this first go to Part-I first where I have discussed the Terraform Modules and what is the need to adopt it.

So here I will start by

Creating our first own module.
Publishing it on Private Registry(Terraform Cloud).
Use Private Modules.
Provision the infrastructure (Locally as well as on Terraform Cloud).
Publish our module on the Public Registry(Terraform Registry)

So let's get started….

Overview of Terraform Modules

Terraform Modules is considered as one of the best practices to provision infrastructure on the cloud. Using Terraform Modules one could reuse, configure, customize, version the infrastructure. There are public modules available on the Terraform public registry by different cloud providers. Import the modules using source and version and apply the changes using public modules.

Creating our first module

Before you start this practical make sure you have terraform installed in your system. If not please follow the instructions here. Secondly, an AWS or any cloud service account would be needed with the secret credentials.
Now in your workspace create a folder named as modules and create a subfolder named as ec2-module. The structure of the ec2-module should be as described below.

tree
.
└── ec2-instance
    ├── LICENSE
    ├── main.tf
    ├── output.tf
    ├── README.md
    └── variable.tf1 directory, 5 files

Open main.tf and create your resources which you want to create in the module. Here I have initialized a VPC, a subnet, a security group, and 2 ec2-instances. So this is my main.tf file.

This is the main module file. It contains a declaration of all that resources which are gonna launch on AWS from our module. So I have specified a VPC, a key pair, a subnet, a security group, and 2 EC2 instances.
Now to declare the input variables open variable.tf file and write the following code.

Now we should decide the outputs of our module. So open outputs.tf file and declare output variables.

Now open LICENSE and paste the following content in it.

Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.

Coming on next prepare a README.md and document your module as it will make others understand that actually what are you doing.

Importing The Modules

Now following the above steps we created a module but it’s time to import it and run.
Come out of the modules folder and create the files as shown below. The main.tf will import the module which we designed inside the modules/ec2-instance.

tree
.
├── main.tf
├── modules
│   └── ec2-instance
│       ├── LICENSE
│       ├── main.tf
│       ├── output.tf
│       ├── README.md
│       └── variable.tf
├── output.tf
├── README.md

Open main.tf and using the module block import the source of the module by specifying the path of the ec2-instance directory. Take help from the below code snippet.

So in the source variable, we passed the relative path of our module directory. Now the variables which we initialized in the module could be defined explicitly in the module block if the user wants to configure his own parameters.
So we configured the module myec2 with our own parameters. Remember that in the modules/ec2-instance/variable.tf we have assigned some default values implicitly but that doesn't mean they can’t be configured. You could change the value of any variable explicitly defined inside the module.
So now let us define the output variables. Open output.tf and initialize the output variables.

Note:- The variables which are declared inside modules/ec2-instance/output.tf could only be shown as output. So according to your need of output, you should first change the module output file then you should define the output variables outside your module.
So now we are good to go.

Provision the Infrastructure

Run terraform init to make initialize your module. Terraform will recognize your module when you will run this command.

Run terraform plan to get a pre-executed plan about what changes will be done in the cloud.

Run terraform apply to apply the changes in the cloud. Confirm the changes by answering yes.

After the changes have been applied you will see the output variables. In my case, you will get your public-ips and subnet-ids as your output.

Check your AWS management console and see the instances getting created.

Publishing your module in a Private registry

Now as you have created your module and its working properly so now we should publish it on a registry. Publishing your modules on a registry helps to version your infrastructure.
Any module release in the registry will be mapped with a version number. So when you will change your infra code and will initiate a new release your infrastructure version will get updated.
Versioning infrastructure is the same as you version your code using certain VCS tools like Github, Mercurial, SVN, etc. So now let us start with publishing the above module into a private registry.

Pre-requisites:-

Should be a user on terraform cloud. If you have not set up Terraform Cloud yet, follow the Getting Started — Terraform Cloud guide on creating a workspace.
A GitHub account.
An AWS account with IAM Access Keys.
OAuth Access to GitHub configured. If you belong to an organization where you don’t have permission to configure this access, you can create a new organization for this guide.

Create a Repository

Create a repository on GitHub with the name terraform-<provider>-<name> and push the files present in your modules/ec2-instance/. You could also clone my repository by the following command and push into your remote repo.

git clone https://github.com/PrajjawalBanati/terraform-aws-ec2.git

Now terraform versions the modules with tag no. associated with it. So to add the module in the terraform cloud it is required that we associate the module with a tag. So on your repo information click on the part which shows 0-releasesand click on Create a new release.

Now in the release tag version give the value 1.0.0. Set the Release title to “First module released!”. Click on Publish Release.

Now that you have created and versioned your module, you need to add a VCS connection to your Terraform Cloud organization.

Import the module

NOTE: The Private Module Registry requires OAuth verification to GitHub. If you haven’t already, follow the OAuth instructions before proceeding.
To create a Terraform module for your private module registry, navigate to the Modules header in Terraform Cloud. Choose “Add Module” from the upper right corner.

Choose the GitHub(Custom) VCS provider you configured and find the name of the module repository. In my case it is terraform-aws-ec2.

Remember that your git repository should be of the name terraform-<PROVIDER>-<NAME> only then it will be identified by the terraform cloud.

Click on Publish Module to publish it on your private registry.

After the module is published you will see the following dashboard. So at the top right-hand side, you will see Provision Instructions block which would be containing the source and version of the module. So to import your published private module use the following source and version.

Versioning Of Modules

Now suppose you changed the infrastructure code and added some more resources in your main.tf file, What will do next? Just Draft a New Release with 1.0.1 tag and after releasing on GitHub, come back to your module and refresh the page. After refreshing it you will see that new version of module is published automatically on your Terraform Registry. Now you can use any either your 1.0.0 version or your newly created 1.0.1 version. This practice is Versioning Infrastructure, where you version your infrastructure code and provision it on the cloud. So this is the best advantage of creating Terraform Modules. Your both versions are remotely stored and you could use any of these.

Use Private Modules

Now you published your module in a private registry, we can import it to our infrastructure code. Let us start by creating a folder named as aws-ec2-instance and make the following files in it.

$ tree aws-ec2-instance/
aws-ec2-instance/
├── main.tf
├── output.tf
└── variable.tf0 directories, 3 files

Open main.tf and import the module by specifying the source and version.

Declare and define the variables in a separate variable.tf file.

Define the root output variables in output.tf for the module.

Ok, so now we will provision the infrastructure by using this module on our cloud. First, we will do it on CLI (Terminal) and then we will do it remotely on terraform cloud workspace. So if you only want to provision the infrastructure remotely you could skip the CLI step.

Provision The Infrastructure

Provision using Terraform CLI 😃
Run terraform init in the workspace. You will observer it will download your created module which is stored in the private registry.

Run terraform plan to get an execution plan of what resources are gonna be created on the cloud.

Run terraform apply to apply the resources on the cloud console. Answer yes to confirm the changes.

After the resources are created in the outputs section you will get the Public IPs and Subnet IDs.

Check your cloud console and see the resources created.
Provision using Terraform Cloud 😄
This part we will be dealing with provisioning the infrastructure by Terraform Cloud. Open variable.tf and remove the default values from variable blocks as now we will use the terraform cloud interface to initialize the variables.

Go to the GitHub create a repo and import the main.tf, output.tf, variable.tf files in that repo or you can clone my repo.

git clone https://github.com/PrajjawalBanati/aws-ec2-instance.git

Create a workspace on Terraform cloud. Click on Workspace tab and click on New Workspace.

Select your VCS provider and select the repository in which your root configuration repository is present. In this case, we have repo named as aws-ec2-instance. Select the repository.

Then the next tab will open specifying your suggested workspace name. Edit the workspace name if you want to and then click on Create Workspace.

Now click on Configure Variables and a page will open where you could assign values to the variables. The variables which you initialized in variables.tf will be assigned these values to provision your infrastructure.

Also to get access on AWS cloud console store the secret credentials as AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.
Now click on Queue Plan and a dialog box will open to specify the reason. Neglect it and click on Queue Plan button present below the text box.

So now your plan will run and if there is no error it will ask for the confirmation from the user to apply the changes.

Click on Confirm & Apply to provision the infrastructure on your cloud. After the Plan is finished you will see that the infrastructure is deployed on the cloud. Check the cloud console.

So that's how you could provision your infrastructure using modules.

Publish Modules on Public Terraform Repository

Publishing modules on terraform registry is quite simple and easy.
Log in with your terraform account credentials on terraform registry and after you have logged in, there will be an option named as Publish on the top right-hand side.
Click on Publish.

Select your module, agree to terms, and publish it.

See your module published on terraform registry.

So till now, we have successfully practiced terraform modules and provisioning. So writing modules could actually help you to Reuse, Code, Configure, Version, and Document your infrastructure.

By this blog, I conclude about Terrafom Modules and provisioning Infrastructure. If there is any issue you could mail me or follow me on twitter.