Power Automation & Management of Infrastructure Using Ansible and Terraform
Whenever we are providing any service to a huge customer base or we are SaaS (Software As A Service) provider, we deal with a huge amount of data daily and for the processing and management of such data we must have some underlying infrastructure on which all of this is being performed. Controlling and Managing that Infrastructure is obviously is one of thing that needs to be done very rapidly and quickly. For example deployment of a group of servers with some particular specification, doing some massive updates to the group of servers, some activity or roll-out that is to be performed on server. For doing all these types of job in a blink of an eye, one must have a smart control over the infrastructure. Here we will have a look at couple of automation, orchestration and provisioning tools that will help us in deploying and managing our infrastructure during our daily, weekly or monthly schedules.
Infrastructure As Code is a concept or a well-defined process of management and provisioning cloud based systems, data centers or such infrastructures through declarative configuration files, rather than physical hardware configuration or some other configuration management tools. We have couple of tools for implementing automation and orchestration on different platforms. In this blog we will be having a look at couple of them in order to understand how this whole concept of automation works.
Overview of Ansible
Ansible is an automation tool widely used by many of the tech companies to automate apps and IT infrastructure. The whole story goes like Application Deployment + Configuration Management + Continuous Delivery. There are lot other tools used for the same purpose, some of the example are Chef, Puppet and Salt-Stack. How ansible differs from all of them is with the term being “Agent-less”. It means for a working ansible model you just have to be up and running with a fine working SSH connection between host and your ansible base machine. This is how it is being able to enable Infrastructure As Code.
For installing Ansible on your local machine, there is a detailed documentation on the website here. So have a look at it, install and be ready to rock and roll. Once you have Ansible installed, you can check using the following command;
$ ansible --versionExample of Ansible with a Bare Metal Machine
Setting up ansible with some basic modules is quite easy, Here we are going with Ubuntu. Ansible uses configuration files called playbooks for a series of tasks. The playbooks are written in YAML syntax.
Creating Inventory
For creating the inventory one must populate the file /etc/ansible/hosts. In order to populate the inventory file, some format like that must be followed;
[testmachines]
192.168.12.1 ansible_connection=ssh ansible_user=myuser
myVM ansible_port=5555
[prodservers]
VMone
VMtwoOnce the host file is setup, we can test multiple modules according to our requirements, Once the inventory is setup we are ready to manage and provision our host nodes/machines as we want.
Testing the ping module
ansible -i hosts all -m pingThis command will ping all the modules and put forward the results as PONG, obviously the ping will give result as PONG.
Playbook for Performing a Telnet Operation
---
- name: Telnet {{ hostname }}
ignore_errors: yes
expect:
timeout: 10
command: telnet 127.0.0.1 10896
responses:
"Escape character is": "\n"
"login:": "root"
"password: ": "admin123"
"# ":
- conf
- hostname {{ hostname }}
- exit
- save allPlaybook for Restarting a Server
---
- hosts:
become: true
tasks:
- name: restart server
shell: sleep 2 && shutdown -r now
async: 1
poll: 0
ignore_errors: true - name: waiting for the server to come back
local_action: wait_for host=testcentos state=started delay=3 timeout=300
Playbook for Creating/Deleting User
---
- hosts:
become: true
tasks:
- name: Creating the User
user: name=testuser password=testuser123 groups=ansible shell=/bin/bash - name: Removing the User
user: name=testuser state=absent remove=yes force=yes
For executing the playbooks written above, we can simply used a single line command in order to execute the playbook and get our desired results, here is the command;
$ ansible-playbook <name-of-playbook>.yml -i <host-file-path>Example of Ansible with a Specific Cloud Provider (AWS, Azure)
Whenever we talk about setting up and provisioning the infrastructure of a specific cloud provider whether it is AWS or Azure. we can do it easily via ansible. If we specifically talk about Azure, we have a huge support with ansible for provisioning the Azure infrastructure. you can also have a look at it here.
Playbook for Setting up an Azure Linux Virtual Machine in Azure
If we specifically look at creating an Azure VM manually via Azure portal, we have to go through couple of steps in order to finally create a VM. I suggest to have a look at the creation of any resource in Azure environment manually first and remember the steps, then try to automate it using ansible playbook. so moving on to the Azure VM, we must have to create a resource group first in Azure, after that we have a create following resources in order to have a up and running machine.
Virtual Network, Subnet, Public IPv4, Network Security Group and a Network Interface Card.
So we will be creating each resource one by one using a playbook, More detailed explanation can also be found here.
- name: Create Azure VM
hosts: localhost
connection: local
tasks: - name: Create resource group
azure_rm_resourcegroup:
name: myResourceGroup
location: eastus - name: Create virtual network
azure_rm_virtualnetwork:
resource_group: myResourceGroup
name: myVnet
address_prefixes: "10.0.0.0/16" - name: Add subnet
azure_rm_subnet:
resource_group: myResourceGroup
name: mySubnet
address_prefix: "10.0.1.0/24"
virtual_network: myVnet - name: Create public IP address
azure_rm_publicipaddress:
resource_group: myResourceGroup
allocation_method: Static
name: myPublicIP
register: output_ip_address - name: Dump public IP for VM which will be created
debug:
msg: "The public IP is {{ output_ip_address.state.ip_address }}." - name: Create Network Security Group that allows SSH
azure_rm_securitygroup:
resource_group: myResourceGroup
name: myNetworkSecurityGroup
rules:
- name: SSH
protocol: Tcp
destination_port_range: 22
access: Allow
priority: 1001
direction: Inbound - name: Create virtual network interface card
azure_rm_networkinterface:
resource_group: myResourceGroup
name: myNIC
virtual_network: myVnet
subnet: mySubnet
public_ip_name: myPublicIP
security_group: myNetworkSecurityGroup - name: Create VM
azure_rm_virtualmachine:
resource_group: myResourceGroup
name: myVM
vm_size: Standard_DS1_v2
admin_username: azureuser
ssh_password_enabled: false
ssh_public_keys:
- path: /home/azureuser/.ssh/authorized_keys
key_data: <your-key-data>
network_interfaces: myNIC
image:
offer: CentOS
publisher: OpenLogic
sku: '7.5'
version: latest
Here are some of the examples for different Ansible modules I have gathered in a repository, Have a look and contribute if you can; Ansible Code Repo.
Overview of Terraform
Terraform is a famous open-source tool which uses the concept of Infrastructure As Code for the automation and provisioning of many cloud, Infrastructure or such services. It was developed by HashiCorp, it follows a Freemium business model. It delivers consistent workflows to provision, secure, connect, and run any infrastructure for any application.
Example of Terraform with AWS
Here we will be looking at an example of writing a terraform configuration for creating a resource in AWS, example creating an EC2 instance in the AWS environment;
# Create a new instance of the latest Ubuntu 20.04 on an
# t2.micro node with an AWS Tag naming it "HelloWorld"terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.0"
}
}
}# Configure the AWS Provider
provider "aws" {
region = "us-east-1"
}# Setting up the EC2 instace named as testingweb
resource "aws_instance" "testingweb" {
ami = "ami-0dba2cb6798deb6d8"
instance_type = "t2.micro"tags = {
Name = "HelloWorld"
}
}- provider “aws”, this is to set the service provider, you are using. So here we are setting up EC2, so we have write aws here.
- resource “aws_instance” “testingweb” resource is to tell what service of AWS we are using, here we are using EC2 which is an AWS Instance so aws_instance would be hardcoded and testingweb is the name we have assign to it.
Detailed explanation of setting up terraform and creating a basic EC2 instance via terraform can be found here in this blog.
Example of Terraform with Microsoft Azure
In this blog above, we have seen an example of creating a resource in Microsoft Azure using ansible, same methodolgy will be followed here along with the same steps of creating a resource group first, after that we have a create following resources in order to have a up and running machine.
Virtual Network, Subnet, Public IPv4, Network Security Group and a Network Interface Card.
# Configure the Microsoft Azure Providerterraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~>2.0"
}
}
}# Configure the Microsoft Azure Providerprovider "azurerm" {
features {}
}# refer to a resource group
data "azurerm_resource_group" "myterraformgroup" {
name = "testing-resource-group"
}# Create virtual network
resource "azurerm_virtual_network" "myterraformnetwork" {
name = "myVnet"
address_space = ["10.0.0.0/16"]
location = "eastus"
resource_group_name = data.azurerm_resource_group.myterraformgroup.name
}# Create subnet
resource "azurerm_subnet" "myterraformsubnet" {
name = "mySubnet"
resource_group_name = data.azurerm_resource_group.myterraformgroup.name
virtual_network_name = azurerm_virtual_network.myterraformnetwork.name
address_prefixes = ["10.0.1.0/24"]
}# Create public IPs
resource "azurerm_public_ip" "myterraformpublicip" {
name = "myPublicIP"
location = "eastus"
resource_group_name = data.azurerm_resource_group.myterraformgroup.name
allocation_method = "Dynamic"
}# Create network interface
resource "azurerm_network_interface" "myterraformnic" {
name = "myNIC"
location = "eastus"
resource_group_name = data.azurerm_resource_group.myterraformgroup.name
ip_configuration {
name = "myNicConfiguration"
subnet_id = azurerm_subnet.myterraformsubnet.id
private_ip_address_allocation = "Dynamic"
public_ip_address_id = azurerm_public_ip.myterraformpublicip.id
}
}# Create Network Security Group and rule
resource "azurerm_network_security_group" "myterraformnsg" {
name = "myNetworkSecurityGroup"
location = "eastus"
resource_group_name = data.azurerm_resource_group.myterraformgroup.name
security_rule {
name = "SSH"
priority = 1001
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_range = "22"
source_address_prefix = "*"
destination_address_prefix = "*"
}
tags = {
environment = "Terraform Demo"
}
}resource "azurerm_network_interface_security_group_association" "example" {
network_interface_id = azurerm_network_interface.myterraformnic.id
network_security_group_id = azurerm_network_security_group.myterraformnsg.id
}# Create virtual machine
resource "azurerm_linux_virtual_machine" "myterraformvm" {
name = "myVM"
location = "eastus"
resource_group_name = data.azurerm_resource_group.myterraformgroup.name
network_interface_ids = [azurerm_network_interface.myterraformnic.id]
size = "Standard_B1s"
os_disk {
name = "myOsDisk"
caching = "ReadWrite"
storage_account_type = "Standard_LRS"
disk_size_gb = "30"
}
source_image_reference {
publisher = "Canonical"
offer = "UbuntuServer"
sku = "18.04-LTS"
version = "latest"
}computer_name = "myvm"
admin_username = "azureuser"
disable_password_authentication = trueadmin_ssh_key {
username = "azureuser"
public_key = file("")
}tags = {
environment = "Terraform Demo"
}
}
Here using this terraform script an Azure VIrtual Machine, specifically Ubuntu 18.04-LTS is being created.
Power of Unification of Ansible and Terraform
After seeing the power of both the tools, one can easily think of merging the usage of both of them in order to take automation to the next step. Just think of using Ansible with Terraform, It is a powerful combo that I use for provisioning cloud infrastructure and for setting up cloud instances.
Yes, you are thinking in a right direction. One can use Terraform to call Ansible. Terraform is an awesome infrastructure provisioning tool, but you may have noticed that it doesn’t come with a configuration management system. This is the point where Ansible comes in to the game. We use Terraform to stand up virtual machines or cloud instances, and then we hand over the reins to Ansible to finish up the configuration of our OS and applications.
As an example let’s think of a simple example of a three-tier application (Front-end, API, Database). What I can do here is I can use Terraform for my infrastructure tasks and Ansible on top of it. Let’s say setting up this three tier infra consist of multiple steps. In a first step, I’m installing the initial necessary server landscape with Terraform. Afterwards, I’m using Ansible playbooks to provision the software components on my infrastructure and make it up and running. Combining this all together, It will also be really easy for me to maintain and optimized build system for integration in a CI/CD Pipeline.
Now Just think of Dockers and Kubernetes environment, we can seamlessly used both of these tools to setup our whole cluster, provision and manage them using these tools. In the upcoming blog, we will be having a look at the implementation of a simple scenario where we will be combining and utilizing the power of ansible and terraform.
Stay Tuned !
