Simplifying Infrastructure Complexity with Zero-Touch Provisioning

Zero-touch provisioning (ZTP) means the automated provisioning of infrastructure components with little or no manual intervention based on predefined steps or configurations. It was prompted by the need to deploy a large number of network devices across distributed and remote networks. The idea behind it is to reduce the human intervention required to bring infrastructure components online with standardized configurations, thus ensuring faster provisioning times while reducing human error.

This technology has become even more important for Internet of Things (IoT) devices and software-defined networks (SDN). Many network vendors, like Cisco and Juniper, now provide devices that support ZTP, and the global market for zero-touch provisioning was valued at $2.5 billion in 2021.

Although ZTP started with network devices, it has become an essential pillar of infrastructure operations and encompasses all components, including compute, storage, network, and security. With automation, you can deploy, manage, and operate very large distributed and complex infrastructures with limited human intervention. 

For infrastructure to be automated, the state of the infrastructure must be written in code that a computer program can understand to produce the desired outcome. Today’s infrastructures are highly dynamic and must adapt to changing requirements continuously. 

In this article, we explore zero-trust provisioning using different tools to perform infrastructure deployment and operations with code: infrastructure as code (IaC). We demonstrate how you can get the benefits of CI/CD as employed in software development for your infrastructure operations. As part of this process, we use version control to maintain the state of the infrastructure, which can help us with proper change management, performing audits, and executing rollbacks, if desired. 

Summary of key concepts

Key concepts	Description
Infrastructure as code (IaC) tools	Terraform and Ansible are essential high-level tools for our zero-touch provisioning workflow.
All-encompassing standardized framework with CloudBolt	Considering the array of tools and technologies that modern enterprises juggle, cloud management tools like CloudBolt integrate with tools like Ansible and Terraform to form a unifying layer of automation across the enterprise.
ZTP with Terraform	Terraform is an IaC tool that can be used to perform Zero-touch deployment and lifecycle management of on-premises and cloud infrastructure.
ZTP with Ansible	Ansible compliments Terraform by providing zero-touch configuration and application management on the deployed infrastructure.
Version control and GitOps	Large and complex environments require a centralized version control system to track changes and maintain a single source of truth.
CI/CD workflow	Jenkins, a CI/CD tool, can integrate GitHub, Terraform, and Ansible to achieve complete zero-touch automation from deployment to infrastructure operations.
Challenges, considerations, and best practices	Integrating multiple IaC tools, and maintaining compliance, visibility, and multi-team operations presents challenges that need to be addressed to successfully achieve zero-touch provisioning.

Infrastructure as code tools

As technology evolves, infrastructure operations evolve along with them, and tools that can perform zero-touch provisioning of infrastructure components are becoming ever more important. There are many IaC tools available, each offering different features and capabilities. Some organizations may choose one or a combination of different IaC tools, such as Ansible, Chef, Puppet, and Terraform. 

In the following sections, we discuss how we can perform ZTP using Terraform and Ansible. We also learn how CloudBolt’s integration with tools such as Terraform and Ansible enhances their deployment and utilization within the wider scope of enterprise IT operations. This standardized framework encompasses cost management, automated self-service, governance, and optimization, all integrated with ease and adaptability over time.

First, we write Terraform code to deploy an Ubuntu virtual machine on the AWS cloud. Then we write an Ansible playbook to install and configure the NGINX web server on the virtual machine. We use GitHub to do version control of our code. Finally, we use Jenkins to interconnect all the tools and perform zero-touch provisioning of our required infrastructure.

Terraform: a declarative IaC tool

Terraform is an IaC tool developed by HashiCorp that can interact with multiple cloud providers and on-premises infrastructure for lifecycle management. Terraform uses plugins to manage the operations on different platforms via its APIs. Terraform Registry provides over 1,000 plugins called providers, which enable interfacing with AWS, Azure, GCP, OCI, and many more. 

Terraform allows you to declare the state of your infrastructure in human-readable code written in HashiCorp Configuration Language (HCL). You define the desired state, like the number of units and resource limits for your compute, desired network connectivity, storage type and capacity to provision, and security rules to attach to your resources. The primary benefit of Terraform is that you write your desired state in one format (HCL), and Terraform itself manages the complexity of interacting with different cloud platforms to execute the desired steps to provision or de-provision the necessary resources.

In mature, large-scale enterprise workflows, CloudBolt can manage and synchronize infrastructure across various environments that Terraform deploys, eventually acting as a provider for Terraform to provision resources using a graphical user interface to embrace users less familiar with code. The unified layer also helps enforce security, compliance, and cost policies across the entire enterprise. Leveraging the strengths of both Terraform and CloudBolt, this unified configuration can be used to implement quotas, power schedules, and expiration dates for full lifecycle management of enterprise infrastructure. 

Ansible: a configuration management and automation tool

Ansible is an open-source community project sponsored by Red Hat. It’s an agentless automation utility that can perform application deployment and configuration management. Ansible is typically deployed on a single host called the control node, which is then used to perform automated operations on remote systems using an SSH connection.

Ansible automation is done via playbooks that define the sequence of steps or operations executed on remote hosts. The playbooks are written in YAML syntax. Ansible is a highly versatile utility that can install applications on virtual machines, perform network configurations on routers, do storage provisioning, and configure security ACLs on firewalls.

With CloudBolt, organizations can better orchestrate Ansible’s deployments, managing them more cohesively and effectively across diverse IT environments. The integration allows for more centralized control of the automation processes, ensuring that Ansible’s diverse applications — from VM installations to network configurations — are executed more seamlessly and in alignment with broader IT strategies. 

Enhancing workflow and resource management with CloudBolt

In medium-sized to large organizations, you typically have multiple teams working to deploy the infrastructure, manage the network, implement security checks, and manage the application stack. With multiple teams comes the challenge of inter-team coordination for an efficient and consistent workflow.

To address this challenge, you must create standard operating procedures (SOPs) and define key performance indicators (KPIs) for each workflow stage. You need to define how a service request will be submitted, what process will be followed for deploying the required systems and applications, and how the security checks and compliance will be met.

CloudBolt helps automate workflow and perform efficient handover across different teams. The platform can automate core components at each stage to increase efficiency and security for a more consistent application delivery experience.

With CloudBolt’s comprehensive framework, you achieve:

• Resource management/Self-service: Terraform scripts can use the CloudBolt API for resource provisioning, orchestrated via its resource handlers. CloudBolt can initiate Terraform scripts, providing a single platform for connections to providers, easing backend complexity management for IT admins and developers.
• Cost control: CloudBolt enables the synchronization of inventory from Terraform-deployed infrastructure. It facilitates checks on resource utilization and user identification, aiding in informed spending decisions.
• Extensibility: Terraform and CloudBolt together empower IT operations with flexible infrastructure provisioning workflows. This can be orchestrated at the CloudBolt level or within Terraform by developers. After developing and hardening Terraform scripts, these can be handed over to IT operations for production management and end-user access.
• Flexibility: Integrating CloudBolt with IaC provisioning allows teams to use their preferred tools while implementing effective guardrails for overall operations. This balance gives control without being overly restrictive.

Zero-touch provisioning with Terraform

Terraform can be installed on Linux, Mac OS, or Windows. Hashicorp documentation provides steps that can be used to install Terraform on different platforms.

Terraform workflow

The Terraform workflow includes three steps. First, you Write your infrastructure state in HCL format, a high-level language that allows you to define the resources deployed on the cloud platform. Second, you provide as input the required infrastructure, and Terraform will Plan the sequence of low-level steps required to get the desired infrastructure up and running. Terraform will generate output that allows you to review the changes that will be done on the cloud platform. The third step is to Apply the changes, initiating the actual execution of infrastructure deployment.

Cloud infrastructure provisioning with Terraform

To deploy your web server on AWS, create the following Terraform code. This will create an Ubuntu instance in the us-east-1 region with the t2.micro instance type. The code will insert an SSH key into the instance for authentication and attach a specific web server security group. It will then create an elastic IP and output the IP on the screen. This elastic IP will be used to create the inventory file for Ansible.

# Configure the AWS provider
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.16"
    }
  }
  required_version = ">= 1.2.0"
}

provider "aws" {
  region = "us-east-1"
}

resource "aws_instance" "webserver01" {
  ami           = "ami-0fc5d935ebf8bc3bc"       # Ubuntu 22.04
  instance_type = "t2.micro"                    # Instance specs
  key_name      = "AWS-SSH-KEY"                 # SSH Key Pair
  vpc_security_group_ids = ["sg-0c5f791faa80eeb0a"] 

  tags = {
    Name = "WebServer01"
  }
}

resource "aws_eip" "elastic01" {
  instance = aws_instance.webserver01.id
}

# Output VM IP address
output "elastic_ip" {
  value = aws_eip.elastic01.public_ip
}

Zero-touch configuration with Ansible

Almost all UNIX-like OSes with Python 3.9 installed can be used as Ansible control nodes. Ansible is available for various Linux distributions as a package. The Ansible documentation outlines the process of installation for specific distributions.

Cloud infrastructure configuration with Ansible

After provisioning the Ubuntu instance on AWS using Terraform, we use Ansible to install NGINX on the instance using the apt package manager and enable the NGINX service via systemd.

---
- name: Install Nginx Web Server
  hosts: web_servers
  become: yes
  tasks:
    - name: Install nginx package
      apt:
        name:
          - nginx
        state: present
        update_cache: yes

    - name: Enable service nginx and ensure it is not masked
      ansible.builtin.systemd:
        name: nginx
        enabled: yes
        masked: no

We also need to create an inventory.yaml file with the IP of the machine created in the previous step. This inventory file will be used by Ansible to connect with the machine.

web_servers:
  hosts:
    webserver01:
      ansible_host: 10.54.101.174
      ansible_user: ubuntu
      ansible_port: 22

Version control and GitOps

In large and complex environments, tracking changes to the infrastructure state is extremely important. With multiple teams, each with multiple team members working and maintaining different parts of your infrastructure, you require a central repository of information to track who changed what and when. 

Git is one of the most popular version control systems. Initially developed for the Linux Kernel by Linus Torvalds, it is now used by many software developers to maintain, collaborate, and do version control of their software applications. 

Once we define our infrastructure state as a code, we can use best software development practices for our infrastructure operations. Like DevOps, we can do GitOps and use Git as a single source of truth. We can use Git to do version control, track changes to the state of the infrastructure, and perform configuration rollbacks if required. 

There are different Git platforms, like GitLab, GitHub, and Bitbucket. We will set up a repository on GitHub as an example to maintain our infrastructure code. 

Setting up Git repositories

First, we need to create repositories on GitHub for the Terraform and Ansible playbooks. Next, we will create the required IaC files and publish them on the GitHub repositories. The following example is for Ansible, and the same process will be carried out for Terraform code. 

To add the Ansible playbook to the GitHub repository, we clone the repository on the local system, create the Ansible playbook with the code shown in the previous section, and commit it back to the repository.

# Create a local clone of the GitHub repo
$ git clone https://github.com/example-repo/ansible.git
Cloning into 'ansible'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), done.
# Create the Ansible playbook for deploying nginx
$ vi nginx-deploy.yaml

# Commit the changes to the GitHub repo
$ git add nginx-deploy.yaml 
$ git commit -m "Add ansible playbook to install nginx"
[main 9009bde] Add ansible playbook to install nginx
 1 file changed, 17 insertions(+)
 create mode 100644 nginx-deploy.yaml
$ git push
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Delta compression using up to 8 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 502 bytes | 502.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
To https://github.com/example-repo/ansible.git
   efee46b..9009bde  main -> main

Continuous integration / continuous deployment (CI/CD)

In DevOps, continuous integration and deployment (CI/CD) workflows ensure the organized and efficient development, testing, and deployment of software applications. We can use similar workflows for the automated deployment and operations of infrastructure.

Jenkins CI/CD tool: installation and configuration

There are many tools available that have varying capabilities for implementing CI/CD; some of the popular ones include Jenkins, CircleCI, GitLab CI, and GitHub Actions. In the following example, we will use Jenkins, a free and open-source CI/CD utility that has hundreds of plugins available. We can use Jenkins to implement our GitOps workflow and achieve zero-touch provisioning of our infrastructure. Jenkins can be installed in a container or on Linux or Windows machines. The Jenkins installation guide provides procedures for installing on different platforms.

Jenkins is an automation engine. It has the concept of Pipelines, which you can use to create processes that range from very simple to highly complex for implementing automated workflows. 

Deployment via Terraform and configuration via Ansible

For our current example, we will implement two simple workflows. For simplicity, we assume that we have Terraform and Ansible installed on the same host where we have Jenkins. Also, before executing the Pipelines, we need to store the credentials of GitHub, AWS Cloud, and SSH user and private keys for the Ubuntu machine on Jenkins using its credential manager. The Pipelines will use these credentials to pull the code from GitHub, using AWS API for deploying the Ubuntu machine and, finally, using SSH credentials to execute the Ansible playbook.

Jenkins Pipeline for Terraform

The first Pipeline will pull the Terraform code from the GitHub repo, initialize the Terraform plugins, and Apply the Terraform code that will perform the deployment of the Ubuntu machine on the AWS cloud. 

pipeline {
    agent any
    stages {
        stage('Checkout from GitHub') {
            steps {
                git branch: 'main', credentialsId: 'github-creds', url: 'https://github.com/example-repo/terraform.git'
            }
        }

        stage('Terraform init') {
            steps {
                sh 'terraform init'
            }
        }
        stage('Terraform apply') {
            steps {
                withCredentials([[
                    $class: 'AmazonWebServicesCredentialsBinding', 
                    accessKeyVariable: 'AWS_ACCESS_KEY_ID', 
                    secretKeyVariable: 'AWS_SECRET_ACCESS_KEY',
                    credentialsId: 'aws-credentials'
                ]])
                {
                     sh 'terraform apply --auto-approve'
                }
            }
        }
    }
}

Jenkins Pipeline for Ansible

The second Pipeline will get the Ansible playbook from GitHub and execute the playbook that will install the NGINX application on the Ubuntu machine created above.

pipeline {
    agent any

    stages {
        stage('Checkout from GitHub') {
            steps {
                // Check out the Ansible repository from GitHub
                script {
                    git branch: 'main', credentialsId: 'github-credentials', url: 'https://github.com/example-repo/ansible.git'
                }
            }
        }

        stage('Run Ansible playbook') {
            steps {
                script {
                    ansiblePlaybook disableHostKeyChecking: true, credentialsId: 'aws-ssh-key', inventory: '$WORKSPACE/inventory.yaml', playbook: '$WORKSPACE/nginx-deploy.yaml'

                }
            }
        }
    }
}

Zero-touch provisioning of infrastructure

With all the prerequisites in place, we are now able to perform zero-touch provisioning of our infrastructure. On the main dashboard of Jenkins, we have the two Pipelines, as shown below. Now, just by clicking the play button, you can have your machine deployed on the AWS cloud. The second click will cause your machine to have NGINX installed.

The following graphics show the high-level flow of the ZTP implementation involving GitHub and Jenkins utilizing the Terraform and Ansible IaC tools.

Challenges, considerations, and best practices

Using multiple tools to perform ZTP versus a unified platform

The descriptions above illustrate the complexity involved in setting up a complete zero-trust provisioning architecture. There are a lot of advantages to making your infrastructure ZTP-ready, but before you can achieve the target, you need to follow a couple of prerequisite steps:

• Train your team to write Terraform code and Ansible playbooks. This includes how Git repositories work, how to check in and check out code using GitHub, and how to create Jenkins Pipelines.
• Standardize your operating environment, which includes the operating system and application versions, infrastructure topology, and access control mechanisms. You can avoid adding exceptions everywhere and simplify the automation code.

CloudBolt offers an alternative to all of these complexities. CloudBolt reduces the learning curve and, as a no-code platform, provides better capabilities for zero-touch provisioning with an all-in-one platform. 

Compliance and governance

One of the biggest challenges with large and complex infrastructure is maintaining the security of the infrastructure and complying with regulations. With automation tools, you can define guardrails within the infrastructure provisioning steps to implement the best practices and security checks.

Some of the checks that are recommended to be made part of the infrastructure automation are as follows:

• All the deployed operating systems and application versions must be under active support to receive updates and security patches.
• The deployed infrastructure must be protected via an approved access control mechanism.
• The infrastructure should be integrated with a centralized monitoring platform for both performance and security events monitoring.

You can also take advantage of CloudBolt’s Cost and Security Management platform, which can provide your environment with over 300 built-in security checks and enable you to stay compliant with regulations such as CIS and PCI-DSS.

Visibility and cost optimization

With traditional automation tools, you can increase your operational efficiency, but these tools generally lack visibility into the cost factor of the deployed infrastructure. This challenge increases further with multi-cloud operations.

Platforms like CloudBolt help your organization with the effective adoption of FinOps programs at scale. CloudBolt’s cost automation can assist in eliminating unused resources, establishing power schedules, rightsizing resources, and executing other cost-reducing activities seamlessly. For accurate cost allocation, the platform’s robust cost logic leverages multiple data sources to automatically identify your workload owners and generate precise show-back / charge-back models, regardless of your organization’s complexity.

Summary of key concepts

In the article, we discussed how you can use IaC tools like Terraform to deploy infrastructure and use Ansible to configure the deployed infrastructure. We looked at the use of Git repositories to maintain the infrastructure code and using Jenkins to implement the zero-touch provisioning of infrastructure. 

Finally, we discussed different challenges related to complexity, compliance, cost, and team coordination while implementing ZTP and how CloudBolt can help overcome these challenges.