Uploading Local Files to an Amazon EC2 Instance with Terraform

by

When working with Amazon EC2 instances, you might often need to transfer local files to an EC2 instance. This is especially critical if the file needs to be immediately available upon the instance’s creation. For instance, in a Terraform project, it was essential to have an Apache virtual host configuration file available on the EC2 instance immediately. This configuration file acted as a template for creating additional virtual host files. To accomplish this, I utilized Terraform’s File Provisioner tool to transfer these files.

It’s important to note that while Terraform does provide the capability to use provisioners, they are generally not recommended unless necessary. Terraform suggests alternative techniques primarily due to the restrictions and considerations outlined on the Terraform website.

What Will We Cover?

In this tutorial, we will explore the method to copy a file from a local machine running Terraform to a newly created Amazon EC2 instance (Ubuntu) on AWS using Terraform’s File utility. We’ll demonstrate this by copying a file to an EC2 instance effectively.

What Are Provisioners Used For?

Provisioners generally automate the execution of scripts to create, destroy, and prepare various resources, making machines production- or development-ready. Provisioning tasks include:

  1. Upgrading the operating system.
  2. Creating user accounts, configuring services, and performing administrative tasks.
  3. Installing software.
  4. Transferring and managing system files.
  5. Configuring system IP addresses, ports, and other network settings.

Prerequisites

  1. Basic knowledge of Terraform.
  2. Terraform and AWS CLI installed on your local machine.
  3. AWS CLI configured with access to your AWS account.

Using the ‘File’ Provisioner for File Transfer

The ‘File Provisioner’ enables transferring files and directories from the machine running Terraform to another machine, such as an EC2 instance. This provisioner can be deployed on both local and remote systems. While Terraform excels at deployment tasks, provisioning isn’t its strongest suit, so it recommends using Provisioners sparingly and as a last resort.

Practical Application of the ‘File’ Provisioner

In this section, we will demonstrate how to utilize the ‘file’ provisioner. We’ll create an EC2 instance resource using Terraform and transfer a file from the local system to this instance. Let’s proceed:

Step 1. Create the file you intend to copy over to the remote system. We’ll do this within the directory containing our configuration files:

$ nano dummy.txt

Add some content to this file:

$ echo “Our dummy file” > dummy.txt

Step 2. Create a file named ‘instance.tf’, or any name with a ‘.tf’ extension:

$ sudo instance.tf

Populate this file with the content below:

provider "aws" {
  region ="us-east-1"
}

resource "aws_instance" "webserver" {
  ami ="ami-id-you-want-to-use"
  instance_type = "t2.micro"
  key_name = "Name-of-your-EC2-keypair"
  vpc_security_group_ids = ["Use-an-existing-SG"]
  associate_public_ip_address = true

  provisioner "file" {
      source     = "dummy.txt"
      destination = "/home/ubuntu/file1.txt"

      connection {
          type        = "ssh"
          user        = "ubuntu"
          private_key = "${file("Path-to-EC2-keyPair-on-Local-system.pem")}"
          host        = "${self.public_ip}"
      }
  }

  tags = {
      Name = "FileProvisionerDemo"
  }
}

The above code transfers the ‘dummy.txt’ file to the EC2 instance, renaming it ‘file1.txt’. It uses an existing security group, which you can replace by creating a new one using Terraform. Here’s an explanation of some of the key arguments used:

  • source: Specifies the absolute or relative path to files and directories compared to the Terraform project directory.
  • destination: The absolute path on the target system where the file will be copied.
  • connection block: Instructs Terraform on server communication details. Supports ‘ssh’ and ‘winrm’ types by file provisioners.
  • type: Indicates the connection type, either ‘ssh’ or ‘winrm’. In this example, an SSH connection is used.
  • user: The user account used during the connection.
  • private_key: Contents of an SSH key to establish the connection.
  • host: Indicates the resource address for the connection.
  • self: Represents the parent resource of a connection and houses all resource attributes.

File provisioners also accommodate other arguments such as certificate, agent, and host_key.

Step 3. Initialize the project directory with:

$ terraform init

Initializing Terraform project directory

Check for any syntax errors using:

$ terraform validate

Validating Terraform configuration

Step 4. Set up this infrastructure with the ‘terraform apply’ command. Preview changes using ‘terraform plan’:

$ terraform apply

Sample Output:

Terraform used the selected providers to generate the following execution plan.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_instance.webserver will be created
  + resource "aws_instance" "webserver"

Step 5. Your instance is being created, as shown below:

AWS EC2 console

SSH into your instance and verify the file’s successful transfer:

Verifying the setup

Conclusion

In this guide, we have demonstrated how to leverage the ‘file’ provisioner to transfer a file to an EC2 instance on AWS, ensuring essential files are immediately available upon instance creation.

FAQs

1. Why are provisioners generally discouraged in Terraform?

Provisioners are discouraged because they can introduce uncertainty and complex dependencies in resource management. Terraform recommends using built-in resource types wherever possible, which provide more predictable and consistent behavior.

2. What are some alternatives to using provisioners in Terraform?

Alternatives include using configuration management tools like Ansible, Puppet, or Chef, or leveraging user data and cloud-init scripts to configure resources without Terraform’s direct involvement.

3. Can I use the ‘file’ provisioner with systems other than AWS EC2?

Yes, the ‘file’ provisioner can be used with various systems that support SSH or WinRM connections, allowing for file transfers to any compatible remote server.

4. What should I do if my file transfer fails during the provisioner execution?

Check the connection parameters, including the user, SSH key, and host address. Ensure the destination path is correct and accessible, and review Terraform debug logs for further details.