Manually clicking through the AWS console to set up infrastructure is a rite of passage for many developers. But it’s a practice fraught with problems. It’s slow, prone to human error, and nearly impossible to replicate perfectly across different environments. One forgotten permission or typo can bring your entire workflow to a halt. This manual “click-ops” approach simply doesn’t scale.
This is exactly the problem that Infrastructure as Code (IaC) solves. By defining your cloud resources in code, you create a reliable, repeatable, and automated process for managing your infrastructure. This guide will provide a hands-on tutorial to build a classic serverless workflow: a Terraform AWS Lambda S3 Trigger. We will create a complete, automated system where a file upload to an S3 bucket automatically triggers a Lambda function, all managed with Terraform.
Foundational Concepts: What is Terraform and IaC?
Before we write any code, let’s clarify the core concepts.
What is Infrastructure as Code (IaC)?
Infrastructure as Code is the practice of managing and provisioning your cloud infrastructure (like servers, databases, and networks) through machine-readable definition files, rather than through physical hardware configuration or interactive configuration tools. Think of it as a blueprint for your cloud resources. This blueprint is version-controlled, testable, and reusable, just like your application code.
Why Use Terraform for Your AWS Infrastructure?
Terraform is an open-source IaC tool created by HashiCorp that has become the industry standard for a reason. It uses a simple, declarative language called HCL (HashiCorp Configuration Language) where you describe the desired end state of your infrastructure, and Terraform figures out how to get there. Its key benefits include:
- Cloud-Agnostic: It works with AWS, Azure, Google Cloud, and many other providers.
- State Management: It keeps a “state file” to track your resources, making updates and deletions predictable and safe.
- Strong Community: It has a massive ecosystem of pre-built modules and extensive documentation.
Prerequisites for This Tutorial
To follow along, you’ll need a few things set up first:
- An AWS account with administrative permissions.
- The AWS CLI installed and configured on your machine.
- Terraform (version 1.0 or newer) installed locally.
- A code editor like VS Code.
Step 1: Setting Up Your Terraform Project Structure
A clean project structure makes your code easier to manage. For this guide, we’ll use a simple setup. Create a main project folder, and inside it, create a src directory.
- main.tf: This file will contain all our Terraform configuration code.
- src/lambda_function.py: This will hold our simple Python code for the Lambda function.
Step 2: Configuring the Terraform AWS Provider and IAM Role
The first step in any Terraform configuration is to declare the provider you’re using. Then, we must address the most common point of failure: permissions. Our Lambda function needs an IAM Role that grants it permission to run and write logs.
Open your main.tf file and add the following code:
Here’s what this code does:
- provider “aws”: Tells Terraform we are working with AWS and specifies the default region.
- aws_iam_role: This creates an IAM role. The crucial part is the assume_role_policy, which essentially says, “I trust the AWS Lambda service to assume this role.”
- aws_iam_role_policy_attachment: This attaches a pre-built AWS policy to our role. The AWSLambdaBasicExecutionRole policy grants our function the necessary permissions to write logs to Amazon CloudWatch, which is essential for debugging.
Step 3: Preparing and Packaging the Lambda Function Code
Terraform provisions the infrastructure, but we still need to provide the application code for our Lambda function.
1. Inside the src directory, create a file named lambda_function.py and add this simple Python code:
This function simply logs a message and the event data it receives from S3.
2.Next, we need to package this code into a .zip file for deployment. Instead of doing this manually, we can have Terraform do it for us. Add the following archive_file data source to your main.tf:
This tells Terraform to create a lambda_function.zip file from our Python script every time we run it.
Step 4: Provisioning the AWS S3 Bucket and Lambda Function
Now we can define the core resources: the S3 bucket that will receive file uploads and the Lambda function that will be triggered.
Add the following to your main.tf file:
Terraform
# main.tf (add this to your file)
- aws_s3_bucket: This creates our S3 bucket. We use a random_id to ensure the bucket name is globally unique, which is an S3 requirement.
- aws_lambda_function: This provisions the Lambda function. We link the role we created in Step 2 and provide the filename and source_code_hash from the archive_file data source.
Step 5: Creating the S3 Trigger for the Lambda Function
This is the final piece of the puzzle, where we connect the S3 bucket to the Lambda function. This requires two more resources.
Add the final block of code to your main.tf:
Terraform
# main.tf (add this to your file)
- aws_lambda_permission: This resource grants the S3 service principal (s3.amazonaws.com) permission to invoke our Lambda function.
- aws_s3_bucket_notification: This resource configures the S3 bucket itself. It tells the bucket, “When a new object is created (s3:ObjectCreated:*), send a notification to this specific Lambda function.”
Step 6: Deploying and Testing Your Automated Infrastructure
With all the code in place, it’s time to deploy. Terraform uses a simple three-command workflow:
Initialize Terraform: First, run terraform init. This command downloads the necessary AWS provider plugin.
Bash
terraform init
1. Plan the Changes: Next, run terraform plan. This command shows you an execution plan of what Terraform will create, change, or destroy. It’s a crucial safety step to review before making any changes.
Bash
terraform plan
2. Apply the Changes: Finally, run terraform apply. This command will execute the plan and build your infrastructure in AWS. Terraform will ask for a confirmation; type yes to proceed.
Bash
terraform apply
3. After a few moments, Terraform will report that it has successfully created all your resources.
To test your new serverless workflow:
- Navigate to the S3 console in your AWS account.
- Find the bucket that Terraform created (e.g., varseno-terraform-s3-trigger-demo-…).
- Upload any file into the bucket.
- Navigate to the Lambda console, find your s3-file-processor function, and click on the “Monitor” tab.
- Click “View CloudWatch logs.” You should see a new log stream with the “Lambda function triggered by S3!” message and the event data from your file upload.
Conclusion: From Manual Clicks to Automated Infrastructure
Congratulations! You have successfully built a fully automated, repeatable, and version-controlled serverless workflow using Infrastructure as Code best practices. You can now destroy and recreate this entire stack with a single command, confident that it will be configured identically every time.
This is a foundational pattern in serverless architecture. From here, you can explore more advanced topics:
- Using Terraform Variables: Parameterize your code to easily change values like the region or bucket name.
- Refactoring into Modules: Turn this configuration into a reusable Terraform module to deploy similar stacks quickly.
- Adding More Triggers: [Connect an API Gateway] to your Lambda function to create a fully serverless API.
Building reliable and scalable cloud infrastructure is our specialty. If you’re looking to adopt IaC, optimize your serverless architecture, or streamline your DevOps processes, talk to the cloud experts at Varseno today.
