-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: add AWS Lambda warmer Terraform module with configuration and d…
…ocumentation
- Loading branch information
Showing
6 changed files
with
482 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,122 @@ | ||
# tag-based-lambda-warmer | ||
# Terraform Module tag-based-lambda-warmer | ||
|
||
A Terraform module for deploying an automated Lambda function warming solution. This module helps prevent cold starts in your Lambda functions by periodically invoking them based on tags. | ||
|
||
## Overview | ||
|
||
This module creates a Lambda function that automatically warms up other Lambda functions based on specified tags. It uses EventBridge Scheduler to trigger the warming process at regular intervals. | ||
|
||
## Features | ||
|
||
- Tag-based function selection for warming | ||
- Configurable warming schedule | ||
- Separate IAM roles for Lambda and EventBridge Scheduler | ||
- CloudWatch logging for monitoring | ||
- Customizable tag key/value pairs for targeting functions | ||
|
||
## Usage | ||
|
||
```hcl | ||
module "lambda_warmer" { | ||
source = "../modules/lambda_warmer" | ||
aws_region = "ap-northeast-1" | ||
environment = "production" | ||
# Optional: Custom configuration | ||
prewarm_tag_key = "Project" | ||
prewarm_tag_value = "MyProject" | ||
lambda_schedule_expression = "rate(5 minutes)" | ||
scheduler_max_retry_attempts = 0 | ||
} | ||
``` | ||
|
||
### Tagging Lambda Functions for Warming | ||
|
||
To mark a Lambda function for warming, add the appropriate tags: | ||
|
||
```hcl | ||
resource "aws_lambda_function" "example" { | ||
# ... other configuration ... | ||
tags = { | ||
Prewarm = "true" # Default tags | ||
# Or use custom tags as configured in the module | ||
Project = "MyProject" | ||
} | ||
} | ||
``` | ||
|
||
## Requirements | ||
|
||
| Name | Version | | ||
|------|---------| | ||
| Terraform | >= 1.8.0 | | ||
| AWS Provider | >= 5.54 | | ||
| Python | >= 3.11 (for Lambda runtime) | | ||
|
||
## Variables | ||
|
||
### Required Variables | ||
|
||
| Name | Description | Type | | ||
|------|-------------|------| | ||
| aws_region | AWS region where the Lambda warmer will be deployed | string | | ||
| environment | Environment name (e.g., prod, dev, staging) | string | | ||
|
||
### Optional Variables | ||
|
||
| Name | Description | Type | Default | | ||
|------|-------------|------|---------| | ||
| lambda_schedule_expression | Schedule expression for the warmer | string | "rate(5 minutes)" | | ||
| scheduler_max_retry_attempts | Max retry attempts for scheduler | number | 0 | | ||
| prewarm_tag_key | Tag key for identifying functions to warm | string | "Prewarm" | | ||
| prewarm_tag_value | Expected value of the warming tag | string | "true" | | ||
|
||
## Outputs | ||
|
||
| Name | Description | | ||
|------|-------------| | ||
| scheduler_group_arn | ARN of the EventBridge Scheduler Group | | ||
| scheduler_arn | ARN of the EventBridge Scheduler | | ||
| lambda_function_name | Name of the Lambda warmer function | | ||
| lambda_function_arn | ARN of the Lambda warmer function | | ||
| lambda_role_name | Name of the Lambda IAM Role | | ||
| lambda_role_arn | ARN of the Lambda IAM Role | | ||
|
||
## Directory Structure | ||
|
||
```plaintext | ||
. | ||
├── README.md | ||
├── main.tf # Main Terraform configuration | ||
├── variables.tf # Input variables | ||
├── outputs.tf # Output definitions | ||
└── lambda_warmer/ # Lambda function code | ||
└── lambda_function.py # Python implementation | ||
``` | ||
|
||
## IAM Roles and Permissions | ||
|
||
The module creates two separate IAM roles: | ||
|
||
1. Lambda Role: | ||
- CloudWatch Logs access | ||
- Lambda function invocation | ||
|
||
2. EventBridge Scheduler Role: | ||
- Permission to invoke the warmer Lambda function | ||
|
||
## Schedule Expression Examples | ||
|
||
- Every 5 minutes: `rate(5 minutes)` | ||
- Every hour: `rate(1 hour)` | ||
- Daily at 2 AM UTC: `cron(0 2 * * ? *)` | ||
|
||
## Monitoring and Logs | ||
|
||
The Lambda warmer function logs its activities to CloudWatch Logs. You can monitor: | ||
|
||
- Functions being warmed | ||
- Warming success/failure | ||
- Number of functions processed |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,120 @@ | ||
import json | ||
import logging | ||
import os | ||
|
||
import boto3 | ||
|
||
# Initialize AWS Lambda client and logger | ||
lambda_client = boto3.client("lambda") | ||
logger = logging.getLogger() | ||
logger.setLevel(logging.INFO) | ||
|
||
# Get tag key and value from environment variables | ||
PREWARM_TAG_KEY = os.environ.get("PREWARM_TAG_KEY", "Prewarm") | ||
PREWARM_TAG_VALUE = os.environ.get("PREWARM_TAG_VALUE", "true") | ||
|
||
|
||
def get_prewarm_functions(): | ||
""" | ||
Fetch all Lambda functions and filter those with the specified tag key and value | ||
from environment variables. | ||
Environment Variables: | ||
PREWARM_TAG_KEY: The tag key to check (e.g., "Project") | ||
PREWARM_TAG_VALUE: The expected value for the tag (e.g., "DemoProject") | ||
Returns: | ||
list: A list of function names that require prewarming. | ||
""" | ||
try: | ||
|
||
logger.info( | ||
"Searching for functions with %s=%s", PREWARM_TAG_KEY, PREWARM_TAG_VALUE | ||
) | ||
|
||
functions = [] | ||
next_marker = None # Marker for pagination | ||
|
||
# Iterate through all pages of Lambda functions | ||
while True: | ||
if next_marker: | ||
response = lambda_client.list_functions(Marker=next_marker) | ||
else: | ||
response = lambda_client.list_functions() | ||
|
||
# Check each function's tags to identify prewarm candidates | ||
for func in response["Functions"]: | ||
function_name = func["FunctionName"] | ||
function_arn = func["FunctionArn"] | ||
|
||
# Retrieve tags for the current function | ||
tags = lambda_client.list_tags(Resource=function_arn) | ||
|
||
# Check if the function has the specified tag and value | ||
if tags.get("Tags", {}).get(PREWARM_TAG_KEY) == PREWARM_TAG_VALUE: | ||
functions.append(function_name) | ||
logger.info( | ||
"Function %s matched criteria: %s=%s", | ||
function_name, | ||
PREWARM_TAG_KEY, | ||
PREWARM_TAG_VALUE, | ||
) | ||
|
||
# Check if there are more pages | ||
next_marker = response.get("NextMarker") | ||
if not next_marker: | ||
break | ||
|
||
logger.info("Found %d functions to prewarm: %s", len(functions), functions) | ||
return functions | ||
|
||
except Exception as e: | ||
logger.error("Error while fetching functions: %s", e) | ||
raise | ||
|
||
|
||
def invoke_lambda(function_name): | ||
""" | ||
Trigger a specified Lambda function to keep it warm. | ||
Args: | ||
function_name (str): The name of the Lambda function to prewarm. | ||
""" | ||
try: | ||
# Use Event invocation type to avoid blocking | ||
lambda_client.invoke( | ||
FunctionName=function_name, | ||
InvocationType="Event", # Asynchronous invocation to reduce latency | ||
Payload=json.dumps({"action": "PREWARM"}), # Send a custom prewarm signal | ||
) | ||
logger.info("Successfully prewarmed %s", function_name) | ||
except Exception as e: | ||
logger.error("Failed to prewarm %s: %s", function_name, e) | ||
|
||
|
||
def lambda_handler(event, context): | ||
""" | ||
Entry point for the Lambda function. Retrieves the list of Lambda functions to prewarm | ||
and invokes them asynchronously. | ||
Args: | ||
event (dict): AWS event data (not used in this implementation). | ||
context (object): AWS Lambda context object (not used in this implementation). | ||
Returns: | ||
dict: A status object with the results of the prewarm operation. | ||
""" | ||
logger.info("Starting prewarmer...") | ||
try: | ||
# Step 1: Get the list of functions to prewarm | ||
prewarm_functions = get_prewarm_functions() | ||
|
||
# Step 2: Invoke each function asynchronously | ||
for function_name in prewarm_functions: | ||
invoke_lambda(function_name) | ||
|
||
logger.info("Prewarm process completed.") | ||
return {"status": "SUCCESS", "prewarmed_functions": prewarm_functions} | ||
except Exception as e: | ||
logger.error("Prewarmer encountered an error: %s", e) | ||
return {"status": "ERROR", "message": str(e)} |
Oops, something went wrong.