Cloud Engineer Guide: Serverless Event-Driven IaC with AWS Lambda, SQS DLQs & DynamoDB
Provisioning serverless event-driven architectures—where Amazon SQS queues trigger AWS Lambda consumers that write to DynamoDB—requires careful alignment between queue visibility timeouts, batch failure reporting, and Dead-Letter Queue (DLQ) retention policies. If `VisibilityTimeout` is shorter than the Lambda function's `Timeout`, in-flight messages become visible again and get processed twice; if `ReportBatchItemFailures` is omitted, a single poison message retries the entire batch of 10 messages. As a Senior Cloud Engineer, here is how I codify production serverless stacks in CloudFormation and Terraform.
1. Production-Hardened SQS + Lambda + DLQ CloudFormation / SAM Template
Notice three critical invariants below: `VisibilityTimeout` is set to 6× the Lambda timeout, `RedrivePolicy` routes poison messages to an encrypted DLQ after 3 attempts, and `FunctionResponseTypes: [ReportBatchItemFailures]` enables partial batch acknowledgement:
AWSTemplateFormatVersion: '2010-09-09'
Description: Production Event-Driven SQS + Lambda + DynamoDB Stack
Resources:
EventsDeadLetterQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: prod-order-events-dlq
MessageRetentionPeriod: 1209600 # 14 days
KmsMasterKeyId: alias/aws/sqs
EventsPrimaryQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: prod-order-events
VisibilityTimeout: 180 # 6x Lambda 30s timeout
KmsMasterKeyId: alias/aws/sqs
RedrivePolicy:
deadLetterTargetArn: !GetAtt EventsDeadLetterQueue.Arn
maxReceiveCount: 3
OrderProcessorMapping:
Type: AWS::Lambda::EventSourceMapping
Properties:
EventSourceArn: !GetAtt EventsPrimaryQueue.Arn
FunctionName: !GetAtt OrderProcessorFunction.Arn
BatchSize: 10
MaximumBatchingWindowInSeconds: 2
FunctionResponseTypes:
- ReportBatchItemFailures
2. Idempotent DynamoDB Conditional Writes (`attribute_not_exists`)
Because SQS standard queues guarantee at-least-once delivery, every Lambda consumer guards state mutations in DynamoDB using `ConditionExpression="attribute_not_exists(event_id)"` so duplicate deliveries are safely ignored.
3. Hire a Senior AWS & GCP Cloud Engineer
Need to harden your serverless AWS or Google Cloud infrastructure with Terraform or CloudFormation? Explore my Cloud Engineer Pillar Guide, review my Terraform Brownfield Import Guide, or book a 1-Day Serverless Architecture Sprint.