Day 4 - Lambda Concurrency Control: The Token Bucket Algorithm and Error Handling Layers

Lambda's concurrency model looks simple at first glance. It seems to be just a question of "how many copies of the function run at the same time." In reality, four different mechanisms are layered on top of each other: the account-wide limit, per-function Reserved concurrency, Provisioned concurrency, and the burst limit. If you do not understand this hierarchy, you cannot explain "why is my Lambda returning 429" or "why is my function slow because of another team's function."

Error handling is the same. The three invocation types (synchronous, asynchronous, ESM) react completely differently when an error occurs. If you do not know this, you end up with incidents like an entire Kinesis shard getting blocked or SQS messages being reprocessed forever.

Defining and Calculating Concurrency

Lambda concurrency is the number of function instances running at the same time at a given moment. The formula is simple.

Concurrency = requests per second (RPS) × average execution time (seconds)

Example:
- RPS = 1,000, average execution time = 200ms (0.2 seconds)
- Concurrency = 1,000 × 0.2 = 200

- RPS = 100, average execution time = 5 seconds
- Concurrency = 100 × 5 = 500

As this formula shows, the longer the execution time, the more concurrency is consumed. If 100 Lambdas with a 15-minute timeout are running at the same time, they occupy 1,500 concurrency during those 15 minutes, which already exceeds the account default limit (1,000).

The Complete Picture of the Four Concurrency Layers

Account/Region-wide limit: 1,000 (default, increase can be requested)
│
├── [Function A] Reserved: 300 → up to 300 instances
│   └── Provisioned: 20 → always 20 pre-initialized
│
├── [Function B] Reserved: 200 → up to 200 instances
│
└── [All other functions] Shared pool: 500 (1000 - 300 - 200)
    └── All functions without Reserved compete for these 500

Reserved Concurrency:

• Both the concurrency upper limit and a dedicated allocation for the function
• This function runs at most N instances, and other functions cannot use those N
• Setting it to 0 blocks the function completely (ThrottlingException)
• Cost: none

Provisioned Concurrency:

• Pre-initializes the specified number of MicroVMs and keeps them waiting
• Runs only the INVOKE phase immediately, with no cold start
• Can be set only on a version or alias (not $LATEST)
• Cost: number of initialized instances × time × memory (GB) × $0.0000097222/GB-second

# Set Reserved Concurrency
aws lambda put-function-concurrency \
  --function-name payment-api \
  --reserved-concurrent-executions 300
 
# Block completely (every request gets an immediate ThrottlingException)
aws lambda put-function-concurrency \
  --function-name maintenance-mode-function \
  --reserved-concurrent-executions 0
 
# Provisioned Concurrency (on an alias)
aws lambda put-provisioned-concurrency-config \
  --function-name payment-api \
  --qualifier prod \
  --provisioned-concurrent-executions 20

⚠️ Trap: When you set Reserved Concurrency, that amount is deducted from the account-wide pool. If you set Reserved=500 on function A, the other functions share only 500. At peak times when many functions run at once, a situation like "why are other functions suddenly being throttled?" can occur because of a Reserved setting.

Burst Concurrency Limit: A Speed Limit on Scale-Out

Lambda automatically increases concurrency when traffic spikes. But it cannot scale up infinitely fast.

Scaling rate: Per function and per Region, up to 1,000 execution environments are added every 10 seconds (in terms of requests per second, that is 10,000 RPS every 10 seconds). It changed to this model in late 2023.

Previous model: At the account level, 500 to 3,000 were added per Region in the first minute, and then 500 more per minute after that. These numbers remain in old materials and practice question sets, so keep the two apart.

What this means: if traffic suddenly arrives that needs 5,000 concurrency, then when the account limit is 5,000 or more, it scales up over about 50 seconds (+1,000 every 10 seconds). If the account limit is the default 1,000, it stops at 1,000. The rate limit is per function, so it applies separately to each function. Unused rate headroom does not carry over to the next 10 seconds.

Requests that exceed this rate or limit receive a throttling error (429).

Diagnosing Concurrency Problems with CloudWatch Metrics

# Query concurrency metrics from CloudWatch
aws cloudwatch get-metric-statistics \
  --namespace AWS/Lambda \
  --metric-name ConcurrentExecutions \
  --dimensions Name=FunctionName,Value=payment-api \
  --start-time 2026-05-31T00:00:00Z \
  --end-time 2026-05-31T23:59:59Z \
  --period 300 \
  --statistics Maximum

💡 Related theory: IteratorAge is the difference between the current time and the Put time of the record being processed from Kinesis/DynamoDB Streams. If this value keeps rising, Lambda cannot keep up with the stream rate. The fixes are to increase the number of Kinesis shards (spread the throughput), increase the Lambda ParallelizationFactor (the number of parallel Lambdas per shard), or optimize the Lambda execution time.

Error Handling Layers: A Full Comparison by Invocation Type

When an error occurs, "who retries" and "where the failure event goes" are completely different for each invocation type.

Synchronous invocation (API Gateway, ALB, direct invoke):

• If Lambda throws an error, it responds with HTTP 200 + a FunctionError header
• Retrying is the caller's responsibility (API Gateway default: no retries)
• No DLQ. The error event goes nowhere
• If the client gets a 503 or 500, it must implement its own retry

Asynchronous invocation (S3, SNS, EventBridge, direct invoke with the Event type):

• The Lambda service holds the event in an internal queue
• Automatic retries: up to 2, at 1-minute/2-minute intervals (exponential backoff)
• On final failure, sent to a DLQ (SQS or SNS) or a Destinations OnFailure target
• Maximum event age is 21,600 seconds (6 hours) — after that it is not processed

ESM polling (SQS, Kinesis, DDB Streams):

• SQS: after the visibility timeout the message returns to the queue. When maxReceiveCount is exceeded, it goes to the SQS DLQ
• Kinesis/DDB Streams: infinite retries by default (the shard gets blocked). You must limit it with MaximumRetryAttempts
• A Lambda function's DLQ and an SQS queue's DLQ are completely different concepts

Error handling flow by invocation type

[Synchronous - API Gateway]
Client → API GW → Lambda → error occurs
                              ↓
              FunctionError response to API GW
                              ↓
              5XX returned to the client (depending on API GW settings)
              Retry = decided by the client code

[Asynchronous - S3]
S3 upload → Lambda internal queue → Lambda runs → error occurs
                               ↓
                         automatic retry after 1 minute
                               ↓ (fails again)
                         automatic retry after 2 minutes
                               ↓ (fails again)
                     → DLQ or Destinations OnFailure

[ESM - SQS]
Message → Lambda polls → Lambda runs → error occurs
                               ↓
         message returns to the queue after the visibility timeout ends
                               ↓ (maxReceiveCount exceeded)
                     → SQS DLQ (not the function DLQ!)

DLQ vs Lambda Destinations: Exact Selection Criteria

Why Destinations are better than a DLQ:

1. Can handle success events: It can send an event when the function succeeds too (auditing, triggering the next step)
2. More kinds of targets: Not just SQS and SNS, but EventBridge and Lambda as well
3. Rich context: JSON that includes the original event + request context + response/error information
4. Standard asynchronous pattern: Fits fan-out in event-driven architectures better

# Event structure delivered by Destinations OnFailure (example)
{
  "version": "1.0",
  "timestamp": "2026-05-31T09:30:00Z",
  "requestContext": {
    "requestId": "abc-123",
    "functionArn": "arn:aws:lambda:...:function:payment-api:prod",
    "condition": "RetriesExhausted",
    "approximateInvokeCount": 3  # 1 original + 2 retries
  },
  "requestPayload": {
    "customerId": "C001",
    "amount": 9999
  },
  "responseContext": {
    "statusCode": 200,  # At the Lambda API level, 200 even if the function errored
    "executedVersion": "$LATEST",
    "functionError": "Unhandled"
  },
  "responsePayload": {
    "errorMessage": "Payment gateway timeout",
    "errorType": "TimeoutError"
  }
}

SQS DLQ vs Lambda DLQ: Same Name, Different Concepts

This is an easy place to get confused on the exam.

SQS DLQ (Dead Letter Queue):

• A property of an SQS queue
• When an SQS message has been consumed more than maxReceiveCount times, it moves to the SQS DLQ
• In an SQS-Lambda ESM, if Lambda fails repeatedly → the message is retried at every visibility timeout → maxReceiveCount exceeded → SQS DLQ

Lambda DLQ:

• A setting of the Lambda function
• Sends the event when an asynchronous invocation finally fails (the original and both retries all fail)
• The Lambda DLQ does not apply to an SQS-Lambda ESM. ESM failures are handled by the SQS DLQ

Scenario: Lambda keeps failing in an SQS → Lambda ESM

Regardless of whether a Lambda DLQ is configured!
  → the message returns to the SQS queue after the visibility timeout
  → after maxReceiveCount (default 3) is exceeded
  → it moves to the DLQ configured on the SQS queue

The Lambda DLQ works for asynchronous invocations such as S3/SNS

A Practical Pattern Using Reserved + Provisioned Concurrency Together

Scenario: Financial API (payment processing)
- Normal traffic: 50 concurrency
- Peak traffic: 200 concurrency
- Cold starts not acceptable (response time SLA: p99 < 500ms)

Configuration:
Reserved Concurrency = 300  (300 dedicated from the account limit)
Provisioned Concurrency = 50  (always 50 waiting in a warm state)

Behavior:
- Normally: 50 Provisioned instances handle the load, no cold starts
- When traffic spikes: requests beyond 50 create new instances (cold starts possible)
- When traffic exceeds 300: ThrottlingException (Reserved acts as the upper limit)
- Other functions cannot use these 300 (dedicated allocation)

Auto Scaling setting:
Automatically add PC when PC is 80% used

# Automatically adjust Provisioned Concurrency with Application Auto Scaling
aws application-autoscaling register-scalable-target \
  --service-namespace lambda \
  --resource-id function:payment-api:prod \
  --scalable-dimension lambda:function:ProvisionedConcurrency \
  --min-capacity 10 \
  --max-capacity 200
 
aws application-autoscaling put-scaling-policy \
  --service-namespace lambda \
  --resource-id function:payment-api:prod \
  --scalable-dimension lambda:function:ProvisionedConcurrency \
  --policy-name pc-tracking-policy \
  --policy-type TargetTrackingScaling \
  --target-tracking-scaling-policy-configuration '{
    "TargetValue": 0.7,
    "PredefinedMetricSpecification": {
      "PredefinedMetricType": "LambdaProvisionedConcurrencyUtilization"
    }
  }'

Asynchronous Error Handling Pattern: Business Errors vs System Errors

You should not handle every error in Lambda the same way.

System errors (retryable): DB connection failures, HTTP timeouts, transient 503s. Throw an exception so you can take advantage of Lambda's retry mechanism.

Business errors (no retry needed): Invalid data, no permission, domain rule violations. Retrying gives the same result, so do not throw an exception; instead return a success response or route the error event to another queue.

import logging
import json
from botocore.exceptions import ClientError
 
logger = logging.getLogger()
logger.setLevel(logging.INFO)
 
class BusinessError(Exception):
    """Domain rule violation — retrying is pointless"""
    pass
 
class RetryableError(Exception):
    """Transient system error — retrying may succeed"""
    pass
 
def lambda_handler(event, context):
    try:
        order_id = event.get('orderId')
        if not order_id:
            # Business error — retrying won't help
            # Return an error response instead of an exception, or send directly to SQS
            logger.error(f"Invalid order ID: {event}")
            return {'status': 'BUSINESS_ERROR', 'reason': 'orderId missing'}
        
        result = process_order(order_id)
        return {'status': 'SUCCESS', 'orderId': order_id}
    
    except RetryableError as e:
        # Retry needed — re-raise the exception to trigger a Lambda retry
        logger.warning(f"Transient error, will retry: {str(e)}")
        raise  # Lambda's asynchronous retry mechanism kicks in
    
    except Exception as e:
        # Unexpected error — retry while investigating the situation
        logger.error(f"Unexpected error: {str(e)}", exc_info=True)
        raise

Throttling Response Patterns

A client that receives 429 on a synchronous invocation:

import boto3
import random
import time
from botocore.exceptions import ClientError
 
lambda_client = boto3.client('lambda')
 
def invoke_with_retry(function_name, payload, max_retries=3):
    for attempt in range(max_retries):
        try:
            return lambda_client.invoke(
                FunctionName=function_name,
                InvocationType='RequestResponse',
                Payload=payload
            )
        except ClientError as e:
            if e.response['Error']['Code'] == 'TooManyRequestsException':
                wait_time = (2 ** attempt) + (random.random() * 0.5)  # jitter
                logger.warning(f"Throttled. Retrying in {wait_time:.2f}s (attempt {attempt+1}/{max_retries})")
                time.sleep(wait_time)
            else:
                raise
    raise Exception("Maximum retry count exceeded")

Using SQS as a buffer: If a sudden traffic spike is expected, instead of having callers invoke Lambda directly, put the requests in SQS and let the Lambda ESM regulate the processing rate. SQS acts as the buffer.

[Traffic spike] → [Direct Lambda invocation] → 429 throttling
                                         ↓
[Traffic spike] → [SQS queue] → [Lambda ESM] → processing rate adjusted automatically

Summary

Lambda concurrency control is a system in which four different mechanisms are layered. Reserved provides per-function isolation and an upper limit, Provisioned removes cold starts, the burst limit caps the scale-out speed, and the account limit sets the overall ceiling. Error handling happens in completely different layers for each invocation type, and you need to know exactly "where retries happen and where failure events go" for each invocation type.

In the next post, we will review all of Week 3 and check every Lambda concept with scenario-based questions.