How to Convert Files in AWS Lambda

The standard advice for file conversion in Lambda is to bundle FFmpeg as a Lambda Layer. This works but has real costs: 50-250MB layer size, layer version management, cold start latency from loading the layer, and the operational burden of updating FFmpeg across all functions. The alternative: call an external API and stay in pure-Python territory.

This guide covers the complete pattern: S3 trigger → Lambda → ChangeThisFile API → S3 output. No Lambda Layers, no system binary installs, no container image packaging.

TL;DR

• Lambda runtime: Python 3.11 (no layers needed)
• Dependencies: only requests (or use urllib3 from stdlib)
• Pattern: S3 upload → S3 trigger → Lambda → API → S3 output
• Store API key in: AWS Systems Manager Parameter Store (SecureString) or Lambda env var
• Cold start: ~200ms (pure Python) vs ~1-3s (FFmpeg layer)

Architecture overview

The complete flow:

User uploads file
       ↓
  S3 (input bucket)
       ↓
  S3 Event Notification → Lambda (python3.11)
       ↓
  Lambda downloads file from S3
       ↓
  POST /v1/convert to ChangeThisFile API
       ↓
  Lambda uploads converted file to S3 (output bucket)
       ↓
  SNS notification / callback URL (optional)

No FFmpeg, no LibreOffice, no Lambda Layers. The conversion server handles all the heavy lifting.

Lambda function: complete implementation

# lambda_function.py
import boto3
import requests
import os
import io
from urllib.parse import unquote_plus

s3 = boto3.client("s3")

# Store API key in Lambda environment variables or SSM Parameter Store
API_KEY = os.environ["CTF_API_KEY"]

# Format mapping: input extension → output format
FORMAT_MAP = {
    ".heic": "jpg",
    ".heif": "jpg",
    ".png": "webp",
    ".wav": "mp3",
    ".aiff": "mp3",
    ".doc": "pdf",
    ".docx": "pdf",
    ".rtf": "pdf",
}

OUTPUT_BUCKET = os.environ.get("OUTPUT_BUCKET", os.environ.get("INPUT_BUCKET"))
OUTPUT_PREFIX = os.environ.get("OUTPUT_PREFIX", "converted/")

def handler(event, context):
    results = []

    for record in event["Records"]:
        bucket = record["s3"]["bucket"]["name"]
        key = unquote_plus(record["s3"]["object"]["key"])
        size = record["s3"]["object"].get("size", 0)

        # Get file extension
        ext = "." + key.rsplit(".", 1)[-1].lower() if "." in key else ""
        target = FORMAT_MAP.get(ext)

        if not target:
            print(f"Skipping {key}: no conversion defined for {ext}")
            continue

        if size > 25 * 1024 * 1024:
            print(f"Skipping {key}: file too large ({size // (1024*1024)}MB, limit 25MB)")
            continue

        print(f"Converting {key} ({ext} → {target})")

        try:
            # Download from S3
            response = s3.get_object(Bucket=bucket, Key=key)
            file_bytes = response["Body"].read()
            filename = key.rsplit("/", 1)[-1]  # just the filename, not full key

            # Call ChangeThisFile API
            api_resp = requests.post(
                "https://changethisfile.com/v1/convert",
                headers={"Authorization": f"Bearer {API_KEY}"},
                files={"file": (filename, io.BytesIO(file_bytes), "application/octet-stream")},
                data={"target": target},
                timeout=120,
            )
            api_resp.raise_for_status()

            # Build output key
            stem = filename.rsplit(".", 1)[0]
            out_key = OUTPUT_PREFIX + stem + "." + target

            # Upload to S3
            s3.put_object(
                Bucket=OUTPUT_BUCKET,
                Key=out_key,
                Body=api_resp.content,
                ContentType=f"image/{target}" if target in ("jpg", "webp", "png") else "application/octet-stream",
            )

            in_kb = len(file_bytes) // 1024
            out_kb = len(api_resp.content) // 1024
            print(f"Done: s3://{OUTPUT_BUCKET}/{out_key} ({in_kb}KB → {out_kb}KB)")
            results.append({"input": key, "output": out_key, "status": "success"})

        except Exception as e:
            print(f"Error processing {key}: {e}")
            results.append({"input": key, "status": "error", "error": str(e)})

    return {"statusCode": 200, "body": results}

Deployment with AWS SAM

# template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Globals:
  Function:
    Timeout: 180  # 3 min — allow for large file conversion
    MemorySize: 512
    Environment:
      Variables:
        CTF_API_KEY: !Ref CtfApiKey
        OUTPUT_BUCKET: !Ref OutputBucket
        OUTPUT_PREFIX: converted/

Parameters:
  CtfApiKey:
    Type: AWS::SSM::Parameter::Value
    Default: /changethisfile/api-key

Resources:
  ConvertFunction:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: python3.11
      Handler: lambda_function.handler
      Policies:
        - S3ReadPolicy: {BucketName: !Ref InputBucket}
        - S3WritePolicy: {BucketName: !Ref OutputBucket}
      Events:
        S3Upload:
          Type: S3
          Properties:
            Bucket: !Ref InputBucket
            Events: s3:ObjectCreated:*
            Filter:
              S3Key:
                Rules:
                  - Name: prefix
                    Value: uploads/

  InputBucket:
    Type: AWS::S3::Bucket

  OutputBucket:
    Type: AWS::S3::Bucket

# requirements.txt
requests==2.31.0

# Deploy
sam build && sam deploy --guided

# Store API key in SSM (not in template.yaml)
aws ssm put-parameter \
  --name /changethisfile/api-key \
  --value ctf_sk_your_key_here \
  --type SecureString

Production error handling

# Enhanced handler with dead letter queue pattern
import json

def handler(event, context):
    failed = []
    for record in event.get("Records", []):
        try:
            process_record(record)
        except Exception as e:
            key = record.get("s3", {}).get("object", {}).get("key", "unknown")
            print(f"CRITICAL: Failed to process {key}: {e}")
            failed.append({"key": key, "error": str(e)})

    if failed:
        # Raise to trigger Lambda retry or DLQ
        raise RuntimeError(f"{len(failed)} records failed: {json.dumps(failed)}")

Configure a Dead Letter Queue (SQS) on the Lambda function to capture failed records for manual review. Lambda retries S3-triggered functions up to 3 times by default.

Edge cases and gotchas

• Lambda timeout. Default Lambda timeout is 3 seconds. File conversion can take 10-120 seconds for large files. Set timeout to 180 seconds minimum. Use the template.yaml Timeout property.
• S3 event loop. If your input and output buckets are the same, the converted output file triggers another Lambda invocation. Either use separate buckets (recommended) or add a key prefix filter to ignore already-converted files.
• Memory allocation. The function loads the file into memory twice (S3 download + API response). For 25MB files, allocate at least 128MB. Set MemorySize: 512 to be safe.
• API key rotation. Store the key in SSM Parameter Store as a SecureString. Update SSM without redeploying Lambda by reading from SSM at function init time (cold start only).
• Large file workaround. Free tier limit is 25MB. For larger files, use a pre-signed S3 URL strategy: generate a pre-signed URL for the S3 object and pass the URL to the API. The API fetches it directly.

Cost comparison: API vs FFmpeg Layer

API approach wins on simplicity, cold start, and operational overhead. FFmpeg layer wins on marginal cost for very high volume (>100K conversions/month).

The API pattern keeps your Lambda functions small, fast, and maintainable. Skip the FFmpeg layer unless you're doing very high volume conversions where the per-conversion cost matters. Get a free API key — free tier covers 1,000 conversions/month.