Unstopped

Local AWS Emulation with LocalStack

Now comes the big daddy of all the AWS cloud mocking and emulation frameworks. LocalStack is a drop-in single-container solution to provide 80+ AWS services at your immediate disposal. In fact, it's a full-fledged enterprise-level solution to emulate AWS services and Lambdas (serverless compute services) entirely on your local machine without any external dependencies. LocalStack is simply your local cloud sandbox for development, testing, and experimentation.

To begin with LocalStack, I use a very popular IaC automation tool known as Terraform. If Terraform is new to you, it provides a declarative configuration language with tunable properties to create resources on services such as AWS. The Terraform ecosystem also provides official modules to create various cloud resource combos as configurable stacks. I'll use the official Terraform AWS module test examples here to familiarize you with LocalStack.

First things first: Create the Dockerfile in Listing 21, the Terraform module in Listing 22, and the script in Listing 23 to create an image to run Terraform actions against the localhost. Then create the necessary Docker image:

docker build -f Dockerfile_TerraformWS . -t terraformws

FROM hashicorp/terraform
COPY aws_override.tf /etc/terraform/
COPY terraform_awsemu_test.sh /usr/local/bin/tfrmawstst

provider "aws" {
  region                      = "us-west-1"
  access_key                  = "mock_access_key"
  s3_use_path_style           = true
  secret_key                  = "mock_secret_key"
  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true
  endpoints {
    acm            = "http://localstack:4566"
    apigateway     = "http://localstack:4566"
    cloudformation = "http://localstack:4566"
    cloudwatch     = "http://localstack:4566"
    dynamodb       = "http://localstack:4566"
    es             = "http://localstack:4571"
    ec2            = "http://localstack:4566"
    firehose       = "http://localstack:4566"
    iam            = "http://localstack:4566"
    kinesis        = "http://localstack:4566"
    kms            = "http://localstack:4566"
    lambda         = "http://localstack:4566"
    route53        = "http://localstack:4566"
    redshift       = "http://localstack:4566"
    s3             = "http://localstack:4566"
    secretsmanager = "http://localstack:4566"
    ses            = "http://localstack:4566"
    sns            = "http://localstack:4566"
    sqs            = "http://localstack:4566"
    ssm            = "http://localstack:4566"
    stepfunctions  = "http://localstack:4566"
    sts            = "http://localstack:4566"
  }
}

#! /bin/sh
TAMD=${1}
EXDR=${2}
OPRN=${3}
NUMOPTNMX=4
TRFRMCNFL=${TRFRMCNFL:-'/etc/terraform/aws_override.tf'}
TRFRMWLOC=${TRFRMWLOC:-'/tmp/terraform'}
printUsage() {
  cat <<EOF
 Usage: $(basename "${0}") <terraform aws module> <example dir> <plan|apply|destroy>
EOF
  exit 0
}
parseArgs() {
  if [[ $# -gt ${NUMOPTNMX} ]]
  then
    printUsage
  fi
  if [[ "${OPRN}" != "plan" ]] && [[ "${OPRN}" != "apply" ]] && [[ "${OPRN}" != "destroy" ]]
  then
    printUsage
  fi
}
preProcess() {
  if [[ -d "/tmp/terraform-aws-${TAMD}" ]]
  then
    cd "/tmp/terraform-aws-${TAMD}"
    git pull
    cd "examples/${EXDR}/"
    terraform init -input=false
  else
    if git clone "https://github.com/terraform-aws-modules/terraform-aws-${TAMD}.git" "/tmp/terraform-aws-${TAMD}"
    then
      if [[ -e "${TRFRMCNFL}" ]]
      then
        if [[ -d "/tmp/terraform-aws-${TAMD}/examples/${EXDR}/" ]]
        then
          if cp -f "${TRFRMCNFL}" "/tmp/terraform-aws-${TAMD}/examples/${EXDR}/"
          then
            cd "/tmp/terraform-aws-${TAMD}/examples/${EXDR}/"
            terraform init -input=false
          fi
        fi
      fi
    fi
  fi
}
runOprtn() {
  cd "/tmp/terraform-aws-${TAMD}/examples/${EXDR}/"
  if [[ "${OPRN}" = "apply" ]] || [[ "${OPRN}" = "destroy" ]]
  then
    terraform ${OPRN} -input=false -auto-approve
  else
    terraform "${OPRN}"
  fi
}
main() {
  parseArgs
  preProcess
  runOprtn
}
main 2>&1

Next, create the YAML file in Listing 24 and execute the command

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock:ro -v ./terraform_localstack_stack.yml:/etc/compose/terraform_localstack_stack.yml:ro docker docker compose -f /etc/compose/terraform_localstack_stack.yml up -d

services:
  localstack:
    image: localstack/localstack:${LSTKTAG:-latest}
    container_name: localstack
    hostname: localstack
    volumes:
      - "${TMPDIR:-/tmp/terraform}:/tmp/terraform"
      - /var/run/docker.sock:/var/run/docker.sock:ro
    ports:
      - "14566:4566"
      - "14510-14559:4510-4559"
    environment:
      - DEBUG=${DEBUG:-0}
    healthcheck:
      test: ["CMD", "curl", "-I", "localhost:4566/_localstack/health"]
      interval: 5s
      timeout: 3s
      retries: 5
    restart: unless-stopped
  terraformws:
    image: terraformws:${TFRMTAG:-latest}
    container_name: terraformws
    hostname: terraformws
    volumes:
      - ./aws.tf:/etc/terraform/aws.tf:ro
      - ./terraform_awsemu_test.sh:/usr/local/bin/tfrmawstst:ro
    entrypoint: ash
    command: "-c 'while true; do sleep 5; done'"
    depends_on:
      localstack:
        condition: service_healthy
networks:
  default:
    name: awsmockemu-demo
    external: true

to bring up the necessary containers.

You are now fully ready to demonstrate the power of LocalStack to create AWS resources with the official Terraform modules. The first demo uses the AWS Relational Database Service (Amazon RDS) designed to simplify the setup, operation, and scaling of a relational database for use in applications. Administration processes such as patching the database software, backing up databases, and enabling point-in-time recovery are managed automatically. To start, execute

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock:ro -v ./terraform_localstack_stack.yml:/etc/compose/terraform_localstack_stack.yml:ro docker docker compose -f /etc/compose/terraform_localstack_stack.yml exec terraformws sh -c "tfrmawstst rdscomplete-postgres apply"

Terraform first shows which AWS resources will be created for a complete RDS Postgres module (Figure 7) and then creates everything.

Figure 7: Terraform in action for an RDS Postgres module.

Next, try your hand with the very popular Amazon Redshift service used by tens of thousands of customers every day to modernize their data analytics workloads. Just execute the command

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock:ro -v ./terraform_localstack_stack.yml:/etc/compose/terraform_localstack_stack.yml:ro docker docker compose -f /etc/compose/terraform_localstack_stack.yml exec terraformws sh -c "tfrmawstst redshift complete apply"

and profit from playing with Redshift locally without spending even a penny.

The next demo is for DynamoDB, a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. If you execute

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock:ro -v ./terraform_localstack_stack.yml:/etc/compose/terraform_localstack_stack.yml:ro docker docker compose -f /etc/compose/terraform_localstack_stack.yml exec terraformws sh -c "tfrmawstst dynamodb-table autoscaling apply"

you create an AWS DynamoDB table with autoscaling and let Terraform create all the necessary cloud resources locally (Figure 8).

Figure 8: Terraform in action for an autoscaled dynamodb-table module. Note that Terraform shows the Plan for adding, changing, and destroying before proceeding.

To wrap up this quick exploration of LocalStack, the next example uses the Elastic Kubernetes Service (Amazon EKS), a managed Kubernetes service to run Kubernetes in the AWS cloud and on-premises data centers. The command

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock:ro -v ./terraform_localstack_stack.yml:/etc/compose/terraform_localstack_stack.yml:ro docker docker compose -f /etc/compose/terraform_localstack_stack.yml exec terraformws sh -c "tfrmawstst eks eks_managed_node_group apply"

illustrates how easy it is to create any cloud service locally with LocalStack.

All these applications are just the tip of the LocalStack iceberg which is full of great functionalities for AWS services, along with a number of integrations, extensions, tools, and more. Please browse through LocalStack's extensive documentation [7] to become familiar with its power.

Conclusion

Mocking and emulation are essential ingredients in the modern API-driven, cloud-native world. Sooner or later, development, QA, and other experimental environments require cost-effective solutions to unblock development and operations teams. GCP provides free official emulators to unblock teams. In the AWS world, Moto is an elegant library to mock cloud services. LocalStack is an enterprise-level solution for full emulation of hundreds of AWS services. These effective solutions will enable your company to move fast on the cloud-native journey without burning holes in your pockets.