Serverless computing with AWS Lambda
Light Work
For a number of reasons, it makes sense to use today's cloud-native infrastructure to run software without employing servers; instead, you can use an arms-length, abstracted serverless platform such as AWS Lambda.
For example, when you create a Lambda function (source code and a run-time configuration) and execute it, the AWS platform only bills you for the execution time, also called the "compute time." Simple tasks usually book only hundreds of milliseconds, as opposed to running an Elastic Compute Cloud (EC2) server instance all month long along with its associated costs.
In addition to reducing the cost and removing the often overlooked administrative burden of maintaining a fleet of servers to run your tasks, AWS Lambda also takes care of the sometimes difficult-to-get-right automatic scaling of your infrastructure. With Lambda, AWS promises that you can sit back with your feet up and rest assured that "your code runs in parallel" and that the platform will be "scaling precisely with the size of the workload" in an efficient and cost-effective manner [1].
In this article, I show you how to get started with AWS Lambda. Once you've seen that external connectivity is working, I'll use a Python script to demonstrate how you might use a Lambda function to monitor a website all year round, without the need of ever running a server.
For more advanced requirements, I'll also touch on how to get the internal networking set up correctly for a Lambda function to communicate with nonpublic resources (e.g., EC2 instances) hosted internally in AWS. Those Lambda functions will also be able to connect to the Internet, which can be challenging to get right.
On an established AWS infrastructures, most resources are usually segregated into their own virtual private clouds (VPCs) for security and organizational requirements, so I'll look at the workflow required to solve both internal and external connectivity headaches. I assume that you're familiar with the basics of the AWS Management Console and have access to an account in which you can test.
Less Is More
As already mentioned, be warned that Lambda function networking in AWS has a few quirks. For example, Internet Control Message Protocol (ICMP) traffic isn't permitted for running pings and other such network discovery services:
Lambda attempts to impose as few restrictions as possible on normal language and operating system activities, but there are a few activities that are disabled: Inbound network connections are blocked by AWS Lambda, and for outbound connections only TCP/IP sockets are supported, and ptrace (debugging) system calls are blocked. TCP port 25 traffic is also blocked as an anti-spam measure.
Digging a little deeper …, the Lambda OS kernel lacks the CAP_NET_RAW kernel capability to manipulate raw sockets.
So, you can't do ICMP or UDP from a Lambda function [2].
(Be warned that this page is a little dated and things may have changed.)
In other words, you're not dealing with the same networking stack that you might find on a friendly Debian box running in EC2. However, as I'll demonstrate in a moment, public Domain Name Service (DNS) lookups do work as you'd hope, usually with the use of the UDP protocol.
Less Said, The Better
The way to prove that DNS lookups work is, as you might have guessed, to use a short script that simply performs a DNS lookup. First, however, you should create your function. Figure 1 shows the AWS Management Console [3] Lambda service page with an orange Create function button.
If you're wearing your reading glasses, you might see that the name of the function I've typed is internet-access-function . I've also chosen Python 3.7 as the preferred run time. I leave the default Author from scratch option alone at the top.
For now, I ignore the execution role at the bottom of the page and visit that again later, because the clever gubbins behind the scenes will automatically assign an IAM profile, trimmed right down, by default: AWS wants you to log in to CloudWatch to check the execution of your Lambda function.
The next screen in Figure 2 shows the new function; you can see its name in the Designer section and that it has Amazon CloudWatch Logs permissions by default. Figure 2 is only the top of a relatively long page that includes the Designer options. Sometimes these options are hidden and you need to expand them with the arrow next to the word Designer .
Next, hide the Designer options by clicking on the aforementioned arrow. After a little scrolling down, you should see where you will paste your function code (Figure 3). A "Hello World" script, which I will run as an example, is already in the code area.
When I run the Hello World Lambda function by clicking Test , I get a big, green welcome box at the top of the screen (I had to scroll up a bit), and I can expand the details to show the output,
{ "statusCode": 200, "body": "\"Hello from Lambda!\"" }
which means the test worked. If you haven't created a test event yet, you'll see a pop-up dialog box the first time you run a test, and you'll be asked to add an Event Name .
If you do that now, you can just leave the default key1 and other information in place. You don't need to change these values just yet, because, to execute both the Hello World and the DNS lookup script, you don't need to pass any variables to your Lambda function from here. I called my Event Name EmptyTest and then clicked the orange Create button at the bottom.
Next, I'll paste the DNS Python lookup script
import socket ** def lambda_handler(event, context): data = socket.gethostbyname_ex('www.devsecops.cc') print (data) return
over the top of the Hello World example and click the orange Save button at the top.
To run the function as it stands (using only the default configuration options and making sure the indentation in your script is correct), simply click the Test button again; you should get another green success bar at the top of the screen.
The green bar will show null , because the script doesn't actually output anything. However, if you look in the Log Output section, you can see some output (Listing 1), with the IP address next to the DNS name you looked up.
Listing 1
DNS Lookup Output
START RequestId: 4e90b424-95d9-4453-a2f4-8f5259f5f263 Version: $LATEST ('www.devsecops.cc', [], [' 138.68.149.181' ]) END RequestId: 4e90b424-95d9-4453-a2f4-8f5259f5f263 REPORT RequestId: 4e90b424-95d9-4453-a2f4-8f5259f5f263 Duration: 70.72 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 55 MB Init Duration: 129.20 ms
More or Less
For the second Lambda task, you'll use a more sophisticated script that will allow you to monitor a website. The script for the Lambda function, with the kind permission of the people behind the base2Services GitHub page [4], will attempt to perform a two-way remote TCP port connection. Copy the handler.py
script (Listing 2) and paste it into the function tab, as before. If you can't copy the Python script easily, then click the Raw
option on the right side of the page and copy all of the raw text.
Listing 2
handler.py
001 import json 002 import os 003 import boto3 004 from time import perf_counter as pc 005 import socket 006 007 class Config: 008 """Lambda function runtime configuration""" 009 010 HOSTNAME = 'HOSTNAME' 011 PORT = 'PORT' 012 TIMEOUT = 'TIMEOUT' 013 REPORT_AS_CW_METRICS = 'REPORT_AS_CW_METRICS' 014 CW_METRICS_NAMESPACE = 'CW_METRICS_NAMESPACE' 015 016 def __init__(self, event): 017 self.event = event 018 self.defaults = { 019 self.HOSTNAME: 'google.com.au', 020 self.PORT: 443, 021 self.TIMEOUT: 120, 022 self.REPORT_AS_CW_METRICS: '1', 023 self.CW_METRICS_NAMESPACE: 'TcpPortCheck', 024 } 025 026 def __get_property(self, property_name): 027 if property_name in self.event: 028 return self.event[property_name] 029 if property_name in os.environ: 030 return os.environ[property_name] 031 if property_name in self.defaults: 032 return self.defaults[property_name] 033 return None 034 035 @property 036 def hostname(self): 037 return self.__get_property(self.HOSTNAME) 038 039 @property 040 def port(self): 041 return self.__get_property(self.PORT) 042 043 @property 044 def timeout(self): 045 return self.__get_property(self.TIMEOUT) 046 047 @property 048 def reportbody(self): 049 return self.__get_property(self.REPORT_RESPONSE_BODY) 050 051 @property 052 def cwoptions(self): 053 return { 054 'enabled': self.__get_property(self.REPORT_AS_CW_METRICS), 055 'namespace': self.__get_property(self.CW_METRICS_NAMESPACE), 056 } 057 058 class PortCheck: 059 """Execution of HTTP(s) request""" 060 061 def __init__(self, config): 062 self.config = config 063 064 def execute(self): 065 sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 066 sock.settimeout(int(self.config.timeout)) 067 try: 068 # start the stopwatch 069 t0 = pc() 070 071 connect_result = sock.connect_ex((self.config.hostname, int(self.config.port))) 072 if connect_result == 0: 073 available = '1' 074 else: 075 available = '0' 076 077 # stop the stopwatch 078 t1 = pc() 079 080 result = { 081 'TimeTaken': int((t1 - t0) * 1000), 082 'Available': available 083 } 084 print(f"Socket connect result: {connect_result}") 085 # return structure with data 086 return result 087 except Exception as e: 088 print(f"Failed to connect to {self.config.hostname}:{self.config.port}\n{e}") 089 return {'Available': 0, 'Reason': str(e)} 090 091 class ResultReporter: 092 """Reporting results to CloudWatch""" 093 094 def __init__(self, config): 095 self.config = config 096 self.options = config.cwoptions 097 098 def report(self, result): 099 if self.options['enabled'] == '1': 100 try: 101 endpoint = f"{self.config.hostname}:{self.config.port}" 102 cloudwatch = boto3.client('cloudwatch') 103 metric_data = [{ 104 'MetricName': 'Available', 105 'Dimensions': [ 106 {'Name': 'Endpoint', 'Value': endpoint} 107 ], 108 'Unit': 'None', 109 'Value': int(result['Available']) 110 }] 111 if result['Available'] == '1': 112 metric_data.append({ 113 'MetricName': 'TimeTaken', 114 'Dimensions': [ 115 {'Name': 'Endpoint', 'Value': endpoint} 116 ], 117 'Unit': 'Milliseconds', 118 'Value': int(result['TimeTaken']) 119 }) 120 121 result = cloudwatch.put_metric_data( 122 MetricData=metric_data, 123 Namespace=self.config.cwoptions['namespace'] 124 ) 125 126 print(f"Sent data to CloudWatch requestId=:{result['ResponseMetadata']['RequestId']}") 127 except Exception as e: 128 print(f"Failed to publish metrics to CloudWatch:{e}") 129 130 def port_check(event, context): 131 """Lambda function handler""" 132 133 config = Config(event) 134 port_check = PortCheck(config) 135 136 result = port_check.execute() 137 138 # report results 139 ResultReporter(config).report(result) 140 141 result_json = json.dumps(result, indent=4) 142 # log results 143 print(f"Result of checking {config.hostname}:{config.port}\n{result_json}") 144 145 # return to caller 146 return result
Now, click Save at the top right and look for the Handler input box on the right-hand side of where you pasted the code. You'll need to change the starting point for the Lambda function from lambda_function.lambda_handler to lambda_function.port_check , which is how the script is written. Be sure to click Save again.
Next, configure a new test event,
{ "HOSTNAME":"www.devsecops.cc", "PORT":"443", "TIMEOUT":5 }
adjusted a bit from the base2Services GitHub example [5]. Once you've adapted the parameters for your own system settings, go back to the EmptyTest box, pull down the menu, and click Configure test events to create a new parameter to pass to a test. I pasted the test event code over the top of the example JSON code, named it PortTest , and clicked Create .
Buy this article as PDF
(incl. VAT)