Light Work

For a number of reasons, it makes sense to use today's cloud-native infrastructure to run software without employing servers; instead, you can use an arms-length, abstracted serverless platform such as AWS Lambda.

For example, when you create a Lambda function (source code and a run-time configuration) and execute it, the AWS platform only bills you for the execution time, also called the "compute time." Simple tasks usually book only hundreds of milliseconds, as opposed to running an Elastic Compute Cloud (EC2) server instance all month long along with its associated costs.

In addition to reducing the cost and removing the often overlooked administrative burden of maintaining a fleet of servers to run your tasks, AWS Lambda also takes care of the sometimes difficult-to-get-right automatic scaling of your infrastructure. With Lambda, AWS promises that you can sit back with your feet up and rest assured that "your code runs in parallel" and that the platform will be "scaling precisely with the size of the workload" in an efficient and cost-effective manner [1].

In this article, I show you how to get started with AWS Lambda. Once you've seen that external connectivity is working, I'll use a Python script to demonstrate how you might use a Lambda function to monitor a website all year round, without the need of ever running a server.

For more advanced requirements, I'll also touch on how to get the internal networking set up correctly for a Lambda function to communicate with nonpublic resources (e.g., EC2 instances) hosted internally in AWS. Those Lambda functions will also be able to connect to the Internet, which can be challenging to get right.

On an established AWS infrastructures, most resources are usually segregated into their own virtual private clouds (VPCs) for security and organizational requirements, so I'll look at the workflow required to solve both internal and external connectivity headaches. I assume that you're familiar with the basics of the AWS Management Console and have access to an account in which you can test.

Less Is More

As already mentioned, be warned that Lambda function networking in AWS has a few quirks. For example, Internet Control Message Protocol (ICMP) traffic isn't permitted for running pings and other such network discovery services:

Lambda attempts to impose as few restrictions as possible on normal language and operating system activities, but there are a few activities that are disabled: Inbound network connections are blocked by AWS Lambda, and for outbound connections only TCP/IP sockets are supported, and ptrace (debugging) system calls are blocked. TCP port 25 traffic is also blocked as an anti-spam measure.

Digging a little deeper …, the Lambda OS kernel lacks the CAP_NET_RAW kernel capability to manipulate raw sockets.

So, you can't do ICMP or UDP from a Lambda function [2].

(Be warned that this page is a little dated and things may have changed.)

In other words, you're not dealing with the same networking stack that you might find on a friendly Debian box running in EC2. However, as I'll demonstrate in a moment, public Domain Name Service (DNS) lookups do work as you'd hope, usually with the use of the UDP protocol.

Less Said, The Better

The way to prove that DNS lookups work is, as you might have guessed, to use a short script that simply performs a DNS lookup. First, however, you should create your function. Figure 1 shows the AWS Management Console [3] Lambda service page with an orange Create function button.

Figure 1: The page where you will create a Lambda function.

If you're wearing your reading glasses, you might see that the name of the function I've typed is internet-access-function . I've also chosen Python 3.7 as the preferred run time. I leave the default Author from scratch option alone at the top.

For now, I ignore the execution role at the bottom of the page and visit that again later, because the clever gubbins behind the scenes will automatically assign an IAM profile, trimmed right down, by default: AWS wants you to log in to CloudWatch to check the execution of your Lambda function.

The next screen in Figure 2 shows the new function; you can see its name in the Designer section and that it has Amazon CloudWatch Logs permissions by default. Figure 2 is only the top of a relatively long page that includes the Designer options. Sometimes these options are hidden and you need to expand them with the arrow next to the word Designer .

Figure 2: A new Lambda function.

Next, hide the Designer options by clicking on the aforementioned arrow. After a little scrolling down, you should see where you will paste your function code (Figure 3). A "Hello World" script, which I will run as an example, is already in the code area.

Figure 3: Your Lambda function code will go here in place of the Hello World example.

When I run the Hello World Lambda function by clicking Test , I get a big, green welcome box at the top of the screen (I had to scroll up a bit), and I can expand the details to show the output,

{
  "statusCode": 200,
  "body": "\"Hello from Lambda!\""
}

which means the test worked. If you haven't created a test event yet, you'll see a pop-up dialog box the first time you run a test, and you'll be asked to add an Event Name .

If you do that now, you can just leave the default key1 and other information in place. You don't need to change these values just yet, because, to execute both the Hello World and the DNS lookup script, you don't need to pass any variables to your Lambda function from here. I called my Event Name EmptyTest and then clicked the orange Create button at the bottom.

Next, I'll paste the DNS Python lookup script

import socket
 **
def lambda_handler(event, context):
    data = socket.gethostbyname_ex('www.devsecops.cc')
    print (data)
    return

over the top of the Hello World example and click the orange Save button at the top.

To run the function as it stands (using only the default configuration options and making sure the indentation in your script is correct), simply click the Test button again; you should get another green success bar at the top of the screen.

The green bar will show null , because the script doesn't actually output anything. However, if you look in the Log Output section, you can see some output (Listing 1), with the IP address next to the DNS name you looked up.

START RequestId: 4e90b424-95d9-4453-a2f4-8f5259f5f263 Version: $LATEST
('www.devsecops.cc', [], [' 138.68.149.181' ])
END RequestId: 4e90b424-95d9-4453-a2f4-8f5259f5f263
REPORT RequestId: 4e90b424-95d9-4453-a2f4-8f5259f5f263     Duration: 70.72 ms     Billed Duration: 100 ms     Memory Size: 128 MB     Max Memory Used: 55 MB     Init Duration: 129.20 ms

More or Less

For the second Lambda task, you'll use a more sophisticated script that will allow you to monitor a website. The script for the Lambda function, with the kind permission of the people behind the base2Services GitHub page [4], will attempt to perform a two-way remote TCP port connection. Copy the handler.py script (Listing 2) and paste it into the function tab, as before. If you can't copy the Python script easily, then click the Raw option on the right side of the page and copy all of the raw text.

001 import json
002 import os
003 import boto3
004 from time import perf_counter as pc
005 import socket
006
007 class Config:
008     """Lambda function runtime configuration"""
009
010     HOSTNAME = 'HOSTNAME'
011     PORT = 'PORT'
012     TIMEOUT = 'TIMEOUT'
013     REPORT_AS_CW_METRICS = 'REPORT_AS_CW_METRICS'
014     CW_METRICS_NAMESPACE = 'CW_METRICS_NAMESPACE'
015
016     def __init__(self, event):
017         self.event = event
018         self.defaults = {
019             self.HOSTNAME: 'google.com.au',
020             self.PORT: 443,
021             self.TIMEOUT: 120,
022             self.REPORT_AS_CW_METRICS: '1',
023             self.CW_METRICS_NAMESPACE: 'TcpPortCheck',
024         }
025
026     def __get_property(self, property_name):
027         if property_name in self.event:
028             return self.event[property_name]
029         if property_name in os.environ:
030             return os.environ[property_name]
031         if property_name in self.defaults:
032             return self.defaults[property_name]
033         return None
034
035     @property
036     def hostname(self):
037         return self.__get_property(self.HOSTNAME)
038
039     @property
040     def port(self):
041         return self.__get_property(self.PORT)
042
043     @property
044     def timeout(self):
045         return self.__get_property(self.TIMEOUT)
046
047     @property
048     def reportbody(self):
049         return self.__get_property(self.REPORT_RESPONSE_BODY)
050
051     @property
052     def cwoptions(self):
053         return {
054             'enabled': self.__get_property(self.REPORT_AS_CW_METRICS),
055             'namespace': self.__get_property(self.CW_METRICS_NAMESPACE),
056         }
057
058 class PortCheck:
059     """Execution of HTTP(s) request"""
060
061     def __init__(self, config):
062         self.config = config
063
064     def execute(self):
065         sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
066         sock.settimeout(int(self.config.timeout))
067         try:
068             # start the stopwatch
069             t0 = pc()
070
071             connect_result = sock.connect_ex((self.config.hostname, int(self.config.port)))
072             if connect_result == 0:
073                 available = '1'
074             else:
075                 available = '0'
076
077             # stop the stopwatch
078             t1 = pc()
079
080             result = {
081                 'TimeTaken': int((t1 - t0) * 1000),
082                 'Available': available
083             }
084             print(f"Socket connect result: {connect_result}")
085             # return structure with data
086             return result
087         except Exception as e:
088             print(f"Failed to connect to {self.config.hostname}:{self.config.port}\n{e}")
089             return {'Available': 0, 'Reason': str(e)}
090
091 class ResultReporter:
092     """Reporting results to CloudWatch"""
093
094     def __init__(self, config):
095         self.config = config
096         self.options = config.cwoptions
097
098     def report(self, result):
099         if self.options['enabled'] == '1':
100             try:
101                 endpoint = f"{self.config.hostname}:{self.config.port}"
102                 cloudwatch = boto3.client('cloudwatch')
103                 metric_data = [{
104                     'MetricName': 'Available',
105                     'Dimensions': [
106                         {'Name': 'Endpoint', 'Value': endpoint}
107                     ],
108                     'Unit': 'None',
109                     'Value': int(result['Available'])
110                 }]
111                 if result['Available'] == '1':
112                     metric_data.append({
113                         'MetricName': 'TimeTaken',
114                         'Dimensions': [
115                             {'Name': 'Endpoint', 'Value': endpoint}
116                         ],
117                         'Unit': 'Milliseconds',
118                         'Value': int(result['TimeTaken'])
119                     })
120
121                 result = cloudwatch.put_metric_data(
122                     MetricData=metric_data,
123                     Namespace=self.config.cwoptions['namespace']
124                 )
125
126                 print(f"Sent data to CloudWatch requestId=:{result['ResponseMetadata']['RequestId']}")
127             except Exception as e:
128                 print(f"Failed to publish metrics to CloudWatch:{e}")
129
130 def port_check(event, context):
131     """Lambda function handler"""
132
133     config = Config(event)
134     port_check = PortCheck(config)
135
136     result = port_check.execute()
137
138     # report results
139     ResultReporter(config).report(result)
140
141     result_json = json.dumps(result, indent=4)
142     # log results
143     print(f"Result of checking  {config.hostname}:{config.port}\n{result_json}")
144
145     # return to caller
146     return result

Now, click Save at the top right and look for the Handler input box on the right-hand side of where you pasted the code. You'll need to change the starting point for the Lambda function from lambda_function.lambda_handler to lambda_function.port_check , which is how the script is written. Be sure to click Save again.

Next, configure a new test event,

{
  "HOSTNAME":"www.devsecops.cc",
  "PORT":"443",
  "TIMEOUT":5
}

adjusted a bit from the base2Services GitHub example [5]. Once you've adapted the parameters for your own system settings, go back to the EmptyTest box, pull down the menu, and click Configure test events to create a new parameter to pass to a test. I pasted the test event code over the top of the example JSON code, named it PortTest , and clicked Create .