arundhaj

regression towards the datascience

Serverless application architecture in Python with AWS Lambda

 

AWS Lambda is a compute service that lets you run code without provisioning or managing servers. In this article I will show how to create and manage simple AWS Lambda function written in Python.

Create files and folders as shown below. Technically, app.py is sufficient to create a function, however the following structure lets you to separate the concerns into its specified layers.

py3lambda
├───mylib
│   ├───__init__.py
│   ├───helper.py
│   ├───model.py
│   └───service.py
├───packages
├───app.py
├───config.json
└───manage.py

mylib folder shall contain your application files.

package folder should contain any 3rd party packages as required by your application. To install a package to this directory use the following command.

$ pip install pymysql -t .\packages

app.py contains the main entry point of AWS Lambda function.

import logging
from mylib import helper

logger = logging.getLogger()
logger.setLevel(logging.INFO)

def lambda_handler(event, context):
    """ Lamdba entry point. """

    logger.info("Inside lambda_handler1")

    helper.some_helper()

    return {'Status': 'Success'}

Add configurations in config.json file, that goes to AWS Lambda environment variables.

{
    "<<VARIABLE_NAME1>>": "<<variable_value1>>",
    "<<VARIABLE_NAME2>>": "<<variable_value2>>"
}

And finally, the script to manage AWS Lambda function. manage.py

import os
import sys
import io
import zipfile
import boto3
import json


def archive():
    buf = io.BytesIO()

    with zipfile.ZipFile(buf, 'w', zipfile.ZIP_DEFLATED) as zfh:
        # Add my main file
        zfh.write('app.py')

        for root, _, files in os.walk('mylib'):
            for file in files:
                zfh.write(os.path.join(root, file))

        os.chdir('packages')
        for root, _, files in os.walk('.'):
            for file in files:
                zfh.write(os.path.join(root, file))

        os.chdir('..')

    buf.seek(0)
    pkg = buf.read()
    return pkg


def environment():
    with open('config.json') as config_file:
        env_json = json.load(config_file)

    return env_json


def zip_create_function():
    lambda_client = boto3.client('lambda')

    pkg = archive()
    env_json = environment()

    response = lambda_client.create_function(
        FunctionName='Py3Lambda',
        Runtime='python3.6',
        Role='<<your role ARN>>',
        Handler='app.lambda_handler',
        Code={'ZipFile': pkg},
        Environment={
            'Variables': env_json
        },
        Description='Python3 Lambda function',
        Timeout=60,
        MemorySize=128,
        Publish=True
    )

    print(json.dumps(response))


def zip_update_function():
    lambda_client = boto3.client('lambda')

    pkg = archive()
    env_json = environment()

    response = lambda_client.update_function_code(
        FunctionName='Py3Lambda',
        ZipFile=pkg,
        Publish=True
    )

    print(json.dumps(response))

    response = lambda_client.update_function_configuration(
        FunctionName='Py3Lambda',
        Environment={
            'Variables': env_json
        }
    )

    print(json.dumps(response))


if __name__ == "__main__":
    if len(sys.argv) != 2:
        print('Invalid option')
    elif sys.argv[1] == 'create':
        zip_create_function()
    elif sys.argv[1] == 'update':
        zip_update_function()
    else:
        print('Invalid option')

To run the above script

$ python manage.py create|update

Hope this helps!

Comments