Template for lambda with localstack and sam integration for localtest

• Python3

• AWS client (pip3 install awscli --upgrade --user)

• AWS SAM client (pip3 install --user aws-sam-cli)

  • Reference: Adjust path to use sam cli command

• Docker - need for the local lambda function testing

git clone https://github.com/uujo/lambda_localstack_python_template.git

This will copy the pre-commit, pre-push files under .git/hooks

> git config --local init.templatedir './.git_templates'
> git init

Pre commit: change the format using Black and Pylint checking (Will fail to commit if pylint score is not 10.

Pre push: check the unittest and code coverage (Will fail to push if either unittest fails or code coverage is less than 95%

For special cases, if you still need to commit, push the code use --no-verify option on git commands

> git commit --no-verify
> git push origin [branch_name] --no-verify

• set the virtual environment (optional, you can use other virtual env tools or without it)

  cd lambda_localstack_python_template

  python3 -m venv .venv

  source .venv/bin/activate - activate virtual environment

  pip install -r requirements.txt - on time setting

• To start local test setup

  bin/start_local_test

  -i option sets interactive mode on localstack instead of running it background, if you want to monitor localstack log use this option. Without this option, it automatically create the sample data in S3 bucket.

• To check the table, sqs and bucket are created (Use this setting only if you use -i option, without -i, these steps runs automatically)

  aws dynamodb list-tables --endpoint-url http://localhost:4569

  aws sqs list-queues --endpoint-url=http://localhost:4576

  aws s3api list-buckets --endpoint-url=http://localhost:4572

  • put data in S3

    aws s3api put-object --endpoint-url=http://localhost:4572 --bucket VARSAP_BUCKET --key annotation/g-snv-7-7578212-G-A/data.json --body tests/s3_test_data.json

  • To check S3 object is created

    aws s3api list-objects --bucket=VARSAP_BUCKET --endpoint-url=http://localhost:4572

• To test annotation lambdas

  • invoke S3 ObjectCreated event

    sam local invoke HandleS3Event -t build/template.yaml --docker-network [repository_name]_local_aws_network -e tests/s3_test_event.json

  • To check whether table has an entry inserted.

    aws dynamodb scan --table-name TEST_TABLE --endpoint-url http://localhost:4569

  • To check sqs messge is queued

    aws sqs receive-message --queue-url http://localhost:4576/queue/TEST_QUEUE --endpoint-url=http://localhost:4576 --max-number-of-messages=10

  • invoke SQS event

    sam local invoke RequestAnnotation -t build/template.yaml --docker-network [repository_name]_local_aws_network -e tests/sqs_test_event.json

  • check S3 Object is updated: (Might take a while to update annotations) - It will store object in s3_test_out.json

    aws s3api get-object --endpoint-url=http://localhost:4572 --bucket TEST_BUCKET --key [path_to_object]/[name_of_object].json s3_test_out.json

• To stop the local test and clean up docker

  bin/stop_local_test

• AWS SAM guick start guide
• SAM template
• SQS
• Lambda function best practice
  • Take advantage of Execution Context reuse to improve the performance of your function.
  • Use AWS Lambda Environment Variables to pass operational parameters to your function.
  • Control the dependencies in your function's deployment package.
  • Minimize your deployment package size to its runtime necessities.
  • Reduce the time it takes Lambda to unpack deployment packages.
  • Minimize the complexity of your dependencies.
• Pros and cons of monorepo
• Directory structure for multiple Lambdas
• Documenting REST API
• Mocking AWS stack locally
• LocalStack
• LocalStack with aws sam
• LocalStack with docker
• Lambda debugging
• Scheduling lambda periodically
• Pyinstrument (python profiling)
• S3 event message structure
• aws xray python