Skip to content

Commit 1bc8328

Browse files
msailesmaxday
msailes
authored and
maxday
committed
feat: profiler
1 parent 0763aa9 commit 1bc8328

42 files changed

Lines changed: 1902 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/aws-lambda-java-profiler.yml‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
name: Run integration tests for aws-lambda-java-profiler
2+
3+
on:
4+
push:
5+
branches: ['*']
6+
paths:
7+
- 'aws-lambda-java-profiler/**'
8+
- '.github/workflows/aws-lambda-java-profiler.yml'
9+
workflow_dispatch:
10+
11+
jobs:
12+
13+
publish:
14+
runs-on: ubuntu-latest
15+
16+
permissions:
17+
id-token: write
18+
contents: read
19+
20+
steps:
21+
- uses: actions/checkout@v4
22+
23+
- name: Set up JDK
24+
uses: actions/setup-java@v4
25+
with:
26+
java-version: 21
27+
distribution: corretto
28+
29+
- name: Issue AWS credentials
30+
uses: aws-actions/configure-aws-credentials@v4
31+
with:
32+
aws-region: ${{ secrets.AWS_REGION }}
33+
role-to-assume: ${{ secrets.AWS_ROLE }}
34+
role-session-name: GitHubActionsRunIntegrationTests
35+
role-duration-seconds: 900
36+
37+
- name: Build layer
38+
working-directory: ./aws-lambda-java-profiler/extension
39+
run: ./build_layer.sh
40+
41+
- name: Publish layer
42+
working-directory: ./aws-lambda-java-profiler
43+
run: ./integration_tests/publish_layer.sh
44+
45+
- name: Create the bucket layer
46+
working-directory: ./aws-lambda-java-profiler
47+
run: ./integration_tests/create_bucket.sh
48+
49+
- name: Create Java function
50+
working-directory: ./aws-lambda-java-profiler
51+
run: ./integration_tests/create_function.sh
52+
53+
- name: Invoke Java function
54+
working-directory: ./aws-lambda-java-profiler
55+
run: ./integration_tests/invoke_function.sh
56+
57+
- name: Download from s3
58+
working-directory: ./aws-lambda-java-profiler
59+
run: ./integration_tests/download_from_s3.sh
60+
61+
- name: Upload profiles
62+
uses: actions/upload-artifact@v4
63+
with:
64+
name: profiles
65+
path: /tmp/s3-artifacts
66+
67+
- name: cleanup
68+
if: always()
69+
working-directory: ./aws-lambda-java-profiler
70+
run: ./integration_tests/cleanup.sh

‎.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,3 +28,8 @@ dependency-reduced-pom.xml
2828

2929
# snapshot process
3030
aws-lambda-java-runtime-interface-client/pom.xml.versionsBackup
31+
32+
# profiler
33+
aws-lambda-java-profiler/integration_tests/helloworld/build
34+
aws-lambda-java-profiler/extension/build/
35+
/scratch/

‎README.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,18 @@ See the [README](aws-lambda-java-log4j2/README.md) or the [official documentatio
139139
</dependency>
140140
```
141141

142+
## Lambda Profiler Extension for Java - aws-lambda-java-profiler
143+
144+
<p align="center">
145+
<img src="aws-lambda-java-profiler/docs/example-cold-start-flame-graph-small.png" alt="A flame graph of a Java Lambda function">
146+
</p>
147+
148+
This project allows you to profile your Java functions invoke by invoke, with high fidelity, and no code changes. It
149+
uses the [async-profiler](https://github.com/async-profiler/async-profiler) project to produce profiling data and
150+
automatically uploads the data as flame graphs to S3.
151+
152+
Follow our [Quick Start](aws-lambda-java-profiler) to profile your functions.
153+
142154
## Java implementation of the Runtime Interface Client API - aws-lambda-java-runtime-interface-client
143155
[![Maven](https://img.shields.io/maven-central/v/com.amazonaws/aws-lambda-java-runtime-interface-client.svg?label=Maven)](https://central.sonatype.com/artifact/com.amazonaws/aws-lambda-java-runtime-interface-client)
144156

‎aws-lambda-java-profiler/.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
*.zip
2+
/.idea/
3+
/target/

‎aws-lambda-java-profiler/.mvn/wrapper/maven-wrapper.properties‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
wrapperVersion=3.3.2
18+
distributionType=only-script
19+
distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.6/apache-maven-3.9.6-bin.zip

‎aws-lambda-java-profiler/README.md‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
<p align="center">
2+
<img src="docs/Arch_AWS-Lambda_64.svg" alt="AWS Lambda service icon">
3+
</p>
4+
5+
<h2 align="center">AWS Lambda Profiler Extension for Java</h2>
6+
7+
The Lambda profiler extension allows you to profile your Java functions invoke by invoke, with high fidelity, and no
8+
code changes. It uses the [async-profiler](https://github.com/async-profiler/async-profiler) project to produce
9+
profiling data and automatically uploads the data as flame graphs to S3.
10+
11+
<p align="center">
12+
<img src="docs/example-cold-start-flame-graph.png" alt="A flame graph of a Java Lambda function">
13+
</p>
14+
15+
### Usage
16+
17+
To use the profiler you need to
18+
19+
1. Build the extension in this repo
20+
2. Deploy it as a Lambda Layer
21+
3. Create an S3 bucket for the results, or reuse an existing one
22+
4. Give your function permission to write to the bucket
23+
5. Configure the required environment variables.
24+
25+
The following [Quick Start](#quick-start) will give you AWS CLI commands to run to get started. There are also [examples](examples)
26+
using infrastructure as code for you to refer to.
27+
28+
### Quick Start
29+
30+
- Clone the repo
31+
32+
```bash
33+
git clone https://github.com/aws/aws-lambda-java-libs
34+
```
35+
36+
- Build the extension
37+
38+
```bash
39+
cd aws-lambda-java-profiler/extension
40+
./build_layer.sh
41+
```
42+
43+
- Run the `update-function.sh` script which will create a new S3 bucket, Lambda layer and all the configuration required.
44+
45+
```bash
46+
cd ..
47+
./update-function.sh YOUR_FUNCTION_NAME
48+
```
49+
50+
### Configuration
51+
52+
#### Required Environment Variables
53+
54+
| Name | Value |
55+
|------------------------------|-----------------------------------------------------------------------------------------------|
56+
| PROFILER_RESULTS_BUCKET_NAME | Your unique bucket name |
57+
| JAVA_TOOL_OPTIONS | -XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints -javaagent:/opt/profiler-extension.jar |
58+
59+
#### Optional Environment Variables
60+
61+
| Name | Default Value | Options |
62+
|-------------------------------|-----------------------------------------------------------|--------------------------------|
63+
| PROFILER_START_COMMAND | start,event=wall,interval=1us | |
64+
| PROFILER_STOP_COMMAND | stop,file=%s,include=*AWSLambda.main,include=start_thread | file=%s is required |
65+
| PROFILER_DEBUG | false | true - to enable debug logging |
66+
| PROFILER_COMMUNICATION_PORT | 1234 | a valid port number |
67+
68+
### How does it work?
69+
70+
In `/src` is the code for a Java agent. It's entry point `AgentEntry.premain()` is executed as the runtime starts up.
71+
The environment variable `JAVA_TOOL_OPTIONS` is used to specify which .jar file the agent is in. The `MANIFEST.MF` file
72+
is used to specify the pre-main class.
73+
74+
When the agent is constructed, it starts the profiler and registers itself as a Lambda extension for `INVOKE` request.
75+
76+
A new thread is created to handle calling `/next` and uploading the results of the profiler to S3. The bucket to upload
77+
the result to is configurable using an environment variable.
78+
79+
### Project Structure
80+
81+
- `Agent.java`: Main class that coordinates profiling and S3 uploads.
82+
- `AgentEntry.java`: Entry point for the Java agent.
83+
- `ExtensionClient.java`: Handles communication with the Lambda Extensions API.
84+
85+
### Troubleshooting
86+
87+
- Ensure the Lambda function has the necessary permissions to write to the S3 bucket.
88+
- Verify that the environment variables are correctly set in your Lambda function configuration.
89+
- Check CloudWatch logs for any error messages from the extension.
90+
91+
### Contributing
92+
93+
Contributions to improve the extension are welcome. Please submit pull requests with your proposed changes.

‎aws-lambda-java-profiler/docs/Arch_AWS-Lambda_64.svg‎

Lines changed: 18 additions & 0 deletions
Loading

‎aws-lambda-java-profiler/docs/example-cold-start-flame-graph-small.png‎

531 KB
Loading

‎aws-lambda-java-profiler/docs/example-cold-start-flame-graph.png‎

704 KB
Loading

‎aws-lambda-java-profiler/examples/cdk/.gitignore‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
.classpath.txt
2+
target
3+
.classpath
4+
.project
5+
.idea
6+
.settings
7+
.vscode
8+
*.iml
9+
10+
# CDK asset staging directory
11+
.cdk.staging
12+
cdk.out
13+

0 commit comments

Comments
 (0)