Skip to content

aws-lambda-java-log4j2 1.6.3 regression breaks logging #612

Closed
Closed
aws-lambda-java-log4j2 1.6.3 regression breaks logging#612

Description

@brandondahler
brandondahler
opened on May 19, 2026

Problem

Log4j2 logging no longer works after upgrading to com.amazonaws:aws-lambda-java-log4j2:1.6.3.

Log4j prints the error ERROR Error processing element Lambda ([Appenders: null]): CLASS_NOT_FOUND to stderr (which is not logged by lambda).

Reproduction Steps

  1. Create a new maven project as described below
  2. Execute the code via mvn package exec:java -Dexec.mainClass="bug.reproduction.App"

Reproduction Files

./pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>bug.reproduction</groupId>
  <artifactId>aws-lambda-java-log4j2</artifactId>
  <version>1.0-SNAPSHOT</version>

  <name>aws-lambda-java-log4j2</name>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.release>17</maven.compiler.release>
  </properties>

  <dependencies>
    <dependency>
      <groupId>com.amazonaws</groupId>
      <artifactId>aws-lambda-java-log4j2</artifactId>
      <version>1.6.3</version> <!-- 1.6.2 works -->
    </dependency>
    <dependency>
      <groupId>org.apache.logging.log4j</groupId>
      <artifactId>log4j-core</artifactId>
      <version>2.25.4</version>
    </dependency>
    <dependency>
      <groupId>org.apache.logging.log4j</groupId>
      <artifactId>log4j-api</artifactId>
      <version>2.25.4</version>
    </dependency>
    <dependency>
      <groupId>org.apache.logging.log4j</groupId>
      <artifactId>log4j-layout-template-json</artifactId>
      <version>2.25.4</version>
    </dependency>
  </dependencies>

  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <artifactId>maven-clean-plugin</artifactId>
          <version>3.4.0</version>
        </plugin>
        <plugin>
          <artifactId>maven-resources-plugin</artifactId>
          <version>3.3.1</version>
        </plugin>
        <plugin>
          <artifactId>maven-compiler-plugin</artifactId>
          <version>3.13.0</version>
        </plugin>
        <plugin>
          <artifactId>maven-jar-plugin</artifactId>
          <version>3.4.2</version>
          <configuration>
            <archive>
              <manifest>
                <mainClass>bug.reproduction.App</mainClass>
              </manifest>
            </archive>
          </configuration>
        </plugin>
      </plugins>
    </pluginManagement>
  </build>
</project>

./src/main/java/bug/reproduction/App.java

package bug.reproduction;

import org.apache.logging.log4j.LogManager;

/**
 * Hello world!
 */
public class App {
    public static void main(String[] args) {
        System.out.println("Should print test below");
        LogManager.getLogger("foo").info("test");
        System.out.println("Should have print test above");
    }
}

./src/main/resources/log4j2.xml

<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
    <Appenders>
        <Lambda name="Console">
            <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1} - %m%n"/>
        </Lambda>
    </Appenders>
    <Loggers>
        <Root level="INFO">
            <AppenderRef ref="Console"/>
        </Root>
    </Loggers>
</Configuration>

Expected Output

[INFO] Scanning for projects...
[INFO]
[INFO] --------------< bug.reproduction:aws-lambda-java-log4j2 >---------------
[INFO] Building aws-lambda-java-log4j2 1.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ aws-lambda-java-log4j2 ---
[INFO] Copying 1 resource from src/main/resources to target/classes
[INFO]
[INFO] --- compiler:3.13.0:compile (default-compile) @ aws-lambda-java-log4j2 ---
[INFO] Nothing to compile - all classes are up to date.
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ aws-lambda-java-log4j2 ---
[INFO] skip non existing resourceDirectory /Users/bnd/reproduction/aws-lambda-java-log4j2/src/test/resources
[INFO]
[INFO] --- compiler:3.13.0:testCompile (default-testCompile) @ aws-lambda-java-log4j2 ---
[INFO] No sources to compile
[INFO]
[INFO] --- surefire:3.5.4:test (default-test) @ aws-lambda-java-log4j2 ---
[INFO] No tests to run.
[INFO]
[INFO] --- jar:3.4.2:jar (default-jar) @ aws-lambda-java-log4j2 ---
[INFO]
[INFO] --- exec:3.6.3:java (default-cli) @ aws-lambda-java-log4j2 ---
Should print test below
2026-05-19 12:18:10 INFO  foo - test
Should have print test above
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.776 s
[INFO] Finished at: 2026-05-19T12:18:10-04:00
[INFO] ------------------------------------------------------------------------

To reproduce expected output, update line 21 in ./pom.xml to <version>1.6.2</version>

Actual Output

[INFO] Scanning for projects...
[INFO]
[INFO] --------------< bug.reproduction:aws-lambda-java-log4j2 >---------------
[INFO] Building aws-lambda-java-log4j2 1.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ aws-lambda-java-log4j2 ---
[INFO] Copying 1 resource from src/main/resources to target/classes
[INFO]
[INFO] --- compiler:3.13.0:compile (default-compile) @ aws-lambda-java-log4j2 ---
[INFO] Nothing to compile - all classes are up to date.
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ aws-lambda-java-log4j2 ---
[INFO] skip non existing resourceDirectory /Users/bnd/reproduction/aws-lambda-java-log4j2/src/test/resources
[INFO]
[INFO] --- compiler:3.13.0:testCompile (default-testCompile) @ aws-lambda-java-log4j2 ---
[INFO] No sources to compile
[INFO]
[INFO] --- surefire:3.5.4:test (default-test) @ aws-lambda-java-log4j2 ---
[INFO] No tests to run.
[INFO]
[INFO] --- jar:3.4.2:jar (default-jar) @ aws-lambda-java-log4j2 ---
[INFO] Building jar: /Users/bnd/reproduction/aws-lambda-java-log4j2/target/aws-lambda-java-log4j2-1.0-SNAPSHOT.jar
[INFO]
[INFO] --- exec:3.6.3:java (default-cli) @ aws-lambda-java-log4j2 ---
Should print test below
2026-05-19T16:19:47.710360Z bug.reproduction.App.main() ERROR Error processing element Lambda ([Appenders: null]): CLASS_NOT_FOUND
2026-05-19T16:19:47.720281Z bug.reproduction.App.main() ERROR Unable to locate appender "Console" for logger config "root"
Should have print test above
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.038 s
[INFO] Finished at: 2026-05-19T12:19:47-04:00
[INFO] ------------------------------------------------------------------------

Note that the two bug.reproduction.App.main() ERROR lines are written to stderr, as evidenced by running it while redirecting stderr to /dev/null via mvn package exec:java -Dexec.mainClass="bug.reproduction.App" 2> /dev/null

Activity

  1. darklight3it commented on May 19, 2026

    @darklight3it
    darklight3it
    on May 19, 2026
    Collaborator

    Found it was a deployment issue. We are going to redeploy a new version with the correct change.

  2. darklight3it commented on May 19, 2026

    @darklight3it
    darklight3it
    on May 19, 2026
    Collaborator

    Checking the artifact is not including the dat file

    mvn dependency:copy -Dartifact=com.amazonaws:aws-lambda-java-log4j2:1.6.3:jar -DoutputDirectory=/tmp/check -q && \
jar tf /tmp/check/aws-lambda-java-log4j2-1.6.3.jar | grep -i "\.dat" || echo "(none)" && \
mvn dependency:copy -Dartifact=com.amazonaws:aws-lambda-java-log4j2:1.6.2:jar -DoutputDirectory=/tmp/check -q && \
jar tf /tmp/check/aws-lambda-java-log4j2-1.6.2.jar | grep -i "\.dat" || echo "(none)"

(none)
META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat

    DAT files are necessary to register Lambda

  3. darklight3it commented on May 19, 2026

    @darklight3it
    darklight3it
    on May 19, 2026
    Collaborator

    Confirmed that maven built 1.6.3 with a newer version of java. In new version of jdk annotation processor are [disabled by default] (https://maven.apache.org/plugins/maven-compiler-plugin-4.x/examples/annotation-processor.html),

    Tested the 1.6.4 pre-release jar. Log4j2Plugins.dat and native-image/reflect-config.json are back. Jar contents are identical to 1.6.2. (except for the updat in log4j that was intended to be released), the check I run:

    $ jar tf aws-lambda-java-log4j2-1.6.4.jar | grep -i "\.dat"
META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat

$ diff <(jar tf aws-lambda-java-log4j2-1.6.2.jar | sort) <(jar tf aws-lambda-java-log4j2-1.6.4.jar | sort)
(no differences)

    Also run repro and it is working

    $ mvn package exec:java -Dexec.mainClass="bug.reproduction.App"
Should print test below
2026-05-19 21:23:16 INFO  foo - test
Should have print test above

    Thanks @brandondahler for the heads up ❤️.

  4. darklight3it commented on May 20, 2026

    @darklight3it
    darklight3it
    on May 20, 2026
    Collaborator

    Version 1.6.4 released.

  5. darklight3it
    closed this as completedon May 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions