Repository navigation
Report JVM crash logs from CI Visibility smoke tests - #12905
daniel-mohedano wants to merge 1 commit into
Conversation
CI Visibility Test Environment - sbt-scalatestJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
This comment has been minimized.
This comment has been minimized.
CI Visibility Test Environment - pass4sJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - reactive-streams-jvmJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - nebula-release-pluginJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - netflix-zuulJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - heliboardJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
CI Visibility Test Environment - sonar-kotlinJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - jolokiaJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - okhttpJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - spring_bootJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
More details
Crash-log collection runs before temporary-project cleanup, and its destination is covered by CI artifact collection.
🤖 Bits Code Review · Commit eb5c531
CI Visibility Test Environment - sonar-javaJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
AlexeyKuznetsov-DD
left a comment
There was a problem hiding this comment.
LGTM, left minor style notes.
| protected static void reportJvmCrashLogs(Path directory) { | ||
| if (directory == null || !Files.isDirectory(directory)) { | ||
| return; | ||
| } | ||
| List<Path> crashLogs; | ||
| try (Stream<Path> files = Files.walk(directory)) { | ||
| crashLogs = | ||
| files | ||
| .filter(p -> p.getFileName().toString().matches("hs_err_pid\\d+\\.log")) | ||
| .collect(Collectors.toList()); | ||
| } catch (IOException | UncheckedIOException e) { | ||
| System.err.println("Failed to look for JVM crash logs in " + directory + ": " + e); | ||
| return; | ||
| } | ||
| for (Path crashLog : crashLogs) { |
There was a problem hiding this comment.
Just a matter of taste, but adding empty lines improve readability an split code in logical blocks visually, instead of wall of code.
| protected static void reportJvmCrashLogs(Path directory) { | |
| if (directory == null || !Files.isDirectory(directory)) { | |
| return; | |
| } | |
| List<Path> crashLogs; | |
| try (Stream<Path> files = Files.walk(directory)) { | |
| crashLogs = | |
| files | |
| .filter(p -> p.getFileName().toString().matches("hs_err_pid\\d+\\.log")) | |
| .collect(Collectors.toList()); | |
| } catch (IOException | UncheckedIOException e) { | |
| System.err.println("Failed to look for JVM crash logs in " + directory + ": " + e); | |
| return; | |
| } | |
| for (Path crashLog : crashLogs) { | |
| protected static void reportJvmCrashLogs(Path directory) { | |
| if (directory == null || !Files.isDirectory(directory)) { | |
| return; | |
| } | |
| List<Path> crashLogs; | |
| try (Stream<Path> files = Files.walk(directory)) { | |
| crashLogs = | |
| files | |
| .filter(p -> p.getFileName().toString().matches("hs_err_pid\\d+\\.log")) | |
| .collect(Collectors.toList()); | |
| } catch (IOException | UncheckedIOException e) { | |
| System.err.println("Failed to look for JVM crash logs in " + directory + ": " + e); | |
| return; | |
| } | |
| for (Path crashLog : crashLogs) { |
| + lines.limit(JVM_CRASH_LOG_PRINTED_LINES).collect(Collectors.joining("\n"))); | ||
| } | ||
| } catch (IOException | UncheckedIOException e) { | ||
| System.err.println("Failed to report JVM crash log " + crashLog + ": " + e); |
There was a problem hiding this comment.
Here and similar places can be a bit better with formatter approach.
| System.err.println("Failed to report JVM crash log " + crashLog + ": " + e); | |
| System.err.printf("Failed to look for JVM crash logs in %s: %s%n", directory, e); |
| System.out.println( | ||
| "JVM crash log " | ||
| + crashLog | ||
| + " (copied to " | ||
| + copy.toAbsolutePath() | ||
| + "):\n" | ||
| + lines.limit(JVM_CRASH_LOG_PRINTED_LINES).collect(Collectors.joining("\n"))); |
There was a problem hiding this comment.
| System.out.println( | |
| "JVM crash log " | |
| + crashLog | |
| + " (copied to " | |
| + copy.toAbsolutePath() | |
| + "):\n" | |
| + lines.limit(JVM_CRASH_LOG_PRINTED_LINES).collect(Collectors.joining("\n"))); | |
| System.out.printf( | |
| "JVM crash log %s (copied to %s):%n%s%n", | |
| crashLog, | |
| copy.toAbsolutePath(), | |
| lines.limit(JVM_CRASH_LOG_PRINTED_LINES).collect(Collectors.joining("\n"))); |
What Does This Do
CI Visibility smoke tests now report JVM crash logs (
hs_err_pid*.log) written by the processes they start: the Gradle launcher, the Gradle test workers, and Maven.build/reports/jvm-crash-logs, which CI keeps as a job artifact.Motivation
Forked JVMs write their crash log to their working directory. In these tests that is a temporary folder that is deleted after each test. The tests also do not print the process output. So when a forked JVM crashed, the failure only showed a non-zero exit code (134), with nothing to tell why.
On
master, about 109 smoke test failures over the last 3 weeks were JVM crashes on Linux arm64. Only 2 of them had any crash details in the job log.Additional Notes
Checked locally by making the forked JVM crash on purpose (an agent that writes to address 0). The crash logs were printed and copied for the Gradle launcher, the Gradle test workers and Maven.
test-environment-trigger: skip