JVM crash (SIGSEGV) at startup with alpine based docker image

See original GitHub issue

Describe the bug When using micrometer v1.8.0-M2 with spring-boot on an alpine based docker image, the JVM crashes at startup.

Snippet from the crash log:

  # A fatal error has been detected by the Java Runtime Environment:
  #
  #  SIGSEGV (0xb) at pc=0x000000000000401e, pid=14, tid=15
  #
  # JRE version: OpenJDK Runtime Environment Zulu11.45+27-CA (11.0.10+9) (build 11.0.10+9-LTS)
  # Java VM: OpenJDK 64-Bit Server VM Zulu11.45+27-CA (11.0.10+9-LTS, mixed mode, tiered, compressed oops, concurrent mark sweep gc, linux-amd64)
  # Problematic frame:
  # C  0x000000000000401e
  #
  # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
  #
  # If you would like to submit a bug report, please visit:
  #   http://www.azulsystems.com/support/
  # The crash happened outside the Java Virtual Machine in native code.
  # See problematic frame for where to report the bug.
  #

More details towards the end of the crash log:

Internal exceptions (20 events):
  Event: 19.278 Thread 0x00007f5426934800 Exception <a 'java/lang/ClassNotFoundException'{0x000000008aede2e8}: io/micrometer/statsd/StatsdConfigCustomizer> (0x000000008aede2e8) thrown at [./src/hotspot/share/classfile/systemDictionary.cpp, line 234]
  Event: 19.286 Thread 0x00007f5426934800 Exception <a 'java/lang/ClassNotFoundException'{0x000000008af741c8}: io/micrometer/core/instrument/config/MeterRegistryConfigCustomizer> (0x000000008af741c8) thrown at [./src/hotspot/share/classfile/systemDictionary.cpp, line 234]
  Event: 19.307 Thread 0x00007f5426934800 Exception <a 'java/lang/ClassNotFoundException'{0x000000008b156f70}: java/lang/ObjectCustomizer> (0x000000008b156f70) thrown at [./src/hotspot/share/classfile/systemDictionary.cpp, line 234]
  Event: 19.310 Thread 0x00007f5426934800 Exception <a 'java/lang/ClassNotFoundException'{0x000000008b1fab98}: org/springframework/boot/actuate/autoconfigure/metrics/MetricsAutoConfigurationCustomizer> (0x000000008b1fab98) thrown at [./src/hotspot/share/classfile/systemDictionary.cpp, line 234]
  Event: 19.319 Thread 0x00007f5426934800 Exception <a 'java/lang/ClassNotFoundException'{0x000000008b2a71d8}: java/lang/ObjectCustomizer> (0x000000008b2a71d8) thrown at [./src/hotspot/share/classfile/systemDictionary.cpp, line 234]
  Event: 19.322 Thread 0x00007f5426934800 Exception <a 'java/lang/ClassNotFoundException'{0x000000008b32ff00}: io/micrometer/core/instrument/Clock$1Customizer> (0x000000008b32ff00) thrown at [./src/hotspot/share/classfile/systemDictionary.cpp, line 234]
  Event: 19.327 Thread 0x00007f5426934800 Exception <a 'java/lang/ClassNotFoundException'{0x000000008b3b74a8}: io/micrometer/core/instrument/ClockCustomizer> (0x000000008b3b74a8) thrown at [./src/hotspot/share/classfile/systemDictionary.cpp, line 234]
  Event: 19.474 Thread 0x00007f5426934800 Exception <a 'java/lang/NoSuchMethodError'{0x000000008c29f9b8}: 'java.lang.Object java.lang.invoke.DirectMethodHandle$Holder.invokeStaticInit(java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object)'> (0x000000008c29f9b8) thrown at [./src/hotspot/share/interpreter/linkResolver.cpp, line 773]
  Event: 19.486 Thread 0x00007f5426934800 Exception <a 'java/lang/IncompatibleClassChangeError'{0x000000008c3ed9b0}: Found class java.lang.Object, but interface was expected> (0x000000008c3ed9b0) thrown at [./src/hotspot/share/interpreter/linkResolver.cpp, line 839]
  Event: 19.489 Thread 0x00007f5426934800 Exception <a 'java/lang/IncompatibleClassChangeError'{0x000000008c412310}: Found class java.lang.Object, but interface was expected> (0x000000008c412310) thrown at [./src/hotspot/share/interpreter/linkResolver.cpp, line 839]
  Event: 19.489 Thread 0x00007f5426934800 Exception <a 'java/lang/IncompatibleClassChangeError'{0x000000008c4160a0}: Found class java.lang.Object, but interface was expected> (0x000000008c4160a0) thrown at [./src/hotspot/share/interpreter/linkResolver.cpp, line 839]
  Event: 19.489 Thread 0x00007f5426934800 Exception <a 'java/lang/IncompatibleClassChangeError'{0x000000008c419940}: Found class java.lang.Object, but interface was expected> (0x000000008c419940) thrown at [./src/hotspot/share/interpreter/linkResolver.cpp, line 839]
  Event: 19.490 Thread 0x00007f5426934800 Exception <a 'java/lang/IncompatibleClassChangeError'{0x000000008c41d650}: Found class java.lang.Object, but interface was expected> (0x000000008c41d650) thrown at [./src/hotspot/share/interpreter/linkResolver.cpp, line 839]
  Event: 19.490 Thread 0x00007f5426934800 Exception <a 'java/lang/IncompatibleClassChangeError'{0x000000008c421188}: Found class java.lang.Object, but interface was expected> (0x000000008c421188) thrown at [./src/hotspot/share/interpreter/linkResolver.cpp, line 839]
  Event: 19.490 Thread 0x00007f5426934800 Exception <a 'java/lang/IncompatibleClassChangeError'{0x000000008c424a10}: Found class java.lang.Object, but interface was expected> (0x000000008c424a10) thrown at [./src/hotspot/share/interpreter/linkResolver.cpp, line 839]
  Event: 19.510 Thread 0x00007f5426934800 Exception <a 'java/lang/IncompatibleClassChangeError'{0x000000008c53e910}: Found class java.lang.Object, but interface was expected> (0x000000008c53e910) thrown at [./src/hotspot/share/interpreter/linkResolver.cpp, line 839]
  Event: 19.729 Thread 0x00007f5426934800 Exception <a 'java/lang/NoSuchMethodError'{0x000000008dd92a08}: 'int java.lang.invoke.DirectMethodHandle$Holder.invokeStaticInit(java.lang.Object)'> (0x000000008dd92a08) thrown at [./src/hotspot/share/interpreter/linkResolver.cpp, line 773]
  Event: 19.819 Thread 0x00007f5426934800 Exception <a 'java/lang/NoSuchMethodError'{0x000000008e773eb8}: 'void java.lang.invoke.DirectMethodHandle$Holder.invokeStaticInit(java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object)'> (0x000000008e773eb8) thrown at [./src/hotspot/share/interpreter/linkResolver.cpp, line 773]
  Event: 20.056 Thread 0x00007f5426934800 Exception <a 'java/lang/UnsatisfiedLinkError'{0x000000008ffce290}: 'int io.micrometer.shaded.io.netty.channel.epoll.Native.offsetofEpollData()'> (0x000000008ffce290) thrown at [./src/hotspot/share/prims/nativeLookup.cpp, line 527]
  Event: 20.059 Thread 0x00007f5426934800 Exception <a 'java/lang/reflect/InvocationTargetException'{0x0000000090027df0}> (0x0000000090027df0) thrown at [./src/hotspot/share/runtime/reflection.cpp, line 1247]

Environment

Micrometer version [1.8.0-M2]
Micrometer registry [statsd]
OS: [Alpine - docker.artifactory.ea.com/azul/zulu-openjdk-alpine:11-jre]
Java version: [zulu11]

To Reproduce A simple Spring-Boot app with the v1.8.0-M2 micrometer dependency on an alpine based docker container. I tested with Spring Boot version 2.5.2 and 2.5.3.

Expected behavior JVM should startup without an error.

Additional context Conversation from micrometer slack: https://micrometer-metrics.slack.com/archives/C662HUJC9/p1631658539054500

Issue Analytics

State:
Created 2 years ago
Reactions:1
Comments:11 (7 by maintainers)

Top GitHub Comments

11reactions

alexfeigincommented, Jan 18, 2022

People that run into this issue using alpine and an alpine built JDK may want a workaround.

Credit belongs to Eric Anderson @ejona86 on github. On this thread on grpc-java with a similar issue.

Installing gcompat package, and running the jdk with LD_PRELOAD=/lib/libgcompat.so.0 fixes this issue (and other glibc related issues most likely)

If using Docker you may add the following to your Dockerfile:

RUN apk --no-cache add gcompat
ENV LD_PRELOAD=/lib/libgcompat.so.0

Hope this helps people.

9reactions

jonatan-ivanovcommented, Sep 21, 2021

For the sake of documentation:

It seems Netty does not support Alpine since Netty depends on glibc but Alpine does not have it, it has musl libc instead.

The issue disappears if you disable Netty’s native support (see previous comment) or if you use an image that has glibc, e.g.:

azul/zulu-openjdk-alpine:11-jre: Alpine-based, no glibc -> does not work
azul/zulu-openjdk:11: Ubuntu-based, has glibc -> works
bellsoft/liberica-openjdk-alpine-musl:17 (see source): Alpine-based, no glibc -> does not work
bellsoft/liberica-openjdk-alpine:17 (see the source): Alpine-based, has glibc -> works

Top Results From Across the Web

Java SDK crashes the JVM on Alpine docker image

We are using the Docker image eclipse-temurin:17-alpine in a Spring boot application. When running Java SDK version 3.0.10 the application ...

Java SIGSEGV error when running from docker - Stack Overflow

This is solved by using the debian based image instead of alpine. ... The crash happened outside the Java Virtual Machine in native...

OS crashes while running as a docker container inside ...

The Spring Boot application crashes on startup with the error below. The only fix is to switch to docker.io/redhat/ubi8:latest.

openjdk - Official Image | Docker Hub

The most straightforward way to use this image is to use a Java container as ... On startup the JVM tries to detect...

تويتر \ التغريدات مع الردود بواسطة Micrometer Metrics (micrometerio ...

... allowing you to instrument your JVM-based app without lock-in. ... Known issue: [statsd] JVM crash (SIGSEGV) at startup with alpine based docker...