Native library fails to load in docker container

I am using gradle for a springboot app that also uses redis. Redis, apparently, relies on org.zerial.snappy.Snappy

The problem is that when I run my tests locally, without docker (on mac) all goes well, when I run them in a docker container it fails.

Please see this stack overflow post for exact details.

The whole thing fails because SnappLoader fails to find the native library. But this is shocking because:

$ docker restart 87aad5c15757 && docker exec -it 87aad5c15757 /bin/bash
87aad5c15757
bash-4.3$ ls
Jenkinsfile                          build.gradle                         docker-compose.yml                   gradle.properties                    infra                                system.properties-Linux-4.9.36-moby  user-login-dto
README.md                            code-style.xml                       google_checks.xml                    gradlew                              out                                  system.properties-Mac OS X-10.12.5   user-login-server
bitbucket-pipelines.yml              docker-compose-test.yml              gradle                               gradlew.bat                          settings.gradle                      user-login-client
bash-4.3$ find /tmp . -name '*.so'
/tmp/snappy-1.1.2-dc393d01-0b0c-4e52-aacc-583405d4119a-libsnappyjava.so
./.gradle/native/25/linux-amd64/libnative-platform.so
./.gradle/native/25/linux-amd64-ncurses5/libnative-platform-curses.so
./.gradle/native/25/linux-amd64-ncurses6/libnative-platform-curses.so
./.gradle/native/jansi/1.14/linux64/libjansi.so
bash-4.3$ ls -al /tmp/snappy-1.1.2-dc393d01-0b0c-4e52-aacc-583405d4119a-libsnappyjava.so
-rwxr--r--    1 gradle   gradle      263376 Jul 27 06:34 /tmp/snappy-1.1.2-dc393d01-0b0c-4e52-aacc-583405d4119a-libsnappyjava.so

The file is actually where the error says it’s looking! I wonder if it’s actually failing to load the .so but reporting it as a not found?

Here is the sample of the stack trace most relevant:

Caused by: java.lang.UnsatisfiedLinkError: /tmp/snappy-1.1.2-dc393d01-0b0c-4e52-aacc-583405d4119a-libsnappyjava.so: Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /tmp/snappy-1.1.2-dc393d01-0b0c-4e52-aacc-583405d4119a-libsnappyjava.so)
build_1  |      at java.lang.ClassLoader$NativeLibrary.load(Native Method) ~[na:1.8.0_131]
build_1  |      at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) ~[na:1.8.0_131]
build_1  |      at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824) ~[na:1.8.0_131]
build_1  |      at java.lang.Runtime.load0(Runtime.java:809) ~[na:1.8.0_131]
build_1  |      at java.lang.System.load(System.java:1086) ~[na:1.8.0_131]
build_1  |      at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:174) ~[snappy-java-1.1.2.6.jar:na]
build_1  |      at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:152) ~[snappy-java-1.1.2.6.jar:na]
build_1  |      at org.xerial.snappy.Snappy.<clinit>(Snappy.java:47) ~[snappy-java-1.1.2.6.jar:na]
build_1  |      at org.redisson.codec.SnappyCodec$2.encode(SnappyCodec.java:68) ~[redisson-2.5.1.jar:na]
build_1  |      at org.redisson.client.handler.CommandEncoder.encode(CommandEncoder.java:103) ~[redisson-2.5.1.jar:na]
build_1  |      at org.redisson.client.handler.CommandEncoder.encode(CommandEncoder.java:45) ~[redisson-2.5.1.jar:na]
build_1  |      at io.netty.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:107) [netty-codec-4.0.42.Final.jar:4.0.42.Final]
build_1  |      ... 31 common frames omitted
build_1  | 
build_1  |     2017-07-27 06:34:23.378  WARN 120 --- [isson-netty-1-6] io.netty.channel.DefaultChannelPipeline  : Failed to mark a promise as failure because it has failed already: DefaultChannelPromise@68316432(failure: io.netty.handler.codec.EncoderException: java.lang.NoClassDefFoundError: Could not initialize class org.xerial.snappy.Snappy), unnotified cause: io.netty.handler.codec.EncoderException: java.lang.NoClassDefFoundError: Could not initialize class org.xerial.snappy.Snappy
build_1  |      at io.netty.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:125)
build_1  |      at org.redisson.client.handler.CommandEncoder.write(CommandEncoder.java:63)

I am also including system property dumps from running it locally and running it in a container so you can see the difference. Since I know the native library location is strongly based on that.

Docker image (gradle:alpine)

PID=372
awt.toolkit=sun.awt.X11.XToolkit
catalina.base=/tmp/tomcat.5265643677339283794.8081
catalina.home=/tmp/tomcat.5265643677339283794.8081
catalina.useNaming=false
com.zaxxer.hikari.pool_number=1
file.encoding=UTF-8
file.encoding.pkg=sun.io
file.separator=/
java.awt.graphicsenv=sun.awt.X11GraphicsEnvironment
java.awt.headless=true
java.awt.printerjob=sun.print.PSPrinterJob
java.class.version=52.0
java.endorsed.dirs=/usr/lib/jvm/java-1.8-openjdk/jre/lib/endorsed
java.ext.dirs=/usr/lib/jvm/java-1.8-openjdk/jre/lib/ext:/usr/java/packages/lib/ext
java.home=/usr/lib/jvm/java-1.8-openjdk/jre
java.io.tmpdir=/tmp
java.library.path=/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
java.runtime.name=OpenJDK Runtime Environment
java.runtime.version=1.8.0_131-b11
java.specification.name=Java Platform API Specification
java.specification.vendor=Oracle Corporation
java.specification.version=1.8
java.vendor=Oracle Corporation
java.vendor.url=http://java.oracle.com/
java.vendor.url.bug=http://bugreport.sun.com/bugreport/
java.version=1.8.0_131
java.vm.info=mixed mode
java.vm.name=OpenJDK 64-Bit Server VM
java.vm.specification.name=Java Virtual Machine Specification
java.vm.specification.vendor=Oracle Corporation
java.vm.specification.version=1.8
java.vm.vendor=Oracle Corporation
java.vm.version=25.131-b11
line.separator=

org.jboss.logging.provider=slf4j
os.arch=amd64
os.name=Linux
os.version=4.9.36-moby
path.separator=:
spring.beaninfo.ignore=true
sun.arch.data.model=64
sun.boot.class.path=/usr/lib/jvm/java-1.8-openjdk/jre/lib/resources.jar:/usr/lib/jvm/java-1.8-openjdk/jre/lib/rt.jar:/usr/lib/jvm/java-1.8-openjdk/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-1.8-openjdk/jre/lib/jsse.jar:/usr/lib/jvm/java-1.8-openjdk/jre/lib/jce.jar:/usr/lib/jvm/java-1.8-openjdk/jre/lib/charsets.jar:/usr/lib/jvm/java-1.8-openjdk/jre/lib/jfr.jar:/usr/lib/jvm/java-1.8-openjdk/jre/classes
sun.boot.library.path=/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64
sun.cpu.endian=little
sun.cpu.isalist=
sun.io.unicode.encoding=UnicodeLittle
sun.java.command=com.sterling.s1.users.server.UserApplication --spring.profiles.active=docker
sun.java.launcher=SUN_STANDARD
sun.jnu.encoding=UTF-8
sun.management.compiler=HotSpot 64-Bit Tiered Compilers
sun.nio.ch.bugLevel=
sun.os.patch.level=unknown
user.country=US
user.dir=/project/user-login-server
user.home=/home/gradle
user.language=en
user.name=gradle
user.timezone=GMT
user.variant=

Localhost on Mac 10.12.5

PID=21394
awt.toolkit=sun.lwawt.macosx.LWCToolkit
catalina.base=/private/var/folders/c4/m28c23954qq1ndnvlkv07rmshkh91h/T/tomcat.2487844341811596065.9090
catalina.home=/private/var/folders/c4/m28c23954qq1ndnvlkv07rmshkh91h/T/tomcat.2487844341811596065.9090
catalina.useNaming=false
com.zaxxer.hikari.pool_number=1
file.encoding=UTF-8
file.encoding.pkg=sun.io
file.separator=/
ftp.nonProxyHosts=local|*.local|169.254/16|*.169.254/16
gopherProxySet=false
http.nonProxyHosts=local|*.local|169.254/16|*.169.254/16
java.awt.graphicsenv=sun.awt.CGraphicsEnvironment
java.awt.headless=true
java.awt.printerjob=sun.lwawt.macosx.CPrinterJob
java.class.version=52.0
java.endorsed.dirs=/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre/lib/endorsed
java.ext.dirs=/Users/cbongiorno/Library/Java/Extensions:/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre/lib/ext:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java
java.home=/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre
java.io.tmpdir=/var/folders/c4/m28c23954qq1ndnvlkv07rmshkh91h/T/
java.library.path=/Users/cbongiorno/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
java.runtime.name=Java(TM) SE Runtime Environment
java.runtime.version=1.8.0_131-b11
java.specification.name=Java Platform API Specification
java.specification.vendor=Oracle Corporation
java.specification.version=1.8
java.vendor=Oracle Corporation
java.vendor.url=http://java.oracle.com/
java.vendor.url.bug=http://bugreport.sun.com/bugreport/
java.version=1.8.0_131
java.vm.info=mixed mode
java.vm.name=Java HotSpot(TM) 64-Bit Server VM
java.vm.specification.name=Java Virtual Machine Specification
java.vm.specification.vendor=Oracle Corporation
java.vm.specification.version=1.8
java.vm.vendor=Oracle Corporation
java.vm.version=25.131-b11
line.separator=

org.jboss.logging.provider=slf4j
os.arch=x86_64
os.name=Mac OS X
os.version=10.12.5
path.separator=:
socksNonProxyHosts=local|*.local|169.254/16|*.169.254/16
spring.beaninfo.ignore=true
spring.output.ansi.enabled=always
spring.profiles.active=dev,clean
sun.arch.data.model=64
sun.boot.class.path=/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre/lib/sunrsasign.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre/classes
sun.boot.library.path=/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre/lib
sun.cpu.endian=little
sun.cpu.isalist=
sun.io.unicode.encoding=UnicodeBig
sun.java.command=com.sterling.s1.users.server.UserApplication --server.port=9090
sun.java.launcher=SUN_STANDARD
sun.jnu.encoding=UTF-8
sun.management.compiler=HotSpot 64-Bit Tiered Compilers
sun.nio.ch.bugLevel=
sun.os.patch.level=unknown
user.country=US
user.dir=/Volumes/dev/sterling/java-user-login-service
user.home=/Users/cbongiorno
user.language=en
user.name=cbongiorno
user.timezone=America/Los_Angeles

I will try using another image and hope for better results