I’m trying to establish a very basic CI pipeline based on Github Actions.
The problem is that I use Couchbase Lite in some of my tests, and they run perfectly fine either on Windows or MacOS. Nevertheless, Github Actions run on Linux and there lies my problem.
I’ve extracted the native libraries from the ZIP distribution and uploaded them to my repo. Then, I’m setting up an env var in the task inside my Actions definition like this:
Running an extra step with a simple ls -laR $LD_LIBRARY_PATH reassures me that my envvar is set and the content is right, as I can list the libraries there.
But when the Github Action is run…
RepositoryTest[jvm] > test()[jvm] FAILED
java.lang.IllegalStateException: Cannot load native library libLiteCore.so @/home/runner/work/snikt/snikt/common/CouchbaseLiteTemp/7eeb7166a5fd30ef3bfdbbd40057ccae/libLiteCore.so for Linux/amd64
at com.couchbase.lite.internal.NativeLibrary.load(NativeLibrary.java:113)
[...]
Caused by:
java.lang.IllegalStateException: Cannot load native library libLiteCore.so @/home/runner/work/snikt/snikt/common/CouchbaseLiteTemp/7eeb7166a5fd30ef3bfdbbd40057ccae/libLiteCore.so for Linux/amd64
at com.couchbase.lite.internal.NativeLibrary.load(NativeLibrary.java:113)
at com.couchbase.lite.internal.CouchbaseLiteInternal.init(CouchbaseLiteInternal.java:80)
[...]
Caused by:
java.lang.UnsatisfiedLinkError: /home/runner/work/snikt/snikt/common/CouchbaseLiteTemp/7eeb7166a5fd30ef3bfdbbd40057ccae/libLiteCore.so: libicuuc.so.54: cannot open shared object file: No such file or directory
at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:388)
Why is CB trying to load the native libraries from that random temp directory instead of from the path I’ve set as LD_LIBRARY_PATH?
It is not a “random” directory. It is a directory whose name is the SHA for the native library that is bundled with the code. That’s necessary for versioning.
Is there a reason that you want to, manually, pull the library apart and explicitly specify the path?
I don’t have any intention of overriding anything manually, I’m just trying to follow the instructions here to make my tests work on Linux and, thus, have a healthy CI loop. Happy to try any other way.
Do you know why the Github Action runner is not honoring my LD_LIBRARY_PATH env var?
Yes. At the moment, this has nothing to do with either GitHub Action Runner or LD_LIBRARY_PATH.
The CBLite library unpacks the native binary that is part of its jar file, stores it on the host file system, and loads it from there. You are attempting to manually override that process and it won’t work.
We use this process so that we can version the native library and be certain that we are loading the right thing. If you have a good reason for overriding that process manually, I can look in to honoring LD_LIBRARY_PATH.
So then it’s not possible to declare your dependency with Gradle and then add the necessary native libraries and point to them via LD_LIBRARY_PATH? Am I misunderstanding the documentation? Because even if I added the JAR file to my project, the one you provide in your ZIP bundle does not contain libz, libicu or libz. Those are in a separate support/linux/x86_64 folder outside that JAR. How can I point to them if it’s not with that env var?
My reason for trying to make this work is to run my code on Linux, simple as that. That also means being able to run my integration tests in Github Actions, but not being able to use my app on Linux is more important than that.
I don’t know what documentation you are looking at; I don’t think there is anything that suggest that you can use LD_LIBRARY_PATH to set the location of libLiteCore. The code certainly does not support that, currently.
Just to be clear, there are two, distinct sets of native libraries: the support libraries (in the /support directory of the zip file, libz, libicu, etc, necessary only to Linux applications) and libLiteCore (in the jar file). LD_LIBRARY_PATH is how, on Linux, one identifies the location of the support libraries (presumably moved by hand, from the zip file to some location on the Linux file system). It has nothing to do with libLiteCore, though.
Again, I’m not completely clear on what you are trying to do but, on all of the Linux applications with which I am familiar, if you put the support libs somewhere, point LD_LIBRARY_PATH at that somewhere, leave the jar file alone (it contains libLiteCore and knows how to load it) the app will work.
@serandel This is generic info, not specific to CB Lite (for which I defer to @blake.meike), but I have been in dependency hell before trying to get multiple products to work together, and one trick I sometimes found useful is to use a sym link to make it appear that some needed library located elsewhere is in whatever directory other libraries are successfully loading from. There are many possible permutations of this trick, such as making other libraries appear local to the directory a JAR file is in or making generic libraries all appear to be in a single directory or using directory links to make it appear a specific directory a JAR file wants to load from exists. Doesn’t always work but maybe worth a try.
It will also be very nice to be able to run my application on Linux, as a general goal, but still prototyping stuff, so it’s not urgent.
That’s a very good remark.
If we go again to my original message, this is the error stack trace I’m seeing:
RepositoryTest[jvm] > test()[jvm] FAILED
java.lang.IllegalStateException: Cannot load native library libLiteCore.so @/home/runner/work/snikt/snikt/common/CouchbaseLiteTemp/7eeb7166a5fd30ef3bfdbbd40057ccae/libLiteCore.so for Linux/amd64
at com.couchbase.lite.internal.NativeLibrary.load(NativeLibrary.java:113)
[...]
Caused by:
java.lang.IllegalStateException: Cannot load native library libLiteCore.so @/home/runner/work/snikt/snikt/common/CouchbaseLiteTemp/7eeb7166a5fd30ef3bfdbbd40057ccae/libLiteCore.so for Linux/amd64
at com.couchbase.lite.internal.NativeLibrary.load(NativeLibrary.java:113)
at com.couchbase.lite.internal.CouchbaseLiteInternal.init(CouchbaseLiteInternal.java:80)
[...]
Caused by:
java.lang.UnsatisfiedLinkError: /home/runner/work/snikt/snikt/common/CouchbaseLiteTemp/7eeb7166a5fd30ef3bfdbbd40057ccae/libLiteCore.so: libicuuc.so.54: cannot open shared object file: No such file or directory
at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:388)
So what I make of this (I may well be reading this wrong!) is that the JVM is not loading libLiteCore.so (not the library I’m trying to point at with LD_LIBRARY_PATH) because it cannot load libicuuc.so.54 (which is inside the $LD_LIBRARY_PATH/libicu folder).
Could it be I’m pointing to the wrong folder? Or perhaps it’s because the support libraries that you provide in your web are for x86_64, and it looks like the JVM is trying to load the native libraries for amd64…
Please, tell me if there’s any additional relevant info you may need. Thanks!
The problem is that when you download the Couchbase Lite ZIP distribution the support native libraries are located inside folders named support/linux/x86_64/libicu, support/linux/x86_64/libc++ and support/linux/x86_64/libz. I just copied the whole support folder to my project and added LD_LIBRARY_PATH pointing to support/linux/x86_64, trying to follow this snippet from the docs:
But no, it doesn’t work that way. The JVM when linking to the native libraries won’t go and look in the subfolders. So the fix is simply moving the contents of these three folders into /support/linux/x86_64/. Now Github Actions are happy and so am I.
For more experienced Linux devs I suppose this is obvious, but every day you learn something new. Could I suggest rewording the forementioned documentation to make this more clear? Thanks!
Could I suggest rewording the forementioned documentation to make this more clear? Thanks!