You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@arrow.apache.org by Michael Mior <mm...@apache.org> on 2021/01/05 16:11:26 UTC
Arrow/Gandiva dependency management in Java
I am working on integrating Arrow and Gandiva with the Apache Calcite
query processing framework so Calcite is able to process SQL queries
using Arrow data sources. The basics are working well but I'm having
trouble getting a reproducible development environment set up.
For example, on a Linux Travis CI right now, I get the following error:
java.lang.RuntimeException: libgandiva_jni.so was not found inside JAR.
I get something similar with AppVeyor on Windows:
java.lang.RuntimeException: gandiva_jni.dll was not found inside JAR.
Starting with a fresh clone of the repository, on my macOS machine I
use for development, I get the following which seems to be related to
a protobuf version mismatch:
java.lang.UnsatisfiedLinkError:
/private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392:
dlopen(/private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392,
1): Library not loaded:
/usr/local/opt/protobuf/lib/libprotobuf.24.dylib
Referenced from:
/private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392
I have dependencies declared on arrow-memory-netty, arrow-vector, and
arrow-gandiva but this doesn't seem to be sufficient. Any ideas what
I'm missing here?
--
Michael Mior
mmior@apache.org
Re: Arrow/Gandiva dependency management in Java
Posted by Michael Mior <mi...@gmail.com>.
You're correct that AppVeyor is Windows. I think it's fine if Windows
isn't supported for the time being. I assume you mean a JIRA issue as
I'm not able to open issues on the Arrow GH. Link to the JIRA issue
below. I'll be adding more details. Unfortunately still waiting on
Travis.
https://issues.apache.org/jira/browse/ARROW-11135
--
Michael Mior
michael.mior@gmail.com
Le mar. 5 janv. 2021 à 13:09, Jacques Nadeau <ja...@apache.org> a écrit :
>
> If I recall correctly, our artifact doesn't support windows (just mac and linux). If AppVeyor is for windows I would assume that would fail.
>
> Let's see what happens with Travis. I'd expect it to succeed.
>
> I suggest you open a Arrow GH issue to track this problem and we continue to record the result of items there as well.
>
> Can you also confirm the difference of the set of native libraries in the Dremio jar versus the Apache one?
>
>
>
>
> On Tue, Jan 5, 2021 at 9:49 AM Michael Mior <mi...@gmail.com> wrote:
>>
>> Thanks Jaques! Using the Dremio artifacts do seem to work on my macOS
>> development machine but failed with the same error on AppVeyor. (Still
>> waiting on Travis.) If there's anything I can do to help with
>> debugging, please let me know! You can see a (somewhat messy) PR in
>> progress at the link below. The last commit on the branch switches to
>> the Dremio repository if you want to try both and see if you can
>> reproduce (./gradlew :arrow:test should succeed if things are working
>> correctly).
>>
>> https://github.com/apache/calcite/pull/2133
>>
>> --
>> Michael Mior
>> michael.mior@gmail.com
>>
>> Le mar. 5 janv. 2021 à 12:08, Jacques Nadeau <ja...@apache.org> a écrit :
>> >
>> > If you're up for troubleshooting, do you want to try the near master jar we use inside of Dremio? I think we're the primary users of the Gandiva Java bindings and I wonder if the Apache build has an issue with artifact packaging(?). The jar we use in Dremio is used at hundreds of customers so it should work without extra items. You can find the reference in the pom here:
>> >
>> > https://github.com/dremio/dremio-oss/blob/master/pom.xml#L41
>> >
>> > I think you'll need to add our public repo to get the jar. I think it is this repo:
>> > https://github.com/dremio/dremio-oss/blob/master/pom.xml#L3243
>> >
>> > I'm only suggesting this to see if there is some kind of disconnect in keeping the Apache artifact complete. If our jar works for you we'll need to figure out what is different between it and the Apache one. (For reference, in general our jars are slightly off master to pull forward or backward certain bugs, etc.)
>> >
>> >
>> >
>> > On Tue, Jan 5, 2021 at 8:55 AM Michael Mior <mm...@apache.org> wrote:
>> >>
>> >> Yes, I'm using the latest released version (2.0.0).
>> >> --
>> >> Michael Mior
>> >> mmior@apache.org
>> >>
>> >> Le mar. 5 janv. 2021 à 11:45, Jacques Nadeau <ja...@apache.org> a écrit :
>> >> >
>> >> > Hey Michael, are you using the released versions of these Arrow artifacts? I recall there being complexity to building the multi-native-binary jar file...
>> >> >
>> >> > On Tue, Jan 5, 2021 at 8:11 AM Michael Mior <mm...@apache.org> wrote:
>> >> >>
>> >> >> I am working on integrating Arrow and Gandiva with the Apache Calcite
>> >> >> query processing framework so Calcite is able to process SQL queries
>> >> >> using Arrow data sources. The basics are working well but I'm having
>> >> >> trouble getting a reproducible development environment set up.
>> >> >>
>> >> >> For example, on a Linux Travis CI right now, I get the following error:
>> >> >>
>> >> >> java.lang.RuntimeException: libgandiva_jni.so was not found inside JAR.
>> >> >>
>> >> >> I get something similar with AppVeyor on Windows:
>> >> >>
>> >> >> java.lang.RuntimeException: gandiva_jni.dll was not found inside JAR.
>> >> >>
>> >> >> Starting with a fresh clone of the repository, on my macOS machine I
>> >> >> use for development, I get the following which seems to be related to
>> >> >> a protobuf version mismatch:
>> >> >>
>> >> >> java.lang.UnsatisfiedLinkError:
>> >> >> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392:
>> >> >> dlopen(/private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392,
>> >> >> 1): Library not loaded:
>> >> >> /usr/local/opt/protobuf/lib/libprotobuf.24.dylib
>> >> >> Referenced from:
>> >> >> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392
>> >> >>
>> >> >> I have dependencies declared on arrow-memory-netty, arrow-vector, and
>> >> >> arrow-gandiva but this doesn't seem to be sufficient. Any ideas what
>> >> >> I'm missing here?
>> >> >>
>> >> >> --
>> >> >> Michael Mior
>> >> >> mmior@apache.org
Re: Arrow/Gandiva dependency management in Java
Posted by Jacques Nadeau <ja...@apache.org>.
If I recall correctly, our artifact doesn't support windows (just mac and
linux). If AppVeyor is for windows I would assume that would fail.
Let's see what happens with Travis. I'd expect it to succeed.
I suggest you open a Arrow GH issue to track this problem and we continue
to record the result of items there as well.
Can you also confirm the difference of the set of native libraries in the
Dremio jar versus the Apache one?
On Tue, Jan 5, 2021 at 9:49 AM Michael Mior <mi...@gmail.com> wrote:
> Thanks Jaques! Using the Dremio artifacts do seem to work on my macOS
> development machine but failed with the same error on AppVeyor. (Still
> waiting on Travis.) If there's anything I can do to help with
> debugging, please let me know! You can see a (somewhat messy) PR in
> progress at the link below. The last commit on the branch switches to
> the Dremio repository if you want to try both and see if you can
> reproduce (./gradlew :arrow:test should succeed if things are working
> correctly).
>
> https://github.com/apache/calcite/pull/2133
>
> --
> Michael Mior
> michael.mior@gmail.com
>
> Le mar. 5 janv. 2021 à 12:08, Jacques Nadeau <ja...@apache.org> a écrit
> :
> >
> > If you're up for troubleshooting, do you want to try the near master jar
> we use inside of Dremio? I think we're the primary users of the Gandiva
> Java bindings and I wonder if the Apache build has an issue with artifact
> packaging(?). The jar we use in Dremio is used at hundreds of customers so
> it should work without extra items. You can find the reference in the pom
> here:
> >
> > https://github.com/dremio/dremio-oss/blob/master/pom.xml#L41
> >
> > I think you'll need to add our public repo to get the jar. I think it is
> this repo:
> > https://github.com/dremio/dremio-oss/blob/master/pom.xml#L3243
> >
> > I'm only suggesting this to see if there is some kind of disconnect in
> keeping the Apache artifact complete. If our jar works for you we'll need
> to figure out what is different between it and the Apache one. (For
> reference, in general our jars are slightly off master to pull forward or
> backward certain bugs, etc.)
> >
> >
> >
> > On Tue, Jan 5, 2021 at 8:55 AM Michael Mior <mm...@apache.org> wrote:
> >>
> >> Yes, I'm using the latest released version (2.0.0).
> >> --
> >> Michael Mior
> >> mmior@apache.org
> >>
> >> Le mar. 5 janv. 2021 à 11:45, Jacques Nadeau <ja...@apache.org> a
> écrit :
> >> >
> >> > Hey Michael, are you using the released versions of these Arrow
> artifacts? I recall there being complexity to building the
> multi-native-binary jar file...
> >> >
> >> > On Tue, Jan 5, 2021 at 8:11 AM Michael Mior <mm...@apache.org> wrote:
> >> >>
> >> >> I am working on integrating Arrow and Gandiva with the Apache Calcite
> >> >> query processing framework so Calcite is able to process SQL queries
> >> >> using Arrow data sources. The basics are working well but I'm having
> >> >> trouble getting a reproducible development environment set up.
> >> >>
> >> >> For example, on a Linux Travis CI right now, I get the following
> error:
> >> >>
> >> >> java.lang.RuntimeException: libgandiva_jni.so was not found inside
> JAR.
> >> >>
> >> >> I get something similar with AppVeyor on Windows:
> >> >>
> >> >> java.lang.RuntimeException: gandiva_jni.dll was not found inside JAR.
> >> >>
> >> >> Starting with a fresh clone of the repository, on my macOS machine I
> >> >> use for development, I get the following which seems to be related to
> >> >> a protobuf version mismatch:
> >> >>
> >> >> java.lang.UnsatisfiedLinkError:
> >> >>
> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392:
> >> >>
> dlopen(/private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392,
> >> >> 1): Library not loaded:
> >> >> /usr/local/opt/protobuf/lib/libprotobuf.24.dylib
> >> >> Referenced from:
> >> >>
> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392
> >> >>
> >> >> I have dependencies declared on arrow-memory-netty, arrow-vector, and
> >> >> arrow-gandiva but this doesn't seem to be sufficient. Any ideas what
> >> >> I'm missing here?
> >> >>
> >> >> --
> >> >> Michael Mior
> >> >> mmior@apache.org
>
Re: Arrow/Gandiva dependency management in Java
Posted by Michael Mior <mi...@gmail.com>.
Thanks Jaques! Using the Dremio artifacts do seem to work on my macOS
development machine but failed with the same error on AppVeyor. (Still
waiting on Travis.) If there's anything I can do to help with
debugging, please let me know! You can see a (somewhat messy) PR in
progress at the link below. The last commit on the branch switches to
the Dremio repository if you want to try both and see if you can
reproduce (./gradlew :arrow:test should succeed if things are working
correctly).
https://github.com/apache/calcite/pull/2133
--
Michael Mior
michael.mior@gmail.com
Le mar. 5 janv. 2021 à 12:08, Jacques Nadeau <ja...@apache.org> a écrit :
>
> If you're up for troubleshooting, do you want to try the near master jar we use inside of Dremio? I think we're the primary users of the Gandiva Java bindings and I wonder if the Apache build has an issue with artifact packaging(?). The jar we use in Dremio is used at hundreds of customers so it should work without extra items. You can find the reference in the pom here:
>
> https://github.com/dremio/dremio-oss/blob/master/pom.xml#L41
>
> I think you'll need to add our public repo to get the jar. I think it is this repo:
> https://github.com/dremio/dremio-oss/blob/master/pom.xml#L3243
>
> I'm only suggesting this to see if there is some kind of disconnect in keeping the Apache artifact complete. If our jar works for you we'll need to figure out what is different between it and the Apache one. (For reference, in general our jars are slightly off master to pull forward or backward certain bugs, etc.)
>
>
>
> On Tue, Jan 5, 2021 at 8:55 AM Michael Mior <mm...@apache.org> wrote:
>>
>> Yes, I'm using the latest released version (2.0.0).
>> --
>> Michael Mior
>> mmior@apache.org
>>
>> Le mar. 5 janv. 2021 à 11:45, Jacques Nadeau <ja...@apache.org> a écrit :
>> >
>> > Hey Michael, are you using the released versions of these Arrow artifacts? I recall there being complexity to building the multi-native-binary jar file...
>> >
>> > On Tue, Jan 5, 2021 at 8:11 AM Michael Mior <mm...@apache.org> wrote:
>> >>
>> >> I am working on integrating Arrow and Gandiva with the Apache Calcite
>> >> query processing framework so Calcite is able to process SQL queries
>> >> using Arrow data sources. The basics are working well but I'm having
>> >> trouble getting a reproducible development environment set up.
>> >>
>> >> For example, on a Linux Travis CI right now, I get the following error:
>> >>
>> >> java.lang.RuntimeException: libgandiva_jni.so was not found inside JAR.
>> >>
>> >> I get something similar with AppVeyor on Windows:
>> >>
>> >> java.lang.RuntimeException: gandiva_jni.dll was not found inside JAR.
>> >>
>> >> Starting with a fresh clone of the repository, on my macOS machine I
>> >> use for development, I get the following which seems to be related to
>> >> a protobuf version mismatch:
>> >>
>> >> java.lang.UnsatisfiedLinkError:
>> >> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392:
>> >> dlopen(/private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392,
>> >> 1): Library not loaded:
>> >> /usr/local/opt/protobuf/lib/libprotobuf.24.dylib
>> >> Referenced from:
>> >> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392
>> >>
>> >> I have dependencies declared on arrow-memory-netty, arrow-vector, and
>> >> arrow-gandiva but this doesn't seem to be sufficient. Any ideas what
>> >> I'm missing here?
>> >>
>> >> --
>> >> Michael Mior
>> >> mmior@apache.org
Re: Arrow/Gandiva dependency management in Java
Posted by Jacques Nadeau <ja...@apache.org>.
If you're up for troubleshooting, do you want to try the near master jar we
use inside of Dremio? I think we're the primary users of the Gandiva Java
bindings and I wonder if the Apache build has an issue with artifact
packaging(?). The jar we use in Dremio is used at hundreds of customers so
it should work without extra items. You can find the reference in the pom
here:
https://github.com/dremio/dremio-oss/blob/master/pom.xml#L41
I think you'll need to add our public repo to get the jar. I think it is
this repo:
https://github.com/dremio/dremio-oss/blob/master/pom.xml#L3243
I'm only suggesting this to see if there is some kind of disconnect in
keeping the Apache artifact complete. If our jar works for you we'll need
to figure out what is different between it and the Apache one. (For
reference, in general our jars are slightly off master to pull forward or
backward certain bugs, etc.)
On Tue, Jan 5, 2021 at 8:55 AM Michael Mior <mm...@apache.org> wrote:
> Yes, I'm using the latest released version (2.0.0).
> --
> Michael Mior
> mmior@apache.org
>
> Le mar. 5 janv. 2021 à 11:45, Jacques Nadeau <ja...@apache.org> a écrit
> :
> >
> > Hey Michael, are you using the released versions of these Arrow
> artifacts? I recall there being complexity to building the
> multi-native-binary jar file...
> >
> > On Tue, Jan 5, 2021 at 8:11 AM Michael Mior <mm...@apache.org> wrote:
> >>
> >> I am working on integrating Arrow and Gandiva with the Apache Calcite
> >> query processing framework so Calcite is able to process SQL queries
> >> using Arrow data sources. The basics are working well but I'm having
> >> trouble getting a reproducible development environment set up.
> >>
> >> For example, on a Linux Travis CI right now, I get the following error:
> >>
> >> java.lang.RuntimeException: libgandiva_jni.so was not found inside JAR.
> >>
> >> I get something similar with AppVeyor on Windows:
> >>
> >> java.lang.RuntimeException: gandiva_jni.dll was not found inside JAR.
> >>
> >> Starting with a fresh clone of the repository, on my macOS machine I
> >> use for development, I get the following which seems to be related to
> >> a protobuf version mismatch:
> >>
> >> java.lang.UnsatisfiedLinkError:
> >>
> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392:
> >>
> dlopen(/private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392,
> >> 1): Library not loaded:
> >> /usr/local/opt/protobuf/lib/libprotobuf.24.dylib
> >> Referenced from:
> >>
> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392
> >>
> >> I have dependencies declared on arrow-memory-netty, arrow-vector, and
> >> arrow-gandiva but this doesn't seem to be sufficient. Any ideas what
> >> I'm missing here?
> >>
> >> --
> >> Michael Mior
> >> mmior@apache.org
>
Re: Arrow/Gandiva dependency management in Java
Posted by Michael Mior <mm...@apache.org>.
Yes, I'm using the latest released version (2.0.0).
--
Michael Mior
mmior@apache.org
Le mar. 5 janv. 2021 à 11:45, Jacques Nadeau <ja...@apache.org> a écrit :
>
> Hey Michael, are you using the released versions of these Arrow artifacts? I recall there being complexity to building the multi-native-binary jar file...
>
> On Tue, Jan 5, 2021 at 8:11 AM Michael Mior <mm...@apache.org> wrote:
>>
>> I am working on integrating Arrow and Gandiva with the Apache Calcite
>> query processing framework so Calcite is able to process SQL queries
>> using Arrow data sources. The basics are working well but I'm having
>> trouble getting a reproducible development environment set up.
>>
>> For example, on a Linux Travis CI right now, I get the following error:
>>
>> java.lang.RuntimeException: libgandiva_jni.so was not found inside JAR.
>>
>> I get something similar with AppVeyor on Windows:
>>
>> java.lang.RuntimeException: gandiva_jni.dll was not found inside JAR.
>>
>> Starting with a fresh clone of the repository, on my macOS machine I
>> use for development, I get the following which seems to be related to
>> a protobuf version mismatch:
>>
>> java.lang.UnsatisfiedLinkError:
>> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392:
>> dlopen(/private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392,
>> 1): Library not loaded:
>> /usr/local/opt/protobuf/lib/libprotobuf.24.dylib
>> Referenced from:
>> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392
>>
>> I have dependencies declared on arrow-memory-netty, arrow-vector, and
>> arrow-gandiva but this doesn't seem to be sufficient. Any ideas what
>> I'm missing here?
>>
>> --
>> Michael Mior
>> mmior@apache.org
Re: Arrow/Gandiva dependency management in Java
Posted by Jacques Nadeau <ja...@apache.org>.
Hey Michael, are you using the released versions of these Arrow artifacts?
I recall there being complexity to building the multi-native-binary jar
file...
On Tue, Jan 5, 2021 at 8:11 AM Michael Mior <mm...@apache.org> wrote:
> I am working on integrating Arrow and Gandiva with the Apache Calcite
> query processing framework so Calcite is able to process SQL queries
> using Arrow data sources. The basics are working well but I'm having
> trouble getting a reproducible development environment set up.
>
> For example, on a Linux Travis CI right now, I get the following error:
>
> java.lang.RuntimeException: libgandiva_jni.so was not found inside JAR.
>
> I get something similar with AppVeyor on Windows:
>
> java.lang.RuntimeException: gandiva_jni.dll was not found inside JAR.
>
> Starting with a fresh clone of the repository, on my macOS machine I
> use for development, I get the following which seems to be related to
> a protobuf version mismatch:
>
> java.lang.UnsatisfiedLinkError:
>
> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392:
>
> dlopen(/private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392,
> 1): Library not loaded:
> /usr/local/opt/protobuf/lib/libprotobuf.24.dylib
> Referenced from:
>
> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392
>
> I have dependencies declared on arrow-memory-netty, arrow-vector, and
> arrow-gandiva but this doesn't seem to be sufficient. Any ideas what
> I'm missing here?
>
> --
> Michael Mior
> mmior@apache.org
>
Re: Arrow/Gandiva dependency management in Java
Posted by Jacques Nadeau <ja...@apache.org>.
Burke...
Send an email to user-unsuscribe@arrow.apache.org
On Tue, Jan 5, 2021 at 8:25 AM Burke Kaltenberger <
burke@firsttalentsearch.com> wrote:
> Please take me off your mailing list
>
> On Tue, Jan 5, 2021, 8:11 AM Michael Mior <mm...@apache.org> wrote:
>
>> I am working on integrating Arrow and Gandiva with the Apache Calcite
>> query processing framework so Calcite is able to process SQL queries
>> using Arrow data sources. The basics are working well but I'm having
>> trouble getting a reproducible development environment set up.
>>
>> For example, on a Linux Travis CI right now, I get the following error:
>>
>> java.lang.RuntimeException: libgandiva_jni.so was not found inside JAR.
>>
>> I get something similar with AppVeyor on Windows:
>>
>> java.lang.RuntimeException: gandiva_jni.dll was not found inside JAR.
>>
>> Starting with a fresh clone of the repository, on my macOS machine I
>> use for development, I get the following which seems to be related to
>> a protobuf version mismatch:
>>
>> java.lang.UnsatisfiedLinkError:
>>
>> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392:
>>
>> dlopen(/private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392,
>> 1): Library not loaded:
>> /usr/local/opt/protobuf/lib/libprotobuf.24.dylib
>> Referenced from:
>>
>> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392
>>
>> I have dependencies declared on arrow-memory-netty, arrow-vector, and
>> arrow-gandiva but this doesn't seem to be sufficient. Any ideas what
>> I'm missing here?
>>
>> --
>> Michael Mior
>> mmior@apache.org
>>
>
Re: Arrow/Gandiva dependency management in Java
Posted by Burke Kaltenberger <bu...@firsttalentsearch.com>.
Please take me off your mailing list
On Tue, Jan 5, 2021, 8:11 AM Michael Mior <mm...@apache.org> wrote:
> I am working on integrating Arrow and Gandiva with the Apache Calcite
> query processing framework so Calcite is able to process SQL queries
> using Arrow data sources. The basics are working well but I'm having
> trouble getting a reproducible development environment set up.
>
> For example, on a Linux Travis CI right now, I get the following error:
>
> java.lang.RuntimeException: libgandiva_jni.so was not found inside JAR.
>
> I get something similar with AppVeyor on Windows:
>
> java.lang.RuntimeException: gandiva_jni.dll was not found inside JAR.
>
> Starting with a fresh clone of the repository, on my macOS machine I
> use for development, I get the following which seems to be related to
> a protobuf version mismatch:
>
> java.lang.UnsatisfiedLinkError:
>
> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392:
>
> dlopen(/private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392,
> 1): Library not loaded:
> /usr/local/opt/protobuf/lib/libprotobuf.24.dylib
> Referenced from:
>
> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392
>
> I have dependencies declared on arrow-memory-netty, arrow-vector, and
> arrow-gandiva but this doesn't seem to be sufficient. Any ideas what
> I'm missing here?
>
> --
> Michael Mior
> mmior@apache.org
>