You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Vishnu Viswanath <vi...@gmail.com> on 2018/07/30 15:44:22 UTC
RocksDB error with KStream
Hi All,
While running Kafka streams application with a "aggregate" function in it.
Running via docker (alpine linux). I am getting
*"*java.lang.UnsatisfiedLinkError:
/tmp/librocksdbjni20844899025353663*88.so: libstdc++.so.6: cannot open
shared object file: No such file or directory". *similar to what is
described here:
https://groups.google.com/forum/#!msg/confluent-platform/uFTFZh3v-kg/luNsCcRbBQAJ
In the Dockerfile I did create a temp directory with +w permission and pass
it as "-Djava.io.tmpdir". If I login into the instance, I could see that
librocksdbjni2084489902535366388.so file is created in the tmpdir, but
application still fails with unsatisfied link error.
Could solve it by changing to in memory store, but is skeptical if this
will be an issue later if the aggregate function uses lot of memory. Is
there any other suggestion on solving this problem (i.e., use rocksdb as
state store and run within a docker container)
Thanks in advance,
Vishnu
Re: RocksDB error with KStream
Posted by Guozhang Wang <wa...@gmail.com>.
Glad to hear that!
On Wed, Aug 22, 2018 at 8:13 AM, Vishnu Viswanath <
vishnu.viswanath25@gmail.com> wrote:
> Thank you Guoshang. I finally solved the issue by shifting from openjdk to
> https://hub.docker.com/r/anapsix/alpine-java/ in my container.
> Looks like alpine openjdk has musl libc implementation but RocksJava
> required glibc.
>
> On Mon, Jul 30, 2018 at 9:31 AM Guozhang Wang <wa...@gmail.com> wrote:
>
> > Hello Vishnu,
> >
> > Maybe this is the root cause of your issue?
> > https://github.com/docker-flink/docker-flink/issues/14
> >
> >
> > Guozhang
> >
> > On Mon, Jul 30, 2018 at 8:44 AM, Vishnu Viswanath <
> > vishnu.viswanath25@gmail.com> wrote:
> >
> > > Hi All,
> > >
> > > While running Kafka streams application with a "aggregate" function in
> > it.
> > > Running via docker (alpine linux). I am getting
> > > *"*java.lang.UnsatisfiedLinkError:
> > > /tmp/librocksdbjni20844899025353663*88.so: libstdc++.so.6: cannot open
> > > shared object file: No such file or directory". *similar to what is
> > > described here:
> > >
> > > https://groups.google.com/forum/#!msg/confluent-platform/uFTFZh3v-kg/
> > > luNsCcRbBQAJ
> > >
> > >
> > > In the Dockerfile I did create a temp directory with +w permission and
> > pass
> > > it as "-Djava.io.tmpdir". If I login into the instance, I could see
> that
> > > librocksdbjni2084489902535366388.so file is created in the tmpdir, but
> > > application still fails with unsatisfied link error.
> > >
> > > Could solve it by changing to in memory store, but is skeptical if this
> > > will be an issue later if the aggregate function uses lot of memory. Is
> > > there any other suggestion on solving this problem (i.e., use rocksdb
> as
> > > state store and run within a docker container)
> > >
> > > Thanks in advance,
> > > Vishnu
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>
--
-- Guozhang
Re: RocksDB error with KStream
Posted by Vishnu Viswanath <vi...@gmail.com>.
Thank you Guoshang. I finally solved the issue by shifting from openjdk to
https://hub.docker.com/r/anapsix/alpine-java/ in my container.
Looks like alpine openjdk has musl libc implementation but RocksJava
required glibc.
On Mon, Jul 30, 2018 at 9:31 AM Guozhang Wang <wa...@gmail.com> wrote:
> Hello Vishnu,
>
> Maybe this is the root cause of your issue?
> https://github.com/docker-flink/docker-flink/issues/14
>
>
> Guozhang
>
> On Mon, Jul 30, 2018 at 8:44 AM, Vishnu Viswanath <
> vishnu.viswanath25@gmail.com> wrote:
>
> > Hi All,
> >
> > While running Kafka streams application with a "aggregate" function in
> it.
> > Running via docker (alpine linux). I am getting
> > *"*java.lang.UnsatisfiedLinkError:
> > /tmp/librocksdbjni20844899025353663*88.so: libstdc++.so.6: cannot open
> > shared object file: No such file or directory". *similar to what is
> > described here:
> >
> > https://groups.google.com/forum/#!msg/confluent-platform/uFTFZh3v-kg/
> > luNsCcRbBQAJ
> >
> >
> > In the Dockerfile I did create a temp directory with +w permission and
> pass
> > it as "-Djava.io.tmpdir". If I login into the instance, I could see that
> > librocksdbjni2084489902535366388.so file is created in the tmpdir, but
> > application still fails with unsatisfied link error.
> >
> > Could solve it by changing to in memory store, but is skeptical if this
> > will be an issue later if the aggregate function uses lot of memory. Is
> > there any other suggestion on solving this problem (i.e., use rocksdb as
> > state store and run within a docker container)
> >
> > Thanks in advance,
> > Vishnu
> >
>
>
>
> --
> -- Guozhang
>
Re: RocksDB error with KStream
Posted by Guozhang Wang <wa...@gmail.com>.
Hello Vishnu,
Maybe this is the root cause of your issue?
https://github.com/docker-flink/docker-flink/issues/14
Guozhang
On Mon, Jul 30, 2018 at 8:44 AM, Vishnu Viswanath <
vishnu.viswanath25@gmail.com> wrote:
> Hi All,
>
> While running Kafka streams application with a "aggregate" function in it.
> Running via docker (alpine linux). I am getting
> *"*java.lang.UnsatisfiedLinkError:
> /tmp/librocksdbjni20844899025353663*88.so: libstdc++.so.6: cannot open
> shared object file: No such file or directory". *similar to what is
> described here:
>
> https://groups.google.com/forum/#!msg/confluent-platform/uFTFZh3v-kg/
> luNsCcRbBQAJ
>
>
> In the Dockerfile I did create a temp directory with +w permission and pass
> it as "-Djava.io.tmpdir". If I login into the instance, I could see that
> librocksdbjni2084489902535366388.so file is created in the tmpdir, but
> application still fails with unsatisfied link error.
>
> Could solve it by changing to in memory store, but is skeptical if this
> will be an issue later if the aggregate function uses lot of memory. Is
> there any other suggestion on solving this problem (i.e., use rocksdb as
> state store and run within a docker container)
>
> Thanks in advance,
> Vishnu
>
--
-- Guozhang