You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Christian Kniep <ck...@gaikai.com> on 2016/01/12 17:56:17 UTC
Docker-Image for 'hello-samza'
Hey there,
I try to replay the hello-samza stuff with my docker image, but failed. :(
Could someone provide me a hint what I do wrong?
I have to admit, that I am not the biggest Java developer, therefore I might do stupid mistakes… :)
My Steps: https://github.com/ChristianKniep/orchestra/tree/master/samza <https://github.com/ChristianKniep/orchestra/tree/master/samza>
Cheers
Christian
--
Christian Kniep | Release Engineer
www.gaikai.com <http://www.gaikai.com/>
Re: Docker-Image for 'hello-samza'
Posted by Navina Ramesh <nr...@linkedin.com.INVALID>.
Hi Christian,
Sorry for chiming in late.
I am new to docker and having trouble building the shared docker image.
I pulled the docker image as per the instructions in
https://github.com/qnib/u-samza/blob/master/SetupDocker.md
I know that the VM is running. Not sure what we are "building" after
installing the image.
I tried the "docker build -t qnib/samza ." command in README and it threw
the following exception:
unable to prepare context: unable to evaluate symlinks in Dockerfile path:
lstat /Users/nramesh/Dockerfile: no such file or directory
Where does the Dockerfile get created? Is this command intended to be run
within the VM? I am kind of confused as I am still catching up the docker
documentation :)
Let me know how I can fix it. I would like to help troubleshoot the issue.
Thanks!
Navina
On Fri, Jan 15, 2016 at 2:04 AM, Christian Kniep <ck...@gaikai.com> wrote:
> Hi Yi,
>
> I added documentation on how to spin up the image, I guess we could skip a
> couple of RTTs by using a common Docker Image. :)
> https://github.com/qnib/u-samza/blob/master/SetupDocker.md
>
> Cheers
> Christian
>
> --
>
>
>
> Christian Kniep | Release Engineer
>
> www.gaikai.com
>
> On 14 Jan 2016, at 19:12, Yi Pan <ni...@gmail.com> wrote:
>
> Hi, Christian,
>
> From the logs I can find on github, I only saw the job is submitted
> successfully, which means that the Kafka systems are working fine. I don't
> see the following information:
> a) How do you determine that the job failed?
> b) There should be an userlogs under ${HELLO-SAMZA}/deploy/yarn/logs, in
> which you should be able to find the SamzaAppMaster and SamzaContainer
> logs. That would tell us more about errors.
>
> Could you post those logs somewhere together w/ your configuration file
> s.t. we can help to dig into it more?
>
> Thanks a lot!
>
> -Yi
>
> On Thu, Jan 14, 2016 at 3:07 AM, Christian Kniep <ck...@gaikai.com>
> wrote:
>
>> The kafka error seems not to be the important piece…
>> https://issues.apache.org/jira/browse/KAFKA-2251
>>
>> I created an image that holds all services needed to rule out that
>> somehow the job assumes ‘localhost’ connections somewhere:
>> https://hub.docker.com/r/qnib/samza-mono/builds/
>>
>> Still get the same error. :/
>>
>> --
>>
>>
>>
>> Christian Kniep | Release Engineer
>>
>> www.gaikai.com
>>
>> On 14 Jan 2016, at 09:50, Christian Kniep <ck...@gaikai.com> wrote:
>>
>> Hey Yi,
>>
>> as you can see within the github repo I explore software stacks by tackle
>> them with docker. Quite fun, but challenging from times to times.
>> Especially if it’s handled by layers of Java - not per se a problem, but
>> my Java knowledge lacks experience.
>>
>> But to the topic… I checked the logs of ZK and Kafka and Kafka throws:
>> ####
>> [2016-01-14 09:42:28,748] INFO Closing socket connection to /172.17.0.7.
>> (kafka.network.Processor)
>> [2016-01-14 09:42:28,807] INFO Closing socket connection to /172.17.0.7.
>> (kafka.network.Processor)
>> [2016-01-14 09:42:29,993] INFO Closing socket connection to /172.17.0.7.
>> (kafka.network.Processor)
>> [2016-01-14 09:42:30,010] INFO Closing socket connection to /172.17.0.7.
>> (kafka.network.Processor)
>> [2016-01-14 09:42:30,010] INFO Closing socket connection to /172.17.0.7.
>> (kafka.network.Processor)
>> [2016-01-14 09:42:36,183] INFO Closing socket connection to /172.17.0.7.
>> (kafka.network.Processor)
>> [2016-01-14 09:42:36,270] INFO Closing socket connection to /172.17.0.7.
>> (kafka.network.Processor)
>> [2016-01-14 09:42:40,591] INFO Closing socket connection to /172.17.0.7.
>> (kafka.network.Processor)
>> [2016-01-14 09:42:40,592] INFO Closing socket connection to /172.17.0.7.
>> (kafka.network.Processor)
>> [2016-01-14 09:42:46,451] ERROR Closing socket for /172.17.0.7 because
>> of error (kafka.network.Processor)
>> java.io.IOException: Connection reset by peer
>> at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
>> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
>> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
>> at sun.nio.ch.IOUtil.read(IOUtil.java:197)
>> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
>> at kafka.utils.Utils$.read(Utils.scala:380)
>> at
>> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
>> at kafka.network.Processor.read(SocketServer.scala:444)
>> at kafka.network.Processor.run(SocketServer.scala:340)
>> at java.lang.Thread.run(Thread.java:745)
>> [2016-01-14 09:42:46,451] INFO Closing socket connection to /172.17.0.7.
>> (kafka.network.Processor)
>> [2016-01-14 09:42:46,452] INFO Closing socket connection to /172.17.0.7.
>> (kafka.network.Processor)
>> ####
>>
>> Even thought the Kafka-Manager seems to work as expected (see picture).
>> The ZookeeperUI also works as (I would) expect it to work. I used the
>> stack of Kafka&Zookeeper plus all the Manager/Monitors and UI for other
>> purposes, therefore I like to think they are fine. :)
>>
>> If we can figure the stack out with hello-szama it would be my pleasure
>> to write this up nicely and create a blog post, add it to the samza
>> documentation and showcase samza as a BigData project during my FOSDEM talk
>> (https://fosdem.org/2016/schedule/track/hpc,_big_data_and_data_science/).
>>
>> Cheers
>> Christian
>>
>>
>> <Screen Shot 2016-01-14 at 09.45.48.png>
>>
>> --
>>
>> <Gaikai_Logo.png>
>>
>>
>> Christian Kniep | Release Engineer
>>
>> www.gaikai.com
>>
>> On 14 Jan 2016, at 06:42, Yi Pan <ni...@gmail.com> wrote:
>>
>> Hi, Christian,
>>
>> This is interesting area (i.e. creating docker image for hello-samza). I
>> did not see the container log, which I assume that would include the error
>> message? How did you hello-samza job fail?
>>
>> -Yi
>>
>> On Tue, Jan 12, 2016 at 8:56 AM, Christian Kniep <ck...@gaikai.com>
>> wrote:
>>
>>> Hey there,
>>>
>>> I try to replay the hello-samza stuff with my docker image, but failed.
>>> :(
>>> Could someone provide me a hint what I do wrong?
>>>
>>> I have to admit, that I am not the biggest Java developer, therefore I
>>> might do stupid mistakes… :)
>>>
>>> My Steps: https://github.com/ChristianKniep/orchestra/tree/master/samza
>>>
>>> Cheers
>>> Christian
>>>
>>> --
>>>
>>> <Gaikai_Logo.png>
>>>
>>>
>>> Christian Kniep | Release Engineer
>>>
>>> www.gaikai.com
>>>
>>
>>
>
>
--
Navina R.
Re: Docker-Image for 'hello-samza'
Posted by Christian Kniep <ck...@gaikai.com>.
Hi Yi,
I added documentation on how to spin up the image, I guess we could skip a couple of RTTs by using a common Docker Image. :)
https://github.com/qnib/u-samza/blob/master/SetupDocker.md
Cheers
Christian
--
Christian Kniep | Release Engineer
www.gaikai.com <http://www.gaikai.com/>
> On 14 Jan 2016, at 19:12, Yi Pan <ni...@gmail.com> wrote:
>
> Hi, Christian,
>
> From the logs I can find on github, I only saw the job is submitted successfully, which means that the Kafka systems are working fine. I don't see the following information:
> a) How do you determine that the job failed?
> b) There should be an userlogs under ${HELLO-SAMZA}/deploy/yarn/logs, in which you should be able to find the SamzaAppMaster and SamzaContainer logs. That would tell us more about errors.
>
> Could you post those logs somewhere together w/ your configuration file s.t. we can help to dig into it more?
>
> Thanks a lot!
>
> -Yi
>
> On Thu, Jan 14, 2016 at 3:07 AM, Christian Kniep <ckniep@gaikai.com <ma...@gaikai.com>> wrote:
> The kafka error seems not to be the important piece… https://issues.apache.org/jira/browse/KAFKA-2251 <https://issues.apache.org/jira/browse/KAFKA-2251>
>
> I created an image that holds all services needed to rule out that somehow the job assumes ‘localhost’ connections somewhere:
> https://hub.docker.com/r/qnib/samza-mono/builds/ <https://hub.docker.com/r/qnib/samza-mono/builds/>
>
> Still get the same error. :/
>
> --
>
>
>
>
> Christian Kniep | Release Engineer
>
> www.gaikai.com <http://www.gaikai.com/>
>> On 14 Jan 2016, at 09:50, Christian Kniep <ckniep@gaikai.com <ma...@gaikai.com>> wrote:
>>
>> Hey Yi,
>>
>> as you can see within the github repo I explore software stacks by tackle them with docker. Quite fun, but challenging from times to times.
>> Especially if it’s handled by layers of Java - not per se a problem, but my Java knowledge lacks experience.
>>
>> But to the topic… I checked the logs of ZK and Kafka and Kafka throws:
>> ####
>> [2016-01-14 09:42:28,748] INFO Closing socket connection to /172.17.0.7 <http://172.17.0.7/>. (kafka.network.Processor)
>> [2016-01-14 09:42:28,807] INFO Closing socket connection to /172.17.0.7 <http://172.17.0.7/>. (kafka.network.Processor)
>> [2016-01-14 09:42:29,993] INFO Closing socket connection to /172.17.0.7 <http://172.17.0.7/>. (kafka.network.Processor)
>> [2016-01-14 09:42:30,010] INFO Closing socket connection to /172.17.0.7 <http://172.17.0.7/>. (kafka.network.Processor)
>> [2016-01-14 09:42:30,010] INFO Closing socket connection to /172.17.0.7 <http://172.17.0.7/>. (kafka.network.Processor)
>> [2016-01-14 09:42:36,183] INFO Closing socket connection to /172.17.0.7 <http://172.17.0.7/>. (kafka.network.Processor)
>> [2016-01-14 09:42:36,270] INFO Closing socket connection to /172.17.0.7 <http://172.17.0.7/>. (kafka.network.Processor)
>> [2016-01-14 09:42:40,591] INFO Closing socket connection to /172.17.0.7 <http://172.17.0.7/>. (kafka.network.Processor)
>> [2016-01-14 09:42:40,592] INFO Closing socket connection to /172.17.0.7 <http://172.17.0.7/>. (kafka.network.Processor)
>> [2016-01-14 09:42:46,451] ERROR Closing socket for /172.17.0.7 <http://172.17.0.7/> because of error (kafka.network.Processor)
>> java.io.IOException: Connection reset by peer
>> at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
>> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
>> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
>> at sun.nio.ch.IOUtil.read(IOUtil.java:197)
>> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
>> at kafka.utils.Utils$.read(Utils.scala:380)
>> at kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
>> at kafka.network.Processor.read(SocketServer.scala:444)
>> at kafka.network.Processor.run(SocketServer.scala:340)
>> at java.lang.Thread.run(Thread.java:745)
>> [2016-01-14 09:42:46,451] INFO Closing socket connection to /172.17.0.7 <http://172.17.0.7/>. (kafka.network.Processor)
>> [2016-01-14 09:42:46,452] INFO Closing socket connection to /172.17.0.7 <http://172.17.0.7/>. (kafka.network.Processor)
>> ####
>>
>> Even thought the Kafka-Manager seems to work as expected (see picture).
>> The ZookeeperUI also works as (I would) expect it to work. I used the stack of Kafka&Zookeeper plus all the Manager/Monitors and UI for other purposes, therefore I like to think they are fine. :)
>>
>> If we can figure the stack out with hello-szama it would be my pleasure to write this up nicely and create a blog post, add it to the samza documentation and showcase samza as a BigData project during my FOSDEM talk (https://fosdem.org/2016/schedule/track/hpc,_big_data_and_data_science/ <https://fosdem.org/2016/schedule/track/hpc,_big_data_and_data_science/>).
>>
>> Cheers
>> Christian
>>
>>
>> <Screen Shot 2016-01-14 at 09.45.48.png>
>>
>> --
>>
>> <Gaikai_Logo.png>
>>
>>
>> Christian Kniep | Release Engineer
>>
>> www.gaikai.com <http://www.gaikai.com/>
>>> On 14 Jan 2016, at 06:42, Yi Pan <nickpan47@gmail.com <ma...@gmail.com>> wrote:
>>>
>>> Hi, Christian,
>>>
>>> This is interesting area (i.e. creating docker image for hello-samza). I did not see the container log, which I assume that would include the error message? How did you hello-samza job fail?
>>>
>>> -Yi
>>>
>>> On Tue, Jan 12, 2016 at 8:56 AM, Christian Kniep <ckniep@gaikai.com <ma...@gaikai.com>> wrote:
>>> Hey there,
>>>
>>> I try to replay the hello-samza stuff with my docker image, but failed. :(
>>> Could someone provide me a hint what I do wrong?
>>>
>>> I have to admit, that I am not the biggest Java developer, therefore I might do stupid mistakes… :)
>>>
>>> My Steps: https://github.com/ChristianKniep/orchestra/tree/master/samza <https://github.com/ChristianKniep/orchestra/tree/master/samza>
>>>
>>> Cheers
>>> Christian
>>>
>>> --
>>>
>>> <Gaikai_Logo.png>
>>>
>>>
>>> Christian Kniep | Release Engineer
>>>
>>> www.gaikai.com <http://www.gaikai.com/>
>
Re: Docker-Image for 'hello-samza'
Posted by Yi Pan <ni...@gmail.com>.
Hi, Christian,
>From the logs I can find on github, I only saw the job is submitted
successfully, which means that the Kafka systems are working fine. I don't
see the following information:
a) How do you determine that the job failed?
b) There should be an userlogs under ${HELLO-SAMZA}/deploy/yarn/logs, in
which you should be able to find the SamzaAppMaster and SamzaContainer
logs. That would tell us more about errors.
Could you post those logs somewhere together w/ your configuration file
s.t. we can help to dig into it more?
Thanks a lot!
-Yi
On Thu, Jan 14, 2016 at 3:07 AM, Christian Kniep <ck...@gaikai.com> wrote:
> The kafka error seems not to be the important piece…
> https://issues.apache.org/jira/browse/KAFKA-2251
>
> I created an image that holds all services needed to rule out that somehow
> the job assumes ‘localhost’ connections somewhere:
> https://hub.docker.com/r/qnib/samza-mono/builds/
>
> Still get the same error. :/
>
> --
>
>
>
> Christian Kniep | Release Engineer
>
> www.gaikai.com
>
> On 14 Jan 2016, at 09:50, Christian Kniep <ck...@gaikai.com> wrote:
>
> Hey Yi,
>
> as you can see within the github repo I explore software stacks by tackle
> them with docker. Quite fun, but challenging from times to times.
> Especially if it’s handled by layers of Java - not per se a problem, but
> my Java knowledge lacks experience.
>
> But to the topic… I checked the logs of ZK and Kafka and Kafka throws:
> ####
> [2016-01-14 09:42:28,748] INFO Closing socket connection to /172.17.0.7.
> (kafka.network.Processor)
> [2016-01-14 09:42:28,807] INFO Closing socket connection to /172.17.0.7.
> (kafka.network.Processor)
> [2016-01-14 09:42:29,993] INFO Closing socket connection to /172.17.0.7.
> (kafka.network.Processor)
> [2016-01-14 09:42:30,010] INFO Closing socket connection to /172.17.0.7.
> (kafka.network.Processor)
> [2016-01-14 09:42:30,010] INFO Closing socket connection to /172.17.0.7.
> (kafka.network.Processor)
> [2016-01-14 09:42:36,183] INFO Closing socket connection to /172.17.0.7.
> (kafka.network.Processor)
> [2016-01-14 09:42:36,270] INFO Closing socket connection to /172.17.0.7.
> (kafka.network.Processor)
> [2016-01-14 09:42:40,591] INFO Closing socket connection to /172.17.0.7.
> (kafka.network.Processor)
> [2016-01-14 09:42:40,592] INFO Closing socket connection to /172.17.0.7.
> (kafka.network.Processor)
> [2016-01-14 09:42:46,451] ERROR Closing socket for /172.17.0.7 because of
> error (kafka.network.Processor)
> java.io.IOException: Connection reset by peer
> at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
> at sun.nio.ch.IOUtil.read(IOUtil.java:197)
> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
> at kafka.utils.Utils$.read(Utils.scala:380)
> at
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
> at kafka.network.Processor.read(SocketServer.scala:444)
> at kafka.network.Processor.run(SocketServer.scala:340)
> at java.lang.Thread.run(Thread.java:745)
> [2016-01-14 09:42:46,451] INFO Closing socket connection to /172.17.0.7.
> (kafka.network.Processor)
> [2016-01-14 09:42:46,452] INFO Closing socket connection to /172.17.0.7.
> (kafka.network.Processor)
> ####
>
> Even thought the Kafka-Manager seems to work as expected (see picture).
> The ZookeeperUI also works as (I would) expect it to work. I used the
> stack of Kafka&Zookeeper plus all the Manager/Monitors and UI for other
> purposes, therefore I like to think they are fine. :)
>
> If we can figure the stack out with hello-szama it would be my pleasure to
> write this up nicely and create a blog post, add it to the samza
> documentation and showcase samza as a BigData project during my FOSDEM talk
> (https://fosdem.org/2016/schedule/track/hpc,_big_data_and_data_science/).
>
> Cheers
> Christian
>
>
> <Screen Shot 2016-01-14 at 09.45.48.png>
>
> --
>
> <Gaikai_Logo.png>
>
>
> Christian Kniep | Release Engineer
>
> www.gaikai.com
>
> On 14 Jan 2016, at 06:42, Yi Pan <ni...@gmail.com> wrote:
>
> Hi, Christian,
>
> This is interesting area (i.e. creating docker image for hello-samza). I
> did not see the container log, which I assume that would include the error
> message? How did you hello-samza job fail?
>
> -Yi
>
> On Tue, Jan 12, 2016 at 8:56 AM, Christian Kniep <ck...@gaikai.com>
> wrote:
>
>> Hey there,
>>
>> I try to replay the hello-samza stuff with my docker image, but failed. :(
>> Could someone provide me a hint what I do wrong?
>>
>> I have to admit, that I am not the biggest Java developer, therefore I
>> might do stupid mistakes… :)
>>
>> My Steps: https://github.com/ChristianKniep/orchestra/tree/master/samza
>>
>> Cheers
>> Christian
>>
>> --
>>
>> <Gaikai_Logo.png>
>>
>>
>> Christian Kniep | Release Engineer
>>
>> www.gaikai.com
>>
>
>
Re: Docker-Image for 'hello-samza'
Posted by Christian Kniep <ck...@gaikai.com>.
The kafka error seems not to be the important piece… https://issues.apache.org/jira/browse/KAFKA-2251
I created an image that holds all services needed to rule out that somehow the job assumes ‘localhost’ connections somewhere:
https://hub.docker.com/r/qnib/samza-mono/builds/ <https://hub.docker.com/r/qnib/samza-mono/builds/>
Still get the same error. :/
--
Christian Kniep | Release Engineer
www.gaikai.com <http://www.gaikai.com/>
> On 14 Jan 2016, at 09:50, Christian Kniep <ck...@gaikai.com> wrote:
>
> Hey Yi,
>
> as you can see within the github repo I explore software stacks by tackle them with docker. Quite fun, but challenging from times to times.
> Especially if it’s handled by layers of Java - not per se a problem, but my Java knowledge lacks experience.
>
> But to the topic… I checked the logs of ZK and Kafka and Kafka throws:
> ####
> [2016-01-14 09:42:28,748] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
> [2016-01-14 09:42:28,807] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
> [2016-01-14 09:42:29,993] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
> [2016-01-14 09:42:30,010] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
> [2016-01-14 09:42:30,010] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
> [2016-01-14 09:42:36,183] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
> [2016-01-14 09:42:36,270] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
> [2016-01-14 09:42:40,591] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
> [2016-01-14 09:42:40,592] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
> [2016-01-14 09:42:46,451] ERROR Closing socket for /172.17.0.7 because of error (kafka.network.Processor)
> java.io.IOException: Connection reset by peer
> at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
> at sun.nio.ch.IOUtil.read(IOUtil.java:197)
> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
> at kafka.utils.Utils$.read(Utils.scala:380)
> at kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
> at kafka.network.Processor.read(SocketServer.scala:444)
> at kafka.network.Processor.run(SocketServer.scala:340)
> at java.lang.Thread.run(Thread.java:745)
> [2016-01-14 09:42:46,451] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
> [2016-01-14 09:42:46,452] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
> ####
>
> Even thought the Kafka-Manager seems to work as expected (see picture).
> The ZookeeperUI also works as (I would) expect it to work. I used the stack of Kafka&Zookeeper plus all the Manager/Monitors and UI for other purposes, therefore I like to think they are fine. :)
>
> If we can figure the stack out with hello-szama it would be my pleasure to write this up nicely and create a blog post, add it to the samza documentation and showcase samza as a BigData project during my FOSDEM talk (https://fosdem.org/2016/schedule/track/hpc,_big_data_and_data_science/ <https://fosdem.org/2016/schedule/track/hpc,_big_data_and_data_science/>).
>
> Cheers
> Christian
>
>
> <Screen Shot 2016-01-14 at 09.45.48.png>
>
> --
>
> <Gaikai_Logo.png>
>
>
> Christian Kniep | Release Engineer
>
> www.gaikai.com <http://www.gaikai.com/>
>> On 14 Jan 2016, at 06:42, Yi Pan <nickpan47@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hi, Christian,
>>
>> This is interesting area (i.e. creating docker image for hello-samza). I did not see the container log, which I assume that would include the error message? How did you hello-samza job fail?
>>
>> -Yi
>>
>> On Tue, Jan 12, 2016 at 8:56 AM, Christian Kniep <ckniep@gaikai.com <ma...@gaikai.com>> wrote:
>> Hey there,
>>
>> I try to replay the hello-samza stuff with my docker image, but failed. :(
>> Could someone provide me a hint what I do wrong?
>>
>> I have to admit, that I am not the biggest Java developer, therefore I might do stupid mistakes… :)
>>
>> My Steps: https://github.com/ChristianKniep/orchestra/tree/master/samza <https://github.com/ChristianKniep/orchestra/tree/master/samza>
>>
>> Cheers
>> Christian
>>
>> --
>>
>> <Gaikai_Logo.png>
>>
>>
>> Christian Kniep | Release Engineer
>>
>> www.gaikai.com <http://www.gaikai.com/>
Re: Docker-Image for 'hello-samza'
Posted by Christian Kniep <ck...@gaikai.com>.
Hey Yi,
as you can see within the github repo I explore software stacks by tackle them with docker. Quite fun, but challenging from times to times.
Especially if it’s handled by layers of Java - not per se a problem, but my Java knowledge lacks experience.
But to the topic… I checked the logs of ZK and Kafka and Kafka throws:
####
[2016-01-14 09:42:28,748] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
[2016-01-14 09:42:28,807] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
[2016-01-14 09:42:29,993] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
[2016-01-14 09:42:30,010] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
[2016-01-14 09:42:30,010] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
[2016-01-14 09:42:36,183] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
[2016-01-14 09:42:36,270] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
[2016-01-14 09:42:40,591] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
[2016-01-14 09:42:40,592] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
[2016-01-14 09:42:46,451] ERROR Closing socket for /172.17.0.7 because of error (kafka.network.Processor)
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
at kafka.utils.Utils$.read(Utils.scala:380)
at kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
at kafka.network.Processor.read(SocketServer.scala:444)
at kafka.network.Processor.run(SocketServer.scala:340)
at java.lang.Thread.run(Thread.java:745)
[2016-01-14 09:42:46,451] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
[2016-01-14 09:42:46,452] INFO Closing socket connection to /172.17.0.7. (kafka.network.Processor)
####
Even thought the Kafka-Manager seems to work as expected (see picture).
The ZookeeperUI also works as (I would) expect it to work. I used the stack of Kafka&Zookeeper plus all the Manager/Monitors and UI for other purposes, therefore I like to think they are fine. :)
If we can figure the stack out with hello-szama it would be my pleasure to write this up nicely and create a blog post, add it to the samza documentation and showcase samza as a BigData project during my FOSDEM talk (https://fosdem.org/2016/schedule/track/hpc,_big_data_and_data_science/).
Cheers
Christian
--
Christian Kniep | Release Engineer
www.gaikai.com <http://www.gaikai.com/>
> On 14 Jan 2016, at 06:42, Yi Pan <ni...@gmail.com> wrote:
>
> Hi, Christian,
>
> This is interesting area (i.e. creating docker image for hello-samza). I did not see the container log, which I assume that would include the error message? How did you hello-samza job fail?
>
> -Yi
>
> On Tue, Jan 12, 2016 at 8:56 AM, Christian Kniep <ckniep@gaikai.com <ma...@gaikai.com>> wrote:
> Hey there,
>
> I try to replay the hello-samza stuff with my docker image, but failed. :(
> Could someone provide me a hint what I do wrong?
>
> I have to admit, that I am not the biggest Java developer, therefore I might do stupid mistakes… :)
>
> My Steps: https://github.com/ChristianKniep/orchestra/tree/master/samza <https://github.com/ChristianKniep/orchestra/tree/master/samza>
>
> Cheers
> Christian
>
> --
>
>
>
>
> Christian Kniep | Release Engineer
>
> www.gaikai.com <http://www.gaikai.com/>
>
Re: Docker-Image for 'hello-samza'
Posted by Yi Pan <ni...@gmail.com>.
Hi, Christian,
This is interesting area (i.e. creating docker image for hello-samza). I
did not see the container log, which I assume that would include the error
message? How did you hello-samza job fail?
-Yi
On Tue, Jan 12, 2016 at 8:56 AM, Christian Kniep <ck...@gaikai.com> wrote:
> Hey there,
>
> I try to replay the hello-samza stuff with my docker image, but failed. :(
> Could someone provide me a hint what I do wrong?
>
> I have to admit, that I am not the biggest Java developer, therefore I
> might do stupid mistakes… :)
>
> My Steps: https://github.com/ChristianKniep/orchestra/tree/master/samza
>
> Cheers
> Christian
>
> --
>
>
>
> Christian Kniep | Release Engineer
>
> www.gaikai.com
>
>