You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by Kirill Bogdanov <ki...@gmail.com> on 2020/07/24 16:11:48 UTC
Question about "streaming receiver start fail"
Hi,
Apologies if this is a wrong thread to post to, but I have a Kylin question
that I couldn't resolve myself yet.
I am deploying Kylin on AWS EMR by following the official tutorial
http://kylin.apache.org/docs/install/kylin_aws_emr.html
I am interested in running Near RT Streaming using Kafka. My Kafka is
deployed on AWS MSK. I managed to deploy Kylin, connect it with Kafka
stream and configure model and the cube, however, I have a problem enabling
the cube to run OLAP.
When I execute: ./bin/kylin.sh streaming start I am getting this error
message in the streaming logs:
2020-07-24T15:58:12,763 ERROR [main]
org.apache.kylin.stream.server.StreamingReceiver - streaming receiver start
fail
java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:9090
at
org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:346)
~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
at
org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:307)
~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
at
org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
at
org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:231)
~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
at org.eclipse.jetty.server.Server.doStart(Server.java:385)
~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
at
org.apache.kylin.stream.server.StreamingReceiver.startHttpServer(StreamingReceiver.java:94)
~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
at
org.apache.kylin.stream.server.StreamingReceiver.start(StreamingReceiver.java:63)
~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
at
org.apache.kylin.stream.server.StreamingReceiver.main(StreamingReceiver.java:51)
[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_252]
at sun.nio.ch.Net.bind(Net.java:433) ~[?:1.8.0_252]
at sun.nio.ch.Net.bind(Net.java:425) ~[?:1.8.0_252]
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:220)
~[?:1.8.0_252]
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
~[?:1.8.0_252]
at
org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342)
~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
... 9 more
This is strange because there is no receiver running and no process is
using port 9090. What else should I check? Did I miss some configuration?
Thanks & best regards,
Kirill
Re: Question about "streaming receiver start fail"
Posted by Xiaoxiang Yu <xx...@apache.org>.
Hi,
Thanks for the update, I am glad to hear your good news.
By the way, the question you asked, "why the default port (9090) is the same as the port used by ThriftServer", I didn't think it is a seroius issue. I guess head node is suitable for deploment of ThriftServer, and worker node is suitable for deployment of Streaming Receiver. These two instances have little chance to be located at same node.
--
Best wishes to you !
From ：Xiaoxiang Yu
At 2020-07-27 23:53:24, "Kirill Bogdanov" <ki...@gmail.com> wrote:
>Hi Xiaoxiang,
>
>Just to give an update, I was able to launch stream consumer after changing
>the port in kylin.stream.node and the rest of functionality worked great.
>However, I still don't understand why the default port (9090) is the same
>as the port used by ThriftServer.
>
>Thanks & Best Regards,
>Kirill
>
>On Mon, 27 Jul 2020 at 11:21, Kirill Bogdanov <ki...@gmail.com> wrote:
>
>> Hi Xiaoxiang,
>>
>> Thanks for your reply. You are right, I did not correctly identified that
>> ThriftServer is already using port 9090, thank you for your commands.
>> However, could you clarify how I can start Kylin streaming now? Should I
>> change streaming port or should I change ThriftServer port from 9090 to
>> something else? What is the correct configuration in this case?
>>
>> [hadoop@ip-172-31-0-218 ~]$ sudo netstat -anp | grep LISTEN | grep 9090
>> tcp 0 0 :::9090 :::*
>> LISTEN 4013/java
>> [hadoop@ip-172-31-0-218 ~]$ sudo jps -mlv | grep 4013
>> 4013 org.apache.hadoop.hbase.thrift.ThriftServer start -Dproc_thrift
>> -XX:OnOutOfMemoryError=kill -9 %p -Xmx1024m -XX:+UseConcMarkSweepGC
>> -XX:+UseParNewGC -XX:CMSInitiatingOccupancyFraction=70
>> -Dnetworkaddress.cache.ttl=5 -Dhbase.log.dir=/var/log/hbase
>> -Dhbase.log.file=hbase-hbase-thrift-ip-172-31-0-218.log
>> -Dhbase.home.dir=/usr/lib/hbase -Dhbase.id.str=hbase
>> -Dhbase.root.logger=INFO,DRFA
>> -Djava.library.path=:/usr/lib/hadoop-lzo/lib/native:/usr/lib/hadoop/lib/native
>> -Dhbase.security.logger=INFO,DRFAS
>>
>> I will write my next questions to the user mailing list.
>>
>> Best regards,
>> Kirill
>>
>> On Mon, 27 Jul 2020 at 03:28, Xiaoxiang Yu <xx...@apache.org> wrote:
>>
>>> Did you confirm that 9090 is not occpied?
>>>
>>> For example, using following command:
>>> ```
>>> [root@cdh-master ~]# netstat -anp | grep LISTEN | grep 9090
>>> tcp 0 0 0.0.0.0:9090 0.0.0.0:*
>>> LISTEN 32274/java
>>> [root@cdh-master ~]#
>>> [root@cdh-master ~]# jps -mlv | grep 32274
>>> 32274 org.apache.hadoop.hbase.thrift.ThriftServer start --port 9090
>>> -threadpool --bind 0.0.0.0 -Dproc_thrift -XX:OnOutOfMemoryError=kill -9 %p
>>> -Djava.net.preferIPv4Stack=true -Xms1073741824 -Xmx1073741824
>>> -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
>>> -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled
>>> -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh
>>> -Dhbase.log.dir=/var/log/hbase
>>> -Dhbase.log.file=hbase-cmf-hbase-HBASETHRIFTSERVER-cdh-master.log.out
>>> -Dhbase.home.dir=/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hbase
>>> -Dhbase.id.str= -Dhbase.root.logger=INFO,RFA
>>> -Djava.library.path=/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hadoop/lib/native:/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hbase/lib/native/Linux-amd64-64
>>> -Dhbase.security.logger=INFO,NullAppender
>>> ```
>>>
>>> Besides, I guess using user mailing maybe more suitable.
>>>
>>>
>>>
>>>
>>>
>>> --
>>> *Best wishes to you ! *
>>> *From ：**Xiaoxiang Yu*
>>>
>>>
>>>
>>> At 2020-07-25 00:11:48, "Kirill Bogdanov" <ki...@gmail.com> wrote:
>>> >Hi,
>>> >
>>> >Apologies if this is a wrong thread to post to, but I have a Kylin question
>>> >that I couldn't resolve myself yet.
>>> >
>>> >I am deploying Kylin on AWS EMR by following the official tutorial
>>> >http://kylin.apache.org/docs/install/kylin_aws_emr.html
>>> >I am interested in running Near RT Streaming using Kafka. My Kafka is
>>> >deployed on AWS MSK. I managed to deploy Kylin, connect it with Kafka
>>> >stream and configure model and the cube, however, I have a problem enabling
>>> >the cube to run OLAP.
>>> >
>>> >When I execute: ./bin/kylin.sh streaming start I am getting this error
>>> >message in the streaming logs:
>>> >
>>> >2020-07-24T15:58:12,763 ERROR [main]
>>> >org.apache.kylin.stream.server.StreamingReceiver - streaming receiver start
>>> >fail
>>> >java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:9090
>>> > at
>>> >org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:346)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at
>>> >org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:307)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at
>>> >org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at
>>> >org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:231)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at
>>> >org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at org.eclipse.jetty.server.Server.doStart(Server.java:385)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at
>>> >org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at
>>> >org.apache.kylin.stream.server.StreamingReceiver.startHttpServer(StreamingReceiver.java:94)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at
>>> >org.apache.kylin.stream.server.StreamingReceiver.start(StreamingReceiver.java:63)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > at
>>> >org.apache.kylin.stream.server.StreamingReceiver.main(StreamingReceiver.java:51)
>>> >[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> >Caused by: java.net.BindException: Address already in use
>>> > at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_252]
>>> > at sun.nio.ch.Net.bind(Net.java:433) ~[?:1.8.0_252]
>>> > at sun.nio.ch.Net.bind(Net.java:425) ~[?:1.8.0_252]
>>> > at
>>> >sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:220)
>>> >~[?:1.8.0_252]
>>> > at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
>>> >~[?:1.8.0_252]
>>> > at
>>> >org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342)
>>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>>> > ... 9 more
>>> >
>>> >This is strange because there is no receiver running and no process is
>>> >using port 9090. What else should I check? Did I miss some configuration?
>>> >
>>> >Thanks & best regards,
>>> >Kirill
>>>
>>>
Re: Question about "streaming receiver start fail"
Posted by Kirill Bogdanov <ki...@gmail.com>.
Hi Xiaoxiang,
Just to give an update, I was able to launch stream consumer after changing
the port in kylin.stream.node and the rest of functionality worked great.
However, I still don't understand why the default port (9090) is the same
as the port used by ThriftServer.
Thanks & Best Regards,
Kirill
On Mon, 27 Jul 2020 at 11:21, Kirill Bogdanov <ki...@gmail.com> wrote:
> Hi Xiaoxiang,
>
> Thanks for your reply. You are right, I did not correctly identified that
> ThriftServer is already using port 9090, thank you for your commands.
> However, could you clarify how I can start Kylin streaming now? Should I
> change streaming port or should I change ThriftServer port from 9090 to
> something else? What is the correct configuration in this case?
>
> [hadoop@ip-172-31-0-218 ~]$ sudo netstat -anp | grep LISTEN | grep 9090
> tcp 0 0 :::9090 :::*
> LISTEN 4013/java
> [hadoop@ip-172-31-0-218 ~]$ sudo jps -mlv | grep 4013
> 4013 org.apache.hadoop.hbase.thrift.ThriftServer start -Dproc_thrift
> -XX:OnOutOfMemoryError=kill -9 %p -Xmx1024m -XX:+UseConcMarkSweepGC
> -XX:+UseParNewGC -XX:CMSInitiatingOccupancyFraction=70
> -Dnetworkaddress.cache.ttl=5 -Dhbase.log.dir=/var/log/hbase
> -Dhbase.log.file=hbase-hbase-thrift-ip-172-31-0-218.log
> -Dhbase.home.dir=/usr/lib/hbase -Dhbase.id.str=hbase
> -Dhbase.root.logger=INFO,DRFA
> -Djava.library.path=:/usr/lib/hadoop-lzo/lib/native:/usr/lib/hadoop/lib/native
> -Dhbase.security.logger=INFO,DRFAS
>
> I will write my next questions to the user mailing list.
>
> Best regards,
> Kirill
>
> On Mon, 27 Jul 2020 at 03:28, Xiaoxiang Yu <xx...@apache.org> wrote:
>
>> Did you confirm that 9090 is not occpied?
>>
>> For example, using following command:
>> ```
>> [root@cdh-master ~]# netstat -anp | grep LISTEN | grep 9090
>> tcp 0 0 0.0.0.0:9090 0.0.0.0:*
>> LISTEN 32274/java
>> [root@cdh-master ~]#
>> [root@cdh-master ~]# jps -mlv | grep 32274
>> 32274 org.apache.hadoop.hbase.thrift.ThriftServer start --port 9090
>> -threadpool --bind 0.0.0.0 -Dproc_thrift -XX:OnOutOfMemoryError=kill -9 %p
>> -Djava.net.preferIPv4Stack=true -Xms1073741824 -Xmx1073741824
>> -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
>> -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled
>> -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh
>> -Dhbase.log.dir=/var/log/hbase
>> -Dhbase.log.file=hbase-cmf-hbase-HBASETHRIFTSERVER-cdh-master.log.out
>> -Dhbase.home.dir=/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hbase
>> -Dhbase.id.str= -Dhbase.root.logger=INFO,RFA
>> -Djava.library.path=/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hadoop/lib/native:/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hbase/lib/native/Linux-amd64-64
>> -Dhbase.security.logger=INFO,NullAppender
>> ```
>>
>> Besides, I guess using user mailing maybe more suitable.
>>
>>
>>
>>
>>
>> --
>> *Best wishes to you ! *
>> *From ：**Xiaoxiang Yu*
>>
>>
>>
>> At 2020-07-25 00:11:48, "Kirill Bogdanov" <ki...@gmail.com> wrote:
>> >Hi,
>> >
>> >Apologies if this is a wrong thread to post to, but I have a Kylin question
>> >that I couldn't resolve myself yet.
>> >
>> >I am deploying Kylin on AWS EMR by following the official tutorial
>> >http://kylin.apache.org/docs/install/kylin_aws_emr.html
>> >I am interested in running Near RT Streaming using Kafka. My Kafka is
>> >deployed on AWS MSK. I managed to deploy Kylin, connect it with Kafka
>> >stream and configure model and the cube, however, I have a problem enabling
>> >the cube to run OLAP.
>> >
>> >When I execute: ./bin/kylin.sh streaming start I am getting this error
>> >message in the streaming logs:
>> >
>> >2020-07-24T15:58:12,763 ERROR [main]
>> >org.apache.kylin.stream.server.StreamingReceiver - streaming receiver start
>> >fail
>> >java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:9090
>> > at
>> >org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:346)
>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>> > at
>> >org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:307)
>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>> > at
>> >org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>> > at
>> >org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:231)
>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>> > at
>> >org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>> > at org.eclipse.jetty.server.Server.doStart(Server.java:385)
>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>> > at
>> >org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>> > at
>> >org.apache.kylin.stream.server.StreamingReceiver.startHttpServer(StreamingReceiver.java:94)
>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>> > at
>> >org.apache.kylin.stream.server.StreamingReceiver.start(StreamingReceiver.java:63)
>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>> > at
>> >org.apache.kylin.stream.server.StreamingReceiver.main(StreamingReceiver.java:51)
>> >[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>> >Caused by: java.net.BindException: Address already in use
>> > at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_252]
>> > at sun.nio.ch.Net.bind(Net.java:433) ~[?:1.8.0_252]
>> > at sun.nio.ch.Net.bind(Net.java:425) ~[?:1.8.0_252]
>> > at
>> >sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:220)
>> >~[?:1.8.0_252]
>> > at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
>> >~[?:1.8.0_252]
>> > at
>> >org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342)
>> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>> > ... 9 more
>> >
>> >This is strange because there is no receiver running and no process is
>> >using port 9090. What else should I check? Did I miss some configuration?
>> >
>> >Thanks & best regards,
>> >Kirill
>>
>>
Re: Question about "streaming receiver start fail"
Posted by Kirill Bogdanov <ki...@gmail.com>.
Hi Xiaoxiang,
Thanks for your reply. You are right, I did not correctly identified that
ThriftServer is already using port 9090, thank you for your commands.
However, could you clarify how I can start Kylin streaming now? Should I
change streaming port or should I change ThriftServer port from 9090 to
something else? What is the correct configuration in this case?
[hadoop@ip-172-31-0-218 ~]$ sudo netstat -anp | grep LISTEN | grep 9090
tcp 0 0 :::9090 :::*
LISTEN 4013/java
[hadoop@ip-172-31-0-218 ~]$ sudo jps -mlv | grep 4013
4013 org.apache.hadoop.hbase.thrift.ThriftServer start -Dproc_thrift
-XX:OnOutOfMemoryError=kill -9 %p -Xmx1024m -XX:+UseConcMarkSweepGC
-XX:+UseParNewGC -XX:CMSInitiatingOccupancyFraction=70
-Dnetworkaddress.cache.ttl=5 -Dhbase.log.dir=/var/log/hbase
-Dhbase.log.file=hbase-hbase-thrift-ip-172-31-0-218.log
-Dhbase.home.dir=/usr/lib/hbase -Dhbase.id.str=hbase
-Dhbase.root.logger=INFO,DRFA
-Djava.library.path=:/usr/lib/hadoop-lzo/lib/native:/usr/lib/hadoop/lib/native
-Dhbase.security.logger=INFO,DRFAS
I will write my next questions to the user mailing list.
Best regards,
Kirill
On Mon, 27 Jul 2020 at 03:28, Xiaoxiang Yu <xx...@apache.org> wrote:
> Did you confirm that 9090 is not occpied?
>
> For example, using following command:
> ```
> [root@cdh-master ~]# netstat -anp | grep LISTEN | grep 9090
> tcp 0 0 0.0.0.0:9090 0.0.0.0:*
> LISTEN 32274/java
> [root@cdh-master ~]#
> [root@cdh-master ~]# jps -mlv | grep 32274
> 32274 org.apache.hadoop.hbase.thrift.ThriftServer start --port 9090
> -threadpool --bind 0.0.0.0 -Dproc_thrift -XX:OnOutOfMemoryError=kill -9 %p
> -Djava.net.preferIPv4Stack=true -Xms1073741824 -Xmx1073741824
> -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
> -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled
> -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh
> -Dhbase.log.dir=/var/log/hbase
> -Dhbase.log.file=hbase-cmf-hbase-HBASETHRIFTSERVER-cdh-master.log.out
> -Dhbase.home.dir=/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hbase
> -Dhbase.id.str= -Dhbase.root.logger=INFO,RFA
> -Djava.library.path=/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hadoop/lib/native:/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hbase/lib/native/Linux-amd64-64
> -Dhbase.security.logger=INFO,NullAppender
> ```
>
> Besides, I guess using user mailing maybe more suitable.
>
>
>
>
>
> --
> *Best wishes to you ! *
> *From ：**Xiaoxiang Yu*
>
>
>
> At 2020-07-25 00:11:48, "Kirill Bogdanov" <ki...@gmail.com> wrote:
> >Hi,
> >
> >Apologies if this is a wrong thread to post to, but I have a Kylin question
> >that I couldn't resolve myself yet.
> >
> >I am deploying Kylin on AWS EMR by following the official tutorial
> >http://kylin.apache.org/docs/install/kylin_aws_emr.html
> >I am interested in running Near RT Streaming using Kafka. My Kafka is
> >deployed on AWS MSK. I managed to deploy Kylin, connect it with Kafka
> >stream and configure model and the cube, however, I have a problem enabling
> >the cube to run OLAP.
> >
> >When I execute: ./bin/kylin.sh streaming start I am getting this error
> >message in the streaming logs:
> >
> >2020-07-24T15:58:12,763 ERROR [main]
> >org.apache.kylin.stream.server.StreamingReceiver - streaming receiver start
> >fail
> >java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:9090
> > at
> >org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:346)
> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> > at
> >org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:307)
> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> > at
> >org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> > at
> >org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:231)
> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> > at
> >org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> > at org.eclipse.jetty.server.Server.doStart(Server.java:385)
> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> > at
> >org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> > at
> >org.apache.kylin.stream.server.StreamingReceiver.startHttpServer(StreamingReceiver.java:94)
> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> > at
> >org.apache.kylin.stream.server.StreamingReceiver.start(StreamingReceiver.java:63)
> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> > at
> >org.apache.kylin.stream.server.StreamingReceiver.main(StreamingReceiver.java:51)
> >[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> >Caused by: java.net.BindException: Address already in use
> > at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_252]
> > at sun.nio.ch.Net.bind(Net.java:433) ~[?:1.8.0_252]
> > at sun.nio.ch.Net.bind(Net.java:425) ~[?:1.8.0_252]
> > at
> >sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:220)
> >~[?:1.8.0_252]
> > at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
> >~[?:1.8.0_252]
> > at
> >org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342)
> >~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> > ... 9 more
> >
> >This is strange because there is no receiver running and no process is
> >using port 9090. What else should I check? Did I miss some configuration?
> >
> >Thanks & best regards,
> >Kirill
>
>
Re:Question about "streaming receiver start fail"
Posted by Xiaoxiang Yu <xx...@apache.org>.
Did you confirm that 9090 is not occpied?
For example, using following command:
```
[root@cdh-master ~]# netstat -anp | grep LISTEN | grep 9090
tcp 0 0 0.0.0.0:9090 0.0.0.0:* LISTEN 32274/java
[root@cdh-master ~]#
[root@cdh-master ~]# jps -mlv | grep 32274
32274 org.apache.hadoop.hbase.thrift.ThriftServer start --port 9090 -threadpool --bind 0.0.0.0 -Dproc_thrift -XX:OnOutOfMemoryError=kill -9 %p -Djava.net.preferIPv4Stack=true -Xms1073741824 -Xmx1073741824 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dhbase.log.dir=/var/log/hbase -Dhbase.log.file=hbase-cmf-hbase-HBASETHRIFTSERVER-cdh-master.log.out -Dhbase.home.dir=/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hbase -Dhbase.id.str= -Dhbase.root.logger=INFO,RFA -Djava.library.path=/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hadoop/lib/native:/opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hbase/lib/native/Linux-amd64-64 -Dhbase.security.logger=INFO,NullAppender
```
Besides, I guess using user mailing maybe more suitable.
--
Best wishes to you !
From ：Xiaoxiang Yu
At 2020-07-25 00:11:48, "Kirill Bogdanov" <ki...@gmail.com> wrote:
>Hi,
>
>Apologies if this is a wrong thread to post to, but I have a Kylin question
>that I couldn't resolve myself yet.
>
>I am deploying Kylin on AWS EMR by following the official tutorial
>http://kylin.apache.org/docs/install/kylin_aws_emr.html
>I am interested in running Near RT Streaming using Kafka. My Kafka is
>deployed on AWS MSK. I managed to deploy Kylin, connect it with Kafka
>stream and configure model and the cube, however, I have a problem enabling
>the cube to run OLAP.
>
>When I execute: ./bin/kylin.sh streaming start I am getting this error
>message in the streaming logs:
>
>2020-07-24T15:58:12,763 ERROR [main]
>org.apache.kylin.stream.server.StreamingReceiver - streaming receiver start
>fail
>java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:9090
> at
>org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:346)
>~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> at
>org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:307)
>~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> at
>org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
>~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> at
>org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:231)
>~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> at
>org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
>~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> at org.eclipse.jetty.server.Server.doStart(Server.java:385)
>~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> at
>org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
>~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> at
>org.apache.kylin.stream.server.StreamingReceiver.startHttpServer(StreamingReceiver.java:94)
>~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> at
>org.apache.kylin.stream.server.StreamingReceiver.start(StreamingReceiver.java:63)
>~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> at
>org.apache.kylin.stream.server.StreamingReceiver.main(StreamingReceiver.java:51)
>[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
>Caused by: java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_252]
> at sun.nio.ch.Net.bind(Net.java:433) ~[?:1.8.0_252]
> at sun.nio.ch.Net.bind(Net.java:425) ~[?:1.8.0_252]
> at
>sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:220)
>~[?:1.8.0_252]
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
>~[?:1.8.0_252]
> at
>org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342)
>~[kylin-stream-receiver-3.1.0-all.jar:3.1.0]
> ... 9 more
>
>This is strange because there is no receiver running and no process is
>using port 9090. What else should I check? Did I miss some configuration?
>
>Thanks & best regards,
>Kirill