You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Vinti Maheshwari <vi...@gmail.com> on 2016/02/22 15:38:22 UTC
Spark Streaming not reading input coming from the other ip
Hi
I am in spark Streaming context, and i am reading input from the the socket
using nc -lk 9999. When i am running it and manually giving input it's
working. But, if input is coming from different ip to this socket then
spark is not reading that input, though it's showing all the input coming
from different source under nc -lk 9999. But somehow spark is not reading
it.
I am not sure what can be issue. Does anyone has idea about it? Thanks in
advance.
Thanks
~Vinti
Re: Spark Streaming not reading input coming from the other ip
Posted by Vinti Maheshwari <vi...@gmail.com>.
Thanks Shixiong, I am not getting any error and telnet is also working fine.
<remote-machine>$ telnet <host-name-where-spark-running> 9999
Trying 192.168.186.97...
Connected to ttsv-
On Mon, Feb 22, 2016 at 1:10 PM, Shixiong(Ryan) Zhu <shixiong@databricks.com
> wrote:
> What's the error info reported by Streaming? And could you use "telnet" to
> test if the network is normal?
>
> On Mon, Feb 22, 2016 at 6:59 AM, Vinti Maheshwari <vi...@gmail.com>
> wrote:
>
>> For reference, my program:
>>
>> def main(args: Array[String]): Unit = {
>> val conf = new SparkConf().setAppName("HBaseStream")
>> val sc = new SparkContext(conf)
>> // create a StreamingContext, the main entry point for all streaming functionality
>> val ssc = new StreamingContext(sc, Seconds(2))
>> val inputStream = ssc.socketTextStream(<hostname>, 9999)
>> .......
>> }
>>
>> Data coming form different host to this socket, But somehow spark is not
>> reading it, while if i copy and paste same data, it works.
>>
>> [root@ ~]# nc -lk 9999
>> 56b4b2b23c24c3608376d1f0,/obj/i386/junos/lib/librtsock/rtslib_gencfg.So.gcda,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
>> 56b4b2b23c24c3608376d1f0,/obj/i386/junos/lib/librtsock/rtslib_idl.So.gcda,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
>>
>> On Mon, Feb 22, 2016 at 6:38 AM, Vinti Maheshwari <vi...@gmail.com>
>> wrote:
>>
>>> Hi
>>>
>>> I am in spark Streaming context, and i am reading input from the the
>>> socket using nc -lk 9999. When i am running it and manually giving input
>>> it's working. But, if input is coming from different ip to this socket then
>>> spark is not reading that input, though it's showing all the input coming
>>> from different source under nc -lk 9999. But somehow spark is not reading
>>> it.
>>>
>>> I am not sure what can be issue. Does anyone has idea about it? Thanks
>>> in advance.
>>>
>>>
>>> Thanks
>>>
>>> ~Vinti
>>>
>>
>>
>
Re: Spark Streaming not reading input coming from the other ip
Posted by "Shixiong(Ryan) Zhu" <sh...@databricks.com>.
What's the error info reported by Streaming? And could you use "telnet" to
test if the network is normal?
On Mon, Feb 22, 2016 at 6:59 AM, Vinti Maheshwari <vi...@gmail.com>
wrote:
> For reference, my program:
>
> def main(args: Array[String]): Unit = {
> val conf = new SparkConf().setAppName("HBaseStream")
> val sc = new SparkContext(conf)
> // create a StreamingContext, the main entry point for all streaming functionality
> val ssc = new StreamingContext(sc, Seconds(2))
> val inputStream = ssc.socketTextStream(<hostname>, 9999)
> .......
> }
>
> Data coming form different host to this socket, But somehow spark is not
> reading it, while if i copy and paste same data, it works.
>
> [root@ ~]# nc -lk 9999
> 56b4b2b23c24c3608376d1f0,/obj/i386/junos/lib/librtsock/rtslib_gencfg.So.gcda,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
> 56b4b2b23c24c3608376d1f0,/obj/i386/junos/lib/librtsock/rtslib_idl.So.gcda,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
>
> On Mon, Feb 22, 2016 at 6:38 AM, Vinti Maheshwari <vi...@gmail.com>
> wrote:
>
>> Hi
>>
>> I am in spark Streaming context, and i am reading input from the the
>> socket using nc -lk 9999. When i am running it and manually giving input
>> it's working. But, if input is coming from different ip to this socket then
>> spark is not reading that input, though it's showing all the input coming
>> from different source under nc -lk 9999. But somehow spark is not reading
>> it.
>>
>> I am not sure what can be issue. Does anyone has idea about it? Thanks in
>> advance.
>>
>>
>> Thanks
>>
>> ~Vinti
>>
>
>
Re: Spark Streaming not reading input coming from the other ip
Posted by Vinti Maheshwari <vi...@gmail.com>.
For reference, my program:
def main(args: Array[String]): Unit = {
val conf = new SparkConf().setAppName("HBaseStream")
val sc = new SparkContext(conf)
// create a StreamingContext, the main entry point for all
streaming functionality
val ssc = new StreamingContext(sc, Seconds(2))
val inputStream = ssc.socketTextStream(<hostname>, 9999)
.......
}
Data coming form different host to this socket, But somehow spark is not
reading it, while if i copy and paste same data, it works.
[root@ ~]# nc -lk 9999
56b4b2b23c24c3608376d1f0,/obj/i386/junos/lib/librtsock/rtslib_gencfg.So.gcda,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
56b4b2b23c24c3608376d1f0,/obj/i386/junos/lib/librtsock/rtslib_idl.So.gcda,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
On Mon, Feb 22, 2016 at 6:38 AM, Vinti Maheshwari <vi...@gmail.com>
wrote:
> Hi
>
> I am in spark Streaming context, and i am reading input from the the
> socket using nc -lk 9999. When i am running it and manually giving input
> it's working. But, if input is coming from different ip to this socket then
> spark is not reading that input, though it's showing all the input coming
> from different source under nc -lk 9999. But somehow spark is not reading
> it.
>
> I am not sure what can be issue. Does anyone has idea about it? Thanks in
> advance.
>
>
> Thanks
>
> ~Vinti
>