"Block input-* already exists on this machine; not re-adding it" warnings

[Aniket Bhatnagar <an...@gmail.com>]

Hi everyone

I back ported kinesis-asl to spark 1.0.2 and ran a quick test on my local
machine. It seems to be working fine but I keep getting the following
warnings. I am not sure what it means and weather it is something to worry
about or not.

2014-08-22 15:53:43,803 [pool-1-thread-7] WARN
 o.apache.spark.storage.BlockManager - Block input-0-1408703023600 already
exists on this machine; not re-adding it

Thoughts?

Thanks,
Aniket

Re: "Block input-* already exists on this machine; not re-adding it" warnings

[Aniket Bhatnagar <an...@gmail.com>]

Answering my own question, it seems that the warnings are expected as
explained by TD @
http://apache-spark-user-list.1001560.n3.nabble.com/streaming-questions-td3281.html
.

Here is what he wrote:
"Spark Streaming is designed to replicate the received data within the
machines in a Spark cluster for fault-tolerance. However, when you are
running in the local mode, since there is only one machine, the
"blocks" of data arent able to replicate. This is expected and safe to
ignore in local mode."

I was indeed running it in local mode and hence it seems that I can safely
ignore such warnings.

Thanks,
Aniket


On 22 August 2014 15:54, Aniket Bhatnagar <an...@gmail.com>
wrote:

> Hi everyone
>
> I back ported kinesis-asl to spark 1.0.2 and ran a quick test on my local
> machine. It seems to be working fine but I keep getting the following
> warnings. I am not sure what it means and weather it is something to worry
> about or not.
>
> 2014-08-22 15:53:43,803 [pool-1-thread-7] WARN
>  o.apache.spark.storage.BlockManager - Block input-0-1408703023600 already
> exists on this machine; not re-adding it
>
> Thoughts?
>
> Thanks,
> Aniket
>