You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by rapelly kartheek <ka...@gmail.com> on 2014/11/10 07:00:44 UTC
Rdd replication
Hi,
I am trying to understand rdd replication code. In the process, I
frequently execute one spark application whenever I make a change to the
code to see effect.
My problem is, after a set of repeated executions of the same application,
I find that my cluster behaves unusually.
Ideally, when I replicate an rdd twice, the webUI displays each partition
twice in the RDD storage info tab. But, sometimes I find that it displays
each partition only once. Also, when it is replicated only once, each
partition gets displayed twice. This happens frequently.
Can someone throw some light in this regard.