You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by WangJianfei <wa...@otcaix.iscas.ac.cn> on 2016/09/16 01:15:25 UTC
Why we get 0 when the key is null?
this func is in Partitioner
def getPartition(key: Any): Int = key match {
case null => 0
// case None => 0
case _ => Utils.nonNegativeMod(key.hashCode, numPartitions)
}
--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Why-we-get-0-when-the-key-is-null-tp18952.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Why we get 0 when the key is null?
Posted by Sean Owen <so...@cloudera.com>.
"null" is a valid value in an RDD, so it has to be partition-able.
On Fri, Sep 16, 2016 at 2:26 AM, WangJianfei
<wa...@otcaix.iscas.ac.cn> wrote:
> When the key is not In the rdd, I can also get an value , I just feel a
> little strange.
>
>
>
> --
> View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Why-we-get-0-when-the-key-is-null-tp18952p18955.html
> Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Why we get 0 when the key is null?
Posted by WangJianfei <wa...@otcaix.iscas.ac.cn>.
When the key is not In the rdd, I can also get an value , I just feel a
little strange.
--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Why-we-get-0-when-the-key-is-null-tp18952p18955.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Why we get 0 when the key is null?
Posted by Reynold Xin <rx...@databricks.com>.
What else do you expect to get? A non-zero hash value?
It can technically be any constant.
On Thu, Sep 15, 2016 at 6:15 PM, WangJianfei <
wangjianfei15@otcaix.iscas.ac.cn> wrote:
> this func is in Partitioner
> def getPartition(key: Any): Int = key match {
> case null => 0
> // case None => 0
> case _ => Utils.nonNegativeMod(key.hashCode, numPartitions)
> }
>
>
>
> --
> View this message in context: http://apache-spark-
> developers-list.1001551.n3.nabble.com/Why-we-get-0-when-
> the-key-is-null-tp18952.html
> Sent from the Apache Spark Developers List mailing list archive at
> Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>