You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Rohit Verma <ro...@rokittech.com> on 2016/11/14 11:21:00 UTC

Spark hash function

Hi All,

One of the miscellaneous functions in spark sql is hash
    expression[Murmur3Hash]("hash"),

I was wondering whether its  which variant of murmurhas3

murmurhash3_x64_128
murmurhash3_x86_32 ( this is also part of spark unsafe package). 

Also what is seed for the hash function.

I am doing a query as as 

dataset = spark.sql(“select hash(name) from employee”);

what would be correct way to check equableness with this like 

dataset.first.equals(#MyHashFunction(val)) is true.

Thanks all.

Regards
Rohit