[jira] [Commented] (PIG-5197) Replace IndexedKey with PigNullableWritable in spark branch

["liyunzhang_intel (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-5197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949885#comment-15949885 ] 

liyunzhang_intel commented on PIG-5197:
---------------------------------------

[~rohini]:  I tried to replace IndexedKey wih PigBNullableWritable but it failed because PigNullableWritable is not serializable. So i will remain IndexedKey in spark package.  Can you give me some suggestion?


exception info
{code}
Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0.0 in stage 32.0 mpl.io.NullableTuple
Serialization stack:
	- object not serializable (class: org.apache.pig.impl.io.NullableTuple, value: Null: false in
[

{code}



> Replace IndexedKey with PigNullableWritable in spark branch
> -----------------------------------------------------------
>
>                 Key: PIG-5197
>                 URL: https://issues.apache.org/jira/browse/PIG-5197
>             Project: Pig
>          Issue Type: Sub-task
>          Components: spark
>            Reporter: liyunzhang_intel
>             Fix For: spark-branch
>
>
> The function of IndexedKey and PigNullableWritable is similar. 
> The difference between these two is  IndexedKey contains Index,key while PigNullableWritable contains index,key,value.
> Besides,the comparators for PigNullableWritable have lot of conditions for the different data types taken care of and IndexedKey can miss some of that. We can try to replace IndexedKey with PigNullableWritable.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)