[jira] [Commented] (PIG-4611) Fix remaining unit test failures about "TestHBaseStorage"

["Mohit Sabharwal (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-4611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611367#comment-14611367 ] 

Mohit Sabharwal commented on PIG-4611:
--------------------------------------

Thanks, [~kellyzly], this looks like a reasonable workaround to the UDFContext issue, where it is not initialized in Spark executor threads.

However, I'm not sure whether it the right thing to do in the case where "pig.hbase.caster" is set by the user.

i.e. For Spark engine, with your workaround, HBaseStorage will always use the default caster (i.e. Utf8StorageConverter). It will never use HBaseBinaryConverter or any other option.



> Fix remaining unit test failures about "TestHBaseStorage"
> ---------------------------------------------------------
>
>                 Key: PIG-4611
>                 URL: https://issues.apache.org/jira/browse/PIG-4611
>             Project: Pig
>          Issue Type: Sub-task
>          Components: spark
>            Reporter: liyunzhang_intel
>            Assignee: liyunzhang_intel
>             Fix For: spark-branch
>
>         Attachments: PIG-4611.patch
>
>
> In https://builds.apache.org/job/Pig-spark/lastCompletedBuild/testReport/, it shows following unit test failures about TestHBaseStorage:
>  org.apache.pig.test.TestHBaseStorage.testStoreToHBase_1_with_delete	
>  org.apache.pig.test.TestHBaseStorage.testLoadWithProjection_1
>  org.apache.pig.test.TestHBaseStorage.testLoadWithProjection_2	
>  org.apache.pig.test.TestHBaseStorage.testStoreToHBase_2_with_projection
>  org.apache.pig.test.TestHBaseStorage.testCollectedGroup	
>  org.apache.pig.test.TestHBaseStorage.testHeterogeneousScans



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)