You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Sankar Hariappan (JIRA)" <ji...@apache.org> on 2018/03/08 09:39:00 UTC
[jira] [Commented] (HIVE-18751) ACID table scan through get_splits
UDF doesn't receive ValidWriteIdList configuration.
[ https://issues.apache.org/jira/browse/HIVE-18751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390977#comment-16390977 ]
Sankar Hariappan commented on HIVE-18751:
-----------------------------------------
Attached 01.patch with changes to pass on the ValidTxnList and ValidTxnWriteIdList configurations from driver to get_splits UDF.
After this change, TestAcidOnTez.testGetSplitsLocks wouldn't through IOException due to non availability of ValidWriteIdList config. But, the test continue to fail with NPE as it used to fail on the base code before merging HIVE-18192.
Request [~ekoifman] to review the patch!
> ACID table scan through get_splits UDF doesn't receive ValidWriteIdList configuration.
> --------------------------------------------------------------------------------------
>
> Key: HIVE-18751
> URL: https://issues.apache.org/jira/browse/HIVE-18751
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
> Affects Versions: 3.0.0
> Reporter: Sankar Hariappan
> Assignee: Sankar Hariappan
> Priority: Major
> Labels: ACID, UDF
> Fix For: 3.0.0
>
> Attachments: HIVE-18751.01.patch
>
>
> Per table write ID (HIVE-18192) have replaced global transaction ID with write ID to version data files in ACID/MM tables,
> To ensure snapshot isolation, need to generate ValidWriteIdList for the given txn/table and use it when scan the ACID/MM tables.
> In case of get_splits UDF which runs on ACID table scan query won't receive it properly through configuration (hive.txn.tables.valid.writeids) and hence throws exception.
> TestAcidOnTez.testGetSplitsLocks is the test failing for the same. Need to fix it.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)