You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Eric Wohlstadter (JIRA)" <ji...@apache.org> on 2018/06/05 22:00:00 UTC
[jira] [Comment Edited] (HIVE-19808) GenericUDTFGetSplits should
support ACID reads in the temp. table read path
[ https://issues.apache.org/jira/browse/HIVE-19808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16502563#comment-16502563 ]
Eric Wohlstadter edited comment on HIVE-19808 at 6/5/18 9:59 PM:
-----------------------------------------------------------------
[~ekoifman] [~jdere]
Yeah.
What happens in GenericUDTFGetSplits is:
{code:java}
"create temporary table " + tableName + " as " + query
{code}
and then the temp table is read into LLAP and exported by the {{LlapOutputFormatService}}.
Currently if {{query}} references an ACID table, then it fails.
The temp table itself is not ACID, the issue occurs when any of the source tables are ACID.
was (Author: ewohlstadter):
[~ekoifman]
Yeah.
What happens in GenericUDTFGetSplits is:
{code:java}
"create temporary table " + tableName + " as " + query
{code}
and then the temp table is read into LLAP and exported by the {{LlapOutputFormatService}}.
Currently if {{query}} references an ACID table, then it fails.
The temp table itself is not ACID, the issue occurs when any of the source tables are ACID.
> GenericUDTFGetSplits should support ACID reads in the temp. table read path
> ---------------------------------------------------------------------------
>
> Key: HIVE-19808
> URL: https://issues.apache.org/jira/browse/HIVE-19808
> Project: Hive
> Issue Type: Bug
> Reporter: Eric Wohlstadter
> Assignee: Eric Wohlstadter
> Priority: Major
>
> 1. Map-only reads work on ACID tables.
> 2. Temp. table reads (for multi-vertex queries) work on non-ACID tables.
> 3. But temp. table reads don't work on ACID tables.
> {code}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to create temp table: java.lang.IllegalStateException: calling recordValidTxn() more than once in the same txnid:420
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDTFGetSplits.createPlanFragment(GenericUDTFGetSplits.java:303)
> at org.apache.hadoop.hive.ql.udf.generic.GenericUDTFGetSplits.process(GenericUDTFGetSplits.java:202)
> at org.apache.hadoop.hive.ql.exec.UDTFOperator.process(UDTFOperator.java:116)
> at org.apache.hadoop.hive.ql.exec.Operator.baseForward(Operator.java:985)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:931)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:918)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.baseForward(Operator.java:985)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:931)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:125)
> at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:492)
> at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:484)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:145)
> ... 16 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)