You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Eugene Koifman (JIRA)" <ji...@apache.org> on 2018/06/03 23:56:00 UTC
[jira] [Comment Edited] (HIVE-19750) Initialize NEXT_WRITE_ID.
NWI_NEXT on converting an existing table to full acid
[ https://issues.apache.org/jira/browse/HIVE-19750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499608#comment-16499608 ]
Eugene Koifman edited comment on HIVE-19750 at 6/3/18 11:55 PM:
----------------------------------------------------------------
This is very odd. Test bot reports a failure:
{noformat}
Regression
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testNonAcidToAcidConversion01
Failing for the past 1 build (Since Failed#11481 )
Took 13 sec.
Error Message
{"writeid":0,"bucketid":536936448,"rowid":0} 1 2 file:/home/hiveptest/35.232.255.26-hiveptest-0/apache-github-source-source/ql/target/tmp/org.apache.hadoop.hive.ql.TestTxnCommands-1528061700968/warehouse/nonacidorctbl/base_10000001/bucket_00001
Stacktrace
java.lang.AssertionError: {"writeid":0,"bucketid":536936448,"rowid":0} 1 2 file:/home/hiveptest/35.232.255.26-hiveptest-0/apache-github-source-source/ql/target/tmp/org.apache.hadoop.hive.ql.TestTxnCommands-1528061700968/warehouse/nonacidorctbl/base_10000001/bucket_00001
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.apache.hadoop.hive.ql.TestTxnCommands.testNonAcidToAcidConversion01(TestTxnCommands.java:915)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{noformat}
but {{TestTxnCommandsWithSplitUpdateAndVectorization.testNonAcidToAcidConversion01}} does not exist.
{{TestTxnCommandsWithSplitUpdateAndVectorization extends TestTxnCommands2}} not {{TestTxnCommands}}
{{org.apache.hadoop.hive.ql.TestTxnCommands.testNonAcidToAcidConversion01}} does exist but the 2 failure messages are identical except for the host name in the file path....
former: https://builds.apache.org/job/PreCommit-HIVE-Build/11481/testReport/org.apache.hadoop.hive.ql/TestTxnCommandsWithSplitUpdateAndVectorization/testNonAcidToAcidConversion01/
latter: https://builds.apache.org/job/PreCommit-HIVE-Build/11481/testReport/org.apache.hadoop.hive.ql/TestTxnCommands/testNonAcidToAcidConversion01/
was (Author: ekoifman):
This is very odd. Test bot reports a failure:
{noformat}
Regression
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testNonAcidToAcidConversion01
Failing for the past 1 build (Since Failed#11481 )
Took 13 sec.
Error Message
{"writeid":0,"bucketid":536936448,"rowid":0} 1 2 file:/home/hiveptest/35.232.255.26-hiveptest-0/apache-github-source-source/ql/target/tmp/org.apache.hadoop.hive.ql.TestTxnCommands-1528061700968/warehouse/nonacidorctbl/base_10000001/bucket_00001
Stacktrace
java.lang.AssertionError: {"writeid":0,"bucketid":536936448,"rowid":0} 1 2 file:/home/hiveptest/35.232.255.26-hiveptest-0/apache-github-source-source/ql/target/tmp/org.apache.hadoop.hive.ql.TestTxnCommands-1528061700968/warehouse/nonacidorctbl/base_10000001/bucket_00001
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.apache.hadoop.hive.ql.TestTxnCommands.testNonAcidToAcidConversion01(TestTxnCommands.java:915)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
{noformat}
but {{TestTxnCommandsWithSplitUpdateAndVectorization.testNonAcidToAcidConversion01}} does not exist.
{{TestTxnCommandsWithSplitUpdateAndVectorization extends TestTxnCommands2}} not {{TestTxnCommands}}
> Initialize NEXT_WRITE_ID. NWI_NEXT on converting an existing table to full acid
> -------------------------------------------------------------------------------
>
> Key: HIVE-19750
> URL: https://issues.apache.org/jira/browse/HIVE-19750
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 3.0.0
> Reporter: Eugene Koifman
> Assignee: Eugene Koifman
> Priority: Critical
> Fix For: 3.1.0
>
> Attachments: HIVE-19750.01.patch, HIVE-19750.02.patch
>
>
> Need to set this to a reasonably high value the the table.
> This will reserve a range of write IDs that will be treated by the system as committed.
> This is needed so that we can assign unique ROW__IDs to each row in files that already exist in the table. For example, if the value is initialized to the number of files currently in the table, we can think of each file as written by a separate transaction and thus a free to assign bucketProperty (BucketCodec) of ROW_ID in whichever way is convenient.
> it's guaranteed that all rows get unique ROW_IDs this way.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)