You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Hive QA (Jira)" <ji...@apache.org> on 2019/09/25 10:58:00 UTC
[jira] [Commented] (HIVE-22234) Hive replication fails with table
already exist error when replicating from old version of hive.
[ https://issues.apache.org/jira/browse/HIVE-22234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937626#comment-16937626 ]
Hive QA commented on HIVE-22234:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981284/HIVE-22234.01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/18718/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18718/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18718/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-09-25 10:56:39.198
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-18718/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-09-25 10:56:39.201
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at d6668c7 HIVE-22221: Llap external client - Need to reduce LlapBaseInputFormat#getSplits() footprint (Shubham Chaurasia, reviewed by Jason Dere)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at d6668c7 HIVE-22221: Llap external client - Need to reduce LlapBaseInputFormat#getSplits() footprint (Shubham Chaurasia, reviewed by Jason Dere)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-09-25 10:56:40.221
+ rm -rf ../yetus_PreCommit-HIVE-Build-18718
+ mkdir ../yetus_PreCommit-HIVE-Build-18718
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-18718
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-18718/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch
fatal: git diff header lacks filename information when removing 0 leading pathname components (line 126)
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl/delta_0000001_0000001_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl/delta_0000001_0000001_0000/bucket_00000' without full index line
error: data/files/test_hcube_2.db/tbl/delta_0000001_0000001_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl1/delta_0000001_0000001_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl1/delta_0000001_0000001_0000/bucket_00000' without full index line
error: data/files/test_hcube_2.db/tbl1/delta_0000001_0000001_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl1/delta_0000002_0000002_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl1/delta_0000002_0000002_0000/bucket_00000' without full index line
error: data/files/test_hcube_2.db/tbl1/delta_0000002_0000002_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl1/delta_0000003_0000003_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl1/delta_0000003_0000003_0000/bucket_00000' without full index line
error: data/files/test_hcube_2.db/tbl1/delta_0000003_0000003_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl2/delta_0000001_0000001_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl2/delta_0000001_0000001_0000/bucket_00000' without full index line
error: data/files/test_hcube_2.db/tbl2/delta_0000001_0000001_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl2/delta_0000002_0000002_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl2/delta_0000002_0000002_0000/bucket_00000' without full index line
error: data/files/test_hcube_2.db/tbl2/delta_0000002_0000002_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl3/delta_0000001_0000001_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl3/delta_0000001_0000001_0000/bucket_00000' without full index line
error: data/files/test_hcube_2.db/tbl3/delta_0000001_0000001_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl4/delta_0000001_0000001_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl4/delta_0000001_0000001_0000/bucket_00000' without full index line
error: data/files/test_hcube_2.db/tbl4/delta_0000001_0000001_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl5/delta_0000001_0000001_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl5/delta_0000001_0000001_0000/bucket_00000' without full index line
error: data/files/test_hcube_2.db/tbl5/delta_0000001_0000001_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl6/delta_0000001_0000001_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'data/files/test_hcube_2.db/tbl6/delta_0000001_0000001_0000/bucket_00000' without full index line
error: data/files/test_hcube_2.db/tbl6/delta_0000001_0000001_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl/delta_0000001_0000001_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl/delta_0000001_0000001_0000/bucket_00000' without full index line
error: files/test_hcube_2.db/tbl/delta_0000001_0000001_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl1/delta_0000001_0000001_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl1/delta_0000001_0000001_0000/bucket_00000' without full index line
error: files/test_hcube_2.db/tbl1/delta_0000001_0000001_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl1/delta_0000002_0000002_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl1/delta_0000002_0000002_0000/bucket_00000' without full index line
error: files/test_hcube_2.db/tbl1/delta_0000002_0000002_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl1/delta_0000003_0000003_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl1/delta_0000003_0000003_0000/bucket_00000' without full index line
error: files/test_hcube_2.db/tbl1/delta_0000003_0000003_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl2/delta_0000001_0000001_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl2/delta_0000001_0000001_0000/bucket_00000' without full index line
error: files/test_hcube_2.db/tbl2/delta_0000001_0000001_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl2/delta_0000002_0000002_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl2/delta_0000002_0000002_0000/bucket_00000' without full index line
error: files/test_hcube_2.db/tbl2/delta_0000002_0000002_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl3/delta_0000001_0000001_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl3/delta_0000001_0000001_0000/bucket_00000' without full index line
error: files/test_hcube_2.db/tbl3/delta_0000001_0000001_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl4/delta_0000001_0000001_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl4/delta_0000001_0000001_0000/bucket_00000' without full index line
error: files/test_hcube_2.db/tbl4/delta_0000001_0000001_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl5/delta_0000001_0000001_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl5/delta_0000001_0000001_0000/bucket_00000' without full index line
error: files/test_hcube_2.db/tbl5/delta_0000001_0000001_0000/bucket_00000: patch does not apply
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl6/delta_0000001_0000001_0000/bucket_00000' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'files/test_hcube_2.db/tbl6/delta_0000001_0000001_0000/bucket_00000' without full index line
error: files/test_hcube_2.db/tbl6/delta_0000001_0000001_0000/bucket_00000: patch does not apply
error: src/java/org/apache/hadoop/hive/ql/ddl/table/creation/CreateTableDesc.java: does not exist in index
error: src/java/org/apache/hadoop/hive/ql/exec/CopyTask.java: does not exist in index
error: src/java/org/apache/hadoop/hive/ql/exec/ReplCopyTask.java: does not exist in index
error: src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/events/filesystem/FSTableEvent.java: does not exist in index
error: src/java/org/apache/hadoop/hive/ql/metadata/Table.java: does not exist in index
error: metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ColumnStatistics.java: does not exist in index
error: metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java: does not exist in index
error: metastore-common/src/gen/thrift/gen-php/metastore/Types.php: does not exist in index
error: metastore-common/src/gen/thrift/gen-py/hive_metastore/ttypes.py: does not exist in index
error: metastore-common/src/gen/thrift/gen-rb/hive_metastore_types.rb: does not exist in index
error: metastore-common/src/main/thrift/hive_metastore.thrift: does not exist in index
error: metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java: does not exist in index
error: metastore-server/src/main/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java: does not exist in index
error: metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestStats.java: does not exist in index
The patch does not appear to apply with p0, p1, or p2
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-18718
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981284 - PreCommit-HIVE-Build
> Hive replication fails with table already exist error when replicating from old version of hive.
> ------------------------------------------------------------------------------------------------
>
> Key: HIVE-22234
> URL: https://issues.apache.org/jira/browse/HIVE-22234
> Project: Hive
> Issue Type: Bug
> Reporter: mahesh kumar behera
> Assignee: mahesh kumar behera
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22234.01.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> HIve replication from old version where HIVE-22046 is not patched will not have engine column set in the table column stats. This causes "ERROR: null value in column "ENGINE" violates not-null constraint" error during create table while updating the column stats. As the column stats are updated after the create table txn is committed, the next retry by HMS client throws table already exist error. Need to update the ENGINE column to default value while importing the table if the column value is not set. The column stat and create table in same txn can be done as part of separate Jira.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)