You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Hive QA (Jira)" <ji...@apache.org> on 2020/05/13 02:39:00 UTC
[jira] [Commented] (HIVE-23455) Improve error message for external
orc table
[ https://issues.apache.org/jira/browse/HIVE-23455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105905#comment-17105905 ]
Hive QA commented on HIVE-23455:
--------------------------------
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m 29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 51s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 23s{color} | {color:blue} ql in master has 1527 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 16s{color} | {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 24s{color} | {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-22299/dev-support/hive-personality.sh |
| git revision | master / 2ff6370 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-22299/yetus.txt |
| Powered by | Apache Yetus http://yetus.apache.org |
This message was automatically generated.
> Improve error message for external orc table
> --------------------------------------------
>
> Key: HIVE-23455
> URL: https://issues.apache.org/jira/browse/HIVE-23455
> Project: Hive
> Issue Type: Improvement
> Reporter: Mustafa Iman
> Assignee: Mustafa Iman
> Priority: Major
> Attachments: HIVE-23455.patch, acid_table_as_external.q, acid_table_as_external.q.out
>
>
> Since there is no schema validation for external tables, users may face various errors if their orc data and external table schema does not match. If orc schema has fewer columns than projection OrcEncodedDataConsumer may receive an incomplete TypeDescription array which will manifest itself as NullPointerException later.
> We can at least verify that OrcEncodedDataConsumer gets enough TypeDescriptions. If assertion fails, user sees there is something wrong with the schema and hopefully resolves the problem quickly. If there are enough columns in the file but the schema of the query does not match, user generally sees a ClassCastException. If there are enough columns and types accidentally match, there is nothing we can do as this is an external table.
> We have seen this when trying to use a managed table as external table location. Although user facing schemas are the same, managed table has acid related metadata. I am adding a q file demonstrating NullPointerException with TestMiniLlapLocalCliDriver and the output after the fix. I haven't added this to precommit tests as it is hard to assert the exception message from mini driver framework and effectively it is just changing the error.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)