You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Hive QA (Jira)" <ji...@apache.org> on 2020/04/05 20:35:00 UTC
[jira] [Commented] (HIVE-23123) Disable export/import of views and
materialized views
[ https://issues.apache.org/jira/browse/HIVE-23123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17075924#comment-17075924 ]
Hive QA commented on HIVE-23123:
--------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m 2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 36s{color} | {color:blue} ql in master has 1528 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 40s{color} | {color:green} ql: The patch generated 0 new + 2 unchanged - 101 fixed = 2 total (was 103) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 56s{color} | {color:red} ql generated 3 new + 1528 unchanged - 0 fixed = 1531 total (was 1528) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 58s{color} | {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Possible null pointer dereference of ts in org.apache.hadoop.hive.ql.parse.ExportSemanticAnalyzer.analyzeExport(ASTNode, String, Hive, HiveConf, Set, Set) Dereferenced at ExportSemanticAnalyzer.java:ts in org.apache.hadoop.hive.ql.parse.ExportSemanticAnalyzer.analyzeExport(ASTNode, String, Hive, HiveConf, Set, Set) Dereferenced at ExportSemanticAnalyzer.java:[line 106] |
| | Nullcheck of ts at line 128 of value previously dereferenced in org.apache.hadoop.hive.ql.parse.ExportSemanticAnalyzer.analyzeExport(ASTNode, String, Hive, HiveConf, Set, Set) At ExportSemanticAnalyzer.java:128 of value previously dereferenced in org.apache.hadoop.hive.ql.parse.ExportSemanticAnalyzer.analyzeExport(ASTNode, String, Hive, HiveConf, Set, Set) At ExportSemanticAnalyzer.java:[line 106] |
| | Unread field:ImportTableDesc.java:[line 47] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-21459/dev-support/hive-personality.sh |
| git revision | master / bc669b3 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| whitespace | http://104.198.109.242/logs//PreCommit-HIVE-Build-21459/yetus/whitespace-eol.txt |
| findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-21459/yetus/new-findbugs-ql.html |
| modules | C: ql U: ql |
| Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-21459/yetus.txt |
| Powered by | Apache Yetus http://yetus.apache.org |
This message was automatically generated.
> Disable export/import of views and materialized views
> -----------------------------------------------------
>
> Key: HIVE-23123
> URL: https://issues.apache.org/jira/browse/HIVE-23123
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Reporter: Miklos Gergely
> Assignee: Miklos Gergely
> Priority: Major
> Attachments: HIVE-23123.01.patch
>
>
> According to [https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ImportExport] import and export can be done by using the
> {code:java}
> export table ...
> import table ...
> {code}
> commands. The document doesn't mention views or materialized views at all, and in fact we don't support commands like
> {code:java}
> export view ...
> import view ...
> export materialized view ...
> import materialized view ...
> {code}
> they can not be parsed at all. The word table is often used though in a broader sense, when it means all table like entities, including views and materialized views. For example the various Table classes may represent any of these as well.
> If I try to export a view with the export table ... command, it goes fine. A _metadata file will be created, but no data directory, which is what we'd expect. If I try to import it back, an exception is thrown due to the lack of the data dir:
> {code:java}
> java.lang.AssertionError: null==getPath() for exim_view
> at org.apache.hadoop.hive.ql.metadata.Hive.loadTable(Hive.java:3088)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:419)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105)
> at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:364)
> at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:335)
> at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:246)
> at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:109)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:722)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:491)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:485)
> {code}
> Still the view gets imported successfully, as data movement wasn't even necessary.
> If we try to export a materialized view which is transactional, then this exception occurs:
> {code:java}
> org.apache.hadoop.hive.ql.parse.SemanticException: org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found exim_materialized_view_da21d41a_9fe4_4446_9c72_d251496abf9d
> at org.apache.hadoop.hive.ql.parse.AcidExportSemanticAnalyzer.analyzeAcidExport(AcidExportSemanticAnalyzer.java:163)
> at org.apache.hadoop.hive.ql.parse.AcidExportSemanticAnalyzer.analyze(AcidExportSemanticAnalyzer.java:71)
> at org.apache.hadoop.hive.ql.parse.RewriteSemanticAnalyzer.analyzeInternal(RewriteSemanticAnalyzer.java:72)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:220)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:183)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:601)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:547)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:541)
> {code}
> So the export process can not handle it, as the temporary table is not getting created.
>
> The import command handling have a lot of codes dedicated to importing views and materialized views, which suggests that we support the importing (and thus also suggests implicitly that we support the exporting) of views and materialiezed views.
>
> So the conclusion is that we have to decide if we support exporting/importing of views and materialized views.
> If we decide not to support them then:
> - export process should throw an exception if a view or materialized view is the subject
> - the codes specific to view imports should be removed
> If we decide to support them, then:
> - the commands mentioned above above should be introduced
> - exception should be thrown if not the proper command used (e.g. export view on a table)
> - the exceptions mentioned above should be fixed
>
> I prefer not to support them, I don't think we should support the exporting / importing of views. The point of exporting / importing is the transfer of data, not DDL, it causes more issues than it solves. Our current documentation also suggests that it is only supported for tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)