You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Zoltan Haindrich (Jira)" <ji...@apache.org> on 2019/12/19 06:32:00 UTC

[jira] [Commented] (HIVE-16907) "INSERT INTO" overwrite old data when destination table encapsulated by backquote

    [ https://issues.apache.org/jira/browse/HIVE-16907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16999775#comment-16999775 ] 

Zoltan Haindrich commented on HIVE-16907:
-----------------------------------------

There are a few cases in which this seems like a regression; to make life easier - there is an SA hook which could be used to fix incorrectly written `db.table` references
the hook transparently rewrites `db.table` references as if they were `db`.`table`.
https://github.com/kgyrtkirk/tablename_fixer_hook
This hook could give more transition time to fix the actual queries.

>  "INSERT INTO"  overwrite old data when destination table encapsulated by backquote 
> ------------------------------------------------------------------------------------
>
>                 Key: HIVE-16907
>                 URL: https://issues.apache.org/jira/browse/HIVE-16907
>             Project: Hive
>          Issue Type: Bug
>          Components: Parser
>    Affects Versions: 1.1.0, 2.1.1
>            Reporter: Nemon Lou
>            Assignee: Zoltan Haindrich
>            Priority: Major
>             Fix For: 4.0.0
>
>         Attachments: HIVE-16907.02.patch, HIVE-16907.03.patch, HIVE-16907.03.patch, HIVE-16907.03.patch, HIVE-16907.03.patch, HIVE-16907.1.patch
>
>
> A way to reproduce:
> {noformat}
> create database tdb;
> use tdb;
> create table t1(id int);
> create table t2(id int);
> explain insert into `tdb.t1` select * from t2;
> {noformat}
> {noformat}
> +---------------------------------------------------------------------------------------------------------------------------------------------------+
> |                                                                      Explain                                                                      |
> +---------------------------------------------------------------------------------------------------------------------------------------------------+
> | STAGE DEPENDENCIES:                                                                                                                               |
> |   Stage-1 is a root stage                                                                                                                         |
> |   Stage-6 depends on stages: Stage-1 , consists of Stage-3, Stage-2, Stage-4                                                                      |
> |   Stage-3                                                                                                                                         |
> |   Stage-0 depends on stages: Stage-3, Stage-2, Stage-5                                                                                            |
> |   Stage-2                                                                                                                                         |
> |   Stage-4                                                                                                                                         |
> |   Stage-5 depends on stages: Stage-4                                                                                                              |
> |                                                                                                                                                   |
> | STAGE PLANS:                                                                                                                                      |
> |   Stage: Stage-1                                                                                                                                  |
> |     Map Reduce                                                                                                                                    |
> |       Map Operator Tree:                                                                                                                          |
> |           TableScan                                                                                                                               |
> |             alias: t2                                                                                                                             |
> |             Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE                                                             |
> |             Select Operator                                                                                                                       |
> |               expressions: id (type: int)                                                                                                         |
> |               outputColumnNames: _col0                                                                                                            |
> |               Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE                                                           |
> |               File Output Operator                                                                                                                |
> |                 compressed: false                                                                                                                 |
> |                 Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE                                                         |
> |                 table:                                                                                                                            |
> |                     input format: org.apache.hadoop.hive.ql.io.RCFileInputFormat                                                                  |
> |                     output format: org.apache.hadoop.hive.ql.io.RCFileOutputFormat                                                                |
> |                     serde: org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe                                                                   |
> |                     name: tdb.t1                                                                                                                  |
> |                                                                                                                                                   |
> |   Stage: Stage-6                                                                                                                                  |
> |     Conditional Operator                                                                                                                          |
> |                                                                                                                                                   |
> |   Stage: Stage-3                                                                                                                                  |
> |     Move Operator                                                                                                                                 |
> |       files:                                                                                                                                      |
> |           hdfs directory: true                                                                                                                    |
> |           destination: hdfs://hacluster/user/hive/warehouse/tdb.db/t1/.hive-staging_hive_2017-06-15_15-52-34_017_849305017872068583-1/-ext-10000  |
> |                                                                                                                                                   |
> |   Stage: Stage-0                                                                                                                                  |
> |     Move Operator                                                                                                                                 |
> |       tables:                                                                                                                                     |
> |           replace: true                                                                                                                           |
> |           table:                                                                                                                                  |
> |               input format: org.apache.hadoop.hive.ql.io.RCFileInputFormat                                                                        |
> |               output format: org.apache.hadoop.hive.ql.io.RCFileOutputFormat                                                                      |
> |               serde: org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe                                                                         |
> |               name: tdb.t1                                                                                                                        |
> |                                                                                                                                                   |
> |   Stage: Stage-2                                                                                                                                  |
> |     Merge File Operator                                                                                                                           |
> |       Map Operator Tree:                                                                                                                          |
> |           RCFile Merge Operator                                                                                                                   |
> |       merge level: block                                                                                                                          |
> |       input format: org.apache.hadoop.hive.ql.io.RCFileInputFormat                                                                                |
> |                                                                                                                                                   |
> |   Stage: Stage-4                                                                                                                                  |
> |     Merge File Operator                                                                                                                           |
> |       Map Operator Tree:                                                                                                                          |
> |           RCFile Merge Operator                                                                                                                   |
> |       merge level: block                                                                                                                          |
> |       input format: org.apache.hadoop.hive.ql.io.RCFileInputFormat                                                                                |
> |                                                                                                                                                   |
> |   Stage: Stage-5                                                                                                                                  |
> |     Move Operator                                                                                                                                 |
> |       files:                                                                                                                                      |
> |           hdfs directory: true                                                                                                                    |
> |           destination: hdfs://hacluster/user/hive/warehouse/tdb.db/t1/.hive-staging_hive_2017-06-15_15-52-34_017_849305017872068583-1/-ext-10000  |
> |                                                                                                                                                   |
> +---------------------------------------------------------------------------------------------------------------------------------------------------+
> {noformat}
> Note that  'replace: true'  in move operator



--
This message was sent by Atlassian Jira
(v8.3.4#803005)