[jira] [Assigned] (IMPALA-9071) When metastore.warehouse.dir != metastore.warehouse.external.dir, Impala writes to the wrong location for external tables

["Quanlong Huang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-9071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Quanlong Huang reassigned IMPALA-9071:
--------------------------------------

    Assignee: Quanlong Huang

> When metastore.warehouse.dir != metastore.warehouse.external.dir, Impala writes to the wrong location for external tables
> -------------------------------------------------------------------------------------------------------------------------
>
>                 Key: IMPALA-9071
>                 URL: https://issues.apache.org/jira/browse/IMPALA-9071
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Frontend
>    Affects Versions: Impala 3.4.0
>            Reporter: Joe McDonnell
>            Assignee: Quanlong Huang
>            Priority: Blocker
>              Labels: broken-build
>
> Hive introduced a translation layer that can convert a normal table to an external table. When doing so without a specified location, the translated external table uses metastore.warehouse.external.dir as the location rather than metastore.warehouse.dir. Impala does not know about this distinction, so it writes to the location it thinks the table should be (under metastore.warehouse.dir). This means I can do the following:
> {noformat}
> [localhost:21000] joetest> select count(*) from functional.alltypes;
> Query: select count(*) from functional.alltypes
> Query submitted at: 2019-10-19 13:08:24 (Coordinator: http://joemcdonnell:25000)
> Query progress can be monitored at: http://joemcdonnell:25000/query_plan?query_id=68434b05e2badd50:a18a2e3000000000
> +----------+
> | count(*) |
> +----------+
> | 7300     |
> +----------+
> Fetched 1 row(s) in 0.14s
> [localhost:21000] joetest> create table testtable as select * from functional.alltypes;
> Query: create table testtable as select * from functional.alltypes
> Query submitted at: 2019-10-19 13:08:36 (Coordinator: http://joemcdonnell:25000)
> Query progress can be monitored at: http://joemcdonnell:25000/query_plan?query_id=794b92fb68f36ab0:910d036400000000
> +----------------------+
> | summary              |
> +----------------------+
> | Inserted 7300 row(s) |
> +----------------------+
> Fetched 1 row(s) in 0.50s
> [localhost:21000] joetest> select count(*) from testtable;
> Query: select count(*) from testtable
> Query submitted at: 2019-10-19 13:08:43 (Coordinator: http://joemcdonnell:25000)
> Query progress can be monitored at: http://joemcdonnell:25000/query_plan?query_id=66423abf016e65af:8362460900000000
> +----------+
> | count(*) |
> +----------+
> | 0        |
> +----------+
> Fetched 1 row(s) in 0.13s
> {noformat}
> We inserted 7300 rows, but we can't select them back because they were written to the wrong location.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org