You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2020/04/14 23:02:00 UTC
[jira] [Commented] (IMPALA-9071) When metastore.warehouse.dir !=
metastore.warehouse.external.dir, Impala writes to the wrong location for
external tables
[ https://issues.apache.org/jira/browse/IMPALA-9071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083676#comment-17083676 ]
ASF subversion and git services commented on IMPALA-9071:
---------------------------------------------------------
Commit 82ca3262b3aeee13d2cc27c5ff9aef3f6c70cc8a in impala's branch refs/heads/master from Csaba Ringhofer
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=82ca326 ]
IMPALA-9071: remove xfail from ACID CTAS test
The test passes at the moment - according to the TODO
it needed HIVE-22158, which is included in CDP dependencies
since a long time.
Change-Id: I4e068879100061bb7e724f0bd1e5e4aa8edd7825
Reviewed-on: http://gerrit.cloudera.org:8080/15729
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> When metastore.warehouse.dir != metastore.warehouse.external.dir, Impala writes to the wrong location for external tables
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-9071
> URL: https://issues.apache.org/jira/browse/IMPALA-9071
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.4.0
> Reporter: Joe McDonnell
> Assignee: Quanlong Huang
> Priority: Blocker
> Labels: broken-build
> Fix For: Impala 3.4.0
>
>
> Hive introduced a translation layer that can convert a normal table to an external table. When doing so without a specified location, the translated external table uses metastore.warehouse.external.dir as the location rather than metastore.warehouse.dir. Impala does not know about this distinction, so it writes to the location it thinks the table should be (under metastore.warehouse.dir). This means I can do the following:
> {noformat}
> [localhost:21000] joetest> select count(*) from functional.alltypes;
> Query: select count(*) from functional.alltypes
> Query submitted at: 2019-10-19 13:08:24 (Coordinator: http://joemcdonnell:25000)
> Query progress can be monitored at: http://joemcdonnell:25000/query_plan?query_id=68434b05e2badd50:a18a2e3000000000
> +----------+
> | count(*) |
> +----------+
> | 7300 |
> +----------+
> Fetched 1 row(s) in 0.14s
> [localhost:21000] joetest> create table testtable as select * from functional.alltypes;
> Query: create table testtable as select * from functional.alltypes
> Query submitted at: 2019-10-19 13:08:36 (Coordinator: http://joemcdonnell:25000)
> Query progress can be monitored at: http://joemcdonnell:25000/query_plan?query_id=794b92fb68f36ab0:910d036400000000
> +----------------------+
> | summary |
> +----------------------+
> | Inserted 7300 row(s) |
> +----------------------+
> Fetched 1 row(s) in 0.50s
> [localhost:21000] joetest> select count(*) from testtable;
> Query: select count(*) from testtable
> Query submitted at: 2019-10-19 13:08:43 (Coordinator: http://joemcdonnell:25000)
> Query progress can be monitored at: http://joemcdonnell:25000/query_plan?query_id=66423abf016e65af:8362460900000000
> +----------+
> | count(*) |
> +----------+
> | 0 |
> +----------+
> Fetched 1 row(s) in 0.13s
> {noformat}
> We inserted 7300 rows, but we can't select them back because they were written to the wrong location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org