You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Fredy Wijaya (JIRA)" <ji...@apache.org> on 2018/03/14 15:39:00 UTC
[jira] [Resolved] (IMPALA-6619) Alter table recover partitions
creates unneeded partitions when faces percent sign
[ https://issues.apache.org/jira/browse/IMPALA-6619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Fredy Wijaya resolved IMPALA-6619.
----------------------------------
Resolution: Fixed
Fix Version/s: Impala 2.13.0
Impala 2.12.0
Impala 3.0
> Alter table recover partitions creates unneeded partitions when faces percent sign
> ----------------------------------------------------------------------------------
>
> Key: IMPALA-6619
> URL: https://issues.apache.org/jira/browse/IMPALA-6619
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 2.11.0
> Reporter: Miklos Szurap
> Assignee: Fredy Wijaya
> Priority: Major
> Fix For: Impala 3.0, Impala 2.12.0, Impala 2.13.0
>
>
> When a table has a partition with a special character in it's name, then the HDFS directory contains a percent sign (due to an escaped/UrlEncoded sequence). This is not decoded and compared properly when running {{alter table recover partitions}}. This creates new, unneeded partitions on each execution.
> The steps to reproduce/demonstrate the issue:
> {noformat}
> [nightly-2:21000] > CREATE TABLE tbl_with_partition(col1 string) partitioned by (p string);
> Query: CREATE TABLE tbl_with_partition(col1 string) partitioned by (p string)
> Fetched 0 row(s) in 1.08s
> [nightly-2:21000] > ALTER TABLE tbl_with_partition add partition (p='100%');
> Query: ALTER TABLE tbl_with_partition add partition (p='100%')
> Fetched 0 row(s) in 5.72s
> [nightly-2:21000] > show partitions tbl_with_partition;
> Query: show partitions tbl_with_partition
> +-------+-------+--------+------+--------------+-------------------+--------+-------------------+------------------------------------------------------------+
> | p | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location |
> +-------+-------+--------+------+--------------+-------------------+--------+-------------------+------------------------------------------------------------+
> | 100% | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%25 |
> | Total | -1 | 0 | 0B | 0B | | | | |
> +-------+-------+--------+------+--------------+-------------------+--------+-------------------+------------------------------------------------------------+
> Fetched 2 row(s) in 0.02s
> [nightly-2:21000] > ALTER TABLE tbl_with_partition recover partitions;
> Query: ALTER TABLE tbl_with_partition recover partitions
> Fetched 0 row(s) in 0.29s
> [nightly-2:21000] > show partitions tbl_with_partition;
> Query: show partitions tbl_with_partition
> +--------+-------+--------+------+--------------+-------------------+--------+-------------------+--------------------------------------------------------------+
> | p | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location |
> +--------+-------+--------+------+--------------+-------------------+--------+-------------------+--------------------------------------------------------------+
> | 100% | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%25 |
> | 100%25 | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%2525 |
> | Total | -1 | 0 | 0B | 0B | | | | |
> +--------+-------+--------+------+--------------+-------------------+--------+-------------------+--------------------------------------------------------------+
> Fetched 3 row(s) in 0.02s
> [nightly-2:21000] > ALTER TABLE tbl_with_partition recover partitions;
> Query: ALTER TABLE tbl_with_partition recover partitions
> Fetched 0 row(s) in 0.27s
> [nightly-2:21000] > show partitions tbl_with_partition;
> Query: show partitions tbl_with_partition
> +----------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------------+
> | p | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location |
> +----------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------------+
> | 100% | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%25 |
> | 100%25 | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%2525 |
> | 100%2525 | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%252525 |
> | Total | -1 | 0 | 0B | 0B | | | | |
> +----------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------------+
> Fetched 4 row(s) in 0.02s{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)