[jira] [Created] (IMPALA-6619) Alter table recover partitions creates unneeded partitions when faces percent sign

["Miklos Szurap (JIRA)" <ji...@apache.org>]

Miklos Szurap created IMPALA-6619:
-------------------------------------

             Summary: Alter table recover partitions creates unneeded partitions when faces percent sign
                 Key: IMPALA-6619
                 URL: https://issues.apache.org/jira/browse/IMPALA-6619
             Project: IMPALA
          Issue Type: Bug
          Components: Frontend
    Affects Versions: Impala 2.11.0
            Reporter: Miklos Szurap


When a table has a partition with a special character in it's name, then the HDFS directory contains a percent sign (due to an escaped/UrlEncoded sequence). This is not decoded and compared properly when running {{alter table recover partitions}}. This creates new, unneeded partitions on each execution.

The steps to reproduce/demonstrate the issue:
{noformat}
[nightly-2:21000] > CREATE TABLE tbl_with_partition(col1 string) partitioned by (p string);
Query: CREATE TABLE tbl_with_partition(col1 string) partitioned by (p string)
Fetched 0 row(s) in 1.08s
[nightly-2:21000] > ALTER TABLE tbl_with_partition add partition (p='100%');
Query: ALTER TABLE tbl_with_partition add partition (p='100%')
Fetched 0 row(s) in 5.72s
[nightly-2:21000] > show partitions tbl_with_partition;
Query: show partitions tbl_with_partition
+-------+-------+--------+------+--------------+-------------------+--------+-------------------+------------------------------------------------------------+
| p | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location |
+-------+-------+--------+------+--------------+-------------------+--------+-------------------+------------------------------------------------------------+
| 100% | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%25 |
| Total | -1 | 0 | 0B | 0B | | | | |
+-------+-------+--------+------+--------------+-------------------+--------+-------------------+------------------------------------------------------------+
Fetched 2 row(s) in 0.02s
[nightly-2:21000] > ALTER TABLE tbl_with_partition recover partitions;
Query: ALTER TABLE tbl_with_partition recover partitions
Fetched 0 row(s) in 0.29s
[nightly-2:21000] > show partitions tbl_with_partition;
Query: show partitions tbl_with_partition
+--------+-------+--------+------+--------------+-------------------+--------+-------------------+--------------------------------------------------------------+
| p | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location |
+--------+-------+--------+------+--------------+-------------------+--------+-------------------+--------------------------------------------------------------+
| 100% | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%25 |
| 100%25 | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%2525 |
| Total | -1 | 0 | 0B | 0B | | | | |
+--------+-------+--------+------+--------------+-------------------+--------+-------------------+--------------------------------------------------------------+
Fetched 3 row(s) in 0.02s
[nightly-2:21000] > ALTER TABLE tbl_with_partition recover partitions;
Query: ALTER TABLE tbl_with_partition recover partitions
Fetched 0 row(s) in 0.27s
[nightly-2:21000] > show partitions tbl_with_partition;
Query: show partitions tbl_with_partition
+----------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------------+
| p | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format | Incremental stats | Location |
+----------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------------+
| 100% | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%25 |
| 100%25 | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%2525 |
| 100%2525 | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT | false | hdfs://ns1/user/hive/warehouse/tbl_with_partition/p=100%252525 |
| Total | -1 | 0 | 0B | 0B | | | | |
+----------+-------+--------+------+--------------+-------------------+--------+-------------------+----------------------------------------------------------------+
Fetched 4 row(s) in 0.02s{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)