You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Eli Vingot (JIRA)" <ji...@apache.org> on 2018/06/07 08:58:00 UTC
[jira] [Updated] (IMPALA-7142) Ability to use expressions on range
partitions
[ https://issues.apache.org/jira/browse/IMPALA-7142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Vingot updated IMPALA-7142:
-------------------------------
Description:
As a devop I would like to have scripts that create tables for me.
Many times when creating tables the partitioning is dependent on calculated values, mainly time stamps.
Currently we need to alter the script manually for every deployment
CREATE TABLE dwh.f_events (
ts BIGINT,
batch_id STRING,
network_id INT,
...
last_updated BIGINT,
PRIMARY KEY (ts,batch_id,network_id)
)
PARTITION BY HASH (network_id) PARTITIONS 8,
RANGE (ts)
(
PARTITION 1524528000000 <= VALUES < 1524614400000
)
STORED AS KUDU
It would be lovely if we could set the ranges with an expression as below (for 7 days partitions starting today)
CREATE TABLE dwh.f_events (
ts BIGINT,
batch_id STRING,
network_id INT,
...
last_updated BIGINT,
PRIMARY KEY (ts,batch_id,network_id)
)
PARTITION BY HASH (network_id) PARTITIONS 8,
RANGE (ts)
(
*PARTITION (unix_timestamp() - (unix_timestamp() % (3600*24) ))*1000) <= VALUES < ((unix_timestamp() - (unix_timestamp() % (3600*24) ))*1000 + (3600000*24*7)))*
STORED AS KUDU
was:
As a devop I would like to have scripts that create tables for me.
Many times when creating tables the partitioning is dependent on calculated values, mainly time stamps.
Currently we need to alter the script manually for every deployment
CREATE TABLE dwh.f_events (
ts BIGINT,
batch_id STRING,
network_id INT,
...
last_updated BIGINT,
PRIMARY KEY (ts,batch_id,network_id)
)
PARTITION BY HASH (network_id) PARTITIONS 8,
RANGE (ts)
(
PARTITION 1524528000000 <= VALUES < 1524614400000
)
STORED AS KUDU
It would be lovely if we could set the ranges with an expression as below (for 7 days partitions starting today)
CREATE TABLE dwh.f_events (
ts BIGINT,
batch_id STRING,
network_id INT,
...
last_updated BIGINT,
PRIMARY KEY (ts,batch_id,network_id)
)
PARTITION BY HASH (network_id) PARTITIONS 8,
RANGE (ts)
(
PARTITION (unix_timestamp() - (unix_timestamp() % (3600*24) ))*1000) <= VALUES < ((unix_timestamp() - (unix_timestamp() % (3600*24) ))*1000 + (3600000*24*7)))
STORED AS KUDU
> Ability to use expressions on range partitions
> ----------------------------------------------
>
> Key: IMPALA-7142
> URL: https://issues.apache.org/jira/browse/IMPALA-7142
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Eli Vingot
> Priority: Major
>
> As a devop I would like to have scripts that create tables for me.
> Many times when creating tables the partitioning is dependent on calculated values, mainly time stamps.
> Currently we need to alter the script manually for every deployment
>
> CREATE TABLE dwh.f_events (
> ts BIGINT,
> batch_id STRING,
> network_id INT,
> ...
> last_updated BIGINT,
> PRIMARY KEY (ts,batch_id,network_id)
> )
> PARTITION BY HASH (network_id) PARTITIONS 8,
> RANGE (ts)
> (
> PARTITION 1524528000000 <= VALUES < 1524614400000
> )
> STORED AS KUDU
>
>
> It would be lovely if we could set the ranges with an expression as below (for 7 days partitions starting today)
> CREATE TABLE dwh.f_events (
> ts BIGINT,
> batch_id STRING,
> network_id INT,
> ...
> last_updated BIGINT,
> PRIMARY KEY (ts,batch_id,network_id)
> )
> PARTITION BY HASH (network_id) PARTITIONS 8,
> RANGE (ts)
> (
> *PARTITION (unix_timestamp() - (unix_timestamp() % (3600*24) ))*1000) <= VALUES < ((unix_timestamp() - (unix_timestamp() % (3600*24) ))*1000 + (3600000*24*7)))*
> STORED AS KUDU
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org