You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Zheng Shao (JIRA)" <ji...@apache.org> on 2018/08/09 23:07:00 UTC
[jira] [Updated] (HIVE-20358) Allow setting variable value from Hive metastore table properties

     [ https://issues.apache.org/jira/browse/HIVE-20358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zheng Shao updated HIVE-20358:
------------------------------
    Description: 
Hive already supports set command as well as variable substitution:

{{set start_ds=2018-08-01;}}

{{SELECT COUNT(*) FROM t WHERE ds >= '${hiveconf:start_ds}';}}

 

Or:

{{set start_ds='2018-08-01';}}

{{SELECT COUNT(*) FROM t WHERE ds >= ${hiveconf:start_ds};}}

 

This issue propose to extend the set syntax to allow running UDF and a UDF that queries metastore:

{{SET <VARIABLE_NAME> <EXPRESSION>;}}

 

For example:

{{set start_ds GET_TABLE_PROPERTY('mydb', 'mytable', 'last_modified_time');}}

{{set start_ds GET_PARTITION_PROPERTY('mydb', 'mytable', 'ds=2018-01-01/hr=12', 'last_modified_time');}}

 

This will allow query workflows like the following:

{{set last_run_time GET_TABLE_PROPERTY('mydb', 'mytable', 'last_modified_time');}}

{{INSERT INTO TABLE mytable SELECT * FROM src WHERE src.time > last_run_time;}}

 

  was:
Hive already supports set command as well as variable substitution:


{{set start_ds=2018-08-01;}}

{{SELECT COUNT(*) FROM t WHERE ds >= '${hiveconf:start_ds}';}}

 

Or:

{{set start_ds='2018-08-01';}}

{{SELECT COUNT(*) FROM t WHERE ds >= ${hiveconf:start_ds};}}

 

This issue propose to extend the set syntax to allow running UDF and a UDF that queries metastore:

{{SET <VARIABLE_NAME> <EXPRESSION>;}}

 

For example:

{{set start_ds GET_TABLE_PROPERTY('mydb', 'mytable', 'last_modified_time');}}

{{set start_ds GET_PARTITION_PROPERTY('mydb', 'mytable', 'ds=2018-01-01/hr=12', 'last_modified_time');}}

 

This will allow query workflows like the following:

{{set last_run_time GET_TABLE_PROPERTY('mydb', 'mytable', 'last_modified_time');}}

{{INSERT INTO TABLE mytable SELECT * FROM src WHERE src.time > last_run_time;}}

 


> Allow setting variable value from Hive metastore table properties
> -----------------------------------------------------------------
>
>                 Key: HIVE-20358
>                 URL: https://issues.apache.org/jira/browse/HIVE-20358
>             Project: Hive
>          Issue Type: New Feature
>          Components: Query Processor
>            Reporter: Zheng Shao
>            Priority: Minor
>
> Hive already supports set command as well as variable substitution:
> {{set start_ds=2018-08-01;}}
> {{SELECT COUNT(*) FROM t WHERE ds >= '${hiveconf:start_ds}';}}
>  
> Or:
> {{set start_ds='2018-08-01';}}
> {{SELECT COUNT(*) FROM t WHERE ds >= ${hiveconf:start_ds};}}
>  
> This issue propose to extend the set syntax to allow running UDF and a UDF that queries metastore:
> {{SET <VARIABLE_NAME> <EXPRESSION>;}}
>  
> For example:
> {{set start_ds GET_TABLE_PROPERTY('mydb', 'mytable', 'last_modified_time');}}
> {{set start_ds GET_PARTITION_PROPERTY('mydb', 'mytable', 'ds=2018-01-01/hr=12', 'last_modified_time');}}
>  
> This will allow query workflows like the following:
> {{set last_run_time GET_TABLE_PROPERTY('mydb', 'mytable', 'last_modified_time');}}
> {{INSERT INTO TABLE mytable SELECT * FROM src WHERE src.time > last_run_time;}}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)