You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Esteban Gutierrez (JIRA)" <ji...@apache.org> on 2011/04/22 05:51:05 UTC
[jira] [Created] (HIVE-2124) Specified functions in the
partitioning predicates should not generate a M/R job.
Specified functions in the partitioning predicates should not generate a M/R job.
---------------------------------------------------------------------------------
Key: HIVE-2124
URL: https://issues.apache.org/jira/browse/HIVE-2124
Project: Hive
Issue Type: Improvement
Components: Query Processor
Affects Versions: 0.7.0, 0.6.0, 0.5.0
Reporter: Esteban Gutierrez
Priority: Minor
For certain situations specifying which functions should be evaluated once would help to make syntax simpler to avoid launching M/R jobs.
Example:
# myhql.time=`date "+%s"` -> constant
# counting rows from the last 30 days generates a M/R job using all the partitions
$ hive -hiveconf myhql.time=`date "+%s"` -e "SELECT COUNT(*) FROM mybigtable WHERE mypartition >= from_unixtime(\${hiveconf:myhql.time}-2592000,'yyyy-MM-dd');
Suggested feature:
# will scan only the right partitions
$ hive -hiveconf hive.partition.evaluateonce=unix_timestamp -e "SELECT COUNT(*) FROM mybigtable WHERE mypartition >= from_unixtime(unix_timestamp()-2592000,'yyyy-MM-dd');
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2124) Specified functions in the
partitioning predicates should not generate a M/R job.
Posted by "Esteban Gutierrez (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Esteban Gutierrez updated HIVE-2124:
------------------------------------
Description:
For certain situations specifying which functions should be evaluated once would help to make syntax simpler to avoid launching M/R jobs.
Example:
\# myhql.time=`date "+%s"` -> constant
\# counting rows from the last 30 days generates a M/R job using all the partitions
$ hive -hiveconf myhql.time=`date "+%s"` -e "SELECT COUNT\(\*\) FROM mybigtable WHERE mypartition >= from_unixtime(\${hiveconf:myhql.time}-2592000,'yyyy-MM-dd');
Suggested feature:
\# will scan only the right partitions
$ hive -hiveconf hive.partition.evaluateonce=unix_timestamp -e "SELECT COUNT\(\*\) FROM mybigtable WHERE mypartition >= from_unixtime(unix_timestamp()-2592000,'yyyy-MM-dd');
was:
For certain situations specifying which functions should be evaluated once would help to make syntax simpler to avoid launching M/R jobs.
Example:
# myhql.time=`date "+%s"` -> constant
# counting rows from the last 30 days generates a M/R job using all the partitions
$ hive -hiveconf myhql.time=`date "+%s"` -e "SELECT COUNT(*) FROM mybigtable WHERE mypartition >= from_unixtime(\${hiveconf:myhql.time}-2592000,'yyyy-MM-dd');
Suggested feature:
# will scan only the right partitions
$ hive -hiveconf hive.partition.evaluateonce=unix_timestamp -e "SELECT COUNT(*) FROM mybigtable WHERE mypartition >= from_unixtime(unix_timestamp()-2592000,'yyyy-MM-dd');
> Specified functions in the partitioning predicates should not generate a M/R job.
> ---------------------------------------------------------------------------------
>
> Key: HIVE-2124
> URL: https://issues.apache.org/jira/browse/HIVE-2124
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
> Affects Versions: 0.5.0, 0.6.0, 0.7.0
> Reporter: Esteban Gutierrez
> Priority: Minor
> Labels: features, new
>
> For certain situations specifying which functions should be evaluated once would help to make syntax simpler to avoid launching M/R jobs.
> Example:
> \# myhql.time=`date "+%s"` -> constant
> \# counting rows from the last 30 days generates a M/R job using all the partitions
> $ hive -hiveconf myhql.time=`date "+%s"` -e "SELECT COUNT\(\*\) FROM mybigtable WHERE mypartition >= from_unixtime(\${hiveconf:myhql.time}-2592000,'yyyy-MM-dd');
> Suggested feature:
> \# will scan only the right partitions
> $ hive -hiveconf hive.partition.evaluateonce=unix_timestamp -e "SELECT COUNT\(\*\) FROM mybigtable WHERE mypartition >= from_unixtime(unix_timestamp()-2592000,'yyyy-MM-dd');
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira