You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "slim bouguerra (JIRA)" <ji...@apache.org> on 2018/02/17 06:00:00 UTC
[jira] [Resolved] (HIVE-15641) Hive/Druid integration: filter on
timestamp not pushed to DruidQuery
[ https://issues.apache.org/jira/browse/HIVE-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
slim bouguerra resolved HIVE-15641.
-----------------------------------
Resolution: Fixed
> Hive/Druid integration: filter on timestamp not pushed to DruidQuery
> --------------------------------------------------------------------
>
> Key: HIVE-15641
> URL: https://issues.apache.org/jira/browse/HIVE-15641
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
> Affects Versions: 2.2.0
> Reporter: Jesus Camacho Rodriguez
> Assignee: Jesus Camacho Rodriguez
> Priority: Major
>
> It seems we are missing opportunity push Filter operation to DruidQuery.
> For instance, for the following query:
> {code:sql}
> EXPLAIN
> SELECT i_brand_id, floor_day(`__time`), max(ss_quantity), sum(ss_wholesale_cost) as s
> FROM store_sales_sold_time_subset
> WHERE floor_day(`__time`) BETWEEN '1999-11-01 00:00:00' AND '1999-11-10 00:00:00'
> GROUP BY i_brand_id, floor_day(`__time`)
> ORDER BY s
> LIMIT 10;
> OK
> Plan optimized by CBO.
> Vertex dependency in root stage
> Reducer 2 <- Map 1 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> Stage-0
> Fetch Operator
> limit:10
> Stage-1
> Reducer 3 vectorized
> File Output Operator [FS_17]
> Limit [LIM_16] (rows=1 width=0)
> Number of rows:10
> Select Operator [SEL_15] (rows=1 width=0)
> Output:["_col0","_col1","_col2","_col3"]
> <-Reducer 2 [SIMPLE_EDGE] vectorized
> SHUFFLE [RS_14]
> Group By Operator [GBY_13] (rows=1 width=0)
> Output:["_col0","_col1","_col2","_col3"],aggregations:["max(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0, KEY._col1
> <-Map 1 [SIMPLE_EDGE]
> SHUFFLE [RS_5]
> PartitionCols:_col0, _col1
> Group By Operator [GBY_4] (rows=1 width=0)
> Output:["_col0","_col1","_col2","_col3"],aggregations:["max(_col2)","sum(_col3)"],keys:_col0, _col1
> Select Operator [SEL_2] (rows=1 width=0)
> Output:["_col0","_col1","_col2","_col3"]
> Filter Operator [FIL_12] (rows=1 width=0)
> predicate:floor_day(__time) BETWEEN '1999-11-01 00:00:00' AND '1999-11-10 00:00:00'
> TableScan [TS_0] (rows=15888 width=0)
> tpcds_druid_10@store_sales_sold_time_subset,store_sales_sold_time_subset,Tbl:PARTIAL,Col:NONE,Output:["__time","i_brand_id","ss_quantity","ss_wholesale_cost"],properties:{"druid.query.json":"{\"queryType\":\"select\",\"dataSource\":\"druid_tpcds_ss_sold_time_subset\",\"descending\":false,\"intervals\":[\"1900-01-01T00:00:00.000Z/3000-01-01T00:00:00.000Z\"],\"dimensions\":[\"i_item_id\",\"i_rec_start_date\",\"i_rec_end_date\",\"i_item_desc\",\"i_brand_id\",\"i_brand\",\"i_class_id\",\"i_class\",\"i_category_id\",\"i_category\",\"i_manufact_id\",\"i_manufact\",\"i_size\",\"i_formulation\",\"i_color\",\"i_units\",\"i_container\",\"i_manager_id\",\"i_product_name\",\"c_customer_id\",\"c_salutation\",\"c_first_name\",\"c_last_name\",\"c_preferred_cust_flag\",\"c_birth_day\",\"c_birth_month\",\"c_birth_year\",\"c_birth_country\",\"c_login\",\"c_email_address\",\"c_last_review_date\",\"ca_address_id\",\"ca_street_number\",\"ca_street_name\",\"ca_street_type\",\"ca_suite_number\",\"ca_city\",\"ca_county\",\"ca_state\",\"ca_zip\",\"ca_country\",\"ca_gmt_offset\",\"ca_location_type\",\"s_store_id\",\"s_rec_start_date\",\"s_rec_end_date\",\"s_store_name\",\"s_hours\",\"s_manager\",\"s_market_id\",\"s_geography_class\",\"s_market_desc\",\"s_market_manager\",\"s_division_id\",\"s_division_name\",\"s_company_id\",\"s_company_name\",\"s_street_number\",\"s_street_name\",\"s_street_type\",\"s_suite_number\",\"s_city\",\"s_county\",\"s_state\",\"s_zip\",\"s_country\",\"s_gmt_offset\"],\"metrics\":[\"ss_ticket_number\",\"ss_quantity\",\"ss_wholesale_cost\",\"ss_list_price\",\"ss_sales_price\",\"ss_ext_discount_amt\",\"ss_ext_sales_price\",\"ss_ext_wholesale_cost\",\"ss_ext_list_price\",\"ss_ext_tax\",\"ss_coupon_amt\",\"ss_net_paid\",\"ss_net_paid_inc_tax\",\"ss_net_profit\",\"i_current_price\",\"i_wholesale_cost\",\"s_number_employees\",\"s_floor_space\",\"s_tax_precentage\"],\"granularity\":\"all\",\"pagingSpec\":{\"threshold\":16384},\"context\":{\"druid.query.fetch\":false}}","druid.query.type":"select"}
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)