You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Raajay <ra...@gmail.com> on 2015/08/10 17:48:25 UTC
Hive CBO - Calcite Interface
For a research project, I am studying the performance gains in Hive queries
achieved through cost based optimization. Rather than just study scenarios
where CBO is switched on or off, I would like to study the performance
across a “collection" of query plans considered by the CBO (Calcite); for
example, the performance of the query plan with the least cost vs the
performance of query plan with next higher cost. Performance here is the
total completion time.
1. Currently, Hive gets only the least cost plan from Calcite. Is there an
interface to get alternate plans ? Say, all plans (read Operator trees)
with cost lesser than a threshold.
2. Is there an interface for Hive to get the absolute cost (based on Hive
Cost Factory) of a operator tree returned by Calcite ?
Thanks,
Raajay
Re: Hive CBO - Calcite Interface
Posted by Sergey Shelukhin <se...@hortonworks.com>.
You can also take a look at https://issues.apache.org/jira/browse/HIVE-11471 (although there’s no patch yet).
From: John Pullokkaran <jp...@hortonworks.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Friday, August 14, 2015 at 12:11
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>, "dev@hive.apache.org<ma...@hive.apache.org>" <de...@hive.apache.org>>
Subject: Re: Hive CBO - Calcite Interface
Hi Raajay,
#1 No, there is no API for this.
#2 If you enable Logging (BaseSemanticAnalyzer) then CalcitePlanner will print out the plan with cost.
John
From: Raajay <ra...@gmail.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Monday, August 10, 2015 at 8:48 AM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>, "dev@hive.apache.org<ma...@hive.apache.org>" <de...@hive.apache.org>>
Subject: Hive CBO - Calcite Interface
nterface for Hive to get the absolute cost (based on Hive Cost Factory) of a operator tree returned by Calcite ?
Re: Hive CBO - Calcite Interface
Posted by Sergey Shelukhin <se...@hortonworks.com>.
You can also take a look at https://issues.apache.org/jira/browse/HIVE-11471 (although there’s no patch yet).
From: John Pullokkaran <jp...@hortonworks.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Friday, August 14, 2015 at 12:11
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>, "dev@hive.apache.org<ma...@hive.apache.org>" <de...@hive.apache.org>>
Subject: Re: Hive CBO - Calcite Interface
Hi Raajay,
#1 No, there is no API for this.
#2 If you enable Logging (BaseSemanticAnalyzer) then CalcitePlanner will print out the plan with cost.
John
From: Raajay <ra...@gmail.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Monday, August 10, 2015 at 8:48 AM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>, "dev@hive.apache.org<ma...@hive.apache.org>" <de...@hive.apache.org>>
Subject: Hive CBO - Calcite Interface
nterface for Hive to get the absolute cost (based on Hive Cost Factory) of a operator tree returned by Calcite ?
Re: Hive CBO - Calcite Interface
Posted by John Pullokkaran <jp...@hortonworks.com>.
Hi Raajay,
#1 No, there is no API for this.
#2 If you enable Logging (BaseSemanticAnalyzer) then CalcitePlanner will print out the plan with cost.
John
From: Raajay <ra...@gmail.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Monday, August 10, 2015 at 8:48 AM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>, "dev@hive.apache.org<ma...@hive.apache.org>" <de...@hive.apache.org>>
Subject: Hive CBO - Calcite Interface
nterface for Hive to get the absolute cost (based on Hive Cost Factory) of a operator tree returned by Calcite ?
Re: Hive CBO - Calcite Interface
Posted by John Pullokkaran <jp...@hortonworks.com>.
Hi Raajay,
#1 No, there is no API for this.
#2 If you enable Logging (BaseSemanticAnalyzer) then CalcitePlanner will print out the plan with cost.
John
From: Raajay <ra...@gmail.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Monday, August 10, 2015 at 8:48 AM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>, "dev@hive.apache.org<ma...@hive.apache.org>" <de...@hive.apache.org>>
Subject: Hive CBO - Calcite Interface
nterface for Hive to get the absolute cost (based on Hive Cost Factory) of a operator tree returned by Calcite ?