You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Edson Ramiro <er...@gmail.com> on 2014/02/27 14:40:59 UTC
Log Progress of Queries
Hi all,
I was using hive-0.11 and I used to get the query status from log files.
But I changed from 0.11.0 to 0.12.0 and, even if it's configured, hive is
not more generating the logs with the progress of the queries. Does the
query status have been disabled or may be I've misconfigured hive? These
are my configs:
<property>
<name>hive.querylog.location</name>
<value>/tmp/${user.name}</value>
<description>
Location of Hive run time structured log file
</description>
</property>
<property>
<name>hive.querylog.enable.plan.progress</name>
<value>true</value>
<description>
Whether to log the plan's progress every time a job's progress is
checked.
These logs are written to the location specified by
hive.querylog.location
</description>
</property>
This is the logging I used to get.
Counters
plan="{"queryId":"xxx_20131213115858_3699e7ff-8ff5-4dd7-91df-983b0588682b","queryType":null,"queryAttributes":{"queryString":"
insert overwrite table q7_volume_shipping_tmp select * from (
select n1.n_name as supp_nation, n2.n_name as cust_nation,
n1.n_nationkey as s_nationkey, n2.n_nationkey as c_nationkey
from nation n1 join nation n2 on n1.n_name = 'FRANCE' and
n2.n_name = 'GERMANY' UNION ALL select n1.n_name as supp_nation,
n2.n_name as cust_nation, n1.n_nationkey as s_nationkey, n2.n_nationkey
as c_nationkey from nation n1 join nation n2 on n2.n_name =
'FRANCE' and n1.n_name = 'GERMANY' )
a"},"queryCounters":"null","stageGraph":{"nodeType":"STAGE","roots":"null","adjacencyList":[{"node":"Stage-1","children":["Stage-2"],"adjacencyType":"CONJUNCTIVE"},{"node":"Stage-10","children":["Stage-2"],"adjacencyType":"CONJUNCTIVE"},{"node":"Stage-2","children":["Stage-8"],"adjacencyType":"CONJUNCTIVE"},{"node":"Stage-2","children":["Stage-8"],"adjacencyType":"CONJUNCTIVE"},{"node":"Stage-8","children":["Stage-5","Stage-4","Stage-6"],"adjacencyType":"DISJUNCTIVE"},{"node":"Stage-8","children":["Stage-5","Stage-4","Stage-6"],"adjacencyType":"DISJUNCTIVE"},{"node":"Stage-5","children":["Stage-0"],"adjacencyType":"CONJUNCTIVE"},{"node":"Stage-4","children":["Stage-
Thanks in advance,
Edson Ramiro
Re: Log Progress of Queries
Posted by Edson Ramiro <er...@gmail.com>.
Moving to dev list.
Edson Ramiro
On Thu, Feb 27, 2014 at 2:40 PM, Edson Ramiro <er...@gmail.com> wrote:
> Hi all,
>
> I was using hive-0.11 and I used to get the query status from log files.
>
> But I changed from 0.11.0 to 0.12.0 and, even if it's configured, hive is
> not more generating the logs with the progress of the queries. Does the
> query status have been disabled or may be I've misconfigured hive? These
> are my configs:
>
> <property>
> <name>hive.querylog.location</name>
> <value>/tmp/${user.name}</value>
> <description>
> Location of Hive run time structured log file
> </description>
> </property>
>
> <property>
> <name>hive.querylog.enable.plan.progress</name>
> <value>true</value>
> <description>
> Whether to log the plan's progress every time a job's progress is
> checked.
> These logs are written to the location specified by
> hive.querylog.location
> </description>
> </property>
>
> This is the logging I used to get.
>
> Counters
> plan="{"queryId":"xxx_20131213115858_3699e7ff-8ff5-4dd7-91df-983b0588682b","queryType":null,"queryAttributes":{"queryString":"
> insert overwrite table q7_volume_shipping_tmp select * from (
> select n1.n_name as supp_nation, n2.n_name as cust_nation,
> n1.n_nationkey as s_nationkey, n2.n_nationkey as c_nationkey
> from nation n1 join nation n2 on n1.n_name = 'FRANCE' and
> n2.n_name = 'GERMANY' UNION ALL select n1.n_name as supp_nation,
> n2.n_name as cust_nation, n1.n_nationkey as s_nationkey, n2.n_nationkey
> as c_nationkey from nation n1 join nation n2 on n2.n_name =
> 'FRANCE' and n1.n_name = 'GERMANY' )
> a"},"queryCounters":"null","stageGraph":{"nodeType":"STAGE","roots":"null","adjacencyList":[{"node":"Stage-1","children":["Stage-2"],"adjacencyType":"CONJUNCTIVE"},{"node":"Stage-10","children":["Stage-2"],"adjacencyType":"CONJUNCTIVE"},{"node":"Stage-2","children":["Stage-8"],"adjacencyType":"CONJUNCTIVE"},{"node":"Stage-2","children":["Stage-8"],"adjacencyType":"CONJUNCTIVE"},{"node":"Stage-8","children":["Stage-5","Stage-4","Stage-6"],"adjacencyType":"DISJUNCTIVE"},{"node":"Stage-8","children":["Stage-5","Stage-4","Stage-6"],"adjacencyType":"DISJUNCTIVE"},{"node":"Stage-5","children":["Stage-0"],"adjacencyType":"CONJUNCTIVE"},{"node":"Stage-4","children":["Stage-
>
> Thanks in advance,
>
> Edson Ramiro
>