You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@oozie.apache.org by Boris Shulman <sh...@gmail.com> on 2012/08/08 07:50:37 UTC
Hadoop counters for Hive actions
Hi,
I am trying to get statistics from a hive action similar to what can be
done for a mr or pig jobs. I am getting the following error:
java.lang.IllegalArgumentException: Hadoop counters not available for
action [hive-node]
at
org.apache.oozie.action.hadoop.HadoopELFunctions.getCounters(HadoopELFunctions.java:65)
at
org.apache.oozie.action.hadoop.HadoopELFunctions.hadoop_counters(HadoopELFunctions.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.commons.el.FunctionInvocation.evaluate(FunctionInvocation.java:172)
at org.apache.commons.el.ComplexValue.evaluate(ComplexValue.java:140)
at
org.apache.commons.el.ExpressionString.evaluate(ExpressionString.java:114)
at
org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:274)
at
org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:190)
at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:203)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:157)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:55)
at org.apache.oozie.command.XCommand.call(XCommand.java:277)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:326)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:255)
at
org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
What statistics are available for Hive action and how can I access it? Are
there any plans for making Hadoop counters available for Hive Action?
Boris.
Re: Hadoop counters for Hive actions
Posted by Mohammad Islam <mi...@yahoo.com>.
Hi Boris,
To the best of my knowledge, Job counters/stats generated by hive is not available through oozie API. Oozie doesn't store those too. It could be a new feature.
Pig and MR are already supported, as you mentioned.
Regards,
Mohammad
________________________________
From: Boris Shulman <sh...@gmail.com>
To: oozie-users@incubator.apache.org; oozie-dev@incubator.apache.org
Sent: Tuesday, August 7, 2012 10:50 PM
Subject: Hadoop counters for Hive actions
Hi,
I am trying to get statistics from a hive action similar to what can be
done for a mr or pig jobs. I am getting the following error:
java.lang.IllegalArgumentException: Hadoop counters not available for
action [hive-node]
at
org.apache.oozie.action.hadoop.HadoopELFunctions.getCounters(HadoopELFunctions.java:65)
at
org.apache.oozie.action.hadoop.HadoopELFunctions.hadoop_counters(HadoopELFunctions.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.commons.el.FunctionInvocation.evaluate(FunctionInvocation.java:172)
at org.apache.commons.el.ComplexValue.evaluate(ComplexValue.java:140)
at
org.apache.commons.el.ExpressionString.evaluate(ExpressionString.java:114)
at
org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:274)
at
org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:190)
at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:203)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:157)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:55)
at org.apache.oozie.command.XCommand.call(XCommand.java:277)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:326)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:255)
at
org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
What statistics are available for Hive action and how can I access it? Are
there any plans for making Hadoop counters available for Hive Action?
Boris.
Re: Hadoop counters for Hive actions
Posted by Mohammad Islam <mi...@yahoo.com>.
Hi Boris,
To the best of my knowledge, Job counters/stats generated by hive is not available through oozie API. Oozie doesn't store those too. It could be a new feature.
Pig and MR are already supported, as you mentioned.
Regards,
Mohammad
________________________________
From: Boris Shulman <sh...@gmail.com>
To: oozie-users@incubator.apache.org; oozie-dev@incubator.apache.org
Sent: Tuesday, August 7, 2012 10:50 PM
Subject: Hadoop counters for Hive actions
Hi,
I am trying to get statistics from a hive action similar to what can be
done for a mr or pig jobs. I am getting the following error:
java.lang.IllegalArgumentException: Hadoop counters not available for
action [hive-node]
at
org.apache.oozie.action.hadoop.HadoopELFunctions.getCounters(HadoopELFunctions.java:65)
at
org.apache.oozie.action.hadoop.HadoopELFunctions.hadoop_counters(HadoopELFunctions.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.commons.el.FunctionInvocation.evaluate(FunctionInvocation.java:172)
at org.apache.commons.el.ComplexValue.evaluate(ComplexValue.java:140)
at
org.apache.commons.el.ExpressionString.evaluate(ExpressionString.java:114)
at
org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:274)
at
org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:190)
at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:203)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:157)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:55)
at org.apache.oozie.command.XCommand.call(XCommand.java:277)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:326)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:255)
at
org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
What statistics are available for Hive action and how can I access it? Are
there any plans for making Hadoop counters available for Hive Action?
Boris.