You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Harish Jaiprakash (JIRA)" <ji...@apache.org> on 2017/10/18 12:07:00 UTC
[jira] [Comment Edited] (HIVE-17771) Implement commands to manage
resource plan.
[ https://issues.apache.org/jira/browse/HIVE-17771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16209205#comment-16209205 ]
Harish Jaiprakash edited comment on HIVE-17771 at 10/18/17 12:06 PM:
---------------------------------------------------------------------
[~sershe], While executing "ALTER RESOURCE PLAN plan_1 RENAME TO plan_3", I'm getting this exception:
2017-10-18T04:54:24,779 ERROR [18742cff-1694-4358-a9f7-ccc3113ac855 main] ql.Driver: FAILED: NullPointerException null
java.lang.NullPointerException
at org.apache.hadoop.hive.ql.stats.StatsUtils.estimateRowSizeFromSchema(StatsUtils.java:777)
at org.apache.hadoop.hive.ql.stats.StatsUtils.getNumRows(StatsUtils.java:308)
at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:346)
at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:161)
at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:147)
at org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$TableScanStatsRule.process(StatsRulesProcFactory.java:134)
at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
at org.apache.hadoop.hive.ql.lib.LevelOrderWalker.walk(LevelOrderWalker.java:143)
at org.apache.hadoop.hive.ql.lib.LevelOrderWalker.startWalking(LevelOrderWalker.java:122)
at org.apache.hadoop.hive.ql.optimizer.stats.annotation.AnnotateWithStatistics.transform(AnnotateWithStatistics.java:78)
at org.apache.hadoop.hive.ql.parse.TezCompiler.runStatsAnnotation(TezCompiler.java:413)
at org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:150)
at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:145)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11621)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11042)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:267)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:597)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1461)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1593)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1360)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1350)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:409)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:342)
at org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1287)
at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1261)
at org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:173)
at org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)
at org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver(TestMiniLlapLocalCliDriver.java:59)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:92)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
Looked up online, it shows up as build issue. Tried doing a clean install after removing .m2/repository/org/apache/hive, still same issue. Submitting patch to apache build to see if it builds. Please advise if any fix is known.
was (Author: harishjp):
[~sershe], I'm getting this exception:
2017-10-18T04:54:24,779 ERROR [18742cff-1694-4358-a9f7-ccc3113ac855 main] ql.Driver: FAILED: NullPointerException null
java.lang.NullPointerException
at org.apache.hadoop.hive.ql.stats.StatsUtils.estimateRowSizeFromSchema(StatsUtils.java:777)
at org.apache.hadoop.hive.ql.stats.StatsUtils.getNumRows(StatsUtils.java:308)
at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:346)
at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:161)
at org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:147)
at org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$TableScanStatsRule.process(StatsRulesProcFactory.java:134)
at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
at org.apache.hadoop.hive.ql.lib.LevelOrderWalker.walk(LevelOrderWalker.java:143)
at org.apache.hadoop.hive.ql.lib.LevelOrderWalker.startWalking(LevelOrderWalker.java:122)
at org.apache.hadoop.hive.ql.optimizer.stats.annotation.AnnotateWithStatistics.transform(AnnotateWithStatistics.java:78)
at org.apache.hadoop.hive.ql.parse.TezCompiler.runStatsAnnotation(TezCompiler.java:413)
at org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:150)
at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:145)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11621)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11042)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:267)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:597)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1461)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1593)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1360)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1350)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:409)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:342)
at org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1287)
at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1261)
at org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:173)
at org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)
at org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver(TestMiniLlapLocalCliDriver.java:59)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:92)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
Looked up online, it shows up as build issue. Tried doing a clean install after removing .m2/repository/org/apache/hive. Still same issue.
> Implement commands to manage resource plan.
> -------------------------------------------
>
> Key: HIVE-17771
> URL: https://issues.apache.org/jira/browse/HIVE-17771
> Project: Hive
> Issue Type: Sub-task
> Reporter: Harish Jaiprakash
> Assignee: Harish Jaiprakash
> Attachments: HIVE-17771.01.patch, HIVE-17771.02.patch
>
>
> Please see parent jira about llap workload management.
> This jira is to implement create and show resource plan commands in hive to configure resource plans for llap workload. The following are the proposed commands implemented as part of the jira:
> CREATE RESOURCE PLAN plan_name WITH QUERY_PARALLELISM parallelism;
> SHOW RESOURCE PLAN plan_name;
> SHOW RESOURCE PLANS;
> ALTER RESOURCE PLAN plan_name SET QUERY_PARALLELISM = parallelism;
> ALTER RESOURCE PLAN plan_name RENAME TO new_name;
> ALTER RESOURCE PLAN plan_name ACTIVATE;
> ALTER RESOURCE PLAN plan_name DISABLE;
> ALTER RESOURCE PLAN plan_name ENABLE;
> DROP RESOURCE PLAN;
> It will be followed up with more jiras to manage pools, triggers and copy resource plans.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)