You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Gopal V (JIRA)" <ji...@apache.org> on 2014/07/12 07:31:05 UTC
[jira] [Commented] (HIVE-7396) BucketingSortingReduceSinkOptimizer
throws NullPointException during ETL
[ https://issues.apache.org/jira/browse/HIVE-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059657#comment-14059657 ]
Gopal V commented on HIVE-7396:
-------------------------------
This is not just a simple NULL check missing.
I made modifications to test this out
{code}
--- a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketingSortingReduceSinkOptimizer.java
+++ b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketingSortingReduceSinkOptimizer.java
@@ -498,7 +498,7 @@ public Object process(Node nd, Stack<Node> stack, NodeProcessorCtx procCtx,
if (srcTable.isPartitioned()) {
PrunedPartitionList prunedParts = pGraphContext.getOpToPartList().get(ts);
- List<Partition> partitions = prunedParts.getNotDeniedPartns();
+ List<Partition> partitions = (prunedParts != null ) ? prunedParts.getNotDeniedPartns() : null;
// Support for dynamic partitions can be added later
// The following is not optimized:
{code}
But then it generates a scan on only one partition instead of a full-table scan.
> BucketingSortingReduceSinkOptimizer throws NullPointException during ETL
> ------------------------------------------------------------------------
>
> Key: HIVE-7396
> URL: https://issues.apache.org/jira/browse/HIVE-7396
> Project: Hive
> Issue Type: Bug
> Reporter: Gopal V
> Priority: Trivial
>
> When a bucketed insert is performed from a partitioned table into a new table
> {code}
> 2014-07-11 22:11:30,176 ERROR [main]: ql.Driver (SessionState.java:printError(660)) - FAILED: NullPointerException null
> java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.optimizer.BucketingSortingReduceSinkOptimizer$BucketSortReduceSinkProcessor.process(BucketingSortingReduceSinkOptimizer.java:501)
> at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
> at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
> at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:78)
> at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:132)
> at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:109)
> at org.apache.hadoop.hive.ql.optimizer.BucketingSortingReduceSinkOptimizer.transform(BucketingSortingReduceSinkOptimizer.java:97)
> at org.apache.hadoop.hive.ql.optimizer.Optimizer.optimize(Optimizer.java:146)
> at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9503)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:328)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)