You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Jesus Camacho Rodriguez (JIRA)" <ji...@apache.org> on 2016/08/04 08:37:21 UTC
[jira] [Commented] (HIVE-14384) CBO: Decimal constant folding is
failing
[ https://issues.apache.org/jira/browse/HIVE-14384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15407421#comment-15407421 ]
Jesus Camacho Rodriguez commented on HIVE-14384:
------------------------------------------------
Error seems to be in Calcite, in the method {{makeExactLiteral(BigDecimal)}} in RexBuilder. Basically, it will infer the incorrect precision when decimal < 1 e.g. for 0.06 it infers the type to be Decimal(1,2) instead of Decimal(3,2).
{code:java}
/**
* Creates a numeric literal.
*/
public RexLiteral makeExactLiteral(BigDecimal bd) {
RelDataType relType;
int scale = bd.scale();
long l = bd.unscaledValue().longValue();
assert scale >= 0;
assert scale <= typeFactory.getTypeSystem().getMaxNumericScale() : scale;
assert BigDecimal.valueOf(l, scale).equals(bd);
if (scale == 0) {
if ((l >= Integer.MIN_VALUE) && (l <= Integer.MAX_VALUE)) {
relType = typeFactory.createSqlType(SqlTypeName.INTEGER);
} else {
relType = typeFactory.createSqlType(SqlTypeName.BIGINT);
}
} else {
int precision = bd.unscaledValue().abs().toString().length();
relType =
typeFactory.createSqlType(SqlTypeName.DECIMAL, precision, scale);
}
return makeExactLiteral(bd, relType);
}
{code}
> CBO: Decimal constant folding is failing
> ----------------------------------------
>
> Key: HIVE-14384
> URL: https://issues.apache.org/jira/browse/HIVE-14384
> Project: Hive
> Issue Type: Bug
> Components: CBO
> Affects Versions: 2.2.0
> Reporter: Gopal V
>
> {code}
> explain select sum(l_extendedprice * l_discount) as revenue from lineitem where l_shipdate >= '1993-01-01' and l_shipdate < '1994-01-01' and l_discount between 0.06 - 0.01 and 0.06 + 0.01 and l_quantity < 25;
> {code}
> Fails CBO because of constant folding errors.
> {code}
> 2016-07-29T17:15:50,921 ERROR [0b3f62eb-8a80-40cd-8cf2-60bc835191a8 main] parse.CalcitePlanner: CBO failed, skipping CBO.
> java.lang.IllegalArgumentException: Decimal scale must be less than or equal to precision
> at org.apache.hadoop.hive.serde2.typeinfo.HiveDecimalUtils.validateParameter(HiveDecimalUtils.java:53) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.hadoop.hive.serde2.typeinfo.DecimalTypeInfo.<init>(DecimalTypeInfo.java:36) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory.createPrimitiveTypeInfo(TypeInfoFactory.java:157) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory.getPrimitiveTypeInfo(TypeInfoFactory.java:109) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory.getDecimalTypeInfo(TypeInfoFactory.java:175) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.optimizer.calcite.translator.ExprNodeConverter.visitLiteral(ExprNodeConverter.java:259) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.optimizer.calcite.translator.ExprNodeConverter.visitLiteral(ExprNodeConverter.java:82) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.calcite.rex.RexLiteral.accept(RexLiteral.java:657) ~[calcite-core-1.6.0.jar:1.6.0]
> at org.apache.hadoop.hive.ql.optimizer.calcite.translator.ExprNodeConverter.visitCall(ExprNodeConverter.java:144) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.optimizer.calcite.translator.ExprNodeConverter.visitCall(ExprNodeConverter.java:82) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:108) ~[calcite-core-1.6.0.jar:1.6.0]
> at org.apache.hadoop.hive.ql.optimizer.calcite.HiveRexExecutorImpl.reduce(HiveRexExecutorImpl.java:58) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveReduceExpressionsRule.reduceExpressionsInternal(HiveReduceExpressionsRule.java:376) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveReduceExpressionsRule.reduceExpressions(HiveReduceExpressionsRule.java:286) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveReduceExpressionsRule$FilterReduceExpressionsRule.onMatch(HiveReduceExpressionsRule.java:141) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:318) ~[calcite-core-1.6.0.jar:1.6.0]
> at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:514) ~[calcite-core-1.6.0.jar:1.6.0]
> at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:392) ~[calcite-core-1.6.0.jar:1.6.0]
> at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:285) ~[calcite-core-1.6.0.jar:1.6.0]
> at org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute(HepInstruction.java:72) ~[calcite-core-1.6.0.jar:1.6.0]
> at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:207) ~[calcite-core-1.6.0.jar:1.6.0]
> at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:194) ~[calcite-core-1.6.0.jar:1.6.0]
> at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.hepPlan(CalcitePlanner.java:1320) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.applyPreJoinOrderingTransforms(CalcitePlanner.java:1191) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:972) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)