You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Navis Ryu <na...@nexr.com> on 2015/08/10 10:43:06 UTC
Review Request 37287: HIVE-11506 Casting varchar/char type to string
cannot be vectorized
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37287/
-----------------------------------------------------------
Review request for hive.
Bugs: HIVE-11506
https://issues.apache.org/jira/browse/HIVE-11506
Repository: hive-git
Description
-------
It's not defined in vectorization context.
{code}
explain
select cast(cast(cstring1 as varchar(10)) as string) x from alltypesorc order by x;
{code}
Mapper is not vectorized by exception,
{noformat}
015-08-10 17:02:08,003 INFO [main]: physical.Vectorizer (Vectorizer.java:validateExprNodeDesc(1299)) - Failed to vectorize
org.apache.hadoop.hive.ql.metadata.HiveException: Unhandled cast input type: varchar(10)
at org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getCastToString(VectorizationContext.java:1543)
at org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getGenericUDFBridgeVectorExpression(VectorizationContext.java:1379)
at org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getGenericUdfVectorExpression(VectorizationContext.java:1177)
at org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:440)
at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateExprNodeDesc(Vectorizer.java:1293)
at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateExprNodeDesc(Vectorizer.java:1284)
at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateSelectOperator(Vectorizer.java:1116)
at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateMapWorkOperator(Vectorizer.java:906)
{noformat}
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java 5b702bb
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/CastStringGroupToString.java PRE-CREATION
ql/src/test/queries/clientpositive/vectorized_casts.q ade3645
ql/src/test/results/clientpositive/tez/vectorized_casts.q.out 1dd07b1
ql/src/test/results/clientpositive/vectorized_casts.q.out 1113453
Diff: https://reviews.apache.org/r/37287/diff/
Testing
-------
Thanks,
Navis Ryu
Re: Review Request 37287: HIVE-11506 Casting varchar/char type to
string cannot be vectorized
Posted by Swarnim Kulkarni <ku...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37287/#review94839
-----------------------------------------------------------
Ship it!
Ship It!
- Swarnim Kulkarni
On Aug. 11, 2015, 12:37 a.m., Navis Ryu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37287/
> -----------------------------------------------------------
>
> (Updated Aug. 11, 2015, 12:37 a.m.)
>
>
> Review request for hive.
>
>
> Bugs: HIVE-11506
> https://issues.apache.org/jira/browse/HIVE-11506
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> It's not defined in vectorization context.
> {code}
> explain
> select cast(cast(cstring1 as varchar(10)) as string) x from alltypesorc order by x;
> {code}
>
> Mapper is not vectorized by exception,
> {noformat}
> 015-08-10 17:02:08,003 INFO [main]: physical.Vectorizer (Vectorizer.java:validateExprNodeDesc(1299)) - Failed to vectorize
> org.apache.hadoop.hive.ql.metadata.HiveException: Unhandled cast input type: varchar(10)
> at org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getCastToString(VectorizationContext.java:1543)
> at org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getGenericUDFBridgeVectorExpression(VectorizationContext.java:1379)
> at org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getGenericUdfVectorExpression(VectorizationContext.java:1177)
> at org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:440)
> at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateExprNodeDesc(Vectorizer.java:1293)
> at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateExprNodeDesc(Vectorizer.java:1284)
> at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateSelectOperator(Vectorizer.java:1116)
> at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateMapWorkOperator(Vectorizer.java:906)
> {noformat}
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java 5b702bb
> ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/CastStringGroupToString.java PRE-CREATION
> ql/src/test/queries/clientpositive/vectorized_casts.q ade3645
> ql/src/test/results/clientpositive/tez/vector_char_mapjoin1.q.out f495f95
> ql/src/test/results/clientpositive/tez/vector_varchar_mapjoin1.q.out ae5be44
> ql/src/test/results/clientpositive/tez/vectorized_casts.q.out 1dd07b1
> ql/src/test/results/clientpositive/vector_char_mapjoin1.q.out 744bfb3
> ql/src/test/results/clientpositive/vector_varchar_mapjoin1.q.out 2994bff
> ql/src/test/results/clientpositive/vectorized_casts.q.out 1113453
>
> Diff: https://reviews.apache.org/r/37287/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Navis Ryu
>
>
Re: Review Request 37287: HIVE-11506 Casting varchar/char type to
string cannot be vectorized
Posted by Navis Ryu <na...@nexr.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37287/
-----------------------------------------------------------
(Updated 8 11, 2015, 12:37 오전)
Review request for hive.
Changes
-------
Updated golden files
Bugs: HIVE-11506
https://issues.apache.org/jira/browse/HIVE-11506
Repository: hive-git
Description
-------
It's not defined in vectorization context.
{code}
explain
select cast(cast(cstring1 as varchar(10)) as string) x from alltypesorc order by x;
{code}
Mapper is not vectorized by exception,
{noformat}
015-08-10 17:02:08,003 INFO [main]: physical.Vectorizer (Vectorizer.java:validateExprNodeDesc(1299)) - Failed to vectorize
org.apache.hadoop.hive.ql.metadata.HiveException: Unhandled cast input type: varchar(10)
at org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getCastToString(VectorizationContext.java:1543)
at org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getGenericUDFBridgeVectorExpression(VectorizationContext.java:1379)
at org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getGenericUdfVectorExpression(VectorizationContext.java:1177)
at org.apache.hadoop.hive.ql.exec.vector.VectorizationContext.getVectorExpression(VectorizationContext.java:440)
at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateExprNodeDesc(Vectorizer.java:1293)
at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateExprNodeDesc(Vectorizer.java:1284)
at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateSelectOperator(Vectorizer.java:1116)
at org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.validateMapWorkOperator(Vectorizer.java:906)
{noformat}
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java 5b702bb
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/CastStringGroupToString.java PRE-CREATION
ql/src/test/queries/clientpositive/vectorized_casts.q ade3645
ql/src/test/results/clientpositive/tez/vector_char_mapjoin1.q.out f495f95
ql/src/test/results/clientpositive/tez/vector_varchar_mapjoin1.q.out ae5be44
ql/src/test/results/clientpositive/tez/vectorized_casts.q.out 1dd07b1
ql/src/test/results/clientpositive/vector_char_mapjoin1.q.out 744bfb3
ql/src/test/results/clientpositive/vector_varchar_mapjoin1.q.out 2994bff
ql/src/test/results/clientpositive/vectorized_casts.q.out 1113453
Diff: https://reviews.apache.org/r/37287/diff/
Testing
-------
Thanks,
Navis Ryu