You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Bogdan Raducanu (JIRA)" <ji...@apache.org> on 2017/02/08 10:23:41 UTC
[jira] [Updated] (SPARK-19512) codegen for compare structs fails
[ https://issues.apache.org/jira/browse/SPARK-19512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bogdan Raducanu updated SPARK-19512:
------------------------------------
Description:
This (1 struct field)
{code:java|title=1 struct field}
spark.range(10)
.selectExpr("named_struct('a', id) as col1", "named_struct('a', id+2) as col2")
.filter("col1 = col2").count
{code}
fails with
{code}
[info] Cause: java.util.concurrent.ExecutionException: java.lang.Exception: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 144, Column 32: Expression "range_value" is not an rvalue
{code}
This (2 struct fields)
{code:java|title=2 struct fields}
spark.range(10).selectExpr("named_struct('a', id, 'b', id) as col1", "named_struct('a',id+2, 'b',id+2) as col2").filter($"col1" === $"col2").count
{code}
fails with
{code}
Caused by: java.lang.IndexOutOfBoundsException: 1
at scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
at scala.collection.immutable.List.apply(List.scala:84)
at org.apache.spark.sql.catalyst.expressions.BoundReference.doGenCode(BoundAttribute.scala:64)
{code}
was:
This (1 struct field)
{code:scala|title=1 struct field}
spark.range(10)
.selectExpr("named_struct('a', id) as col1", "named_struct('a', id+2) as col2")
.filter("col1 = col2").count
{code}
fails with
{code}
[info] Cause: java.util.concurrent.ExecutionException: java.lang.Exception: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 144, Column 32: Expression "range_value" is not an rvalue
{code}
This (2 struct fields)
{code:scala|title=2 struct fields}
spark.range(10).selectExpr("named_struct('a', id, 'b', id) as col1", "named_struct('a',id+2, 'b',id+2) as col2").filter($"col1" === $"col2").count
{code}
fails with
{code}
Caused by: java.lang.IndexOutOfBoundsException: 1
at scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
at scala.collection.immutable.List.apply(List.scala:84)
at org.apache.spark.sql.catalyst.expressions.BoundReference.doGenCode(BoundAttribute.scala:64)
{code}
> codegen for compare structs fails
> ---------------------------------
>
> Key: SPARK-19512
> URL: https://issues.apache.org/jira/browse/SPARK-19512
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.0.0, 2.1.0
> Reporter: Bogdan Raducanu
>
> This (1 struct field)
> {code:java|title=1 struct field}
> spark.range(10)
> .selectExpr("named_struct('a', id) as col1", "named_struct('a', id+2) as col2")
> .filter("col1 = col2").count
> {code}
> fails with
> {code}
> [info] Cause: java.util.concurrent.ExecutionException: java.lang.Exception: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 144, Column 32: Expression "range_value" is not an rvalue
> {code}
> This (2 struct fields)
> {code:java|title=2 struct fields}
> spark.range(10).selectExpr("named_struct('a', id, 'b', id) as col1", "named_struct('a',id+2, 'b',id+2) as col2").filter($"col1" === $"col2").count
> {code}
> fails with
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: 1
> at scala.collection.LinearSeqOptimized$class.apply(LinearSeqOptimized.scala:65)
> at scala.collection.immutable.List.apply(List.scala:84)
> at org.apache.spark.sql.catalyst.expressions.BoundReference.doGenCode(BoundAttribute.scala:64)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org