[jira] [Commented] (SPARK-20111) codegen bug surfaced by GraphFrames issue 165

["Timothy Hunter (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-20111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15944004#comment-15944004 ] 

Timothy Hunter commented on SPARK-20111:
----------------------------------------

As Spark SQL is making more and more forays into code generation, I have been wondering if it would make sense to start adopting practical compiler technologies, such as generating first an intermediate representation, instead of doing string manipulation as we currently do. This is of course much beyond the scope of this particular ticket.

> codegen bug surfaced by GraphFrames issue 165
> ---------------------------------------------
>
>                 Key: SPARK-20111
>                 URL: https://issues.apache.org/jira/browse/SPARK-20111
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.0.2, 2.1.0, 2.2.0
>            Reporter: Joseph K. Bradley
>
> In GraphFrames, test {{test("named edges")}} in {{PatternMatchSuite}} surfaces a SQL codegen bug.
> This is described in https://github.com/graphframes/graphframes/issues/165
> Summary
> * The unit test does a simple motif query on a graph.  Essentially, this means taking 2 DataFrames, doing a few joins, selecting 2 columns, and collecting the (tiny) DataFrame.
> * The test runs, but codegen fails.  See the linked GraphFrames issue for the stacktrace.
> To reproduce this:
> * Check out GraphFrames https://github.com/graphframes/graphframes
> * Run {{sbt assembly}} to compile it and run tests
> Copying [~felixcheung]'s comment from the GraphFrames issue 165:
> {quote}
> Seems like codegen bug; it looks like at least 2 issues:
> 1. At L472, inputadapter_value is not defined within scope
> 2. inputadapter_value is an InternalRow, for this statement to work
> {{bhj_primitiveA = inputadapter_value;}}
> it should be
> {{bhj_primitiveA = inputadapter_value.getLong(0);}}
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org