[GitHub] bersprockets opened a new pull request #23617: [SPARK-26680][SQL] Eagerly create inputVars while conditions are appropriate

[GitBox <gi...@apache.org>]

bersprockets opened a new pull request #23617: [SPARK-26680][SQL] Eagerly create inputVars while conditions are appropriate
URL: https://github.com/apache/spark/pull/23617
 
 
   
   ## What changes were proposed in this pull request?
   
   When a user passes a Stream to groupBy, ```CodegenSupport.consume``` ends up lazily generating ```inputVars``` from a Stream, since the field ```output``` will be a Stream. At the time ```output.zipWithIndex.map``` is called, conditions are correct. However, by the time the map operation actually executes, conditions are no longer appropriate. The closure used by the map operation ends up using a reference to the partially created ```inputVars```. As a result, a StackOverflowError occurs.
   
   This PR ensures that ```inputVars``` is eagerly created while conditions are appropriate. It seems this was also an issue with the code path for creating ```inputVars``` from ```outputVars``` (SPARK-25767). I simply extended the solution for that code path to encompass both code paths.
   
   ## How was this patch tested?
   
   SQL unit tests
   new test
   python tests
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org