[spark] branch branch-3.2 updated: [SPARK-39184][SQL][FOLLOWUP] Make interpreted and codegen paths for date/timestamp sequences the same

[ma...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

maxgekk pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.2 by this push:
     new 19e90859790 [SPARK-39184][SQL][FOLLOWUP] Make interpreted and codegen paths for date/timestamp sequences the same
19e90859790 is described below

commit 19e908597907a458c317c3472b9ceb89dff94c2a
Author: Bruce Robbins <be...@gmail.com>
AuthorDate: Mon Aug 22 19:17:26 2022 +0300

    [SPARK-39184][SQL][FOLLOWUP] Make interpreted and codegen paths for date/timestamp sequences the same
    
    ### What changes were proposed in this pull request?
    
    Change how the length of the new result array is calculated in `InternalSequenceBase.eval` to match how the same is calculated in the generated code.
    
    ### Why are the changes needed?
    
    This change brings the interpreted mode code in line with the generated code. Although I am not aware of any case where the current interpreted mode code fails, the generated code is more correct (it handles the case where the result array must grow more than once, whereas the current interpreted mode code does not).
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Existing unit tests.
    
    Closes #37542 from bersprockets/date_sequence_array_size_issue_follow_up.
    
    Authored-by: Bruce Robbins <be...@gmail.com>
    Signed-off-by: Max Gekk <ma...@gmail.com>
    (cherry picked from commit d718867a16754c62cb8c30a750485f4856481efc)
    Signed-off-by: Max Gekk <ma...@gmail.com>
---
 .../apache/spark/sql/catalyst/expressions/collectionOperations.scala    | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
index 361137ad91b..db45a00086c 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
@@ -2936,7 +2936,7 @@ object Sequence {
           // "spring forward" without a corresponding "fall back", make a copy
           // that's larger by 1
           if (i == arr.length) {
-            arr = arr.padTo(estimatedArrayLength + 1, fromLong(0L))
+            arr = arr.padTo(i + 1, fromLong(0L))
           }
           arr(i) = fromLong(result)
           i += 1


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org