(spark) branch master updated: [SPARK-46509][CORE][SS] Replace `.reverse.find` with `.findLast`

[do...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 9671abbc1dc9 [SPARK-46509][CORE][SS] Replace `.reverse.find` with `.findLast`
9671abbc1dc9 is described below

commit 9671abbc1dc997f1d88a9f62c05a9d799dae8139
Author: yangjie01 <ya...@baidu.com>
AuthorDate: Fri Dec 29 16:52:23 2023 -0800

    [SPARK-46509][CORE][SS] Replace `.reverse.find` with `.findLast`
    
    ### What changes were proposed in this pull request?
    This pr aims to replace `.reverse.find` with `.findLast` in Spark code.
    
    ### Why are the changes needed?
    1. The `.findLast` seems more concise.
    2. The `.reverse` involves a copy of the collection, while `StringOps/ArrayOps.findLast` wraps the input data into a `ReverseIterator` to avoid collection copying.
    
    ### Does this PR introduce _any_ user-facing change?
    No
    
    ### How was this patch tested?
    Pass GitHub Actions
    
    ### Was this patch authored or co-authored using generative AI tooling?
    No
    
    Closes #44495 from LuciferYang/find-last.
    
    Lead-authored-by: yangjie01 <ya...@baidu.com>
    Co-authored-by: YangJie <ya...@baidu.com>
    Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
 .../org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala      | 2 +-
 core/src/main/scala/org/apache/spark/util/Utils.scala                   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/connector/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala b/connector/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala
index 5b4567aa2881..cee0d9a3dd72 100644
--- a/connector/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala
+++ b/connector/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala
@@ -1789,7 +1789,7 @@ class KafkaMicroBatchV2SourceSuite extends KafkaMicroBatchSourceSuiteBase {
       CheckAnswer(data: _*),
       Execute { query =>
         // The rate limit is 1, so there must be some delay in offsets per partition.
-        val progressWithDelay = query.recentProgress.map(_.sources.head).reverse.find { progress =>
+        val progressWithDelay = query.recentProgress.map(_.sources.head).findLast { progress =>
           // find the metrics that has non-zero average offsetsBehindLatest greater than 0.
           !progress.metrics.isEmpty && progress.metrics.get("avgOffsetsBehindLatest").toDouble > 0
         }
diff --git a/core/src/main/scala/org/apache/spark/util/Utils.scala b/core/src/main/scala/org/apache/spark/util/Utils.scala
index e3d30725d1c9..a55539c0a235 100644
--- a/core/src/main/scala/org/apache/spark/util/Utils.scala
+++ b/core/src/main/scala/org/apache/spark/util/Utils.scala
@@ -2851,7 +2851,7 @@ private[spark] object Utils
     else {
       // The last char is a dollar sign
       // Find last non-dollar char
-      val lastNonDollarChar = s.reverse.find(_ != '$')
+      val lastNonDollarChar = s.findLast(_ != '$')
       lastNonDollarChar match {
         case None => s
         case Some(c) =>


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org