[GitHub] [spark] beliefer edited a comment on issue #26875: [SPARK-30245][SQL] Add cache for Like and RLike when pattern is not static

[GitBox <gi...@apache.org>]

beliefer edited a comment on issue #26875: [SPARK-30245][SQL] Add cache for Like and RLike when pattern is not static
URL: https://github.com/apache/spark/pull/26875#issuecomment-601999002
 
 
   @HyukjinKwon After many tests, I found that this modification did not bring a clear performance improvement.
   ```
   spark-sql> select count(*) from bigtable;
   360014110
   ```
   I test the sql show below:
   `select count(*) from bigtable where string(ws_ship_customer_sk) like '/%SystemDrive/%//Users%';`
   
   Spark3.1.0
   Time | Min | 25th percentile | Median | 75th percentile | Max
   -- | -- | -- | -- | -- | --
   2.4 min | 4 s | 9 s | 13 s | 32 s | 2.2 min
   1.3 min | 6 s | 9 s | 12 s | 17 s | 56 s
   1.4 min | 6 s | 11 s | 13 s | 17 s | 1.1 min
   1.2 min | 6 s | 11 s | 14 s | 20 s | 40 s
   1.1 min | 4 s | 10 s | 12 s | 15 s | 38 s
   2.6 min | 5 s | 9 s | 12 s | 18 s | 2.5 min
   
   Spark3.1.0 apply this PR
   Time | Min | 25th percentile | Median | 75th percentile | Max
   -- | -- | -- | -- | -- | --
   1.1 min | 5 s | 9 s | 12 s | 16 s | 40 s
   46 s | 7 s | 9 s | 11 s | 14 s | 37 s
   1.4 min | 7 s | 11 s | 14 s | 18 s | 1.0 min
   2.2 min | 5 s | 11 s | 13 s | 19 s | 1.9 min
   1.0 min | 4 s | 10 s | 12 s | 14 s | 50 s
   2.0 min | 6 s | 9 s | 12 s | 15 s | 1.9 min
   
   So I think we should not make this change and I will close https://github.com/apache/spark/pull/27497
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org