(spark) branch branch-4.0 updated: [SPARK-51283][SQL][TESTS] Add test cases for LZ4 and SNAPPY for text

[do...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.0 by this push:
     new 59287c017b6c [SPARK-51283][SQL][TESTS] Add test cases for LZ4 and SNAPPY for text
59287c017b6c is described below

commit 59287c017b6c74c633efb7fa928d50fda315bb85
Author: beliefer <be...@163.com>
AuthorDate: Fri Feb 21 08:10:10 2025 -0800

    [SPARK-51283][SQL][TESTS] Add test cases for LZ4 and SNAPPY for text
    
    ### What changes were proposed in this pull request?
    This PR proposes to add test cases for `LZ4` and `SNAPPY` for text.
    
    ### Why are the changes needed?
    Currently, Spark missing the test cases for `LZ4` and `SNAPPY`.
    
    ### Does this PR introduce _any_ user-facing change?
    'No'.
    Add test.
    
    ### How was this patch tested?
    GA.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    'No'.
    
    Closes #50043 from beliefer/SPARK-51283.
    
    Authored-by: beliefer <be...@163.com>
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
    (cherry picked from commit 140a69b4c21321107a6879ccd48aac6bfdaf280d)
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
 .../org/apache/spark/sql/execution/datasources/text/TextSuite.scala   | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala
index 5c373a2de973..dd254dbc7da8 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/text/TextSuite.scala
@@ -26,7 +26,7 @@ import org.apache.hadoop.io.compress.GzipCodec
 
 import org.apache.spark.{SparkConf, SparkIllegalArgumentException, TestUtils}
 import org.apache.spark.sql.{AnalysisException, DataFrame, QueryTest, Row, SaveMode}
-import org.apache.spark.sql.catalyst.util.HadoopCompressionCodec.{BZIP2, DEFLATE, GZIP, NONE}
+import org.apache.spark.sql.catalyst.util.HadoopCompressionCodec.{BZIP2, DEFLATE, GZIP, LZ4, NONE, SNAPPY}
 import org.apache.spark.sql.execution.datasources.CommonFileDataSourceSuite
 import org.apache.spark.sql.internal.SQLConf
 import org.apache.spark.sql.test.SharedSparkSession
@@ -93,7 +93,7 @@ abstract class TextSuite extends QueryTest with SharedSparkSession with CommonFi
 
   test("SPARK-13503 Support to specify the option for compression codec for TEXT") {
     val testDf = spark.read.text(testFile)
-    val extensionNameMap = Seq(BZIP2, DEFLATE, GZIP)
+    val extensionNameMap = Seq(BZIP2, DEFLATE, GZIP, LZ4, SNAPPY)
       .map(codec => codec.lowerCaseName() -> codec.getCompressionCodec.getDefaultExtension)
     extensionNameMap.foreach {
       case (codecName, extension) =>


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org