[GitHub] [incubator-seatunnel] BenJFan commented on a change in pull request #1534: [Bug] [connector-spark-file] Fix config param issues of spark file #1533

[GitBox <gi...@apache.org>]

BenJFan commented on a change in pull request #1534:
URL: https://github.com/apache/incubator-seatunnel/pull/1534#discussion_r831796240



##########
File path: seatunnel-connectors/seatunnel-connectors-spark/seatunnel-connector-spark-file/src/main/scala/org/apache/seatunnel/spark/source/File.scala
##########
@@ -30,7 +33,15 @@ import scala.util.{Failure, Success, Try}
 class File extends SparkBatchSource {
 
   override def checkConfig(): CheckResult = {
-    checkAllExists(config, PATH, FORMAT)
+    checkAllExists(config, PATH)
+  }
+
+  override def prepare(prepareEnv: SparkEnvironment): Unit = {
+    val defaultConfig = ConfigFactory.parseMap(
+      Map(
+        FORMAT -> DEFAULT_FORMAT

Review comment:
       Seem like add default format is JSON, why not TEXT? TEXT can support any file data.

##########
File path: seatunnel-connectors/seatunnel-connectors-spark/seatunnel-connector-spark-file/src/main/scala/org/apache/seatunnel/spark/Config.scala
##########
@@ -23,7 +23,7 @@ object Config extends Serializable {
   final val SAVE_MODE = "save_mode"
   final val SERIALIZER = "serializer"
   final val PATH_TIME_FORMAT = "path_time_format"
-  final val DEFAULT_TIME_FORMAT = "path_time_format"
+  final val DEFAULT_TIME_FORMAT = "yyyyMMddHHmmss"

Review comment:
       DEFAULT_TIME_FORMAT should be "yyyy-MM-dd HH:mm:ss"




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org