[GitHub] [spark] CodingCat commented on a change in pull request #30972: [SPARK-33940][SQL] allow configuring the max column name length in csv writer

[GitBox <gi...@apache.org>]

CodingCat commented on a change in pull request #30972:
URL: https://github.com/apache/spark/pull/30972#discussion_r550847454



##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CsvOutputWriter.scala
##########
@@ -47,3 +50,15 @@ class CsvOutputWriter(
 
   override def close(): Unit = gen.close()
 }
+
+object CsvOutputWriter {
+
+  def createUnivocityGenerator(
+      maxLength: Int,
+      writer: OutputStreamWriter,
+      dataSchema: StructType,
+      params: CSVOptions): UnivocityGenerator = this.synchronized {
+    NormalizedString.getCache.setMaxStringLength(maxLength)

Review comment:
       eh..may I ask why? is it a convention that we should expose all static configuration as static SQL config?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org