[GitHub] [spark] wayneguow edited a comment on pull request #34853: [SPARK-37575][SQL] null values should be saved as nothing rather than quoted empty Strings "" by default settings

[GitBox <gi...@apache.org>]

wayneguow edited a comment on pull request #34853:
URL: https://github.com/apache/spark/pull/34853#issuecomment-992615673


   @MaxGekk Thank you for your careful reviews and providing a significant test case.
   
   But in my opinion, the root cause is  that the parameter `skipEmptyLines` is set to true by default.
   
   Before the 2.4 version, when a row has only one column with null values, this row is also not to be saved in the final csv files.
   
   Since 2.4 version, we save null values and empty strings both as quoted empty strings "" _by mistake_, so the row which has only one null values column can be saved in csv files. 
    
   The change what I made this time is just to distinguish null values and empty strings in saved in csv files really and it's stiil to keep the behavior with before.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org