[GitHub] bersprockets opened a new pull request #23323: [SPARK-26372][SQL] Don't reuse value from previous row when parsing bad CSV input field

[GitBox <gi...@apache.org>]

bersprockets opened a new pull request #23323: [SPARK-26372][SQL] Don't reuse value from previous row when parsing bad CSV input field
URL: https://github.com/apache/spark/pull/23323
 
 
   ## What changes were proposed in this pull request?
   
   CSV parsing accidentally uses the previous good value for a bad input field. See example in Jira.
   
   This PR ensures that the associated column is set to null when an input field cannot be converted.
   
   ## How was this patch tested?
   
   Added new test.
   Ran all SQL unit tests (testOnly org.apache.spark.sql.*).
   Ran pyspark tests for pyspark-sql
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org