[GitHub] [spark] srowen commented on pull request #35573: [SPARK-38060] [SQL] Respect allowNonNumericNumbers when parsing quoted NaN and Infinity values in JSON reader

[GitBox <gi...@apache.org>]

srowen commented on pull request #35573:
URL: https://github.com/apache/spark/pull/35573#issuecomment-1046116321


   I see, so the most consistent behavior would be to not allow `"NaN"`, whether allowing non-numeric inputs to be parsed as numbers or not?
   
   So a raw literal like `NaN` is not handled by this section of code in Spark, but by Jackson?
   
   Well in any event, I suppose it's a little better to be permissive in this narrow case, to achieve a little more consistency, rather than 'break' existing functionality.
   
   `allowNonNumericNumbers` is on by default, so adding the additional constraint here wouldn't affect most paths. If someone had explicitly disallowed it, now Spark would parse things like `"NaN"` as NaN. It could be worth a note in core-migration-guide.md for good measure.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org