You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/03/01 17:18:15 UTC

[GitHub] [spark] peter-toth commented on pull request #31682: [WIP][SPARK-34545][SQL] Fix issues with valueCompare feature of pyrolite

peter-toth commented on pull request #31682:
URL: https://github.com/apache/spark/pull/31682#issuecomment-788121131


   > > This is because during serialization (1, 1) is replaced to (1.0, 1.0) and return type of c4 is expected to be IntegerType and if a different type comes back from python then it is discarded
   > 
   > How is this related to the `valueCompare` flag? It casts all numeric to double?
   
   No. During serialization `GenericRowWithSchema(1.0, 1.0)` will be memoized first and when `GenericRowWithSchema(1, 1)` comes then it is replaced to some short code of the first one (instead of writing the second out) as the they are equal and `valueCompare` is enabled.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org