You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2024/04/24 12:23:09 UTC

[I] Implement Spark-compatible CAST float/double to string [datafusion-comet]

andygrove opened a new issue, #312:
URL: https://github.com/apache/datafusion-comet/issues/312

   ### What is the problem the feature request solves?
   
   We currently delegate to DataFusion when casting float/double to string and there are some differences in behavior compared to Spark.
   
   - Spark uses `Infinity` and `-Infinity` and DataFusion uses `inf` and `-inf`
   - Spark uses `E` for exponent and DataFusion uses `e`
   
   There can also be differences in formatting but where the values are semantically equivalent. For example, for the input `1.0499721536516571E-4`:
   
   Spark produces `"1.0499721536516571E-4"`
   DataFusion produces: `"0.00010499721536516571"`
   
   We should create a compatibility guide and include this information.
   
   Here are some suggested values to test:
   
   ```scala
   val testData = Seq(Float.NegativeInfinity, Float.PositiveInfinity,
         Float.MinPositiveValue, Float.MinValue, Float.MaxValue, Float.NaN, -0.0f, 0.0f)
   ```
   
   
   
   ### Describe the potential solution
   
   _No response_
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Implement Spark-compatible CAST float/double to string [datafusion-comet]

Posted by "mattharder91 (via GitHub)" <gi...@apache.org>.
mattharder91 commented on issue #312:
URL: https://github.com/apache/datafusion-comet/issues/312#issuecomment-2083225823

   Here you go https://github.com/apache/datafusion-comet/pull/346


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Implement Spark-compatible CAST float/double to string [datafusion-comet]

Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #312: Implement Spark-compatible CAST float/double to string
URL: https://github.com/apache/datafusion-comet/issues/312


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Implement Spark-compatible CAST float/double to string [datafusion-comet]

Posted by "mattharder91 (via GitHub)" <gi...@apache.org>.
mattharder91 commented on issue #312:
URL: https://github.com/apache/datafusion-comet/issues/312#issuecomment-2080587569

   Can I try this one out?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Implement Spark-compatible CAST float/double to string [datafusion-comet]

Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on issue #312:
URL: https://github.com/apache/datafusion-comet/issues/312#issuecomment-2081104450

   Of course. Thanks @mattharder91. Please let us know if you have any question.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org