You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2024/04/24 00:14:24 UTC

[I] Implement Spark-compatible CAST between integer types [datafusion-comet]

andygrove opened a new issue, #311:
URL: https://github.com/apache/datafusion-comet/issues/311

   ### What is the problem the feature request solves?
   
   We currently delegate to DataFusion for casting between integer types and there are differences in behavior when compared to Spark.
   
   Here is a test that can be added to `CometCastSuite` to demonstrate the issue:
   
   ```scala
     test("cast int to short") {
       castTest(Seq(Int.MaxValue, Short.MaxValue.toInt).toDF("a"), DataTypes.ShortType)
     }
   ```
   
   Output:
   
   ```
   == Results ==
   !== Correct Answer - 2 ==           == Spark Answer - 2 ==
    struct<a:int,converted:smallint>   struct<a:int,converted:smallint>
   ![2147483647,-1]                    [2147483647,null]
    [32767,32767]                      [32767,32767]
   ```
   
   Note that there are different code paths in Spark depending on whether ANSI mode is enabled or not, and we need to handle both cases.
   
   ### Describe the potential solution
   
   _No response_
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Implement Spark-compatible CAST between integer types [datafusion-comet]

Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #311: Implement Spark-compatible CAST between integer types
URL: https://github.com/apache/datafusion-comet/issues/311


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Implement Spark-compatible CAST between integer types [datafusion-comet]

Posted by "ganeshkumar269 (via GitHub)" <gi...@apache.org>.
ganeshkumar269 commented on issue #311:
URL: https://github.com/apache/datafusion-comet/issues/311#issuecomment-2079769379

   Hi I would like to work on this


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Implement Spark-compatible CAST between integer types [datafusion-comet]

Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on issue #311:
URL: https://github.com/apache/datafusion-comet/issues/311#issuecomment-2079973730

   Thank you @ganeshkumar269 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org