You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2025/10/20 18:24:49 UTC

Re: [I] [EPIC] Improve Comet Correctness Testing for Expressions [datafusion-comet]

andygrove commented on issue #2611:
URL: https://github.com/apache/datafusion-comet/issues/2611#issuecomment-3423263022

   As an example of the benefit of fuzz testing, I just found a correctness issue while experimenting with the proposed solution. 
   
   Issue: https://github.com/apache/datafusion-comet/issues/2612
   
   We had an existing test for `array_reverse` but it was only testing with inputs that Comet supports:
   
   ```
             val fieldNames =
               table.schema.fields
                 .filter(field => CometArrayReverse.isTypeSupported(field.dataType))
                 .map(_.name)
             for (fieldName <- fieldNames) {
               sql(s"SELECT $fieldName as a FROM t1")
                 .createOrReplaceTempView("t2")
               checkSparkAnswer(sql("SELECT reverse(a) FROM t2"))
             }
   ```
   
   Those cases work fine. The correctness issue happens when we fall back to Spark for the projection due to unsupported types.
   
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org