You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2025/05/21 16:46:09 UTC
[I] [native_iceberg_compat] Spark SQL core-2 "filter pushdown - StringPredicate" failure [datafusion-comet]
andygrove opened a new issue, #1767:
URL: https://github.com/apache/datafusion-comet/issues/1767
### Describe the bug
Test fails:
```
[info] - filter pushdown - StringPredicate *** FAILED *** (3 seconds, 333 milliseconds)
[info] 0 was not greater than 0 (ParquetFilterSuite.scala:242)
```
Debug logging shows the filter that should have been pushed down:
```
testStringPredicate filter=value like 'a%' shouldFilterOut=true
```
Comet plan shows that the filter is pushed down:
```
*(1) CometColumnarToRow
+- CometFilter [value#14], (isnotnull(value#14) AND StartsWith(value#14, a))
+- CometScan parquet [value#14] Batched: true, DataFilters: [isnotnull(value#14), StartsWith(value#14, a)], Format: CometParquet, Location: InMemoryFileIndex(1 paths)[file:/Users/andy/git/apache/apache-spark/target/tmp/spark-e8f77490-5e9..., PartitionFilters: [], PushedFilters: [IsNotNull(value), StringStartsWith(value,a)], ReadSchema: struct<value:string>
```
### Steps to reproduce
_No response_
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] [native_iceberg_compat] Spark SQL core-2 "filter pushdown - StringPredicate" failure [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on issue #1767:
URL: https://github.com/apache/datafusion-comet/issues/1767#issuecomment-2920364682
Fixed in https://github.com/apache/datafusion-comet/pull/1768
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] [native_iceberg_compat] Spark SQL core-2 "filter pushdown - StringPredicate" failure [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #1767: [native_iceberg_compat] Spark SQL core-2 "filter pushdown - StringPredicate" failure
URL: https://github.com/apache/datafusion-comet/issues/1767
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org