You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2026/03/28 14:50:50 UTC
[PR] fix: use IgnoreCometNativeScan for StringPredicate test in 3.4.3 diff [datafusion-comet]
andygrove opened a new pull request, #3830:
URL: https://github.com/apache/datafusion-comet/pull/3830
## Which issue does this PR close?
N/A - minor diff consistency fix.
## Rationale for this change
The `filter pushdown - StringPredicate` test in the 3.4.3 Spark SQL test diff uses `IgnoreCometNativeDataFusion` while the same test in the 3.5.8 and 4.0.1 diffs uses `IgnoreCometNativeScan`. The test fails because the native scan doesn't propagate the `NumRowGroupsAcc` Spark accumulator, so `IgnoreCometNativeScan` is the more appropriate tag.
## What changes are included in this PR?
Changed the tag on the "filter pushdown - StringPredicate" test in `dev/diffs/3.4.3.diff` from `IgnoreCometNativeDataFusion` to `IgnoreCometNativeScan` for consistency with the other Spark version diffs.
## How are these changes tested?
Verified the test still fails with `COMET_PARQUET_SCAN_IMPL=native_datafusion` (accumulator returns 0), confirming the ignore tag is still needed. The change only narrows the scope of the ignore to match the other diffs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: link accumulator-related IgnoreCometNativeScan tests to #3867 [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on code in PR #3830:
URL: https://github.com/apache/datafusion-comet/pull/3830#discussion_r3021336925
##########
dev/diffs/3.4.3.diff:
##########
@@ -2069,7 +2069,7 @@ index 104b4e416cd..d865077684f 100644
- test("filter pushdown - StringPredicate") {
+ test("filter pushdown - StringPredicate",
-+ IgnoreCometNativeDataFusion("cannot be pushed down")) {
++ IgnoreCometNativeScan("cannot be pushed down")) {
Review Comment:
I created a new specific issue and updated all relevant tests to link to the new issue #3867
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] minor: use `IgnoreCometNativeScan` for StringPredicate test in 3.4.3 diff [datafusion-comet]
Posted by "kazuyukitanimura (via GitHub)" <gi...@apache.org>.
kazuyukitanimura commented on code in PR #3830:
URL: https://github.com/apache/datafusion-comet/pull/3830#discussion_r3011160616
##########
dev/diffs/3.4.3.diff:
##########
@@ -2069,7 +2069,7 @@ index 104b4e416cd..d865077684f 100644
- test("filter pushdown - StringPredicate") {
+ test("filter pushdown - StringPredicate",
-+ IgnoreCometNativeDataFusion("cannot be pushed down")) {
++ IgnoreCometNativeScan("cannot be pushed down")) {
Review Comment:
Thanks.
Is https://github.com/apache/datafusion-comet/issues/3321 the tracking issue?
`The test fails because the native scan doesn't propagate the NumRowGroupsAcc Spark accumulator` So sounds like the gap is still there even it is not a bug? I was thinking we can include the link like `IgnoreCometNativeScan("https://github.com/apache/datafusion-comet/issues/3321")` so that we can revisit, just like other skipped tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] minor: use `IgnoreCometNativeScan` for StringPredicate test in 3.4.3 diff [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove commented on code in PR #3830:
URL: https://github.com/apache/datafusion-comet/pull/3830#discussion_r3010650613
##########
dev/diffs/3.4.3.diff:
##########
@@ -2069,7 +2069,7 @@ index 104b4e416cd..d865077684f 100644
- test("filter pushdown - StringPredicate") {
+ test("filter pushdown - StringPredicate",
-+ IgnoreCometNativeDataFusion("cannot be pushed down")) {
++ IgnoreCometNativeScan("cannot be pushed down")) {
Review Comment:
There is no issue for this one. It isn't due to a bug in Comet, but a limitation of how the Spark test is written.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: link accumulator-related IgnoreCometNativeScan tests to #3867 [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed pull request #3830: fix: link accumulator-related IgnoreCometNativeScan tests to #3867
URL: https://github.com/apache/datafusion-comet/pull/3830
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] minor: use `IgnoreCometNativeScan` for StringPredicate test in 3.4.3 diff [datafusion-comet]
Posted by "kazuyukitanimura (via GitHub)" <gi...@apache.org>.
kazuyukitanimura commented on code in PR #3830:
URL: https://github.com/apache/datafusion-comet/pull/3830#discussion_r3011160616
##########
dev/diffs/3.4.3.diff:
##########
@@ -2069,7 +2069,7 @@ index 104b4e416cd..d865077684f 100644
- test("filter pushdown - StringPredicate") {
+ test("filter pushdown - StringPredicate",
-+ IgnoreCometNativeDataFusion("cannot be pushed down")) {
++ IgnoreCometNativeScan("cannot be pushed down")) {
Review Comment:
Thanks.
Is https://github.com/apache/datafusion-comet/issues/3321 the tracking issue?
`The test fails because the native scan doesn't propagate the NumRowGroupsAcc Spark accumulator` So sounds like the gap is still there even it is not a bug?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] minor: use `IgnoreCometNativeScan` for StringPredicate test in 3.4.3 diff [datafusion-comet]
Posted by "kazuyukitanimura (via GitHub)" <gi...@apache.org>.
kazuyukitanimura commented on code in PR #3830:
URL: https://github.com/apache/datafusion-comet/pull/3830#discussion_r3010643176
##########
dev/diffs/3.4.3.diff:
##########
@@ -2069,7 +2069,7 @@ index 104b4e416cd..d865077684f 100644
- test("filter pushdown - StringPredicate") {
+ test("filter pushdown - StringPredicate",
-+ IgnoreCometNativeDataFusion("cannot be pushed down")) {
++ IgnoreCometNativeScan("cannot be pushed down")) {
Review Comment:
Would you mind putting a link to the issue please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] minor: use `IgnoreCometNativeScan` for StringPredicate test in 3.4.3 diff [datafusion-comet]
Posted by "kazuyukitanimura (via GitHub)" <gi...@apache.org>.
kazuyukitanimura commented on code in PR #3830:
URL: https://github.com/apache/datafusion-comet/pull/3830#discussion_r3011160616
##########
dev/diffs/3.4.3.diff:
##########
@@ -2069,7 +2069,7 @@ index 104b4e416cd..d865077684f 100644
- test("filter pushdown - StringPredicate") {
+ test("filter pushdown - StringPredicate",
-+ IgnoreCometNativeDataFusion("cannot be pushed down")) {
++ IgnoreCometNativeScan("cannot be pushed down")) {
Review Comment:
Is https://github.com/apache/datafusion-comet/issues/3321 the tracking issue?
`The test fails because the native scan doesn't propagate the NumRowGroupsAcc Spark accumulator` So sounds like the gap is still there?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org