You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "ayushi-agarwal (via GitHub)" <gi...@apache.org> on 2025/03/04 16:46:35 UTC

[I] Filter not being pushed into scan [incubator-gluten]

ayushi-agarwal opened a new issue, #8897:
URL: https://github.com/apache/incubator-gluten/issues/8897

   ### Backend
   
   VL (Velox)
   
   ### Bug description
   
   select * from store_sales where ss_list_price<0;
   
   When this query is run, filter is not being pushed to scan resulting in higher latency for the query.
   
   Plan comes out to be on latest main today:
   TableScan[0][table: hive_table, remaining filter: (and(isnotnull("ss_list_price"),lessthan("ss_list_price",0)))] -> n0_0:INTEGER, n0_1:INTEGER, n0_2:INTEGER, n0_3:INTEGER, n0_4:INTEGER, n0_5:INTEGER, n0_6:INTEGER, n0_7:INTEGER, n0_8:BIGINT, n0_9:INTEGER, n0_10:DOUBLE, n0_11:DOUBLE, n0_12:DOUBLE, n0_13:DOUBLE, n0_14:DOUBLE, n0_15:DOUBLE, n0_16:DOUBLE, n0_17:DOUBLE, n0_18:DOUBLE, n0_19:DOUBLE, n0_20:DOUBLE, n0_21:DOUBLE, n0_22:INTEGER
      Input: 0 rows (0B, 0 batches), Raw Input: 85327543 rows (4.22GB), Output: 0 rows (0B, 0 batches), Cpu time: 10.87s, Wall time: 50.51s, Blocked wall time: 0ns, Peak memory: 132.29MB, Memory allocations: 25232, Threads: 1, Splits: 98, CPU breakdown: B/I/O/F (74.47us/0ns/10.87s/28.52us)
   
   Previously without this change https://github.com/apache/incubator-gluten/issues/6666 it was coming like below where the filter was being pushed as range fliter.
   TableScan[0][table: hive_table, range filters: [(ss_list_price, DoubleRange: (-inf, 0.000000) no nulls)]] -> n0_0:INTEGER, n0_1:INTEGER, n0_2:INTEGER, n0_3:INTEGER, n0_4:INTEGER, n0_5:INTEGER, n0_6:INTEGER, n0_7:INTEGER, n0_8:BIGINT, n0_9:INTEGER, n0_10:DOUBLE, n0_11:DOUBLE, n0_12:DOUBLE, n0_13:DOUBLE, n0_14:DOUBLE, n0_15:DOUBLE, n0_16:DOUBLE, n0_17:DOUBLE, n0_18:DOUBLE, n0_19:DOUBLE, n0_20:DOUBLE, n0_21:DOUBLE, n0_22:INTEGER 
   Input: 0 rows (0B, 0 batches), Output: 0 rows (0B, 0 batches), Cpu time: 1.36s, Blocked wall time: 0ns, Peak memory: 34.88KB, Memory allocations: 121, Threads: 1, Splits: 98
   
   @rui-mo Is this a known issue?
   
   ### Spark version
   
   Spark-3.5.x
   
   ### Spark configurations
   
   _No response_
   
   ### System information
   
   _No response_
   
   ### Relevant logs
   
   ```bash
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [I] [VL] Filter not being pushed into scan [incubator-gluten]

Posted by "ayushi-agarwal (via GitHub)" <gi...@apache.org>.
ayushi-agarwal commented on issue #8897:
URL: https://github.com/apache/incubator-gluten/issues/8897#issuecomment-2700377992

   @rui-mo We are also seeing higher latency for the query. Attaching the event log snapshot for the same query.
   
   This is on the older version where scan time is less, also skipped row groups metrics can be seen, raw input bytes total is also less.
   ![Image](https://github.com/user-attachments/assets/149d3521-2521-4287-bc7d-177135424567)
   
   This is on latest version:
   ![Image](https://github.com/user-attachments/assets/3116f88c-0670-4c67-b297-2966e09520b5)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [I] [VL] Filter not being pushed into scan [incubator-gluten]

Posted by "ayushi-agarwal (via GitHub)" <gi...@apache.org>.
ayushi-agarwal closed issue #8897: [VL] Filter not being pushed into scan
URL: https://github.com/apache/incubator-gluten/issues/8897


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [I] [VL] Filter not being pushed into scan [incubator-gluten]

Posted by "ayushi-agarwal (via GitHub)" <gi...@apache.org>.
ayushi-agarwal commented on issue #8897:
URL: https://github.com/apache/incubator-gluten/issues/8897#issuecomment-2703645101

   This is the commit ID in gluten which caused regression https://github.com/apache/incubator-gluten/commit/624752de220672b7c1ecfe882419fe8bc79156b3 - still need to find out the commit in velox


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [I] [VL] Filter not being pushed into scan [incubator-gluten]

Posted by "rui-mo (via GitHub)" <gi...@apache.org>.
rui-mo commented on issue #8897:
URL: https://github.com/apache/incubator-gluten/issues/8897#issuecomment-2700447240

   @ayushi-agarwal Would you like to try two closer commits like 61033b719f8494ef80dd0c46fa6e738afa7e732d and 0b899c046f2f0b9c141518f4ef43d2075540d93c and compare the performance? I will also check them locally, but as I verified in https://github.com/apache/incubator-gluten/issues/8897#issuecomment-2699049143 the filter push-down actually works.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [I] [VL] Filter not being pushed into scan [incubator-gluten]

Posted by "rui-mo (via GitHub)" <gi...@apache.org>.
rui-mo commented on issue #8897:
URL: https://github.com/apache/incubator-gluten/issues/8897#issuecomment-2698869397

   @ayushi-agarwal I reproduced this issue, and will investigate. Thanks for reporting!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [I] [VL] Filter not being pushed into scan [incubator-gluten]

Posted by "ayushi-agarwal (via GitHub)" <gi...@apache.org>.
ayushi-agarwal commented on issue #8897:
URL: https://github.com/apache/incubator-gluten/issues/8897#issuecomment-2700944805

   @rui-mo I also added the log line as you mentioned and it's getting converted to push-down filters. It is possible that some other change has caused this regression. We saw this regression and while trying to figure root cause, because of this plan change we assumed that may this is causing it. After youclaified it seems there might be some other commit.
   
   Are you also able to reproduce this regression at your end?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [I] [VL] Filter not being pushed into scan [incubator-gluten]

Posted by "rui-mo (via GitHub)" <gi...@apache.org>.
rui-mo commented on issue #8897:
URL: https://github.com/apache/incubator-gluten/issues/8897#issuecomment-2706581342

   @ayushi-agarwal Thanks for keeping me updated! If the stats are not used, row group skip could not take effect so the performance regress happens.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [I] [VL] Filter not being pushed into scan [incubator-gluten]

Posted by "rui-mo (via GitHub)" <gi...@apache.org>.
rui-mo commented on issue #8897:
URL: https://github.com/apache/incubator-gluten/issues/8897#issuecomment-2698549479

   @ayushi-agarwal This is not the expected behaviour. In https://github.com/apache/incubator-gluten/pull/6754 I verified TPC-DS and TPC-H workloads to make sure the push-down works correctly. I will try this query, and could you please check if `SparkExprToSubfieldFilterParser::leafCallToSubfieldFilter` has been entered in your test case?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [I] [VL] Filter not being pushed into scan [incubator-gluten]

Posted by "rui-mo (via GitHub)" <gi...@apache.org>.
rui-mo commented on issue #8897:
URL: https://github.com/apache/incubator-gluten/issues/8897#issuecomment-2699049143

   Hi @ayushi-agarwal, let me explain the context here. The reason for this plan change as you noticed is that before https://github.com/apache/incubator-gluten/pull/6754 Gluten decides which filters can be pushed down, which cannot, so the TableScan node is created with that information and is like below.
   
   > TableScan[0][table: hive_table, range filters: [(ss_list_price, DoubleRange: (-inf, 0.000000) no nulls)]]
   
   However, after #6754 Gluten depends on 'extractFiltersFromRemainingFilter' to extract the push-down filters internally in Velox, so in Gluten the TableScan node is created with remaining filter only, and the plan is like below.
   
    > TableScan[0][table: hive_table, remaining filter: (and(isnotnull("ss_list_price"),lessthan("ss_list_price",0)))]
   
   This plan does not imply that all filters are not pushed down. If we try to print the 'remainingFilter' and 'filters_' at [HiveDataSource](https://github.com/facebookincubator/velox/blob/55257c24487ae0002a0e8ea5ee25972fc955699f/velox/connectors/hive/HiveDataSource.cpp#L135-L140), we will find the 'remainingFilter' is nullptr and 'filters_' contains the `DoubleRange: (-inf, 0.000000) no nulls`, which means the remaining filter is converted to push-down filters completely.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org

Re: [I] [VL] Filter not being pushed into scan [incubator-gluten]

Posted by "ayushi-agarwal (via GitHub)" <gi...@apache.org>.
ayushi-agarwal commented on issue #8897:
URL: https://github.com/apache/incubator-gluten/issues/8897#issuecomment-2706555224

   This commit was causing regression for us as this commit added a check for parquet-mr version to determine whether to use stats or not, our parquet files were created using photon, therefore the regex check failed for it and stats were not being used.
   created_by: parquet-mr compatible Photon version 0.2 (build 14.3)
   https://github.com/oap-project/velox/commit/d20ca4bd254c86dae908f6c239fd6ae799cd55e4#diff-5f006b6d4d0ed3a13d2140176367fe887429135da51cead9f48f5f1051a0514b
   
   I will mark this issue as closed. Thanks.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org