You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "Yizhou-Yang (via GitHub)" <gi...@apache.org> on 2026/02/25 02:58:06 UTC

[PR] feat:support gluten-level approx_percentile [incubator-gluten]

Yizhou-Yang opened a new pull request, #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651

   velox pr: https://github.com/facebookincubator/velox/pull/16320
   inspired by https://github.com/apache/incubator-gluten/pull/5007
   
   Since this function is already being used in the production environment,  we add a config to manually fallback for its safety in production environments.
   
   @jinchengchenghh 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4028129345

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4411076812

   This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on code in PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#discussion_r2903512762


##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
     "split"};
 
 const std::unordered_set<std::string> kBlackList =
-    {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+    {"split_part", "sequence", "map_from_arrays"};

Review Comment:
   ok, I will look into it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on code in PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#discussion_r2903512762


##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
     "split"};
 
 const std::unordered_set<std::string> kBlackList =
-    {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+    {"split_part", "sequence", "map_from_arrays"};

Review Comment:
   sure, I will look into it. @jinchengchenghh 
   https://github.com/facebookincubator/velox/pull/13183
   I think that the velox part is compatible with spark as of 2025/05, it's just for some reason the velox fix is not adopted in gluten.  Right now I think the only gluten fix needed is to remove the kblacklist and test it... 
   
   I'll do a comparison with spark to see if it is the case, but it should be a trivial fix.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3983217661

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3983179123

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4523775413

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4574305033

   bugfix: accuracy can be 0, leading to kFromAccuracy inifite, and instantly oom. this is the reason for spark 41 failure....


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3983273654

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3983193666

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on code in PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#discussion_r2903512762


##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
     "split"};
 
 const std::unordered_set<std::string> kBlackList =
-    {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+    {"split_part", "sequence", "map_from_arrays"};

Review Comment:
   sure, I will look into it. @jinchengchenghh 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on code in PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#discussion_r2903512762


##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
     "split"};
 
 const std::unordered_set<std::string> kBlackList =
-    {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+    {"split_part", "sequence", "map_from_arrays"};

Review Comment:
   sure, I will look into it. @jinchengchenghh 
   https://github.com/facebookincubator/velox/pull/13183
   I think that the velox part is compatible with spark as of 2025/05, it's just for some reason the velox fix is not adopted in gluten.  Right now I think the only gluten fix needed is to remove the kblacklist and test it... 
   
   I'll do a comparison with spark, post the results and add appropriate ut, but I think it might essentially be removing the kblacklist.



##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
     "split"};
 
 const std::unordered_set<std::string> kBlackList =
-    {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+    {"split_part", "sequence", "map_from_arrays"};

Review Comment:
   sure, I will look into it. @jinchengchenghh 
   https://github.com/facebookincubator/velox/pull/13183
   I think that the velox part is compatible with spark as of 2025/05, it's just for some reason the velox fix is not adopted in gluten.  Right now I think the only gluten fix needed is to remove the kblacklist and test it... 
   
   I'll do a comparison with spark, post the results and add appropriate ut, but I think it might essentially just be removing the kblacklist.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4009362498

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4021628168

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on code in PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#discussion_r2895170724


##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
     "split"};
 
 const std::unordered_set<std::string> kBlackList =
-    {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+    {"split_part", "sequence", "map_from_arrays"};

Review Comment:
   I find also the map_from_arrays in black list, another topic, would you like to implement it in Gluten?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3958241316

   Please update get-velox.sh to test your PR, then you can verify if both can work well, you may update this line https://github.com/apache/incubator-gluten/blob/5d3f7145cd7fc258aa10b434ea4ec651bd82c764/ep/build-velox/src/get-velox.sh#L28


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3990529567

   > Please update the PR description to describe the KLL Sketch is different so that we handle fallback separately.
   
   done
   
   > ack separately.
   
   done~


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4419137557

   Sorry for missing this PR, looks good to me, do you have further comments? @zhztheplayer 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #11651: [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function
URL: https://github.com/apache/gluten/pull/11651


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3988315395

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4473669990

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4021592298

   > Please resolve CH CI https://github.com/apache/incubator-gluten/blob/main/docs/get-started/ClickHouse.md#new-ci-system
   
   <img width="1618" height="871" alt="Clipboard_Screenshot_1773039609" src="https://github.com/user-attachments/assets/ed9cae9a-d7ac-4d33-9a89-2a173a2cb56d" />
   I don't think the issue is related with the code, HDFS DataNode did not init, maybe retrying the ci would help...?
   
   The code runs fine in internal pipeline:
   (editing this after passing internal pipeline)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3990420216

   Please update the PR description to describe the KLL Sketch is different so that we handle fallback separately. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4030722121

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4574334465

   retest this please


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4030749468

   https://github.com/apache/incubator-gluten/actions/runs/22893811220/job/66432061435?pr=11651
   There was a test failure in some x86 environments. Fixed, but the pipeline might need restart...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4030716571

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4009249734

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3956443095

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4483972176

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4574325503

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3958786689

   > Please update get-velox.sh to test your PR, then you can verify if both can work well, you may update this line
   > 
   > https://github.com/apache/incubator-gluten/blob/5d3f7145cd7fc258aa10b434ea4ec651bd82c764/ep/build-velox/src/get-velox.sh#L28
   
   added the 16320 and removed the config


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3956418087

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3981827798

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4523841011

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4029592441

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4494057887

   hopefully I fixed spark 40/41 related cases:
   <img width="1351" height="645" alt="Clipboard_Screenshot_1779245317" src="https://github.com/user-attachments/assets/76c2e6d8-11a6-4a1b-a546-0c69a687638d" />
   
   PTAL again...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4485469109

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-5086619078

   This PR was auto-closed because it has been stalled for 10 days with no activity. Please feel free to reopen if it is still valid. Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4009335889

   @jinchengchenghh
   hi! The gluten side needs to be reviewed as well for the entire pr to work...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]

Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on code in PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#discussion_r2910156771


##########
ep/build-velox/src/get-velox.sh:
##########
@@ -25,7 +25,7 @@ RUN_SETUP_SCRIPT=ON
 ENABLE_ENHANCED_FEATURES=OFF
 
 # Developer use only for testing Velox PR.
-UPSTREAM_VELOX_PR_ID=""
+UPSTREAM_VELOX_PR_ID="16320"

Review Comment:
   Update here



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on code in PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#discussion_r2903512762


##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
     "split"};
 
 const std::unordered_set<std::string> kBlackList =
-    {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+    {"split_part", "sequence", "map_from_arrays"};

Review Comment:
   sure, I will look into it. @jinchengchenghh 
   https://github.com/facebookincubator/velox/pull/13183
   I think that the velox part is compatible with spark as of 2025/05, it's just for some reason the velox fix is not adopted in gluten.  Right now I think the only gluten fix needed is to remove the kblacklist and test it... 
   
   I'll do a comparison with spark, post the results and add appropriate ut.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "jiangjiangtian (via GitHub)" <gi...@apache.org>.
jiangjiangtian commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4037243895

   @Yizhou-Yang Thanks for your implementation! Recently I am cherrying pick your PR and I find that the KLL implementation in Gluten has only one level and discards items in odd position when compacting. I am wondering if this implementation can meet the `accuracy` requirement. Will the relative error rate be too high?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4088781439

   ptal again... @jinchengchenghh 
   
   > > @Yizhou-Yang Thanks for your implementation! Recently I am cherrying pick your PR and I find that the KLL implementation in Gluten has only one level and discards items in odd position when compacting. I am wondering if this implementation can meet the `accuracy` requirement. Will the relative error rate be too high?
   > 
   > @jiangjiangtian Hi, the code has been fixed. Here's what changed:
   > 
   > **Changelog:**
   > 
   > 1. Rewrote KLL sketch with proper multi-level compaction — the old implementation only had one level and discarded odd-position items, which is essentially random downsampling with no error bound guarantee. Now it uses the standard KLL algorithm: items across multiple levels, level-0 inserts, sort-and-halve compaction promoting to higher levels, with each level-_i_ item representing 2^_i_ original values.
   > 2. Merge correctly combines multi-level sketches and re-compacts, instead of simple concatenation + truncation.
   > 
   > **Why some tests are disabled:**
   > 
   > KLL and Spark's native GK (Greenwald-Khanna) are fundamentally different algorithms. Both satisfy the `1/accuracy` error bound, but they produce **different concrete values** at the same percentile boundary. For example, given integers 1–1000, the exact 25th percentile is 250.25 — GK returns 250, KLL may return 251. Both are correct within the error bound, but a strict equality assertion like `assert(result == 250)` will fail.
   > 
   > For those tests I added TestGluten in the approxpercentile suite. The off-by one problem can't be simply solved by adding more layers in partial result, I tried to not disable any test when developing, but the current version is the best I can do for now.
   > 
   > The 4 disabled tests (`approx_percentile`, `summary`, `SPARK-35480`, `SPARK-32908`) all use exact-match assertions against GK's specific output. Rather than modifying upstream Spark tests, we excluded them and added a dedicated `GlutenApproximatePercentileQuerySuite` with tolerance-based assertions that validate correctness for both algorithms.
   > 
   > There are some other tests in collect_list that assumes approx_percentile will fallback, changed to percentile.
   > 
   > I tested only spark35/spark35smj and their slow versions, hopefully it also passes spark33 etc...
   
   PTAL again... @jinchengchenghh 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3958792469

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3981795118

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4450512950

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4477657939

   @jinchengchenghh 
   
   sorry was not ready... didn't actually run local tests. 
   I reran all the local tests so hopefully it works now.
   
   <img width="507" height="98" alt="Clipboard_Screenshot_1779107274" src="https://github.com/user-attachments/assets/aab4270d-3903-4fa0-90dd-82047d979436" />
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3958252989

   Do we need the config? Usually we offload the function to native by default


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4011083414

   Please resolve CH CI https://github.com/apache/incubator-gluten/blob/main/docs/get-started/ClickHouse.md#new-ci-system


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4531163201

   reran spark 40/spark41 total 1124 cases and fixed issues in new ut. hopefully this is the last time... sorry about rerunning pipeline multiple times... @jinchengchenghh 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4485416849

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4429011349

   Could you help resolve the conflict? Then we can merge it, thanks! @Yizhou-Yang 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3988334436

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3988472430

   Run Gluten Clickhouse CI on x86


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "jiangjiangtian (via GitHub)" <gi...@apache.org>.
jiangjiangtian commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4037244922

   @Yizhou-Yang Thanks for your implementation! Recently I am cherrying pick your PR and I find that the KLL implementation in Gluten has only one level and discards items in odd position when compacting. I am wondering if this implementation can meet the `accuracy` requirement. Will the relative error rate be too high?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4037340387

   > @Yizhou-Yang Thanks for your implementation! Recently I am cherrying pick your PR and I find that the KLL implementation in Gluten has only one level and discards items in odd position when compacting. I am wondering if this implementation can meet the `accuracy` requirement. Will the relative error rate be too high?
   
   yes, this is part of the reason for the ut failures. The gluten version of kll is written rather hastily and was primarily concerned with simplicity and fallback compatibiltity with velox, and now it seems to have caused problems including but not limited to ut failures and lots of off-by-one issues. I am editing this implementation and doing a line-by-line comparison with the velox one to ensure the gluten side quality, although still some existing uts might need to be excluded.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on code in PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#discussion_r2903512762


##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
     "split"};
 
 const std::unordered_set<std::string> kBlackList =
-    {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+    {"split_part", "sequence", "map_from_arrays"};

Review Comment:
   sure, I will look into it. @jinchengchenghh 
   https://github.com/facebookincubator/velox/pull/13183
   I think that the velox part is compatible with spark in 2025/05, it's just for some reason the velox spark fix is not adopted in gluten.  Right now I think the only gluten fix needed is to remove the kblacklist and test it... 
   
   I'll do a comparison with spark to see if it is the case, but it should be a trivial fix.



##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
     "split"};
 
 const std::unordered_set<std::string> kBlackList =
-    {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+    {"split_part", "sequence", "map_from_arrays"};

Review Comment:
   sure, I will look into it. @jinchengchenghh 
   https://github.com/facebookincubator/velox/pull/13183
   I think that the velox part is compatible with spark as of 2025/05, it's just for some reason the velox spark fix is not adopted in gluten.  Right now I think the only gluten fix needed is to remove the kblacklist and test it... 
   
   I'll do a comparison with spark to see if it is the case, but it should be a trivial fix.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4459679064

   > Could you help resolve the conflict? Then we can merge it, thanks! @Yizhou-Yang
   
   ready @jinchengchenghh 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4998324348

   This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org