You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "Yizhou-Yang (via GitHub)" <gi...@apache.org> on 2026/02/25 02:58:06 UTC
[PR] feat:support gluten-level approx_percentile [incubator-gluten]
Yizhou-Yang opened a new pull request, #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651
velox pr: https://github.com/facebookincubator/velox/pull/16320
inspired by https://github.com/apache/incubator-gluten/pull/5007
Since this function is already being used in the production environment, we add a config to manually fallback for its safety in production environments.
@jinchengchenghh
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4028129345
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4411076812
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on code in PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#discussion_r2903512762
##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
"split"};
const std::unordered_set<std::string> kBlackList =
- {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+ {"split_part", "sequence", "map_from_arrays"};
Review Comment:
ok, I will look into it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on code in PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#discussion_r2903512762
##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
"split"};
const std::unordered_set<std::string> kBlackList =
- {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+ {"split_part", "sequence", "map_from_arrays"};
Review Comment:
sure, I will look into it. @jinchengchenghh
https://github.com/facebookincubator/velox/pull/13183
I think that the velox part is compatible with spark as of 2025/05, it's just for some reason the velox fix is not adopted in gluten. Right now I think the only gluten fix needed is to remove the kblacklist and test it...
I'll do a comparison with spark to see if it is the case, but it should be a trivial fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3983217661
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3983179123
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4523775413
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4574305033
bugfix: accuracy can be 0, leading to kFromAccuracy inifite, and instantly oom. this is the reason for spark 41 failure....
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3983273654
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3983193666
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on code in PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#discussion_r2903512762
##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
"split"};
const std::unordered_set<std::string> kBlackList =
- {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+ {"split_part", "sequence", "map_from_arrays"};
Review Comment:
sure, I will look into it. @jinchengchenghh
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on code in PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#discussion_r2903512762
##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
"split"};
const std::unordered_set<std::string> kBlackList =
- {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+ {"split_part", "sequence", "map_from_arrays"};
Review Comment:
sure, I will look into it. @jinchengchenghh
https://github.com/facebookincubator/velox/pull/13183
I think that the velox part is compatible with spark as of 2025/05, it's just for some reason the velox fix is not adopted in gluten. Right now I think the only gluten fix needed is to remove the kblacklist and test it...
I'll do a comparison with spark, post the results and add appropriate ut, but I think it might essentially be removing the kblacklist.
##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
"split"};
const std::unordered_set<std::string> kBlackList =
- {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+ {"split_part", "sequence", "map_from_arrays"};
Review Comment:
sure, I will look into it. @jinchengchenghh
https://github.com/facebookincubator/velox/pull/13183
I think that the velox part is compatible with spark as of 2025/05, it's just for some reason the velox fix is not adopted in gluten. Right now I think the only gluten fix needed is to remove the kblacklist and test it...
I'll do a comparison with spark, post the results and add appropriate ut, but I think it might essentially just be removing the kblacklist.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4009362498
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4021628168
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on code in PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#discussion_r2895170724
##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
"split"};
const std::unordered_set<std::string> kBlackList =
- {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+ {"split_part", "sequence", "map_from_arrays"};
Review Comment:
I find also the map_from_arrays in black list, another topic, would you like to implement it in Gluten?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3958241316
Please update get-velox.sh to test your PR, then you can verify if both can work well, you may update this line https://github.com/apache/incubator-gluten/blob/5d3f7145cd7fc258aa10b434ea4ec651bd82c764/ep/build-velox/src/get-velox.sh#L28
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3990529567
> Please update the PR description to describe the KLL Sketch is different so that we handle fallback separately.
done
> ack separately.
done~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4419137557
Sorry for missing this PR, looks good to me, do you have further comments? @zhztheplayer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #11651: [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function
URL: https://github.com/apache/gluten/pull/11651
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3988315395
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4473669990
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4021592298
> Please resolve CH CI https://github.com/apache/incubator-gluten/blob/main/docs/get-started/ClickHouse.md#new-ci-system
<img width="1618" height="871" alt="Clipboard_Screenshot_1773039609" src="https://github.com/user-attachments/assets/ed9cae9a-d7ac-4d33-9a89-2a173a2cb56d" />
I don't think the issue is related with the code, HDFS DataNode did not init, maybe retrying the ci would help...?
The code runs fine in internal pipeline:
(editing this after passing internal pipeline)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3990420216
Please update the PR description to describe the KLL Sketch is different so that we handle fallback separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4030722121
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4574334465
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4030749468
https://github.com/apache/incubator-gluten/actions/runs/22893811220/job/66432061435?pr=11651
There was a test failure in some x86 environments. Fixed, but the pipeline might need restart...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4030716571
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4009249734
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3956443095
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4483972176
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4574325503
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3958786689
> Please update get-velox.sh to test your PR, then you can verify if both can work well, you may update this line
>
> https://github.com/apache/incubator-gluten/blob/5d3f7145cd7fc258aa10b434ea4ec651bd82c764/ep/build-velox/src/get-velox.sh#L28
added the 16320 and removed the config
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3956418087
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3981827798
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4523841011
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4029592441
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4494057887
hopefully I fixed spark 40/41 related cases:
<img width="1351" height="645" alt="Clipboard_Screenshot_1779245317" src="https://github.com/user-attachments/assets/76c2e6d8-11a6-4a1b-a546-0c69a687638d" />
PTAL again...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4485469109
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-5086619078
This PR was auto-closed because it has been stalled for 10 days with no activity. Please feel free to reopen if it is still valid. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4009335889
@jinchengchenghh
hi! The gluten side needs to be reviewed as well for the entire pr to work...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]
Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on code in PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#discussion_r2910156771
##########
ep/build-velox/src/get-velox.sh:
##########
@@ -25,7 +25,7 @@ RUN_SETUP_SCRIPT=ON
ENABLE_ENHANCED_FEATURES=OFF
# Developer use only for testing Velox PR.
-UPSTREAM_VELOX_PR_ID=""
+UPSTREAM_VELOX_PR_ID="16320"
Review Comment:
Update here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on code in PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#discussion_r2903512762
##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
"split"};
const std::unordered_set<std::string> kBlackList =
- {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+ {"split_part", "sequence", "map_from_arrays"};
Review Comment:
sure, I will look into it. @jinchengchenghh
https://github.com/facebookincubator/velox/pull/13183
I think that the velox part is compatible with spark as of 2025/05, it's just for some reason the velox fix is not adopted in gluten. Right now I think the only gluten fix needed is to remove the kblacklist and test it...
I'll do a comparison with spark, post the results and add appropriate ut.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "jiangjiangtian (via GitHub)" <gi...@apache.org>.
jiangjiangtian commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4037243895
@Yizhou-Yang Thanks for your implementation! Recently I am cherrying pick your PR and I find that the KLL implementation in Gluten has only one level and discards items in odd position when compacting. I am wondering if this implementation can meet the `accuracy` requirement. Will the relative error rate be too high?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4088781439
ptal again... @jinchengchenghh
> > @Yizhou-Yang Thanks for your implementation! Recently I am cherrying pick your PR and I find that the KLL implementation in Gluten has only one level and discards items in odd position when compacting. I am wondering if this implementation can meet the `accuracy` requirement. Will the relative error rate be too high?
>
> @jiangjiangtian Hi, the code has been fixed. Here's what changed:
>
> **Changelog:**
>
> 1. Rewrote KLL sketch with proper multi-level compaction — the old implementation only had one level and discarded odd-position items, which is essentially random downsampling with no error bound guarantee. Now it uses the standard KLL algorithm: items across multiple levels, level-0 inserts, sort-and-halve compaction promoting to higher levels, with each level-_i_ item representing 2^_i_ original values.
> 2. Merge correctly combines multi-level sketches and re-compacts, instead of simple concatenation + truncation.
>
> **Why some tests are disabled:**
>
> KLL and Spark's native GK (Greenwald-Khanna) are fundamentally different algorithms. Both satisfy the `1/accuracy` error bound, but they produce **different concrete values** at the same percentile boundary. For example, given integers 1–1000, the exact 25th percentile is 250.25 — GK returns 250, KLL may return 251. Both are correct within the error bound, but a strict equality assertion like `assert(result == 250)` will fail.
>
> For those tests I added TestGluten in the approxpercentile suite. The off-by one problem can't be simply solved by adding more layers in partial result, I tried to not disable any test when developing, but the current version is the best I can do for now.
>
> The 4 disabled tests (`approx_percentile`, `summary`, `SPARK-35480`, `SPARK-32908`) all use exact-match assertions against GK's specific output. Rather than modifying upstream Spark tests, we excluded them and added a dedicated `GlutenApproximatePercentileQuerySuite` with tolerance-based assertions that validate correctness for both algorithms.
>
> There are some other tests in collect_list that assumes approx_percentile will fallback, changed to percentile.
>
> I tested only spark35/spark35smj and their slow versions, hopefully it also passes spark33 etc...
PTAL again... @jinchengchenghh
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3958792469
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3981795118
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4450512950
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4477657939
@jinchengchenghh
sorry was not ready... didn't actually run local tests.
I reran all the local tests so hopefully it works now.
<img width="507" height="98" alt="Clipboard_Screenshot_1779107274" src="https://github.com/user-attachments/assets/aab4270d-3903-4fa0-90dd-82047d979436" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3958252989
Do we need the config? Usually we offload the function to native by default
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-4011083414
Please resolve CH CI https://github.com/apache/incubator-gluten/blob/main/docs/get-started/ClickHouse.md#new-ci-system
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4531163201
reran spark 40/spark41 total 1124 cases and fixed issues in new ut. hopefully this is the last time... sorry about rerunning pipeline multiple times... @jinchengchenghh
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4485416849
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4429011349
Could you help resolve the conflict? Then we can merge it, thanks! @Yizhou-Yang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3988334436
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat:support gluten-level approx_percentile [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#issuecomment-3988472430
Run Gluten Clickhouse CI on x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "jiangjiangtian (via GitHub)" <gi...@apache.org>.
jiangjiangtian commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4037244922
@Yizhou-Yang Thanks for your implementation! Recently I am cherrying pick your PR and I find that the KLL implementation in Gluten has only one level and discards items in odd position when compacting. I am wondering if this implementation can meet the `accuracy` requirement. Will the relative error rate be too high?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4037340387
> @Yizhou-Yang Thanks for your implementation! Recently I am cherrying pick your PR and I find that the KLL implementation in Gluten has only one level and discards items in odd position when compacting. I am wondering if this implementation can meet the `accuracy` requirement. Will the relative error rate be too high?
yes, this is part of the reason for the ut failures. The gluten version of kll is written rather hastily and was primarily concerned with simplicity and fallback compatibiltity with velox, and now it seems to have caused problems including but not limited to ut failures and lots of off-by-one issues. I am editing this implementation and doing a line-by-line comparison with the velox one to ensure the gluten side quality, although still some existing uts might need to be excluded.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [incubator-gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on code in PR #11651:
URL: https://github.com/apache/incubator-gluten/pull/11651#discussion_r2903512762
##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
"split"};
const std::unordered_set<std::string> kBlackList =
- {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+ {"split_part", "sequence", "map_from_arrays"};
Review Comment:
sure, I will look into it. @jinchengchenghh
https://github.com/facebookincubator/velox/pull/13183
I think that the velox part is compatible with spark in 2025/05, it's just for some reason the velox spark fix is not adopted in gluten. Right now I think the only gluten fix needed is to remove the kblacklist and test it...
I'll do a comparison with spark to see if it is the case, but it should be a trivial fix.
##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -60,7 +60,7 @@ const std::unordered_set<std::string> kRegexFunctions = {
"split"};
const std::unordered_set<std::string> kBlackList =
- {"split_part", "sequence", "approx_percentile", "map_from_arrays"};
+ {"split_part", "sequence", "map_from_arrays"};
Review Comment:
sure, I will look into it. @jinchengchenghh
https://github.com/facebookincubator/velox/pull/13183
I think that the velox part is compatible with spark as of 2025/05, it's just for some reason the velox spark fix is not adopted in gluten. Right now I think the only gluten fix needed is to remove the kblacklist and test it...
I'll do a comparison with spark to see if it is the case, but it should be a trivial fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "Yizhou-Yang (via GitHub)" <gi...@apache.org>.
Yizhou-Yang commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4459679064
> Could you help resolve the conflict? Then we can merge it, thanks! @Yizhou-Yang
ready @jinchengchenghh
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-4889][VL] feat: Support approx_percentile aggregate function [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #11651:
URL: https://github.com/apache/gluten/pull/11651#issuecomment-4998324348
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org