You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "taiyang-li (via GitHub)" <gi...@apache.org> on 2024/03/13 06:52:17 UTC
[I] [CH] Support skewness aggregate function [incubator-gluten]
taiyang-li opened a new issue, #4945:
URL: https://github.com/apache/incubator-gluten/issues/4945
### Description
sss
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [CH] Support skewness aggregate function [gluten]
Posted by "navaneethsujith09 (via GitHub)" <gi...@apache.org>.
navaneethsujith09 commented on issue #4945:
URL: https://github.com/apache/gluten/issues/4945#issuecomment-4702686359
Hi, I have submitted a PR ([CH] Support skewness aggregate function #12294 ). Please let me know if I should make any changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [CH] Support skewness aggregate function [gluten]
Posted by "navaneethsujith09 (via GitHub)" <gi...@apache.org>.
navaneethsujith09 commented on issue #4945:
URL: https://github.com/apache/gluten/issues/4945#issuecomment-4764025452
@taiyang-li Thank you for reviewing my PR. I have removed the comment. However, I have a question regarding enabling the skewness UTs in Spark.
From what I can tell, the only Spark SQL golden-file UT covering skewness is in group-by.sql, which is currently listed in IGNORE_SQL_QUERY_LIST in ClickHouseSQLQueryTestSettings.scala (Spark 3.5/4.0/4.1). I'm happy to re-open it similar to how #12033 re-opened other queries, but I wanted to confirm the expected approach first.
The challenge is that the aggregate query in group-by.sql also uses KURTOSIS, which is still unsupported by the CH backend. As a result, that query would fall back to vanilla Spark even though skewness now executes natively. Since the golden-file tests compare query results rather than asserting no-fallback, I would expect the test to still pass, but enabling it would mean re-opening the entire group-by.sql file rather than only the skewness query.
Would you prefer that I:
(a) re-open the full group-by.sql file and rely on CI to validate the remaining queries, or
(b) add a dedicated skewness SQL test so that only skewness is enabled?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [CH] Support skewness aggregate function [gluten]
Posted by "navaneethsujith09 (via GitHub)" <gi...@apache.org>.
navaneethsujith09 commented on issue #4945:
URL: https://github.com/apache/gluten/issues/4945#issuecomment-4802991984
@taiyang-li Sorry, I didn't catch that issue. It should be fixed now since the issue was that line 108 was greater than 100 characters. Can you please check if anything else needs to be done and if not, approve and integrate my changes to the repository.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [CH] Support skewness aggregate function [gluten]
Posted by "navaneethsujith09 (via GitHub)" <gi...@apache.org>.
navaneethsujith09 commented on issue #4945:
URL: https://github.com/apache/gluten/issues/4945#issuecomment-4941826745
@taiyang-li Could you close this issue since it is already completeted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [CH] Support skewness aggregate function [gluten]
Posted by "navaneethsujith09 (via GitHub)" <gi...@apache.org>.
navaneethsujith09 commented on issue #4945:
URL: https://github.com/apache/gluten/issues/4945#issuecomment-4776088649
@taiyang-li Ok, I have added the new test logic as per what you said. Can you please check if anything else needs to be done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [CH] Support skewness aggregate function [gluten]
Posted by "navaneethsujith09 (via GitHub)" <gi...@apache.org>.
navaneethsujith09 commented on issue #4945:
URL: https://github.com/apache/gluten/issues/4945#issuecomment-4785089569
Ok, I removed the duplicate checks. Can you please check if anything else needs to be done and if not, approve and integrate my changes to the repository.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [CH] Support skewness aggregate function [gluten]
Posted by "taiyang-li (via GitHub)" <gi...@apache.org>.
taiyang-li closed issue #4945: [CH] Support skewness aggregate function
URL: https://github.com/apache/gluten/issues/4945
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [CH] Support skewness aggregate function [gluten]
Posted by "taiyang-li (via GitHub)" <gi...@apache.org>.
taiyang-li commented on issue #4945:
URL: https://github.com/apache/gluten/issues/4945#issuecomment-4766592427
@navaneethsujith09 thanks you for you response. I think we can ignore group-by.sql, since it can't be offloaded into gluten. Can you please add a new test logic in "check count distinct with agg fallback", e.g, "select a, count(distinct(b)) , skewness(b) from". In this way, skewness aggregates with keys can also be covered.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [CH] Support skewness aggregate function [gluten]
Posted by "taiyang-li (via GitHub)" <gi...@apache.org>.
taiyang-li commented on issue #4945:
URL: https://github.com/apache/gluten/issues/4945#issuecomment-4697770461
@navaneethsujith09 Thank you very much, I've already assigned it to you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [CH] Support skewness aggregate function [gluten]
Posted by "navaneethsujith09 (via GitHub)" <gi...@apache.org>.
navaneethsujith09 commented on issue #4945:
URL: https://github.com/apache/gluten/issues/4945#issuecomment-4697448119
Hi, I would like to take on this issue if it is not taken yet. Can you please assign me to this issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org