You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/03/11 03:37:04 UTC
[GitHub] [arrow-datafusion] Ted-Jiang opened a new issue #1985: [Ballista] support datafusion built_in UDAF work in ballista cluster
Ted-Jiang opened a new issue #1985:
URL: https://github.com/apache/arrow-datafusion/issues/1985
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
support datafusion built_in UDAF work in ballista cluster, It will make ballista more like a high-performance OLAP system for
production.
```
❯ select APPROX_DISTINCT(14) from table1;
[2022-03-11T03:27:22Z INFO ballista_core::execution_plans::distributed_query] Connecting to Ballista scheduler at http://localhost:50050
Execution("Status { code: Internal, message: \"Job Gs7JJH3 failed due to General error:
Could not save stage plan: Not implemented: Aggregate function not supported: ApproxDistinct { name: \\\"APPROXDISTINCT(Int64(14))\\\", input_data_type: Int64, expr: Literal { value: Int64(14) } }\",
metadata: MetadataMap { headers: {\"content-type\": \"application/grpc\", \"date\": \"Fri, 11 Mar 2022 03:27:22 GMT\"} }, source: None }")
❯
```
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] liukun4515 commented on issue #1985: [Ballista] support datafusion built_in UDAF work in ballista cluster
Posted by GitBox <gi...@apache.org>.
liukun4515 commented on issue #1985:
URL: https://github.com/apache/arrow-datafusion/issues/1985#issuecomment-1065078319
I think many functions are missed in the ballista but implemented in the datafusion.
@Ted-Jiang
You can give a plan to find them and fix them through the community.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] liukun4515 commented on issue #1985: [Ballista] support datafusion built_in UDAF work in ballista cluster
Posted by GitBox <gi...@apache.org>.
liukun4515 commented on issue #1985:
URL: https://github.com/apache/arrow-datafusion/issues/1985#issuecomment-1065092596
@Ted-Jiang
Do you mean the same thing with this issue https://github.com/apache/arrow-datafusion/issues/1988？
We should make it clear we just have two type functions, one is the built-in function and the other is UDF/UDAF.
You can refer to https://spark.apache.org/docs/latest/sql-ref-functions.html and get some knowledge about that.
https://github.com/apache/arrow-datafusion/pull/1881 is just wanting to resolve the issue of adding a new UDF/UDAF.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Ted-Jiang commented on issue #1985: [Ballista] support datafusion built_in UDAF work in ballista cluster
Posted by GitBox <gi...@apache.org>.
Ted-Jiang commented on issue #1985:
URL: https://github.com/apache/arrow-datafusion/issues/1985#issuecomment-1064813730
@houqp plz assign this to me. We need this in our situation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jiangzhx commented on issue #1985: [Ballista] support datafusion built_in UDAF work in ballista cluster
Posted by GitBox <gi...@apache.org>.
jiangzhx commented on issue #1985:
URL: https://github.com/apache/arrow-datafusion/issues/1985#issuecomment-1064794213
FYI，It seems to be the same problem
https://github.com/apache/arrow-datafusion/pull/1881
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Ted-Jiang commented on issue #1985: [Ballista] support datafusion built_in UDAF work in ballista cluster
Posted by GitBox <gi...@apache.org>.
Ted-Jiang commented on issue #1985:
URL: https://github.com/apache/arrow-datafusion/issues/1985#issuecomment-1064813207
> FYI，It seems to be the same problem #1881
IMO, this PR not work for built in UDAF.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] liukun4515 edited a comment on issue #1985: [Ballista] support datafusion built_in UDAF work in ballista cluster
Posted by GitBox <gi...@apache.org>.
liukun4515 edited a comment on issue #1985:
URL: https://github.com/apache/arrow-datafusion/issues/1985#issuecomment-1065092596
@Ted-Jiang
Do you mean the same thing with this issue https://github.com/apache/arrow-datafusion/issues/1988？
We should make it clear database system just have two type of functions, one is the built-in function and the other is UDF/UDAF.
You can refer to https://spark.apache.org/docs/latest/sql-ref-functions.html and get some knowledge about that.
https://github.com/apache/arrow-datafusion/pull/1881 is just wanting to resolve the issue of adding a new UDF/UDAF.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] liukun4515 edited a comment on issue #1985: [Ballista] support datafusion built_in UDAF work in ballista cluster
Posted by GitBox <gi...@apache.org>.
liukun4515 edited a comment on issue #1985:
URL: https://github.com/apache/arrow-datafusion/issues/1985#issuecomment-1065092596
@Ted-Jiang
Do you mean the same thing with this issue https://github.com/apache/arrow-datafusion/issues/1988？
We should make it clear database system just have two type of functions, one is the built-in function and the other is UDF/UDAF.
You can refer to https://spark.apache.org/docs/latest/sql-ref-functions.html and get some knowledge about that.
https://github.com/apache/arrow-datafusion/pull/1881 just want to resolve the issue of adding a new UDF/UDAF.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Ted-Jiang commented on issue #1985: [Ballista] support datafusion built_in UDAF work in ballista cluster
Posted by GitBox <gi...@apache.org>.
Ted-Jiang commented on issue #1985:
URL: https://github.com/apache/arrow-datafusion/issues/1985#issuecomment-1064741911
work in client without ballista
```
select APPROX_DISTINCT("14") from table1;
+---------------------------+
| APPROXDISTINCT(table1.14) |
+---------------------------+
| 55 |
+---------------------------+
1 row in set. Query took 0.015 seconds.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org