You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by twalthr <gi...@git.apache.org> on 2016/07/18 15:46:32 UTC
[GitHub] flink pull request #2265: [FLINK-3097] [table] Add support for custom functi...
GitHub user twalthr opened a pull request:
https://github.com/apache/flink/pull/2265
[FLINK-3097] [table] Add support for custom functions in Table API
Thanks for contributing to Apache Flink. Before you open your pull request, please take the following check list into consideration.
If your changes take all of the items into account, feel free to open your pull request. For more information and/or questions please refer to the [How To Contribute guide](http://flink.apache.org/how-to-contribute.html).
In addition to going through the list, please provide a meaningful description of your changes.
- [x] General
- The pull request references the related JIRA issue ("[FLINK-XXX] Jira title text")
- The pull request addresses only one issue
- Each commit in the PR has a meaningful commit message (including the JIRA id)
- [ ] Documentation
- Documentation has been added for new functionality
- Old documentation affected by the pull request has been updated
- JavaDoc for public methods has been added
- [x] Tests & Build
- Functionality added by the pull request is covered by tests
- `mvn clean verify` has been executed successfully locally or a Travis build has passed
This PR introduces user-defined scalar functions for the Table and SQL API.
I will add documentation soon, but this is the general syntax so far:
In Java:
```java
public class HashCode extends ScalarFunction {
public int eval(String s) {
return s.hashCode();
}
}
tableEnv.registerFunction("hashCode", new HashCode());
Table result = table.select("text.hashCode()");
Table result = tableEnv.sql("SELECT hashCode(text) FROM MyTable")
```
In Scala:
```scala
object hashCode extends ScalarFunction {
def eval(s: String): Int = {
s.hashCode()
}
}
val result = table.select(hashCode(text));
val result = tableEnv.sql("SELECT hashCode(text) FROM MyTable")
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/twalthr/flink FLINK-3097
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2265.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2265
----
commit 3b792569bccea843646d8c88592e9c20ecf0ed37
Author: twalthr <tw...@apache.org>
Date: 2016-07-12T10:22:41Z
[FLINK-3097] [table] Add support for custom functions in Table API
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2265: [FLINK-3097] [table] Add support for custom functions in ...
Posted by wuchong <gi...@git.apache.org>.
Github user wuchong commented on the issue:
https://github.com/apache/flink/pull/2265
Do we have any google docs or FLIP talking about this design ?
I think the `ScalarFunction` has too many internal functions, and should not be exposed to users. Maybe we can create a new interface for custom functions, such as `UDF` or `UserDefinedFunction`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2265: [FLINK-3097] [table] Add support for custom functions in ...
Posted by twalthr <gi...@git.apache.org>.
Github user twalthr commented on the issue:
https://github.com/apache/flink/pull/2265
Merging...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2265: [FLINK-3097] [table] Add support for custom functions in ...
Posted by wuchong <gi...@git.apache.org>.
Github user wuchong commented on the issue:
https://github.com/apache/flink/pull/2265
Yes, you are right. I'm just a little concerned about the class name of `ScalarFunction`, haha..
In addition, Java Table API should be `table.select("hashCode(text)");` which is better I think. Assume that the eval function takes two or more parameters, `"udf(a,b)"` will be satisfied and be consistent with Scala Table API and SQL on syntax.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2265: [FLINK-3097] [table] Add support for custom functions in ...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the issue:
https://github.com/apache/flink/pull/2265
Ah ok, thats perfect! (about infix and postfix)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2265: [FLINK-3097] [table] Add support for custom functions in ...
Posted by twalthr <gi...@git.apache.org>.
Github user twalthr commented on the issue:
https://github.com/apache/flink/pull/2265
No, there is no FLIP about it. I think a discussion in JIRA or in this PR should be enough. That's why I haven't documented it yet. I was inspired by your [document](https://docs.google.com/document/d/1KMUzvBAWSyQ39T8MyxUi0zNHyvLUnyGMPA7_RLSDpFw/edit). You are right, `ScalarFunction` has many internal functions but they are not exposed to the user, only 2 methods can be overriden.
An interface is not enough as it might be sometimes necessary to override `getReturnType` and `getParameterType`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #2265: [FLINK-3097] [table] Add support for custom functi...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/2265
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2265: [FLINK-3097] [table] Add support for custom functions in ...
Posted by wuchong <gi...@git.apache.org>.
Github user wuchong commented on the issue:
https://github.com/apache/flink/pull/2265
Yes, I see. That's great!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2265: [FLINK-3097] [table] Add support for custom functions in ...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the issue:
https://github.com/apache/flink/pull/2265
Yes, @wuchong's suggestion for the Java Table API seems more extensible.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2265: [FLINK-3097] [table] Add support for custom functions in ...
Posted by twalthr <gi...@git.apache.org>.
Github user twalthr commented on the issue:
https://github.com/apache/flink/pull/2265
I was also thinking a lot about the names, because we have currently many `Function`s in Flink. I chose `UserDefinedFunction` as the top-level function for all user-defined functions such as `ScalarFunction`, `TableFunction`, `AggregateFunction`, or what ever will come in future.
If you have a look into the tests you will see that the Java API supports both: postfix and infix notation. So you can also call functions `hashCode(text)` if you like.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---