You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/11/16 09:59:24 UTC
[GitHub] [flink] HuangXingBo opened a new pull request, #21332: [FLINK-21223][python] Support to specify the output types of Python UDFs via string
HuangXingBo opened a new pull request, #21332:
URL: https://github.com/apache/flink/pull/21332
## What is the purpose of the change
*This pull request will support to specify the output types of Python UDFs via string*
## Brief change log
- *Support to specify the output types of Python UDFs via string*
## Verifying this change
This change added tests and can be verified as follows:
- *`test_all_data_types_string` and orginal tests in `test_udtf` and `test_udaf`*
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] HuangXingBo closed pull request #21332: [FLINK-21223][python] Support to specify the output types of Python UDFs via string
Posted by GitBox <gi...@apache.org>.
HuangXingBo closed pull request #21332: [FLINK-21223][python] Support to specify the output types of Python UDFs via string
URL: https://github.com/apache/flink/pull/21332
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #21332: [FLINK-21223][python] Support to specify the output types of Python UDFs via string
Posted by GitBox <gi...@apache.org>.
flinkbot commented on PR #21332:
URL: https://github.com/apache/flink/pull/21332#issuecomment-1316738912
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a61f2582e19a56393d9df4f9d658683bba2fb00",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5a61f2582e19a56393d9df4f9d658683bba2fb00",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5a61f2582e19a56393d9df4f9d658683bba2fb00 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] HuangXingBo commented on pull request #21332: [FLINK-21223][python] Support to specify the output types of Python UDFs via string
Posted by GitBox <gi...@apache.org>.
HuangXingBo commented on PR #21332:
URL: https://github.com/apache/flink/pull/21332#issuecomment-1318477532
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dianfu commented on a diff in pull request #21332: [FLINK-21223][python] Support to specify the output types of Python UDFs via string
Posted by GitBox <gi...@apache.org>.
dianfu commented on code in PR #21332:
URL: https://github.com/apache/flink/pull/21332#discussion_r1030214339
##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/functions/python/PythonAggregateFunction.java:
##########
@@ -120,6 +163,15 @@ public TypeInference getTypeInference(DataTypeFactory typeFactory) {
if (inputTypes != null) {
builder.typedArguments(inputTypes);
}
+
+ if (resultType == null) {
Review Comment:
What about moving this to the constructor?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dianfu commented on a diff in pull request #21332: [FLINK-21223][python] Support to specify the output types of Python UDFs via string
Posted by GitBox <gi...@apache.org>.
dianfu commented on code in PR #21332:
URL: https://github.com/apache/flink/pull/21332#discussion_r1030214339
##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/functions/python/PythonAggregateFunction.java:
##########
@@ -120,6 +163,15 @@ public TypeInference getTypeInference(DataTypeFactory typeFactory) {
if (inputTypes != null) {
builder.typedArguments(inputTypes);
}
+
+ if (resultType == null) {
Review Comment:
What about moving this to the constructor?
##########
flink-python/pyflink/table/udf.py:
##########
@@ -516,8 +539,14 @@ def _create_judf(self, serialized_func, j_input_types, j_function_kind):
gateway = get_gateway()
j_input_types = java_utils.to_jarray(
gateway.jvm.DataType, [_to_java_data_type(i) for i in self._input_types])
Review Comment:
`line 536: self._accumulator_type = DataTypes.ARRAY(self._result_type)`
What happens if self._result_type is a string?
##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/functions/python/PythonTableAggregateFunction.java:
##########
@@ -120,6 +163,15 @@ public TypeInference getTypeInference(DataTypeFactory typeFactory) {
if (inputTypes != null) {
builder.typedArguments(inputTypes);
}
+
Review Comment:
`line 152: TypeConversions.fromDataTypeToLegacyInfo(resultType)`
Does it still work?
##########
flink-python/pyflink/table/udf.py:
##########
@@ -516,8 +539,14 @@ def _create_judf(self, serialized_func, j_input_types, j_function_kind):
gateway = get_gateway()
j_input_types = java_utils.to_jarray(
gateway.jvm.DataType, [_to_java_data_type(i) for i in self._input_types])
Review Comment:
Will we support specifying input types via string?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org