You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2021/07/16 11:27:44 UTC
[GitHub] [flink] HuangXingBo opened a new pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
HuangXingBo opened a new pull request #16516:
URL: https://github.com/apache/flink/pull/16516
## What is the purpose of the change
*This pull request will expose new APIs about TableDescriptor in PyFlink*
## Brief change log
- *Add ConfigOptions/TableDescriptor/FormatDescriptor/Schema*
- *Expose create_table/create_temporary_table/from_descriptor of TableEnvironment*
- *Expose Table#execute_insert and StatementSet#add_insert in Python Table API*
## Verifying this change
This change added tests and can be verified as follows:
- *`test_schema.py` , `test_table_descriptor.py`, `test_table_environment_api.py`*
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-881388489
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d1de6d708b5ff108968142549b0a9074a13834db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20568",
"triggerID" : "d1de6d708b5ff108968142549b0a9074a13834db",
"triggerType" : "PUSH"
}, {
"hash" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20652",
"triggerID" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"triggerType" : "PUSH"
}, {
"hash" : "1eeef1fd87a930bafc95d564455b07640e3e22ab",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20692",
"triggerID" : "1eeef1fd87a930bafc95d564455b07640e3e22ab",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 570dd8123f39a9f8c3d17e578d7c564497836f26 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20652)
* 1eeef1fd87a930bafc95d564455b07640e3e22ab Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20692)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dianfu commented on a change in pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
dianfu commented on a change in pull request #16516:
URL: https://github.com/apache/flink/pull/16516#discussion_r672881206
##########
File path: flink-python/pyflink/table/statement_set.py
##########
@@ -52,21 +55,68 @@ def add_insert_sql(self, stmt: str) -> 'StatementSet':
self._j_statement_set.addInsertSql(stmt)
return self
- def add_insert(self, target_path: str, table, overwrite: bool = False) -> 'StatementSet':
+ def add_insert(self,
+ target_path_or_descriptor: Union[str, TableDescriptor],
+ table,
Review comment:
```suggestion
table: Table,
```
##########
File path: flink-python/pyflink/common/config_options.py
##########
@@ -0,0 +1,125 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+from typing import TypeVar, Generic
+
+from pyflink.java_gateway import get_gateway
+
+T = TypeVar('T')
+
+__all__ = ['ConfigOptions', 'ConfigOption']
+
+
+class ConfigOptions(object):
+ """
+ {@code ConfigOptions} are used to build a :class:`~pyflink.common.ConfigOption`. The option is
+ typically built in one of the following patterns:
+
+ Example:
+ ::
+
+ # simple string-valued option with a default value
+ >>> ConfigOptions.key("tmp.dir").string_type().default_value("/tmp")
+ # simple integer-valued option with a default value
+ >>> ConfigOptions.key("application.parallelism").int_type().default_value(100)
+ # option with no default value
+ >>> ConfigOptions.key("user.name").string_type().no_default_value()
+ """
+
+ def __init__(self, j_config_options):
+ self._j_config_options = j_config_options
+
+ @staticmethod
+ def key(key: str):
+ """
+ Starts building a new ConfigOption.
+
+ :param key: The key for the config option.
+ :return: The builder for the config option with the given key.
+ """
+ gateway = get_gateway()
+ j_option_builder = gateway.jvm.org.apache.flink.configuration.ConfigOptions.key(key)
+ return ConfigOptions.OptionBuilder(j_option_builder)
+
+ class OptionBuilder(object):
+ def __init__(self, j_option_builder):
+ self._j_option_builder = j_option_builder
+
+ def boolean_type(self) -> 'ConfigOptions.TypedConfigOptionBuilder[bool]':
+ """
+ Defines that the value of the option should be of bool type.
+ """
+ return ConfigOptions.TypedConfigOptionBuilder(self._j_option_builder.booleanType())
+
+ def int_type(self) -> 'ConfigOptions.TypedConfigOptionBuilder[int]':
+ """
+ Defines that the value of the option should be of int type
+ (from -2,147,483,648 to 2,147,483,647).
+ """
+ return ConfigOptions.TypedConfigOptionBuilder(self._j_option_builder.intType())
+
+ def long_type(self) -> 'ConfigOptions.TypedConfigOptionBuilder[int]':
+ """
+ Defines that the value of the option should be of int type
+ (from -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807).
+ """
+ return ConfigOptions.TypedConfigOptionBuilder(self._j_option_builder.longType())
+
+ def float_type(self) -> 'ConfigOptions.TypedConfigOptionBuilder[float]':
+ """
+ Defines that the value of the option should be of float type
+ (4-byte single precision floating point number).
+ :return:
Review comment:
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-881388489
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d1de6d708b5ff108968142549b0a9074a13834db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20568",
"triggerID" : "d1de6d708b5ff108968142549b0a9074a13834db",
"triggerType" : "PUSH"
}, {
"hash" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20652",
"triggerID" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"triggerType" : "PUSH"
}, {
"hash" : "1eeef1fd87a930bafc95d564455b07640e3e22ab",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1eeef1fd87a930bafc95d564455b07640e3e22ab",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 570dd8123f39a9f8c3d17e578d7c564497836f26 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20652)
* 1eeef1fd87a930bafc95d564455b07640e3e22ab UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-881388489
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d1de6d708b5ff108968142549b0a9074a13834db",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20568",
"triggerID" : "d1de6d708b5ff108968142549b0a9074a13834db",
"triggerType" : "PUSH"
}, {
"hash" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d1de6d708b5ff108968142549b0a9074a13834db Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20568)
* 570dd8123f39a9f8c3d17e578d7c564497836f26 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dianfu commented on a change in pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
dianfu commented on a change in pull request #16516:
URL: https://github.com/apache/flink/pull/16516#discussion_r671985396
##########
File path: flink-python/pyflink/table/schema.py
##########
@@ -0,0 +1,265 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+from typing import Union, List
+
+from pyflink.java_gateway import get_gateway
+from pyflink.table import Expression
+from pyflink.table.expression import _get_java_expression
+from pyflink.table.types import DataType, _to_java_data_type
+from pyflink.util.java_utils import to_jarray
+
+__all__ = ['Schema']
Review comment:
also added in __init__.py?
##########
File path: flink-python/pyflink/table/schema.py
##########
@@ -0,0 +1,265 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+from typing import Union, List
+
+from pyflink.java_gateway import get_gateway
+from pyflink.table import Expression
+from pyflink.table.expression import _get_java_expression
+from pyflink.table.types import DataType, _to_java_data_type
+from pyflink.util.java_utils import to_jarray
+
+__all__ = ['Schema']
+
+
+class Schema(object):
+ """
+ Schema of a table or view.
+
+ A schema represents the schema part of a {@code CREATE TABLE (schema) WITH (options)} DDL
+ statement in SQL. It defines columns of different kind, constraints, time attributes, and
+ watermark strategies. It is possible to reference objects (such as functions or types) across
+ different catalogs.
+
+ This class is used in the API and catalogs to define an unresolved schema that will be
+ translated to ResolvedSchema. Some methods of this class perform basic validation, however, the
+ main validation happens during the resolution. Thus, an unresolved schema can be incomplete and
+ might be enriched or merged with a different schema at a later stage.
+
+ Since an instance of this class is unresolved, it should not be directly persisted. The str()
+ shows only a summary of the contained objects.
+ """
+
+ def __init__(self, j_schema):
+ self._j_schema = j_schema
+
+ @staticmethod
+ def new_builder() -> 'Schema.Builder':
+ gateway = get_gateway()
+ j_builder = gateway.jvm.Schema.newBuilder()
+ return Schema.Builder(j_builder)
+
+ def __str__(self):
+ return self._j_schema.toString()
+
+ def __eq__(self, other):
+ return self.__class__ == other.__class__ and self._j_schema.equals(other._j_schema)
+
+ def __hash__(self):
+ return self._j_schema.hashCode()
+
+ class Builder(object):
+ """
+ A builder for constructing an immutable but still unresolved Schema.
+ """
+
+ def __init__(self, j_builder):
+ self._j_builder = j_builder
+
+ def from_schema(self, unresolved_schema: 'Schema') -> 'Schema.Builder':
+ """
+ Adopts all members from the given unresolved schema.
+ """
+ self._j_builder.fromSchema(unresolved_schema)
Review comment:
```suggestion
self._j_builder.fromSchema(unresolved_schema._j_schema)
```
##########
File path: flink-python/pyflink/common/config_options.py
##########
@@ -0,0 +1,125 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+from typing import TypeVar, Generic
+
+from pyflink.java_gateway import get_gateway
+
+T = TypeVar('T')
+
+__all__ = ['ConfigOptions', 'ConfigOption']
+
+
+class ConfigOptions(object):
+ """
+ {@code ConfigOptions} are used to build a :class:`~pyflink.table.ConfigOption`. The option is
Review comment:
```suggestion
{@code ConfigOptions} are used to build a :class:`~pyflink.common.ConfigOption`. The option is
```
##########
File path: flink-python/pyflink/table/statement_set.py
##########
@@ -52,21 +55,70 @@ def add_insert_sql(self, stmt: str) -> 'StatementSet':
self._j_statement_set.addInsertSql(stmt)
return self
- def add_insert(self, target_path: str, table, overwrite: bool = False) -> 'StatementSet':
+ def add_insert(self,
+ target_path_or_descriptor: Union[str, TableDescriptor],
+ table,
+ overwrite: bool = False) -> 'StatementSet':
"""
- add Table with the given sink table name to the set.
+ Adds a statement that the pipeline defined by the given Table object should be written to a
+ table (backed by a DynamicTableSink) that was registered under the specified path or
+ expressed via the given TableDescriptor.
+
+ 1. When target_path_or_descriptor is a tale path:
+
+ See the documentation of :func:`~TableEnvironment.use_database` or
+ :func:`~TableEnvironment.use_catalog` for the rules on the path resolution.
+
+ 2. When target_path_or_descriptor is a table descriptor:
+
+ The given TableDescriptor is registered as an inline (i.e. anonymous) temporary catalog
+ table (see :func:`~TableEnvironment.create_temporary_table`).
+
+ Then a statement is added to the statement set that inserts the Table object's pipeline
+ into that temporary table.
+
+ This method allows to declare a Schema for the sink descriptor. The declaration is
+ similar to a {@code CREATE TABLE} DDL in SQL and allows to:
+
+ <ul>
+ <li>overwrite automatically derived columns with a custom DataType
+ <li>add metadata columns next to the physical columns
+ <li>declare a primary key
+ </ul>
+
+ It is possible to declare a schema without physical/regular columns. In this case, those
+ columns will be automatically derived and implicitly put at the beginning of the schema
+ declaration.
+
+ Examples:
+ ::
+
+ >>>stmt_set = table_env.create_statement_set()
Review comment:
```suggestion
>>> stmt_set = table_env.create_statement_set()
```
##########
File path: flink-python/pyflink/common/config_options.py
##########
@@ -0,0 +1,125 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+from typing import TypeVar, Generic
+
+from pyflink.java_gateway import get_gateway
+
+T = TypeVar('T')
+
+__all__ = ['ConfigOptions', 'ConfigOption']
Review comment:
What about also adding it in __init__.py? otherwise, I'm afraid that it will not appear in the Python doc
##########
File path: flink-python/pyflink/table/schema.py
##########
@@ -0,0 +1,265 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+from typing import Union, List
+
+from pyflink.java_gateway import get_gateway
+from pyflink.table import Expression
+from pyflink.table.expression import _get_java_expression
+from pyflink.table.types import DataType, _to_java_data_type
+from pyflink.util.java_utils import to_jarray
+
+__all__ = ['Schema']
+
+
+class Schema(object):
+ """
+ Schema of a table or view.
+
+ A schema represents the schema part of a {@code CREATE TABLE (schema) WITH (options)} DDL
+ statement in SQL. It defines columns of different kind, constraints, time attributes, and
+ watermark strategies. It is possible to reference objects (such as functions or types) across
+ different catalogs.
+
+ This class is used in the API and catalogs to define an unresolved schema that will be
+ translated to ResolvedSchema. Some methods of this class perform basic validation, however, the
+ main validation happens during the resolution. Thus, an unresolved schema can be incomplete and
+ might be enriched or merged with a different schema at a later stage.
+
+ Since an instance of this class is unresolved, it should not be directly persisted. The str()
+ shows only a summary of the contained objects.
+ """
+
+ def __init__(self, j_schema):
+ self._j_schema = j_schema
+
+ @staticmethod
+ def new_builder() -> 'Schema.Builder':
+ gateway = get_gateway()
+ j_builder = gateway.jvm.Schema.newBuilder()
+ return Schema.Builder(j_builder)
+
+ def __str__(self):
+ return self._j_schema.toString()
+
+ def __eq__(self, other):
+ return self.__class__ == other.__class__ and self._j_schema.equals(other._j_schema)
+
+ def __hash__(self):
+ return self._j_schema.hashCode()
+
+ class Builder(object):
+ """
+ A builder for constructing an immutable but still unresolved Schema.
+ """
+
+ def __init__(self, j_builder):
+ self._j_builder = j_builder
+
+ def from_schema(self, unresolved_schema: 'Schema') -> 'Schema.Builder':
+ """
+ Adopts all members from the given unresolved schema.
+ """
+ self._j_builder.fromSchema(unresolved_schema)
+ return self
+
+ def from_row_data_type(self, data_type: DataType) -> 'Schema.Builder':
+ """
+ Adopts all fields of the given row as physical columns of the schema.
+ """
+ self._j_builder.fromRowDataType(data_type)
Review comment:
It would be great to improve the test case to also cover this method
##########
File path: flink-python/pyflink/table/schema.py
##########
@@ -0,0 +1,265 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+from typing import Union, List
+
+from pyflink.java_gateway import get_gateway
+from pyflink.table import Expression
+from pyflink.table.expression import _get_java_expression
+from pyflink.table.types import DataType, _to_java_data_type
+from pyflink.util.java_utils import to_jarray
+
+__all__ = ['Schema']
+
+
+class Schema(object):
+ """
+ Schema of a table or view.
+
+ A schema represents the schema part of a {@code CREATE TABLE (schema) WITH (options)} DDL
+ statement in SQL. It defines columns of different kind, constraints, time attributes, and
+ watermark strategies. It is possible to reference objects (such as functions or types) across
+ different catalogs.
+
+ This class is used in the API and catalogs to define an unresolved schema that will be
+ translated to ResolvedSchema. Some methods of this class perform basic validation, however, the
+ main validation happens during the resolution. Thus, an unresolved schema can be incomplete and
+ might be enriched or merged with a different schema at a later stage.
+
+ Since an instance of this class is unresolved, it should not be directly persisted. The str()
+ shows only a summary of the contained objects.
+ """
+
+ def __init__(self, j_schema):
+ self._j_schema = j_schema
+
+ @staticmethod
+ def new_builder() -> 'Schema.Builder':
+ gateway = get_gateway()
+ j_builder = gateway.jvm.Schema.newBuilder()
+ return Schema.Builder(j_builder)
+
+ def __str__(self):
+ return self._j_schema.toString()
+
+ def __eq__(self, other):
+ return self.__class__ == other.__class__ and self._j_schema.equals(other._j_schema)
+
+ def __hash__(self):
+ return self._j_schema.hashCode()
+
+ class Builder(object):
+ """
+ A builder for constructing an immutable but still unresolved Schema.
+ """
+
+ def __init__(self, j_builder):
+ self._j_builder = j_builder
+
+ def from_schema(self, unresolved_schema: 'Schema') -> 'Schema.Builder':
+ """
+ Adopts all members from the given unresolved schema.
+ """
+ self._j_builder.fromSchema(unresolved_schema)
+ return self
+
+ def from_row_data_type(self, data_type: DataType) -> 'Schema.Builder':
+ """
+ Adopts all fields of the given row as physical columns of the schema.
+ """
+ self._j_builder.fromRowDataType(data_type)
Review comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dianfu commented on a change in pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
dianfu commented on a change in pull request #16516:
URL: https://github.com/apache/flink/pull/16516#discussion_r672881206
##########
File path: flink-python/pyflink/table/statement_set.py
##########
@@ -52,21 +55,68 @@ def add_insert_sql(self, stmt: str) -> 'StatementSet':
self._j_statement_set.addInsertSql(stmt)
return self
- def add_insert(self, target_path: str, table, overwrite: bool = False) -> 'StatementSet':
+ def add_insert(self,
+ target_path_or_descriptor: Union[str, TableDescriptor],
+ table,
Review comment:
```suggestion
table: Table,
```
##########
File path: flink-python/pyflink/common/config_options.py
##########
@@ -0,0 +1,125 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+from typing import TypeVar, Generic
+
+from pyflink.java_gateway import get_gateway
+
+T = TypeVar('T')
+
+__all__ = ['ConfigOptions', 'ConfigOption']
+
+
+class ConfigOptions(object):
+ """
+ {@code ConfigOptions} are used to build a :class:`~pyflink.common.ConfigOption`. The option is
+ typically built in one of the following patterns:
+
+ Example:
+ ::
+
+ # simple string-valued option with a default value
+ >>> ConfigOptions.key("tmp.dir").string_type().default_value("/tmp")
+ # simple integer-valued option with a default value
+ >>> ConfigOptions.key("application.parallelism").int_type().default_value(100)
+ # option with no default value
+ >>> ConfigOptions.key("user.name").string_type().no_default_value()
+ """
+
+ def __init__(self, j_config_options):
+ self._j_config_options = j_config_options
+
+ @staticmethod
+ def key(key: str):
+ """
+ Starts building a new ConfigOption.
+
+ :param key: The key for the config option.
+ :return: The builder for the config option with the given key.
+ """
+ gateway = get_gateway()
+ j_option_builder = gateway.jvm.org.apache.flink.configuration.ConfigOptions.key(key)
+ return ConfigOptions.OptionBuilder(j_option_builder)
+
+ class OptionBuilder(object):
+ def __init__(self, j_option_builder):
+ self._j_option_builder = j_option_builder
+
+ def boolean_type(self) -> 'ConfigOptions.TypedConfigOptionBuilder[bool]':
+ """
+ Defines that the value of the option should be of bool type.
+ """
+ return ConfigOptions.TypedConfigOptionBuilder(self._j_option_builder.booleanType())
+
+ def int_type(self) -> 'ConfigOptions.TypedConfigOptionBuilder[int]':
+ """
+ Defines that the value of the option should be of int type
+ (from -2,147,483,648 to 2,147,483,647).
+ """
+ return ConfigOptions.TypedConfigOptionBuilder(self._j_option_builder.intType())
+
+ def long_type(self) -> 'ConfigOptions.TypedConfigOptionBuilder[int]':
+ """
+ Defines that the value of the option should be of int type
+ (from -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807).
+ """
+ return ConfigOptions.TypedConfigOptionBuilder(self._j_option_builder.longType())
+
+ def float_type(self) -> 'ConfigOptions.TypedConfigOptionBuilder[float]':
+ """
+ Defines that the value of the option should be of float type
+ (4-byte single precision floating point number).
+ :return:
Review comment:
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-881388489
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] HuangXingBo closed pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
HuangXingBo closed pull request #16516:
URL: https://github.com/apache/flink/pull/16516
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-881388489
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d1de6d708b5ff108968142549b0a9074a13834db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20568",
"triggerID" : "d1de6d708b5ff108968142549b0a9074a13834db",
"triggerType" : "PUSH"
}, {
"hash" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20652",
"triggerID" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"triggerType" : "PUSH"
}, {
"hash" : "1eeef1fd87a930bafc95d564455b07640e3e22ab",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20692",
"triggerID" : "1eeef1fd87a930bafc95d564455b07640e3e22ab",
"triggerType" : "PUSH"
}, {
"hash" : "59da1ae61fa8835062651623110a6abdb30ca4fd",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20713",
"triggerID" : "59da1ae61fa8835062651623110a6abdb30ca4fd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 59da1ae61fa8835062651623110a6abdb30ca4fd Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20713)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dianfu commented on a change in pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
dianfu commented on a change in pull request #16516:
URL: https://github.com/apache/flink/pull/16516#discussion_r672881206
##########
File path: flink-python/pyflink/table/statement_set.py
##########
@@ -52,21 +55,68 @@ def add_insert_sql(self, stmt: str) -> 'StatementSet':
self._j_statement_set.addInsertSql(stmt)
return self
- def add_insert(self, target_path: str, table, overwrite: bool = False) -> 'StatementSet':
+ def add_insert(self,
+ target_path_or_descriptor: Union[str, TableDescriptor],
+ table,
Review comment:
```suggestion
table: Table,
```
##########
File path: flink-python/pyflink/common/config_options.py
##########
@@ -0,0 +1,125 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+from typing import TypeVar, Generic
+
+from pyflink.java_gateway import get_gateway
+
+T = TypeVar('T')
+
+__all__ = ['ConfigOptions', 'ConfigOption']
+
+
+class ConfigOptions(object):
+ """
+ {@code ConfigOptions} are used to build a :class:`~pyflink.common.ConfigOption`. The option is
+ typically built in one of the following patterns:
+
+ Example:
+ ::
+
+ # simple string-valued option with a default value
+ >>> ConfigOptions.key("tmp.dir").string_type().default_value("/tmp")
+ # simple integer-valued option with a default value
+ >>> ConfigOptions.key("application.parallelism").int_type().default_value(100)
+ # option with no default value
+ >>> ConfigOptions.key("user.name").string_type().no_default_value()
+ """
+
+ def __init__(self, j_config_options):
+ self._j_config_options = j_config_options
+
+ @staticmethod
+ def key(key: str):
+ """
+ Starts building a new ConfigOption.
+
+ :param key: The key for the config option.
+ :return: The builder for the config option with the given key.
+ """
+ gateway = get_gateway()
+ j_option_builder = gateway.jvm.org.apache.flink.configuration.ConfigOptions.key(key)
+ return ConfigOptions.OptionBuilder(j_option_builder)
+
+ class OptionBuilder(object):
+ def __init__(self, j_option_builder):
+ self._j_option_builder = j_option_builder
+
+ def boolean_type(self) -> 'ConfigOptions.TypedConfigOptionBuilder[bool]':
+ """
+ Defines that the value of the option should be of bool type.
+ """
+ return ConfigOptions.TypedConfigOptionBuilder(self._j_option_builder.booleanType())
+
+ def int_type(self) -> 'ConfigOptions.TypedConfigOptionBuilder[int]':
+ """
+ Defines that the value of the option should be of int type
+ (from -2,147,483,648 to 2,147,483,647).
+ """
+ return ConfigOptions.TypedConfigOptionBuilder(self._j_option_builder.intType())
+
+ def long_type(self) -> 'ConfigOptions.TypedConfigOptionBuilder[int]':
+ """
+ Defines that the value of the option should be of int type
+ (from -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807).
+ """
+ return ConfigOptions.TypedConfigOptionBuilder(self._j_option_builder.longType())
+
+ def float_type(self) -> 'ConfigOptions.TypedConfigOptionBuilder[float]':
+ """
+ Defines that the value of the option should be of float type
+ (4-byte single precision floating point number).
+ :return:
Review comment:
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] HuangXingBo commented on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
HuangXingBo commented on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-882503174
@dianfu Thanks a lot for the review. I have addressed the comments at the corresponding commits. About the documentation part, I plan to add the content after the java doc has been updated in https://issues.apache.org/jira/browse/FLINK-23116
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-881388489
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d1de6d708b5ff108968142549b0a9074a13834db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20568",
"triggerID" : "d1de6d708b5ff108968142549b0a9074a13834db",
"triggerType" : "PUSH"
}, {
"hash" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20652",
"triggerID" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 570dd8123f39a9f8c3d17e578d7c564497836f26 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20652)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-881380178
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit d1de6d708b5ff108968142549b0a9074a13834db (Fri Jul 16 11:31:03 UTC 2021)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-881388489
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d1de6d708b5ff108968142549b0a9074a13834db",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20568",
"triggerID" : "d1de6d708b5ff108968142549b0a9074a13834db",
"triggerType" : "PUSH"
}, {
"hash" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20652",
"triggerID" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d1de6d708b5ff108968142549b0a9074a13834db Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20568)
* 570dd8123f39a9f8c3d17e578d7c564497836f26 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20652)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-881388489
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d1de6d708b5ff108968142549b0a9074a13834db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20568",
"triggerID" : "d1de6d708b5ff108968142549b0a9074a13834db",
"triggerType" : "PUSH"
}, {
"hash" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20652",
"triggerID" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"triggerType" : "PUSH"
}, {
"hash" : "1eeef1fd87a930bafc95d564455b07640e3e22ab",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20692",
"triggerID" : "1eeef1fd87a930bafc95d564455b07640e3e22ab",
"triggerType" : "PUSH"
}, {
"hash" : "59da1ae61fa8835062651623110a6abdb30ca4fd",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20713",
"triggerID" : "59da1ae61fa8835062651623110a6abdb30ca4fd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1eeef1fd87a930bafc95d564455b07640e3e22ab Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20692)
* 59da1ae61fa8835062651623110a6abdb30ca4fd Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20713)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] HuangXingBo closed pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
HuangXingBo closed pull request #16516:
URL: https://github.com/apache/flink/pull/16516
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-881380178
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 59da1ae61fa8835062651623110a6abdb30ca4fd (Thu Sep 23 18:07:16 UTC 2021)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-881388489
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d1de6d708b5ff108968142549b0a9074a13834db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20568",
"triggerID" : "d1de6d708b5ff108968142549b0a9074a13834db",
"triggerType" : "PUSH"
}, {
"hash" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20652",
"triggerID" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"triggerType" : "PUSH"
}, {
"hash" : "1eeef1fd87a930bafc95d564455b07640e3e22ab",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20692",
"triggerID" : "1eeef1fd87a930bafc95d564455b07640e3e22ab",
"triggerType" : "PUSH"
}, {
"hash" : "59da1ae61fa8835062651623110a6abdb30ca4fd",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "59da1ae61fa8835062651623110a6abdb30ca4fd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1eeef1fd87a930bafc95d564455b07640e3e22ab Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20692)
* 59da1ae61fa8835062651623110a6abdb30ca4fd UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] HuangXingBo commented on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
HuangXingBo commented on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-881802414
Thanks @Airblader for the review. @dianfu is familiar with PyFlink and he will help review this PR too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] HuangXingBo closed pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
HuangXingBo closed pull request #16516:
URL: https://github.com/apache/flink/pull/16516
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] HuangXingBo commented on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
HuangXingBo commented on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-882503174
@dianfu Thanks a lot for the review. I have addressed the comments at the corresponding commits. About the documentation part, I plan to add the content after the java doc has been updated in https://issues.apache.org/jira/browse/FLINK-23116
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-881388489
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] HuangXingBo commented on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
HuangXingBo commented on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-882503174
@dianfu Thanks a lot for the review. I have addressed the comments at the corresponding commits. About the documentation part, I plan to add the content after the java doc has been updated in https://issues.apache.org/jira/browse/FLINK-23116
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-881388489
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d1de6d708b5ff108968142549b0a9074a13834db",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20568",
"triggerID" : "d1de6d708b5ff108968142549b0a9074a13834db",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d1de6d708b5ff108968142549b0a9074a13834db Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20568)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-881388489
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d1de6d708b5ff108968142549b0a9074a13834db",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20568",
"triggerID" : "d1de6d708b5ff108968142549b0a9074a13834db",
"triggerType" : "PUSH"
}, {
"hash" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20652",
"triggerID" : "570dd8123f39a9f8c3d17e578d7c564497836f26",
"triggerType" : "PUSH"
}, {
"hash" : "1eeef1fd87a930bafc95d564455b07640e3e22ab",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20692",
"triggerID" : "1eeef1fd87a930bafc95d564455b07640e3e22ab",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1eeef1fd87a930bafc95d564455b07640e3e22ab Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20692)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] Airblader commented on a change in pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
Airblader commented on a change in pull request #16516:
URL: https://github.com/apache/flink/pull/16516#discussion_r671391007
##########
File path: flink-python/pyflink/table/catalog.py
##########
@@ -733,9 +734,22 @@ def get_schema(self) -> TableSchema:
Get the schema of the table.
:return: Schema of the table/view.
+
+ . note:: Deprecated in 1.14. This method returns the deprecated ableSchema class. The old
Review comment:
nit: missing "T" in TableSchema"
##########
File path: flink-python/pyflink/common/config_options.py
##########
@@ -0,0 +1,125 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+from typing import TypeVar, Generic
+
+from pyflink.java_gateway import get_gateway
+
+T = TypeVar('T')
+
+__all__ = ['ConfigOptions', 'ConfigOption']
+
+
+class ConfigOptions(object):
+ """
+ {@code ConfigOptions} are used to build a :class:`~pyflink.table.ConfigOption`. The option is
+ typically built in one of the following pattern:
Review comment:
nit: "pattern" → "patterns"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-881388489
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d1de6d708b5ff108968142549b0a9074a13834db",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20568",
"triggerID" : "d1de6d708b5ff108968142549b0a9074a13834db",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d1de6d708b5ff108968142549b0a9074a13834db Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=20568)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #16516: [FLINK-23115][python] Expose new APIs about TableDescriptor in PyFlink
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #16516:
URL: https://github.com/apache/flink/pull/16516#issuecomment-881388489
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "d1de6d708b5ff108968142549b0a9074a13834db",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d1de6d708b5ff108968142549b0a9074a13834db",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d1de6d708b5ff108968142549b0a9074a13834db UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org