You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/02/24 07:12:20 UTC
[GitHub] [flink] hequn8128 opened a new pull request #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
hequn8128 opened a new pull request #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198
## What is the purpose of the change
Align interface for MLEnvironment and MLEnvironmentFactory, so Python users can use Python MLEnvironmentFactory to maintain execution environment and table environment.
## Brief change log
- Add MLEnvironment and MLEnvironmentFactory interface.
- Remove is_blink_planner parameter for TableEnvironment init method.
- Add tests.
## Verifying this change
This change added tests and can be verified as follows:
- Add MLEnvironmentTest to test the logic in MLEnvironment.
- Add MLEnvironmentFactoryTest to test the logic in MLEnvironmentFactory.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (Yes)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (yes)
- If yes, how is the feature documented? (PythonDocs)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-590216973
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5496",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150251595",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150434494",
"triggerID" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"triggerType" : "PUSH"
}, {
"hash" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5549",
"triggerID" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"triggerType" : "PUSH"
}, {
"hash" : "99cac2eb28fb2907b40012969ff9c7b640c91ad1",
"status" : "PENDING",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150459467",
"triggerID" : "99cac2eb28fb2907b40012969ff9c7b640c91ad1",
"triggerType" : "PUSH"
}, {
"hash" : "99cac2eb28fb2907b40012969ff9c7b640c91ad1",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5566",
"triggerID" : "99cac2eb28fb2907b40012969ff9c7b640c91ad1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 99cac2eb28fb2907b40012969ff9c7b640c91ad1 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/150459467) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5566)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot commented on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
flinkbot commented on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-590216973
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2683ef7e092d4026bad2307c4cbb204aa052be1d UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] hequn8128 commented on a change in pull request #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
hequn8128 commented on a change in pull request #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#discussion_r383918946
##########
File path: flink-python/pyflink/table/table_environment.py
##########
@@ -76,14 +76,24 @@ class TableEnvironment(object):
__metaclass__ = ABCMeta
- def __init__(self, j_tenv, is_blink_planner, serializer=PickleSerializer()):
+ def __init__(self, j_tenv, serializer=PickleSerializer()):
Review comment:
It is an internal constructor which takes a java table environment object as a parameter. For users, the `StreamTableEnvironment.create()` is called to create the Environment.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-590216973
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5496",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "PENDING",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150251595",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2683ef7e092d4026bad2307c4cbb204aa052be1d Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/150251595) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5496)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] hequn8128 merged pull request #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
hequn8128 merged pull request #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] hequn8128 commented on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
hequn8128 commented on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-590192859
@becketqin @walterddr Would be great if you can also take a look.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] hequn8128 edited a comment on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
hequn8128 edited a comment on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-590736162
@becketqin Thanks a lot for your review. I have updated the PR according to your comments. (Have also fixed some checkstyle problems.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] hequn8128 commented on a change in pull request #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
hequn8128 commented on a change in pull request #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#discussion_r383706073
##########
File path: flink-python/pyflink/ml/tests/__init__.py
##########
@@ -0,0 +1,58 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+import glob
+import os
+import unittest
+
+from pyflink.testing.test_case_utils import PyFlinkTestCase
+from pyflink.find_flink_home import _find_flink_source_root
+
+
+class MLTestCase(PyFlinkTestCase):
+ """
+ Base class for testing ML.
+ """
+
+ _inited = False
+
+ @staticmethod
+ def _ensure_path(pattern):
Review comment:
It sounds like a good idea. I will move it there.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] hequn8128 edited a comment on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
hequn8128 edited a comment on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-590736162
@becketqin Thanks a lot for your review. I have updated the PR according to your comments. (Have also fixed some checkstyle problem.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-590216973
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "PENDING",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5496",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "PENDING",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150251595",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2683ef7e092d4026bad2307c4cbb204aa052be1d Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/150251595) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5496)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-590216973
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5496",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150251595",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"status" : "PENDING",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150434494",
"triggerID" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2683ef7e092d4026bad2307c4cbb204aa052be1d Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/150251595) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5496)
* 7c09a7798bf12fa375c06a06772d265bfe881ab8 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/150434494)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-590216973
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5496",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150251595",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150434494",
"triggerID" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"triggerType" : "PUSH"
}, {
"hash" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5549",
"triggerID" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7c09a7798bf12fa375c06a06772d265bfe881ab8 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/150434494) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5549)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] walterddr commented on a change in pull request #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#discussion_r384229751
##########
File path: flink-python/pyflink/ml/ml_environment_factory.py
##########
@@ -0,0 +1,113 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+from typing import Optional
+from pyflink.ml.ml_environment import MLEnvironment
+from pyflink.dataset import ExecutionEnvironment
+from pyflink.datastream import StreamExecutionEnvironment
+from pyflink.table import BatchTableEnvironment, StreamTableEnvironment
+from pyflink.java_gateway import get_gateway
+import threading
+
+
+class MLEnvironmentFactory:
+ """
+ Factory to get the MLEnvironment using a MLEnvironmentId.
+ """
+ _lock = threading.RLock()
+ _default_ml_environment_id = 0
+ _next_id = 1
+ _map = {_default_ml_environment_id: None}
+
+ @staticmethod
+ def get(ml_env_id: int) -> Optional[MLEnvironment]:
Review comment:
Looks like the ID in Java is `Long` type?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-590216973
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5496",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150251595",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150434494",
"triggerID" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"triggerType" : "PUSH"
}, {
"hash" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5549",
"triggerID" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"triggerType" : "PUSH"
}, {
"hash" : "99cac2eb28fb2907b40012969ff9c7b640c91ad1",
"status" : "PENDING",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150459467",
"triggerID" : "99cac2eb28fb2907b40012969ff9c7b640c91ad1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7c09a7798bf12fa375c06a06772d265bfe881ab8 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/150434494) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5549)
* 99cac2eb28fb2907b40012969ff9c7b640c91ad1 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/150459467)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] becketqin commented on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
becketqin commented on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-591188935
The patch LGTM.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] becketqin commented on a change in pull request #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
becketqin commented on a change in pull request #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#discussion_r383792886
##########
File path: flink-python/pyflink/table/table_environment.py
##########
@@ -76,14 +76,24 @@ class TableEnvironment(object):
__metaclass__ = ABCMeta
- def __init__(self, j_tenv, is_blink_planner, serializer=PickleSerializer()):
+ def __init__(self, j_tenv, serializer=PickleSerializer()):
Review comment:
Do we expect others to construct `TableEnvironment`? Isn't this a public constructor?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] hequn8128 commented on a change in pull request #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
hequn8128 commented on a change in pull request #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#discussion_r384232990
##########
File path: flink-python/pyflink/ml/ml_environment_factory.py
##########
@@ -0,0 +1,113 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+from typing import Optional
+from pyflink.ml.ml_environment import MLEnvironment
+from pyflink.dataset import ExecutionEnvironment
+from pyflink.datastream import StreamExecutionEnvironment
+from pyflink.table import BatchTableEnvironment, StreamTableEnvironment
+from pyflink.java_gateway import get_gateway
+import threading
+
+
+class MLEnvironmentFactory:
+ """
+ Factory to get the MLEnvironment using a MLEnvironmentId.
+ """
+ _lock = threading.RLock()
+ _default_ml_environment_id = 0
+ _next_id = 1
+ _map = {_default_ml_environment_id: None}
+
+ @staticmethod
+ def get(ml_env_id: int) -> Optional[MLEnvironment]:
Review comment:
Hi, appreciate your feedback! There is no `Long` type in Python3 and we drop Python2 support a few months ago. Integers in Python 3 are of unlimited size.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] becketqin commented on a change in pull request #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
becketqin commented on a change in pull request #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#discussion_r384229247
##########
File path: flink-python/pyflink/table/table_environment.py
##########
@@ -76,14 +76,24 @@ class TableEnvironment(object):
__metaclass__ = ABCMeta
- def __init__(self, j_tenv, is_blink_planner, serializer=PickleSerializer()):
+ def __init__(self, j_tenv, serializer=PickleSerializer()):
Review comment:
Got it. Thanks for the explanation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] hequn8128 commented on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
hequn8128 commented on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-591195928
@becketqin @walterddr Hi, thanks a lot for your review and nice feedback. 👍
I will merge this PR late today. Let me know if there are any other problems. Thanks.
Best, Hequn
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-590216973
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5496",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150251595",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150434494",
"triggerID" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"triggerType" : "PUSH"
}, {
"hash" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5549",
"triggerID" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"triggerType" : "PUSH"
}, {
"hash" : "99cac2eb28fb2907b40012969ff9c7b640c91ad1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "99cac2eb28fb2907b40012969ff9c7b640c91ad1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7c09a7798bf12fa375c06a06772d265bfe881ab8 Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/150434494) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5549)
* 99cac2eb28fb2907b40012969ff9c7b640c91ad1 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-590216973
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5496",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150251595",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2683ef7e092d4026bad2307c4cbb204aa052be1d Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/150251595) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5496)
* 7c09a7798bf12fa375c06a06772d265bfe881ab8 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] becketqin commented on a change in pull request #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
becketqin commented on a change in pull request #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#discussion_r383638628
##########
File path: flink-python/pyflink/table/table_environment.py
##########
@@ -76,14 +76,24 @@ class TableEnvironment(object):
__metaclass__ = ABCMeta
- def __init__(self, j_tenv, is_blink_planner, serializer=PickleSerializer()):
+ def __init__(self, j_tenv, serializer=PickleSerializer()):
Review comment:
Are the changes in this class relevant to the `MLEnvironment` and `MLEnvironmentFactory`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-590216973
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5496",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "FAILURE",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150251595",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2683ef7e092d4026bad2307c4cbb204aa052be1d Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/150251595) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5496)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] becketqin commented on a change in pull request #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
becketqin commented on a change in pull request #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#discussion_r383637498
##########
File path: flink-python/pyflink/ml/tests/__init__.py
##########
@@ -0,0 +1,58 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+import glob
+import os
+import unittest
+
+from pyflink.testing.test_case_utils import PyFlinkTestCase
+from pyflink.find_flink_home import _find_flink_source_root
+
+
+class MLTestCase(PyFlinkTestCase):
+ """
+ Base class for testing ML.
+ """
+
+ _inited = False
+
+ @staticmethod
+ def _ensure_path(pattern):
Review comment:
Should this method be in `pyflink.testing.test_case_util`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-590216973
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5496",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150251595",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150434494",
"triggerID" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"triggerType" : "PUSH"
}, {
"hash" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5549",
"triggerID" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"triggerType" : "PUSH"
}, {
"hash" : "99cac2eb28fb2907b40012969ff9c7b640c91ad1",
"status" : "SUCCESS",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150459467",
"triggerID" : "99cac2eb28fb2907b40012969ff9c7b640c91ad1",
"triggerType" : "PUSH"
}, {
"hash" : "99cac2eb28fb2907b40012969ff9c7b640c91ad1",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5566",
"triggerID" : "99cac2eb28fb2907b40012969ff9c7b640c91ad1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 99cac2eb28fb2907b40012969ff9c7b640c91ad1 Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/150459467) Azure: [SUCCESS](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5566)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] becketqin commented on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
becketqin commented on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-591189134
@flinkbot approve all
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot commented on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
flinkbot commented on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-590196380
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 2683ef7e092d4026bad2307c4cbb204aa052be1d (Mon Feb 24 07:31:21 UTC 2020)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] hequn8128 commented on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
hequn8128 commented on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-590736162
@becketqin Thanks a lot for your review. I have updated the PR according to your comments.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#issuecomment-590216973
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "DELETED",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5496",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"status" : "DELETED",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150251595",
"triggerID" : "2683ef7e092d4026bad2307c4cbb204aa052be1d",
"triggerType" : "PUSH"
}, {
"hash" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"status" : "PENDING",
"url" : "https://travis-ci.com/flink-ci/flink/builds/150434494",
"triggerID" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"triggerType" : "PUSH"
}, {
"hash" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5549",
"triggerID" : "7c09a7798bf12fa375c06a06772d265bfe881ab8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7c09a7798bf12fa375c06a06772d265bfe881ab8 Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/150434494) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5549)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] hequn8128 commented on a change in pull request #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
hequn8128 commented on a change in pull request #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#discussion_r383707337
##########
File path: flink-python/pyflink/table/table_environment.py
##########
@@ -76,14 +76,24 @@ class TableEnvironment(object):
__metaclass__ = ABCMeta
- def __init__(self, j_tenv, is_blink_planner, serializer=PickleSerializer()):
+ def __init__(self, j_tenv, serializer=PickleSerializer()):
Review comment:
Yes, it will be more friendly to create the corresponding table environment in `MLEnvironmentFactory.get_default()`, e.g., we don not need to pass the is_blink_planner variable when creating `StreamTableEnvironment`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] hequn8128 commented on a change in pull request #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
hequn8128 commented on a change in pull request #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#discussion_r383706073
##########
File path: flink-python/pyflink/ml/tests/__init__.py
##########
@@ -0,0 +1,58 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+import glob
+import os
+import unittest
+
+from pyflink.testing.test_case_utils import PyFlinkTestCase
+from pyflink.find_flink_home import _find_flink_source_root
+
+
+class MLTestCase(PyFlinkTestCase):
+ """
+ Base class for testing ML.
+ """
+
+ _inited = False
+
+ @staticmethod
+ def _ensure_path(pattern):
Review comment:
Yes, it would be better.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] hequn8128 commented on a change in pull request #11198:
[FLINK-16248][python][ml] Add interfaces for MLEnvironment and
MLEnvironmentFactory
Posted by GitBox <gi...@apache.org>.
hequn8128 commented on a change in pull request #11198: [FLINK-16248][python][ml] Add interfaces for MLEnvironment and MLEnvironmentFactory
URL: https://github.com/apache/flink/pull/11198#discussion_r384232990
##########
File path: flink-python/pyflink/ml/ml_environment_factory.py
##########
@@ -0,0 +1,113 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+from typing import Optional
+from pyflink.ml.ml_environment import MLEnvironment
+from pyflink.dataset import ExecutionEnvironment
+from pyflink.datastream import StreamExecutionEnvironment
+from pyflink.table import BatchTableEnvironment, StreamTableEnvironment
+from pyflink.java_gateway import get_gateway
+import threading
+
+
+class MLEnvironmentFactory:
+ """
+ Factory to get the MLEnvironment using a MLEnvironmentId.
+ """
+ _lock = threading.RLock()
+ _default_ml_environment_id = 0
+ _next_id = 1
+ _map = {_default_ml_environment_id: None}
+
+ @staticmethod
+ def get(ml_env_id: int) -> Optional[MLEnvironment]:
Review comment:
There is no `Long` type in Python3 and we drop Python2 support a few months ago. Integers in Python 3 are of unlimited size.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services