You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/10/09 14:18:25 UTC
[GitHub] [flink] SteNicholas opened a new pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
SteNicholas opened a new pull request #13572:
URL: https://github.com/apache/flink/pull/13572
## What is the purpose of the change
*Currently Python DataStream doesn't support Python datetime object in `from_collection` of `StreamExecutionEnvironment`. It should support datetime object by coverting datetime elements in collection based on `to_sql_type` of `DataType`.*
## Brief change log
- *`from_collection` of `StreamExecutionEnvironment` adds converter of elements in collection with `to_sql_type` of `DataType` to support datetime object.*
## Verifying this change
- *`test_from_collection_with_data_types` and `test_from_collection_without_data_types` in `StreamExecutionEnvironmentTests` adds test cases for datetime object to verify whether `from_collection` of `StreamExecutionEnvironment` supports datetime object in DataStream.*
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / **no**)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**)
- The serializers: (yes / **no** / don't know)
- The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
- The S3 file system connector: (yes / **no** / don't know)
## Documentation
- Does this pull request introduce a new feature? (yes / **no**)
- If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
}, {
"hash" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458",
"triggerID" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"triggerType" : "PUSH"
}, {
"hash" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"triggerType" : "PUSH"
}, {
"hash" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7479",
"triggerID" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"triggerType" : "PUSH"
}, {
"hash" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a598a13d463908f3ffe49a201f0f0fb863592c0c UNKNOWN
* 5744e4aaf4df85dd1b080c3fed2cf5fe944133cd Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7479)
* e400d2930762569fe1270c4cd085beec2fe78b87 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
}, {
"hash" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458",
"triggerID" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"triggerType" : "PUSH"
}, {
"hash" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"triggerType" : "PUSH"
}, {
"hash" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7479",
"triggerID" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"triggerType" : "PUSH"
}, {
"hash" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7499",
"triggerID" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"triggerType" : "PUSH"
}, {
"hash" : "d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a598a13d463908f3ffe49a201f0f0fb863592c0c UNKNOWN
* 5744e4aaf4df85dd1b080c3fed2cf5fe944133cd Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7479)
* e400d2930762569fe1270c4cd085beec2fe78b87 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7499)
* d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
}, {
"hash" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458",
"triggerID" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"triggerType" : "PUSH"
}, {
"hash" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"triggerType" : "PUSH"
}, {
"hash" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7479",
"triggerID" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"triggerType" : "PUSH"
}, {
"hash" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7499",
"triggerID" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"triggerType" : "PUSH"
}, {
"hash" : "d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7515",
"triggerID" : "d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83",
"triggerType" : "PUSH"
}, {
"hash" : "8a2dd9e221eb323f402fa22a01ccf72cc78efde4",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8a2dd9e221eb323f402fa22a01ccf72cc78efde4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a598a13d463908f3ffe49a201f0f0fb863592c0c UNKNOWN
* e400d2930762569fe1270c4cd085beec2fe78b87 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7499)
* d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7515)
* 8a2dd9e221eb323f402fa22a01ccf72cc78efde4 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] SteNicholas commented on a change in pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
SteNicholas commented on a change in pull request #13572:
URL: https://github.com/apache/flink/pull/13572#discussion_r503111486
##########
File path: flink-python/pyflink/datastream/stream_execution_environment.py
##########
@@ -35,6 +35,8 @@
from pyflink.datastream.time_characteristic import TimeCharacteristic
from pyflink.java_gateway import get_gateway
from pyflink.serializers import PickleSerializer
+from pyflink.table.types import _from_java_type, _infer_schema_from_data, _create_converter,\
Review comment:
@shuiqiangchen Of course, I would like to extract these methods to util methods.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2a573f226d23ae494719691d82a173fd562372de Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338)
* bcac7231cd0a4c79288e68b9974104c5933385c9 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706185991
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] SteNicholas commented on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
SteNicholas commented on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-708051325
@dianfu CI of flink-python has already been passed. Please help to merge.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
}, {
"hash" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458",
"triggerID" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"triggerType" : "PUSH"
}, {
"hash" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"triggerType" : "PUSH"
}, {
"hash" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7479",
"triggerID" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8b99efde7ceb666b35d49a5c9be02583a4d9d81f Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458)
* a598a13d463908f3ffe49a201f0f0fb863592c0c UNKNOWN
* 5744e4aaf4df85dd1b080c3fed2cf5fe944133cd Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7479)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
}, {
"hash" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458",
"triggerID" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"triggerType" : "PUSH"
}, {
"hash" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"triggerType" : "PUSH"
}, {
"hash" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7479",
"triggerID" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"triggerType" : "PUSH"
}, {
"hash" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7499",
"triggerID" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a598a13d463908f3ffe49a201f0f0fb863592c0c UNKNOWN
* 5744e4aaf4df85dd1b080c3fed2cf5fe944133cd Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7479)
* e400d2930762569fe1270c4cd085beec2fe78b87 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7499)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] shuiqiangchen commented on a change in pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
shuiqiangchen commented on a change in pull request #13572:
URL: https://github.com/apache/flink/pull/13572#discussion_r503749442
##########
File path: flink-python/pyflink/datastream/tests/test_stream_execution_environment.py
##########
@@ -226,50 +227,55 @@ def test_execute(self):
self.assertIsNotNone(str(execution_result))
def test_from_collection_without_data_types(self):
- ds = self.env.from_collection([(1, 'Hi', 'Hello', datetime.date(2014, 9, 13),
- datetime.time(hour=11, minute=0, second=0,
- microsecond=123000),
- datetime.datetime(2018, 3, 11, 3, 0, 0, 123000)),
- (2, 'Hello', 'Hi', datetime.date(2015, 10, 14),
- datetime.time(hour=12, minute=0, second=0,
- microsecond=234000),
- datetime.datetime(2019, 4, 12, 4, 1, 1, 234000))])
+ ds = self.env.from_collection([(1, 'Hi', 'Hello'), (2, 'Hello', 'Hi')])
ds.add_sink(self.test_sink)
self.env.execute("test from collection")
results = self.test_sink.get_results(True)
# user does not specify data types for input data, the collected result should be in
# in tuple format as inputs.
- expected = ["(1, 'Hi', 'Hello', datetime.date(2014, 9, 13), "
- "datetime.time(11, 0, 0, 123000), "
- "datetime.datetime(2018, 3, 11, 3, 0, 0, 123000))",
- "(2, 'Hello', 'Hi', datetime.date(2015, 10, 14), "
- "datetime.time(12, 0, 0, 234000), "
- "datetime.datetime(2019, 4, 12, 4, 1, 1, 234000))"]
+ expected = ["(1, 'Hi', 'Hello')", "(2, 'Hello', 'Hi')"]
results.sort()
expected.sort()
self.assertEqual(expected, results)
def test_from_collection_with_data_types(self):
- ds = self.env.from_collection([(1, 'Hi', 'Hello', datetime.date(2014, 9, 13),
- datetime.time(hour=11, minute=0, second=0,
- microsecond=123000),
- datetime.datetime(2018, 3, 11, 3, 0, 0, 123000)),
- (2, 'Hello', 'Hi', datetime.date(2015, 10, 14),
- datetime.time(hour=12, minute=0, second=0,
- microsecond=234000),
- datetime.datetime(2019, 4, 12, 4, 1, 1, 234000))],
- type_info=Types.ROW([Types.INT(),
- Types.STRING(),
- Types.STRING(),
- Types.SQL_DATE(),
- Types.SQL_TIME(),
- Types.SQL_TIMESTAMP()]))
+ # verify from_collection for the collection with single object.
+ ds = self.env.from_collection(['Hi', 'Hello'], type_info=Types.STRING())
ds.add_sink(self.test_sink)
- self.env.execute("test from collection")
+ self.env.execute("test from collection with single object")
+ results = self.test_sink.get_results(False)
+ expected = ['Hello', 'Hi']
+ results.sort()
+ expected.sort()
+ self.assertEqual(expected, results)
+
+ # verify from_collection for the collection with multiple objects like tuple.
+ ds = self.env.from_collection([(1, None, 1, True, 32767, -2147483648, 1.23, 1.98932,
Review comment:
The test looks good. Here all input elements are wrapped by rows, what if they were basic types? Such as self.env.from_collection([1,2,3], type_info=Types.INT()), or self.env.from_collection(["hello", "python", "flink"], type_info=Types.STRING()), it would be better to cover this scene.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bcac7231cd0a4c79288e68b9974104c5933385c9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415)
* c841a04e852cdfca1522352138f38538d92c6faf UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
}, {
"hash" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458",
"triggerID" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"triggerType" : "PUSH"
}, {
"hash" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"triggerType" : "PUSH"
}, {
"hash" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7479",
"triggerID" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"triggerType" : "PUSH"
}, {
"hash" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7499",
"triggerID" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"triggerType" : "PUSH"
}, {
"hash" : "d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7515",
"triggerID" : "d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83",
"triggerType" : "PUSH"
}, {
"hash" : "8a2dd9e221eb323f402fa22a01ccf72cc78efde4",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7536",
"triggerID" : "8a2dd9e221eb323f402fa22a01ccf72cc78efde4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a598a13d463908f3ffe49a201f0f0fb863592c0c UNKNOWN
* d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7515)
* 8a2dd9e221eb323f402fa22a01ccf72cc78efde4 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7536)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
}, {
"hash" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458",
"triggerID" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8b99efde7ceb666b35d49a5c9be02583a4d9d81f Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
}, {
"hash" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458",
"triggerID" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"triggerType" : "PUSH"
}, {
"hash" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"triggerType" : "PUSH"
}, {
"hash" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8b99efde7ceb666b35d49a5c9be02583a4d9d81f Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458)
* a598a13d463908f3ffe49a201f0f0fb863592c0c UNKNOWN
* 5744e4aaf4df85dd1b080c3fed2cf5fe944133cd UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2a573f226d23ae494719691d82a173fd562372de Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338)
* bcac7231cd0a4c79288e68b9974104c5933385c9 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dianfu commented on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
dianfu commented on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706902859
@shuiqiangchen Could you help to take a look at this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] shuiqiangchen commented on a change in pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
shuiqiangchen commented on a change in pull request #13572:
URL: https://github.com/apache/flink/pull/13572#discussion_r503108048
##########
File path: flink-python/pyflink/datastream/stream_execution_environment.py
##########
@@ -35,6 +35,8 @@
from pyflink.datastream.time_characteristic import TimeCharacteristic
from pyflink.java_gateway import get_gateway
from pyflink.serializers import PickleSerializer
+from pyflink.table.types import _from_java_type, _infer_schema_from_data, _create_converter,\
Review comment:
Importing table types in StreamExecutionEnvironment might be confused logically since table API is based on StreamExecutionEnvironment.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] shuiqiangchen commented on a change in pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
shuiqiangchen commented on a change in pull request #13572:
URL: https://github.com/apache/flink/pull/13572#discussion_r503626715
##########
File path: flink-python/pyflink/common/typeinfo.py
##########
@@ -549,6 +662,8 @@ def _from_java_type(j_type_info: JavaObject) -> TypeInformation:
return Types.BASIC_ARRAY(Types.DOUBLE())
elif _is_instance_of(j_type_info, JBasicArrayTypeInfo.CHAR_ARRAY_TYPE_INFO):
return Types.BASIC_ARRAY(Types.CHAR())
+ elif _is_instance_of(j_type_info, JBasicArrayTypeInfo.STRING_ARRAY_TYPE_INFO):
+ return Types.BASIC_ARRAY(Types.CHAR())
Review comment:
The element type may be `Types.STRING()`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
}, {
"hash" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458",
"triggerID" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c841a04e852cdfca1522352138f38538d92c6faf Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448)
* 8b99efde7ceb666b35d49a5c9be02583a4d9d81f Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] shuiqiangchen commented on a change in pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
shuiqiangchen commented on a change in pull request #13572:
URL: https://github.com/apache/flink/pull/13572#discussion_r503108048
##########
File path: flink-python/pyflink/datastream/stream_execution_environment.py
##########
@@ -35,6 +35,8 @@
from pyflink.datastream.time_characteristic import TimeCharacteristic
from pyflink.java_gateway import get_gateway
from pyflink.serializers import PickleSerializer
+from pyflink.table.types import _from_java_type, _infer_schema_from_data, _create_converter,\
Review comment:
Importing table types in StreamExecutionEnvironment might be confusing logically since table API is based on StreamExecutionEnvironment.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
}, {
"hash" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458",
"triggerID" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bcac7231cd0a4c79288e68b9974104c5933385c9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415)
* c841a04e852cdfca1522352138f38538d92c6faf Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448)
* 8b99efde7ceb666b35d49a5c9be02583a4d9d81f Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
}, {
"hash" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458",
"triggerID" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"triggerType" : "PUSH"
}, {
"hash" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"triggerType" : "PUSH"
}, {
"hash" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7479",
"triggerID" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"triggerType" : "PUSH"
}, {
"hash" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7499",
"triggerID" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"triggerType" : "PUSH"
}, {
"hash" : "d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7515",
"triggerID" : "d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a598a13d463908f3ffe49a201f0f0fb863592c0c UNKNOWN
* 5744e4aaf4df85dd1b080c3fed2cf5fe944133cd Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7479)
* e400d2930762569fe1270c4cd085beec2fe78b87 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7499)
* d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7515)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dianfu commented on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
dianfu commented on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-708110654
@SteNicholas Thanks. The test failures are not related to this PR. Merging....
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
}, {
"hash" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458",
"triggerID" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"triggerType" : "PUSH"
}, {
"hash" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"triggerType" : "PUSH"
}, {
"hash" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7479",
"triggerID" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"triggerType" : "PUSH"
}, {
"hash" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7499",
"triggerID" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"triggerType" : "PUSH"
}, {
"hash" : "d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7515",
"triggerID" : "d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a598a13d463908f3ffe49a201f0f0fb863592c0c UNKNOWN
* e400d2930762569fe1270c4cd085beec2fe78b87 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7499)
* d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7515)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bcac7231cd0a4c79288e68b9974104c5933385c9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] shuiqiangchen commented on a change in pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
shuiqiangchen commented on a change in pull request #13572:
URL: https://github.com/apache/flink/pull/13572#discussion_r503118365
##########
File path: flink-python/pyflink/datastream/stream_execution_environment.py
##########
@@ -711,6 +713,17 @@ def from_collection(self, collection: List[Any],
:param type_info: The TypeInformation for the produced data stream
:return: the data stream representing the given collection.
"""
+ if type_info is None:
+ if any(isinstance(element, (dict, tuple, list)) for element in collection) or \
+ any(hasattr(element, "__dict__") for element in collection):
+ data_type = _infer_schema_from_data(collection)
+ else:
+ data_type = _infer_type(collection)
+ converter = _create_converter(data_type)
Review comment:
Maybe we can design a covert util for DataStream collection elements based on user provided typeInfo. Introducing DataTypes for Table/SQL API in DataStream API is also a bit confusing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] shuiqiangchen commented on a change in pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
shuiqiangchen commented on a change in pull request #13572:
URL: https://github.com/apache/flink/pull/13572#discussion_r503742983
##########
File path: flink-python/src/main/java/org/apache/flink/api/common/python/PythonBridgeUtils.java
##########
@@ -74,6 +75,20 @@
return unpickledData;
}
+ public static List<?> readPythonObjects(String fileName) throws IOException {
Review comment:
This util method will also be used in StreamTableEnvironment._from_elements(), have you tested it？Maybe we can keep it here, and create a new method for reading DataStream element objects.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
}, {
"hash" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458",
"triggerID" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"triggerType" : "PUSH"
}, {
"hash" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"triggerType" : "PUSH"
}, {
"hash" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7479",
"triggerID" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"triggerType" : "PUSH"
}, {
"hash" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7499",
"triggerID" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"triggerType" : "PUSH"
}, {
"hash" : "d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7515",
"triggerID" : "d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83",
"triggerType" : "PUSH"
}, {
"hash" : "8a2dd9e221eb323f402fa22a01ccf72cc78efde4",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7536",
"triggerID" : "8a2dd9e221eb323f402fa22a01ccf72cc78efde4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a598a13d463908f3ffe49a201f0f0fb863592c0c UNKNOWN
* e400d2930762569fe1270c4cd085beec2fe78b87 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7499)
* d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7515)
* 8a2dd9e221eb323f402fa22a01ccf72cc78efde4 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7536)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2a573f226d23ae494719691d82a173fd562372de Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] SteNicholas removed a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
SteNicholas removed a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706904807
> @shuiqiangchen Could you help to take a look at this PR?
@dianfu @shuiqiangchen Wait for my changes. I would have a mirror change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
}, {
"hash" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458",
"triggerID" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"triggerType" : "PUSH"
}, {
"hash" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8b99efde7ceb666b35d49a5c9be02583a4d9d81f Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458)
* a598a13d463908f3ffe49a201f0f0fb863592c0c UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
}, {
"hash" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458",
"triggerID" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"triggerType" : "PUSH"
}, {
"hash" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"triggerType" : "PUSH"
}, {
"hash" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7479",
"triggerID" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"triggerType" : "PUSH"
}, {
"hash" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7499",
"triggerID" : "e400d2930762569fe1270c4cd085beec2fe78b87",
"triggerType" : "PUSH"
}, {
"hash" : "d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7515",
"triggerID" : "d3c1773f6b6dce9ff8e0e8d80b0e9807eef43f83",
"triggerType" : "PUSH"
}, {
"hash" : "8a2dd9e221eb323f402fa22a01ccf72cc78efde4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7536",
"triggerID" : "8a2dd9e221eb323f402fa22a01ccf72cc78efde4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a598a13d463908f3ffe49a201f0f0fb863592c0c UNKNOWN
* 8a2dd9e221eb323f402fa22a01ccf72cc78efde4 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7536)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dianfu commented on a change in pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
dianfu commented on a change in pull request #13572:
URL: https://github.com/apache/flink/pull/13572#discussion_r503813488
##########
File path: flink-python/pyflink/common/typeinfo.py
##########
@@ -70,6 +72,18 @@ def __hash__(self) -> int:
def __str__(self):
return self._j_typeinfo.toString()
+ def need_conversion(self):
+ """
+ Does this type need to conversion between Python object and internal Wrapper object.
+ """
+ return False
+
+ def to_wrapper_type(self, obj):
Review comment:
What about renamed to to_internal_type? The "Wrapper" prefix of the class name means wrapping the Java TypeInformation, however the functionality of "to_wrapper_type" means converting the Python Date/Time/Timestamp objects to int/long type. They are not the same thing and the name "wrapper" causes confusing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
}, {
"hash" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7458",
"triggerID" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"triggerType" : "PUSH"
}, {
"hash" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a598a13d463908f3ffe49a201f0f0fb863592c0c",
"triggerType" : "PUSH"
}, {
"hash" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7479",
"triggerID" : "5744e4aaf4df85dd1b080c3fed2cf5fe944133cd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a598a13d463908f3ffe49a201f0f0fb863592c0c UNKNOWN
* 5744e4aaf4df85dd1b080c3fed2cf5fe944133cd Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7479)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2a573f226d23ae494719691d82a173fd562372de Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] SteNicholas commented on a change in pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
SteNicholas commented on a change in pull request #13572:
URL: https://github.com/apache/flink/pull/13572#discussion_r503302051
##########
File path: flink-python/pyflink/datastream/stream_execution_environment.py
##########
@@ -711,6 +713,17 @@ def from_collection(self, collection: List[Any],
:param type_info: The TypeInformation for the produced data stream
:return: the data stream representing the given collection.
"""
+ if type_info is None:
+ if any(isinstance(element, (dict, tuple, list)) for element in collection) or \
+ any(hasattr(element, "__dict__") for element in collection):
+ data_type = _infer_schema_from_data(collection)
+ else:
+ data_type = _infer_type(collection)
+ converter = _create_converter(data_type)
Review comment:
@shuiqiangchen Yes, I extend the WrapperTypeInfo to support to convert the wrapper object.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706203264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "2a573f226d23ae494719691d82a173fd562372de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7338",
"triggerID" : "2a573f226d23ae494719691d82a173fd562372de",
"triggerType" : "PUSH"
}, {
"hash" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415",
"triggerID" : "bcac7231cd0a4c79288e68b9974104c5933385c9",
"triggerType" : "PUSH"
}, {
"hash" : "c841a04e852cdfca1522352138f38538d92c6faf",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448",
"triggerID" : "c841a04e852cdfca1522352138f38538d92c6faf",
"triggerType" : "PUSH"
}, {
"hash" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8b99efde7ceb666b35d49a5c9be02583a4d9d81f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bcac7231cd0a4c79288e68b9974104c5933385c9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7415)
* c841a04e852cdfca1522352138f38538d92c6faf Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=7448)
* 8b99efde7ceb666b35d49a5c9be02583a4d9d81f UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] SteNicholas commented on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
SteNicholas commented on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706904807
> @shuiqiangchen Could you help to take a look at this PR?
Wait for my changes. I would have a mirror change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] SteNicholas edited a comment on pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
SteNicholas edited a comment on pull request #13572:
URL: https://github.com/apache/flink/pull/13572#issuecomment-706904807
> @shuiqiangchen Could you help to take a look at this PR?
@dianfu @shuiqiangchen Wait for my changes. I would have a mirror change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] SteNicholas commented on a change in pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
SteNicholas commented on a change in pull request #13572:
URL: https://github.com/apache/flink/pull/13572#discussion_r503821712
##########
File path: flink-python/pyflink/common/typeinfo.py
##########
@@ -70,6 +72,18 @@ def __hash__(self) -> int:
def __str__(self):
return self._j_typeinfo.toString()
+ def need_conversion(self):
+ """
+ Does this type need to conversion between Python object and internal Wrapper object.
+ """
+ return False
+
+ def to_wrapper_type(self, obj):
Review comment:
@dianfu OK. I would like to rename `to_wrapper_type` to `to_internal_type `.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dianfu closed pull request #13572: [FLINK-19416][python] Support Python datetime object in from_collection of Python DataStream
Posted by GitBox <gi...@apache.org>.
dianfu closed pull request #13572:
URL: https://github.com/apache/flink/pull/13572
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org