You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2021/10/22 07:11:00 UTC
[GitHub] [flink] slinkydeveloper opened a new pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
slinkydeveloper opened a new pull request #17544:
URL: https://github.com/apache/flink/pull/17544
## What is the purpose of the change
This PR adds support for metadata in filesystem connector. Only `filesystem` metadata is supported, but more will be supported in next iterations of the feature (see https://issues.apache.org/jira/browse/FLINK-19903 for more details).
**Note**: the feature is not implemented for formats using `FileSystemFormatFactory`, which is being removed https://issues.apache.org/jira/browse/FLINK-19845
## Brief change log
- Now `FileSystemTableSource` implements `SupportMetadataReading`
- When metadata is used, a bulk format called `FileInfoExtractorBulkFormat` takes care of adding metadata to the input row.
- Add `ExtendedRowData` to support merging 2 rows providing an index mapping.
## Verifying this change
* `ExtendedRowData` is tested with a new unit test
* A new test case in `FileSystemTableSourceTest` checks everything works during the planning
* `FileSystemITCaseBase` now has 2 new test cases to check metadata works both in combination with partition keys or without. These test cases run for every format that supports metadata.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? docs, JavaDocs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
* 7e8209d6c05454c4ad642a2f3e71d044a2bb65b6 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] slinkydeveloper commented on a change in pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
slinkydeveloper commented on a change in pull request #17544:
URL: https://github.com/apache/flink/pull/17544#discussion_r739121306
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -217,7 +283,12 @@ public String getDefaultPartName() {
@Override
public int[] getProjectFields() {
- return readFields();
+ return projectFields == null
+ ? IntStream.range(0, DataType.getFieldCount(getPhysicalDataType()))
Review comment:
yep. Note that is related to the deprecated `FileInputFormatFactory`, that we're hopefully going to remove soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24615][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
}, {
"hash" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784",
"triggerID" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"triggerType" : "PUSH"
}, {
"hash" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26053",
"triggerID" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"triggerType" : "PUSH"
}, {
"hash" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26061",
"triggerID" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"triggerType" : "PUSH"
}, {
"hash" : "89b9e48aacedba0ff4d47dc8f7fdb3e5315945be",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "89b9e48aacedba0ff4d47dc8f7fdb3e5315945be",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8ed8594acccabd5105cb3cb6bb046e6b1394fb7d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26061)
* 89b9e48aacedba0ff4d47dc8f7fdb3e5315945be UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
* ba7ae1015a95b39276744f6cd0aaf19638e61509 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d23020004351008490b234654859363929e2cdac Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
* 7e8209d6c05454c4ad642a2f3e71d044a2bb65b6 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d23020004351008490b234654859363929e2cdac Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521)
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
}, {
"hash" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784",
"triggerID" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"triggerType" : "PUSH"
}, {
"hash" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26053",
"triggerID" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"triggerType" : "PUSH"
}, {
"hash" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d98738ad4d70cc734983a0dc46da41f2c057b059 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26053)
* 8ed8594acccabd5105cb3cb6bb046e6b1394fb7d UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24615][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
}, {
"hash" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784",
"triggerID" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"triggerType" : "PUSH"
}, {
"hash" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26053",
"triggerID" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"triggerType" : "PUSH"
}, {
"hash" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26061",
"triggerID" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"triggerType" : "PUSH"
}, {
"hash" : "89b9e48aacedba0ff4d47dc8f7fdb3e5315945be",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26166",
"triggerID" : "89b9e48aacedba0ff4d47dc8f7fdb3e5315945be",
"triggerType" : "PUSH"
}, {
"hash" : "50b077bbd46a5a0e7c99dcdd1a252558df2dc526",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26205",
"triggerID" : "50b077bbd46a5a0e7c99dcdd1a252558df2dc526",
"triggerType" : "PUSH"
}, {
"hash" : "d09a0c1190c3da8b7ffa87e11bf266961bcb8695",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d09a0c1190c3da8b7ffa87e11bf266961bcb8695",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 50b077bbd46a5a0e7c99dcdd1a252558df2dc526 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26205)
* d09a0c1190c3da8b7ffa87e11bf266961bcb8695 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
* ba7ae1015a95b39276744f6cd0aaf19638e61509 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
* 7e8209d6c05454c4ad642a2f3e71d044a2bb65b6 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] slinkydeveloper commented on a change in pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
slinkydeveloper commented on a change in pull request #17544:
URL: https://github.com/apache/flink/pull/17544#discussion_r743812037
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -340,64 +406,27 @@ public String asSummaryString() {
return map;
}
- private int[] readFields() {
- if (this.producedDataType != null) {
- return IntStream.range(
- 0,
- (int)
- DataType.getFields(this.producedDataType).stream()
- .filter(
- field ->
- !usedMetadataKeys.contains(
- field.getName()))
- .count())
- .toArray();
- }
- return projectedFields == null
- ? IntStream.range(0, DataType.getFieldCount(getPhysicalDataType())).toArray()
- : Arrays.stream(projectedFields).mapToInt(array -> array[0]).toArray();
- }
+ // --------------------------------------------------------------------------------------------
+ // Methods to apply projections and metadata,
+ // will influence the final output and physical type used by formats
+ // --------------------------------------------------------------------------------------------
@Override
- DataType getPhysicalDataType() {
- if (this.usedMetadataKeys != null) {
- return DataTypes.ROW(
- DataType.getFields(super.getPhysicalDataType()).stream()
- .filter(field -> !usedMetadataKeys.contains(field.getName()))
- .toArray(DataTypes.Field[]::new));
- }
- return super.getPhysicalDataType();
- }
-
- private DataType getProjectedDataType() {
- final DataType physicalDataType =
- this.producedDataType != null
- ? DataTypes.ROW(
- DataType.getFields(this.producedDataType).stream()
- .filter(
- field ->
- !usedMetadataKeys.contains(field.getName()))
- .toArray(DataTypes.Field[]::new))
- : super.getPhysicalDataType();
-
- // If we haven't projected fields, we just return the original physical data type,
- // otherwise we need to compute the physical data type depending on the projected fields.
- if (projectedFields == null) {
- return physicalDataType;
- }
- return DataType.projectFields(physicalDataType, projectedFields);
+ public void applyProjection(int[][] projectedFields) {
+ this.projectFields = projectedFields;
}
@Override
- DataType getPhysicalDataTypeWithoutPartitionColumns() {
- if (this.producedDataType != null) {
- return DataTypes.ROW(
- DataType.getFields(this.producedDataType).stream()
- .filter(field -> !usedMetadataKeys.contains(field.getName()))
- .filter(field -> !partitionKeys.contains(field.getName()))
- .toArray(DataTypes.Field[]::new));
+ public void applyReadableMetadata(List<String> metadataKeys, DataType producedDataType) {
+ if (!metadataKeys.isEmpty()) {
+ this.metadataKeys = metadataKeys;
+ this.fullOutputDataType = producedDataType;
+ // If a projection was pushed down, we need to remove it here because producedDataType
+ // is already projected
+ if (projectFields != null) {
Review comment:
I've updated this code, now it's even better with the new `applyProjections` interface
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
}, {
"hash" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784",
"triggerID" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"triggerType" : "PUSH"
}, {
"hash" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a40d8242fcdc183cada6687f62722c356ad4e36e Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784)
* d98738ad4d70cc734983a0dc46da41f2c057b059 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
* ba7ae1015a95b39276744f6cd0aaf19638e61509 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
* ba7ae1015a95b39276744f6cd0aaf19638e61509 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
* 7e8209d6c05454c4ad642a2f3e71d044a2bb65b6 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
* 7e8209d6c05454c4ad642a2f3e71d044a2bb65b6 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379)
* bbe23062517c6ecd1ff3dd55a221da63fe00b3d3 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
* 7e8209d6c05454c4ad642a2f3e71d044a2bb65b6 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] slinkydeveloper commented on a change in pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
slinkydeveloper commented on a change in pull request #17544:
URL: https://github.com/apache/flink/pull/17544#discussion_r739060715
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -340,64 +406,27 @@ public String asSummaryString() {
return map;
}
- private int[] readFields() {
- if (this.producedDataType != null) {
- return IntStream.range(
- 0,
- (int)
- DataType.getFields(this.producedDataType).stream()
- .filter(
- field ->
- !usedMetadataKeys.contains(
- field.getName()))
- .count())
- .toArray();
- }
- return projectedFields == null
- ? IntStream.range(0, DataType.getFieldCount(getPhysicalDataType())).toArray()
- : Arrays.stream(projectedFields).mapToInt(array -> array[0]).toArray();
- }
+ // --------------------------------------------------------------------------------------------
+ // Methods to apply projections and metadata,
+ // will influence the final output and physical type used by formats
+ // --------------------------------------------------------------------------------------------
@Override
- DataType getPhysicalDataType() {
- if (this.usedMetadataKeys != null) {
- return DataTypes.ROW(
- DataType.getFields(super.getPhysicalDataType()).stream()
- .filter(field -> !usedMetadataKeys.contains(field.getName()))
- .toArray(DataTypes.Field[]::new));
- }
- return super.getPhysicalDataType();
- }
-
- private DataType getProjectedDataType() {
- final DataType physicalDataType =
- this.producedDataType != null
- ? DataTypes.ROW(
- DataType.getFields(this.producedDataType).stream()
- .filter(
- field ->
- !usedMetadataKeys.contains(field.getName()))
- .toArray(DataTypes.Field[]::new))
- : super.getPhysicalDataType();
-
- // If we haven't projected fields, we just return the original physical data type,
- // otherwise we need to compute the physical data type depending on the projected fields.
- if (projectedFields == null) {
- return physicalDataType;
- }
- return DataType.projectFields(physicalDataType, projectedFields);
+ public void applyProjection(int[][] projectedFields) {
+ this.projectFields = projectedFields;
}
@Override
- DataType getPhysicalDataTypeWithoutPartitionColumns() {
- if (this.producedDataType != null) {
- return DataTypes.ROW(
- DataType.getFields(this.producedDataType).stream()
- .filter(field -> !usedMetadataKeys.contains(field.getName()))
- .filter(field -> !partitionKeys.contains(field.getName()))
- .toArray(DataTypes.Field[]::new));
+ public void applyReadableMetadata(List<String> metadataKeys, DataType producedDataType) {
+ if (!metadataKeys.isEmpty()) {
+ this.metadataKeys = metadataKeys;
+ this.fullOutputDataType = producedDataType;
+ // If a projection was pushed down, we need to remove it here because producedDataType
+ // is already projected
+ if (projectFields != null) {
Review comment:
I think we discussed it during our offline discussion, I also added more comments to explain it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379)
* bbe23062517c6ecd1ff3dd55a221da63fe00b3d3 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] slinkydeveloper commented on a change in pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
slinkydeveloper commented on a change in pull request #17544:
URL: https://github.com/apache/flink/pull/17544#discussion_r743812037
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -340,64 +406,27 @@ public String asSummaryString() {
return map;
}
- private int[] readFields() {
- if (this.producedDataType != null) {
- return IntStream.range(
- 0,
- (int)
- DataType.getFields(this.producedDataType).stream()
- .filter(
- field ->
- !usedMetadataKeys.contains(
- field.getName()))
- .count())
- .toArray();
- }
- return projectedFields == null
- ? IntStream.range(0, DataType.getFieldCount(getPhysicalDataType())).toArray()
- : Arrays.stream(projectedFields).mapToInt(array -> array[0]).toArray();
- }
+ // --------------------------------------------------------------------------------------------
+ // Methods to apply projections and metadata,
+ // will influence the final output and physical type used by formats
+ // --------------------------------------------------------------------------------------------
@Override
- DataType getPhysicalDataType() {
- if (this.usedMetadataKeys != null) {
- return DataTypes.ROW(
- DataType.getFields(super.getPhysicalDataType()).stream()
- .filter(field -> !usedMetadataKeys.contains(field.getName()))
- .toArray(DataTypes.Field[]::new));
- }
- return super.getPhysicalDataType();
- }
-
- private DataType getProjectedDataType() {
- final DataType physicalDataType =
- this.producedDataType != null
- ? DataTypes.ROW(
- DataType.getFields(this.producedDataType).stream()
- .filter(
- field ->
- !usedMetadataKeys.contains(field.getName()))
- .toArray(DataTypes.Field[]::new))
- : super.getPhysicalDataType();
-
- // If we haven't projected fields, we just return the original physical data type,
- // otherwise we need to compute the physical data type depending on the projected fields.
- if (projectedFields == null) {
- return physicalDataType;
- }
- return DataType.projectFields(physicalDataType, projectedFields);
+ public void applyProjection(int[][] projectedFields) {
+ this.projectFields = projectedFields;
}
@Override
- DataType getPhysicalDataTypeWithoutPartitionColumns() {
- if (this.producedDataType != null) {
- return DataTypes.ROW(
- DataType.getFields(this.producedDataType).stream()
- .filter(field -> !usedMetadataKeys.contains(field.getName()))
- .filter(field -> !partitionKeys.contains(field.getName()))
- .toArray(DataTypes.Field[]::new));
+ public void applyReadableMetadata(List<String> metadataKeys, DataType producedDataType) {
+ if (!metadataKeys.isEmpty()) {
+ this.metadataKeys = metadataKeys;
+ this.fullOutputDataType = producedDataType;
+ // If a projection was pushed down, we need to remove it here because producedDataType
+ // is already projected
+ if (projectFields != null) {
Review comment:
I've updated this code, now it's even better with the new `applyProjections` interface
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0cd3dcf68c5e470489388cacad95ec40fde6f74d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
* 7e8209d6c05454c4ad642a2f3e71d044a2bb65b6 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
* ba7ae1015a95b39276744f6cd0aaf19638e61509 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24615][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
}, {
"hash" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784",
"triggerID" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"triggerType" : "PUSH"
}, {
"hash" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26053",
"triggerID" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"triggerType" : "PUSH"
}, {
"hash" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26061",
"triggerID" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"triggerType" : "PUSH"
}, {
"hash" : "89b9e48aacedba0ff4d47dc8f7fdb3e5315945be",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26166",
"triggerID" : "89b9e48aacedba0ff4d47dc8f7fdb3e5315945be",
"triggerType" : "PUSH"
}, {
"hash" : "50b077bbd46a5a0e7c99dcdd1a252558df2dc526",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26205",
"triggerID" : "50b077bbd46a5a0e7c99dcdd1a252558df2dc526",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 50b077bbd46a5a0e7c99dcdd1a252558df2dc526 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26205)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
* 7e8209d6c05454c4ad642a2f3e71d044a2bb65b6 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7e8209d6c05454c4ad642a2f3e71d044a2bb65b6 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602)
* 0cd3dcf68c5e470489388cacad95ec40fde6f74d UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379)
* bbe23062517c6ecd1ff3dd55a221da63fe00b3d3 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509)
* efc206d83ea51d690d045cb3dc0529dcb9002534 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] slinkydeveloper commented on a change in pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
slinkydeveloper commented on a change in pull request #17544:
URL: https://github.com/apache/flink/pull/17544#discussion_r739062795
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSink.java
##########
@@ -281,32 +282,46 @@ private RowDataPartitionComputer partitionComputer() {
}
private Optional<CompactReader.Factory<RowData>> createCompactReaderFactory(Context context) {
- if (bulkReaderFormat != null) {
- final BulkFormat<RowData, FileSourceSplit> format =
- bulkReaderFormat.createRuntimeDecoder(
- createSourceContext(context), getPhysicalDataType());
- return Optional.of(CompactBulkReader.factory(format));
- } else if (formatFactory != null) {
+ // TODO FLINK-19845 old format factory, to be removed soon.
+ if (formatFactory != null) {
final InputFormat<RowData, ?> format =
formatFactory.createReader(createReaderContext());
if (format instanceof FileInputFormat) {
//noinspection unchecked
return Optional.of(
FileInputFormatCompactReader.factory((FileInputFormat<RowData>) format));
}
+ return Optional.empty();
+ }
+
+ // Compute fullOutputDataType (including partition fields) and physicalDataType (excluding
+ // partition fields)
+ final DataType fullOutputDataType = getPhysicalDataType();
Review comment:
Renamed to `producedDataType` (same in source)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
* ba7ae1015a95b39276744f6cd0aaf19638e61509 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
* ba7ae1015a95b39276744f6cd0aaf19638e61509 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
* ba7ae1015a95b39276744f6cd0aaf19638e61509 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ba7ae1015a95b39276744f6cd0aaf19638e61509 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362)
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bbe23062517c6ecd1ff3dd55a221da63fe00b3d3 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509)
* efc206d83ea51d690d045cb3dc0529dcb9002534 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379)
* bbe23062517c6ecd1ff3dd55a221da63fe00b3d3 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509)
* efc206d83ea51d690d045cb3dc0529dcb9002534 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
}, {
"hash" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0cd3dcf68c5e470489388cacad95ec40fde6f74d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634)
* a40d8242fcdc183cada6687f62722c356ad4e36e UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] slinkydeveloper commented on a change in pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
slinkydeveloper commented on a change in pull request #17544:
URL: https://github.com/apache/flink/pull/17544#discussion_r737726141
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -315,13 +340,44 @@ public String asSummaryString() {
}
private int[] readFields() {
+ if (this.producedDataType != null) {
+ return IntStream.range(
+ 0,
+ (int)
+ DataType.getFields(this.producedDataType).stream()
+ .filter(
+ field ->
+ !usedMetadataKeys.contains(
+ field.getName()))
+ .count())
+ .toArray();
+ }
return projectedFields == null
? IntStream.range(0, DataType.getFieldCount(getPhysicalDataType())).toArray()
: Arrays.stream(projectedFields).mapToInt(array -> array[0]).toArray();
}
+ @Override
+ DataType getPhysicalDataType() {
+ if (this.usedMetadataKeys != null) {
+ return DataTypes.ROW(
+ DataType.getFields(super.getPhysicalDataType()).stream()
+ .filter(field -> !usedMetadataKeys.contains(field.getName()))
+ .toArray(DataTypes.Field[]::new));
+ }
+ return super.getPhysicalDataType();
+ }
+
private DataType getProjectedDataType() {
- final DataType physicalDataType = super.getPhysicalDataType();
+ final DataType physicalDataType =
+ this.producedDataType != null
+ ? DataTypes.ROW(
+ DataType.getFields(this.producedDataType).stream()
+ .filter(
+ field ->
+ !usedMetadataKeys.contains(field.getName()))
+ .toArray(DataTypes.Field[]::new))
+ : super.getPhysicalDataType();
Review comment:
I think this one can be closed now, reopen if you need it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24615][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
}, {
"hash" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784",
"triggerID" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"triggerType" : "PUSH"
}, {
"hash" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26053",
"triggerID" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"triggerType" : "PUSH"
}, {
"hash" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26061",
"triggerID" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"triggerType" : "PUSH"
}, {
"hash" : "89b9e48aacedba0ff4d47dc8f7fdb3e5315945be",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26166",
"triggerID" : "89b9e48aacedba0ff4d47dc8f7fdb3e5315945be",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 89b9e48aacedba0ff4d47dc8f7fdb3e5315945be Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26166)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] slinkydeveloper commented on a change in pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
slinkydeveloper commented on a change in pull request #17544:
URL: https://github.com/apache/flink/pull/17544#discussion_r743812037
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -340,64 +406,27 @@ public String asSummaryString() {
return map;
}
- private int[] readFields() {
- if (this.producedDataType != null) {
- return IntStream.range(
- 0,
- (int)
- DataType.getFields(this.producedDataType).stream()
- .filter(
- field ->
- !usedMetadataKeys.contains(
- field.getName()))
- .count())
- .toArray();
- }
- return projectedFields == null
- ? IntStream.range(0, DataType.getFieldCount(getPhysicalDataType())).toArray()
- : Arrays.stream(projectedFields).mapToInt(array -> array[0]).toArray();
- }
+ // --------------------------------------------------------------------------------------------
+ // Methods to apply projections and metadata,
+ // will influence the final output and physical type used by formats
+ // --------------------------------------------------------------------------------------------
@Override
- DataType getPhysicalDataType() {
- if (this.usedMetadataKeys != null) {
- return DataTypes.ROW(
- DataType.getFields(super.getPhysicalDataType()).stream()
- .filter(field -> !usedMetadataKeys.contains(field.getName()))
- .toArray(DataTypes.Field[]::new));
- }
- return super.getPhysicalDataType();
- }
-
- private DataType getProjectedDataType() {
- final DataType physicalDataType =
- this.producedDataType != null
- ? DataTypes.ROW(
- DataType.getFields(this.producedDataType).stream()
- .filter(
- field ->
- !usedMetadataKeys.contains(field.getName()))
- .toArray(DataTypes.Field[]::new))
- : super.getPhysicalDataType();
-
- // If we haven't projected fields, we just return the original physical data type,
- // otherwise we need to compute the physical data type depending on the projected fields.
- if (projectedFields == null) {
- return physicalDataType;
- }
- return DataType.projectFields(physicalDataType, projectedFields);
+ public void applyProjection(int[][] projectedFields) {
+ this.projectFields = projectedFields;
}
@Override
- DataType getPhysicalDataTypeWithoutPartitionColumns() {
- if (this.producedDataType != null) {
- return DataTypes.ROW(
- DataType.getFields(this.producedDataType).stream()
- .filter(field -> !usedMetadataKeys.contains(field.getName()))
- .filter(field -> !partitionKeys.contains(field.getName()))
- .toArray(DataTypes.Field[]::new));
+ public void applyReadableMetadata(List<String> metadataKeys, DataType producedDataType) {
+ if (!metadataKeys.isEmpty()) {
+ this.metadataKeys = metadataKeys;
+ this.fullOutputDataType = producedDataType;
+ // If a projection was pushed down, we need to remove it here because producedDataType
+ // is already projected
+ if (projectFields != null) {
Review comment:
I've updated this code, now it's even better with the new `applyProjections` interface
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24615][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
}, {
"hash" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784",
"triggerID" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"triggerType" : "PUSH"
}, {
"hash" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26053",
"triggerID" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"triggerType" : "PUSH"
}, {
"hash" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26061",
"triggerID" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"triggerType" : "PUSH"
}, {
"hash" : "89b9e48aacedba0ff4d47dc8f7fdb3e5315945be",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26166",
"triggerID" : "89b9e48aacedba0ff4d47dc8f7fdb3e5315945be",
"triggerType" : "PUSH"
}, {
"hash" : "50b077bbd46a5a0e7c99dcdd1a252558df2dc526",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26205",
"triggerID" : "50b077bbd46a5a0e7c99dcdd1a252558df2dc526",
"triggerType" : "PUSH"
}, {
"hash" : "d09a0c1190c3da8b7ffa87e11bf266961bcb8695",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26221",
"triggerID" : "d09a0c1190c3da8b7ffa87e11bf266961bcb8695",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 50b077bbd46a5a0e7c99dcdd1a252558df2dc526 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26205)
* d09a0c1190c3da8b7ffa87e11bf266961bcb8695 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26221)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] slinkydeveloper commented on a change in pull request #17544: [FLINK-24615][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
slinkydeveloper commented on a change in pull request #17544:
URL: https://github.com/apache/flink/pull/17544#discussion_r745401218
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -101,54 +109,123 @@ public FileSystemTableSource(
this.bulkReaderFormat = bulkReaderFormat;
this.deserializationFormat = deserializationFormat;
this.formatFactory = formatFactory;
+
+ this.producedDataType = context.getPhysicalRowDataType();
}
@Override
public ScanRuntimeProvider getScanRuntimeProvider(ScanContext scanContext) {
+ // When this table has no partition, just return a empty source.
if (!partitionKeys.isEmpty() && getOrFetchPartitions().isEmpty()) {
- // When this table has no partition, just return a empty source.
return InputFormatProvider.of(new CollectionInputFormat<>(new ArrayList<>(), null));
- } else if (bulkReaderFormat != null) {
- if (bulkReaderFormat instanceof BulkDecodingFormat
- && filters != null
- && filters.size() > 0) {
- ((BulkDecodingFormat<RowData>) bulkReaderFormat).applyFilters(filters);
+ }
+
+ // Physical type is computed from the full data type, filtering out partition and
+ // metadata columns. This type is going to be used by formats to parse the input.
+ List<DataTypes.Field> producedDataTypeFields = DataType.getFields(producedDataType);
+ if (metadataKeys != null) {
+ // If metadata keys are present, then by SupportsReadingMetadata contract all the
+ // metadata columns will be at the end of the producedDataType, so we can just remove
+ // from the list the last metadataKeys.size() fields.
+ producedDataTypeFields =
+ producedDataTypeFields.subList(
+ 0, producedDataTypeFields.size() - metadataKeys.size());
+ }
+ DataType physicalDataType =
+ producedDataTypeFields.stream()
Review comment:
There you go https://issues.apache.org/jira/browse/FLINK-24843
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7e8209d6c05454c4ad642a2f3e71d044a2bb65b6 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602)
* 0cd3dcf68c5e470489388cacad95ec40fde6f74d Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379)
* bbe23062517c6ecd1ff3dd55a221da63fe00b3d3 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509)
* efc206d83ea51d690d045cb3dc0529dcb9002534 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379)
* bbe23062517c6ecd1ff3dd55a221da63fe00b3d3 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379)
* bbe23062517c6ecd1ff3dd55a221da63fe00b3d3 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509)
* efc206d83ea51d690d045cb3dc0529dcb9002534 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
* 7e8209d6c05454c4ad642a2f3e71d044a2bb65b6 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] slinkydeveloper commented on a change in pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
slinkydeveloper commented on a change in pull request #17544:
URL: https://github.com/apache/flink/pull/17544#discussion_r734292104
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -330,4 +386,89 @@ private DataType getProjectedDataType() {
}
return DataType.projectFields(physicalDataType, projectedFields);
}
+
+ @Override
+ DataType getPhysicalDataTypeWithoutPartitionColumns() {
+ if (this.producedDataType != null) {
+ return DataTypes.ROW(
+ DataType.getFields(this.producedDataType).stream()
+ .filter(field -> !usedMetadataKeys.contains(field.getName()))
+ .filter(field -> !partitionKeys.contains(field.getName()))
+ .toArray(DataTypes.Field[]::new));
+ }
+ return super.getPhysicalDataTypeWithoutPartitionColumns();
+ }
+
+ // --------------------------------------------------------------------------------------------
+ // Metadata handling
+ // --------------------------------------------------------------------------------------------
+
+ @Override
+ public Map<String, DataType> listReadableMetadata() {
+ return Arrays.stream(ReadableFileInfo.values())
+ .collect(Collectors.toMap(ReadableFileInfo::getKey, ReadableFileInfo::getDataType));
+ }
+
+ @Override
+ public void applyReadableMetadata(List<String> metadataKeys, DataType producedDataType) {
+ if (metadataKeys.isEmpty()) {
+ // This method should be idempotent
+ this.usedMetadataKeys = null;
+ this.usedMetadata = null;
+ this.producedDataType = null;
+ return;
+ }
+
+ this.usedMetadataKeys = metadataKeys;
+ this.usedMetadata =
+ metadataKeys.stream().map(ReadableFileInfo::resolve).collect(Collectors.toList());
+ this.producedDataType = producedDataType;
+ }
+
+ interface FileInfoAccessor extends Serializable {
+ /**
+ * Access the information from the {@link org.apache.flink.core.fs.FileInputSplit}. The
+ * return value type must be an internal type.
+ */
+ Object getValue(FileSourceSplit split);
+ }
+
+ enum ReadableFileInfo implements Serializable {
+ FILENAME(
+ "filename",
Review comment:
Perhaps should we name this `filepath` or just `path`? From `filename` I expect the filename including the file extension, but excluding the whole path.
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -315,13 +340,44 @@ public String asSummaryString() {
}
private int[] readFields() {
+ if (this.producedDataType != null) {
+ return IntStream.range(
+ 0,
+ (int)
+ DataType.getFields(this.producedDataType).stream()
+ .filter(
+ field ->
+ !usedMetadataKeys.contains(
+ field.getName()))
+ .count())
+ .toArray();
+ }
return projectedFields == null
? IntStream.range(0, DataType.getFieldCount(getPhysicalDataType())).toArray()
: Arrays.stream(projectedFields).mapToInt(array -> array[0]).toArray();
}
+ @Override
+ DataType getPhysicalDataType() {
+ if (this.usedMetadataKeys != null) {
+ return DataTypes.ROW(
+ DataType.getFields(super.getPhysicalDataType()).stream()
+ .filter(field -> !usedMetadataKeys.contains(field.getName()))
+ .toArray(DataTypes.Field[]::new));
+ }
+ return super.getPhysicalDataType();
+ }
+
private DataType getProjectedDataType() {
- final DataType physicalDataType = super.getPhysicalDataType();
+ final DataType physicalDataType =
+ this.producedDataType != null
+ ? DataTypes.ROW(
+ DataType.getFields(this.producedDataType).stream()
+ .filter(
+ field ->
+ !usedMetadataKeys.contains(field.getName()))
+ .toArray(DataTypes.Field[]::new))
+ : super.getPhysicalDataType();
Review comment:
All these changes are unfortunately necessary to support all the different combinations of the types our formats expect now: only physical columns, physical columns + partition columns, projected columns + physical columns, then all the combinations with metadata as well etc... All this code is hopefully going to disappear with https://issues.apache.org/jira/browse/FLINK-24617
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
}, {
"hash" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784",
"triggerID" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"triggerType" : "PUSH"
}, {
"hash" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26053",
"triggerID" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a40d8242fcdc183cada6687f62722c356ad4e36e Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784)
* d98738ad4d70cc734983a0dc46da41f2c057b059 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26053)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* efc206d83ea51d690d045cb3dc0529dcb9002534 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513)
* d23020004351008490b234654859363929e2cdac Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 7e8209d6c05454c4ad642a2f3e71d044a2bb65b6 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379)
* bbe23062517c6ecd1ff3dd55a221da63fe00b3d3 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509)
* efc206d83ea51d690d045cb3dc0529dcb9002534 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] twalthr commented on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
twalthr commented on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-954564321
@slinkydeveloper please update the title and commit to use the correct JIRA issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] slinkydeveloper commented on a change in pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
slinkydeveloper commented on a change in pull request #17544:
URL: https://github.com/apache/flink/pull/17544#discussion_r739053775
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileInfoExtractorBulkFormat.java
##########
@@ -93,10 +129,36 @@ public boolean isSplittable() {
}
private Reader<RowData> wrapReader(Reader<RowData> superReader, FileSourceSplit split) {
- // Fill the metadata row
- final GenericRowData metadataRowData = new GenericRowData(metadataColumnsFunctions.size());
- for (int i = 0; i < metadataColumnsFunctions.size(); i++) {
- metadataRowData.setField(i, metadataColumnsFunctions.get(i).getValue(split));
+ // Fill the metadata + partition columns row
+ final GenericRowData metadataRowData =
Review comment:
Named `fileInfoRowData`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
}, {
"hash" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784",
"triggerID" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"triggerType" : "PUSH"
}, {
"hash" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26053",
"triggerID" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"triggerType" : "PUSH"
}, {
"hash" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26061",
"triggerID" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d98738ad4d70cc734983a0dc46da41f2c057b059 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26053)
* 8ed8594acccabd5105cb3cb6bb046e6b1394fb7d Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26061)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24615][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
}, {
"hash" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784",
"triggerID" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"triggerType" : "PUSH"
}, {
"hash" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26053",
"triggerID" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"triggerType" : "PUSH"
}, {
"hash" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26061",
"triggerID" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"triggerType" : "PUSH"
}, {
"hash" : "89b9e48aacedba0ff4d47dc8f7fdb3e5315945be",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26166",
"triggerID" : "89b9e48aacedba0ff4d47dc8f7fdb3e5315945be",
"triggerType" : "PUSH"
}, {
"hash" : "50b077bbd46a5a0e7c99dcdd1a252558df2dc526",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26205",
"triggerID" : "50b077bbd46a5a0e7c99dcdd1a252558df2dc526",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 89b9e48aacedba0ff4d47dc8f7fdb3e5315945be Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26166)
* 50b077bbd46a5a0e7c99dcdd1a252558df2dc526 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26205)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ba7ae1015a95b39276744f6cd0aaf19638e61509 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362)
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
* ba7ae1015a95b39276744f6cd0aaf19638e61509 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ba7ae1015a95b39276744f6cd0aaf19638e61509 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362)
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ba7ae1015a95b39276744f6cd0aaf19638e61509 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362)
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bbe23062517c6ecd1ff3dd55a221da63fe00b3d3 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509)
* efc206d83ea51d690d045cb3dc0529dcb9002534 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513)
* d23020004351008490b234654859363929e2cdac UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
* ba7ae1015a95b39276744f6cd0aaf19638e61509 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
}, {
"hash" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784",
"triggerID" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"triggerType" : "PUSH"
}, {
"hash" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26053",
"triggerID" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"triggerType" : "PUSH"
}, {
"hash" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26061",
"triggerID" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8ed8594acccabd5105cb3cb6bb046e6b1394fb7d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26061)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24615][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
}, {
"hash" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784",
"triggerID" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"triggerType" : "PUSH"
}, {
"hash" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26053",
"triggerID" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"triggerType" : "PUSH"
}, {
"hash" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26061",
"triggerID" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"triggerType" : "PUSH"
}, {
"hash" : "89b9e48aacedba0ff4d47dc8f7fdb3e5315945be",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26166",
"triggerID" : "89b9e48aacedba0ff4d47dc8f7fdb3e5315945be",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8ed8594acccabd5105cb3cb6bb046e6b1394fb7d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26061)
* 89b9e48aacedba0ff4d47dc8f7fdb3e5315945be Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26166)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] twalthr closed pull request #17544: [FLINK-24615][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
twalthr closed pull request #17544:
URL: https://github.com/apache/flink/pull/17544
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379)
* bbe23062517c6ecd1ff3dd55a221da63fe00b3d3 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] slinkydeveloper commented on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
slinkydeveloper commented on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-953184692
The last commit I made https://github.com/apache/flink/pull/17544/commits/a20fe2427c03ceb71b8d4b5f30bdc282aba530fe fixes the behaviour of `TestCsvDeserializationSchema`. In particular, before this PR, `TestCsvDeserializationSchema` was implicitly assuming that the input data type was a 1:1 mapping to the parsed csv row. This is not the case, and the DeserializationSchema should take care of performing the proper mapping between the input records (in whatever format) and the output `RowData`, which may or may not contain all the fields of the input record and may or may not contain those ordered in the same way the input is. This PR addresses the issue. Note that the problem exists for this particular csv format, because `flink-csv` is still using the old interface (hence, no change) and because json deserialization schema properly map the input object fields to the output row data, without performing any bad assumption: https://github.com/apache/flink/blob/master/flink-formats/fli
nk-json/src/main/java/org/apache/flink/formats/json/JsonToRowDataConverters.java#L334
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bbe23062517c6ecd1ff3dd55a221da63fe00b3d3 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509)
* efc206d83ea51d690d045cb3dc0529dcb9002534 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513)
* d23020004351008490b234654859363929e2cdac Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
* 7e8209d6c05454c4ad642a2f3e71d044a2bb65b6 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] twalthr commented on a change in pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
twalthr commented on a change in pull request #17544:
URL: https://github.com/apache/flink/pull/17544#discussion_r738492595
##########
File path: flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/rules/FlinkBatchRuleSets.scala
##########
@@ -174,13 +174,13 @@ object FlinkBatchRuleSets {
* RuleSet to do push predicate/partition into table scan
*/
val FILTER_TABLESCAN_PUSHDOWN_RULES: RuleSet = RuleSets.ofList(
- // push a filter down into the table scan
- PushFilterIntoTableSourceScanRule.INSTANCE,
- PushFilterIntoLegacyTableSourceScanRule.INSTANCE,
// push partition into the table scan
PushPartitionIntoLegacyTableSourceScanRule.INSTANCE,
// push partition into the dynamic table scan
- PushPartitionIntoTableSourceScanRule.INSTANCE
+ PushPartitionIntoTableSourceScanRule.INSTANCE,
Review comment:
please convert this commit in a hotfix commit before the actual commits for FLINK-24165
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSink.java
##########
@@ -281,32 +282,46 @@ private RowDataPartitionComputer partitionComputer() {
}
private Optional<CompactReader.Factory<RowData>> createCompactReaderFactory(Context context) {
- if (bulkReaderFormat != null) {
- final BulkFormat<RowData, FileSourceSplit> format =
- bulkReaderFormat.createRuntimeDecoder(
- createSourceContext(context), getPhysicalDataType());
- return Optional.of(CompactBulkReader.factory(format));
- } else if (formatFactory != null) {
+ // TODO FLINK-19845 old format factory, to be removed soon.
+ if (formatFactory != null) {
final InputFormat<RowData, ?> format =
formatFactory.createReader(createReaderContext());
if (format instanceof FileInputFormat) {
//noinspection unchecked
return Optional.of(
FileInputFormatCompactReader.factory((FileInputFormat<RowData>) format));
}
+ return Optional.empty();
+ }
+
+ // Compute fullOutputDataType (including partition fields) and physicalDataType (excluding
+ // partition fields)
+ final DataType fullOutputDataType = getPhysicalDataType();
Review comment:
reading this is very confusing `fullOutputDataType = getPhysicalDataType()`? `fullOutputDataType` should be `consumedDataType` or even `physicalRowWithPartitions` everywhere
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileInfoExtractorBulkFormat.java
##########
@@ -93,10 +129,36 @@ public boolean isSplittable() {
}
private Reader<RowData> wrapReader(Reader<RowData> superReader, FileSourceSplit split) {
- // Fill the metadata row
- final GenericRowData metadataRowData = new GenericRowData(metadataColumnsFunctions.size());
- for (int i = 0; i < metadataColumnsFunctions.size(); i++) {
- metadataRowData.setField(i, metadataColumnsFunctions.get(i).getValue(split));
+ // Fill the metadata + partition columns row
+ final GenericRowData metadataRowData =
Review comment:
call this `rowWithMetadataAndPartition`, the name might be long but only `metadataRowData` is confusing
##########
File path: flink-table/flink-table-planner/src/test/resources/org/apache/flink/table/planner/plan/nodes/exec/stream/TableSourceJsonPlanTest_jsonplan/testFilterPushDown.out
##########
@@ -113,4 +113,4 @@
},
"shuffleMode" : "PIPELINED"
} ]
-}
+}
Review comment:
unrelated change?
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -340,64 +406,27 @@ public String asSummaryString() {
return map;
}
- private int[] readFields() {
- if (this.producedDataType != null) {
- return IntStream.range(
- 0,
- (int)
- DataType.getFields(this.producedDataType).stream()
- .filter(
- field ->
- !usedMetadataKeys.contains(
- field.getName()))
- .count())
- .toArray();
- }
- return projectedFields == null
- ? IntStream.range(0, DataType.getFieldCount(getPhysicalDataType())).toArray()
- : Arrays.stream(projectedFields).mapToInt(array -> array[0]).toArray();
- }
+ // --------------------------------------------------------------------------------------------
+ // Methods to apply projections and metadata,
+ // will influence the final output and physical type used by formats
+ // --------------------------------------------------------------------------------------------
@Override
- DataType getPhysicalDataType() {
- if (this.usedMetadataKeys != null) {
- return DataTypes.ROW(
- DataType.getFields(super.getPhysicalDataType()).stream()
- .filter(field -> !usedMetadataKeys.contains(field.getName()))
- .toArray(DataTypes.Field[]::new));
- }
- return super.getPhysicalDataType();
- }
-
- private DataType getProjectedDataType() {
- final DataType physicalDataType =
- this.producedDataType != null
- ? DataTypes.ROW(
- DataType.getFields(this.producedDataType).stream()
- .filter(
- field ->
- !usedMetadataKeys.contains(field.getName()))
- .toArray(DataTypes.Field[]::new))
- : super.getPhysicalDataType();
-
- // If we haven't projected fields, we just return the original physical data type,
- // otherwise we need to compute the physical data type depending on the projected fields.
- if (projectedFields == null) {
- return physicalDataType;
- }
- return DataType.projectFields(physicalDataType, projectedFields);
+ public void applyProjection(int[][] projectedFields) {
+ this.projectFields = projectedFields;
}
@Override
- DataType getPhysicalDataTypeWithoutPartitionColumns() {
- if (this.producedDataType != null) {
- return DataTypes.ROW(
- DataType.getFields(this.producedDataType).stream()
- .filter(field -> !usedMetadataKeys.contains(field.getName()))
- .filter(field -> !partitionKeys.contains(field.getName()))
- .toArray(DataTypes.Field[]::new));
+ public void applyReadableMetadata(List<String> metadataKeys, DataType producedDataType) {
+ if (!metadataKeys.isEmpty()) {
+ this.metadataKeys = metadataKeys;
+ this.fullOutputDataType = producedDataType;
+ // If a projection was pushed down, we need to remove it here because producedDataType
+ // is already projected
+ if (projectFields != null) {
Review comment:
this looks incorrect to me. how can a metadata pushdown suddenly make a pushed projection invalid. we are not recomputing `projectFields` from `fullOutputDataType`, right?
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -292,24 +363,19 @@ public boolean supportsNestedProjection() {
return false;
}
- @Override
- public void applyProjection(int[][] projectedFields) {
- this.projectedFields = projectedFields;
- }
-
@Override
public FileSystemTableSource copy() {
FileSystemTableSource source =
new FileSystemTableSource(
context, bulkReaderFormat, deserializationFormat, formatFactory);
- source.projectedFields = projectedFields;
source.remainingPartitions = remainingPartitions;
source.filters = filters;
source.limit = limit;
Review comment:
remove empty line?
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -217,7 +283,12 @@ public String getDefaultPartName() {
@Override
public int[] getProjectFields() {
- return readFields();
+ return projectFields == null
+ ? IntStream.range(0, DataType.getFieldCount(getPhysicalDataType()))
Review comment:
this will include partitions, is that correct? btw should we rename `getPhysicalDataType()`?
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -83,15 +83,13 @@
@Nullable private final DecodingFormat<DeserializationSchema<RowData>> deserializationFormat;
@Nullable private final FileSystemFormatFactory formatFactory;
- private int[][] projectedFields;
private List<Map<String, String>> remainingPartitions;
private List<ResolvedExpression> filters;
private Long limit;
Review comment:
nit: remove empty line? or is there a logical separation between the upper and lower member field groups
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -292,24 +363,19 @@ public boolean supportsNestedProjection() {
return false;
}
- @Override
- public void applyProjection(int[][] projectedFields) {
- this.projectedFields = projectedFields;
- }
-
@Override
public FileSystemTableSource copy() {
FileSystemTableSource source =
new FileSystemTableSource(
context, bulkReaderFormat, deserializationFormat, formatFactory);
- source.projectedFields = projectedFields;
source.remainingPartitions = remainingPartitions;
source.filters = filters;
source.limit = limit;
- source.producedDataType = producedDataType;
- source.usedMetadata = usedMetadata;
- source.usedMetadataKeys = usedMetadataKeys;
+ source.projectFields = projectFields;
+ source.metadataKeys = metadataKeys;
+ source.partitionKeys = partitionKeys;
Review comment:
nit: order the variables and variable assignment according to the application order? first partitions, then project, then metadata?
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/AbstractFileSystemTable.java
##########
@@ -40,7 +40,7 @@
final ObjectIdentifier tableIdentifier;
final Configuration tableOptions;
final ResolvedSchema schema;
- final List<String> partitionKeys;
+ List<String> partitionKeys;
Review comment:
nit: separate mutable members from constant members, maybe be a empty line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
}, {
"hash" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784",
"triggerID" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a40d8242fcdc183cada6687f62722c356ad4e36e Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ba7ae1015a95b39276744f6cd0aaf19638e61509 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362)
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] slinkydeveloper commented on a change in pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
slinkydeveloper commented on a change in pull request #17544:
URL: https://github.com/apache/flink/pull/17544#discussion_r734647200
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -315,13 +340,44 @@ public String asSummaryString() {
}
private int[] readFields() {
+ if (this.producedDataType != null) {
+ return IntStream.range(
+ 0,
+ (int)
+ DataType.getFields(this.producedDataType).stream()
+ .filter(
+ field ->
+ !usedMetadataKeys.contains(
+ field.getName()))
+ .count())
+ .toArray();
+ }
return projectedFields == null
? IntStream.range(0, DataType.getFieldCount(getPhysicalDataType())).toArray()
: Arrays.stream(projectedFields).mapToInt(array -> array[0]).toArray();
}
+ @Override
+ DataType getPhysicalDataType() {
+ if (this.usedMetadataKeys != null) {
+ return DataTypes.ROW(
+ DataType.getFields(super.getPhysicalDataType()).stream()
+ .filter(field -> !usedMetadataKeys.contains(field.getName()))
+ .toArray(DataTypes.Field[]::new));
+ }
+ return super.getPhysicalDataType();
+ }
+
private DataType getProjectedDataType() {
- final DataType physicalDataType = super.getPhysicalDataType();
+ final DataType physicalDataType =
+ this.producedDataType != null
Review comment:
That's something I wanted to do as well, but then we still have to provide different types depending on the factory we use https://github.com/apache/flink/pull/17544/files#diff-c0ba40e7006e9c6b816201174e8cccc46c3766025d919acf6b15bb537775321dR117. So either we save all the different combinations, or we compute them every time. I rather prefer to keep this as temporary solution, as https://issues.apache.org/jira/browse/FLINK-24617 and https://issues.apache.org/jira/browse/FLINK-19845 should allow us to do a proper cleanup of all the different combinations, and save just two of them:
* only physical projected fields, which is what format factories will accept
* physical + partition + metadata, which is what FileInfoExtractorBulkFormat will accept
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
* ba7ae1015a95b39276744f6cd0aaf19638e61509 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cb72fd0b0045437e947bfde4e729d0abb1df5b22 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ba7ae1015a95b39276744f6cd0aaf19638e61509 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] slinkydeveloper commented on a change in pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
slinkydeveloper commented on a change in pull request #17544:
URL: https://github.com/apache/flink/pull/17544#discussion_r734637968
##########
File path: flink-table/flink-table-common/src/main/java/org/apache/flink/table/data/utils/ExtendedRowData.java
##########
@@ -0,0 +1,312 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.data.utils;
+
+import org.apache.flink.annotation.PublicEvolving;
+import org.apache.flink.table.data.ArrayData;
+import org.apache.flink.table.data.DecimalData;
+import org.apache.flink.table.data.MapData;
+import org.apache.flink.table.data.RawValueData;
+import org.apache.flink.table.data.RowData;
+import org.apache.flink.table.data.StringData;
+import org.apache.flink.table.data.TimestampData;
+import org.apache.flink.types.RowKind;
+
+import java.util.List;
+import java.util.Objects;
+
+/**
+ * An implementation of {@link RowData} which is backed by two {@link RowData} with a well-defined
+ * index mapping, One of the rows is fixed, while the other can be swapped for performant changes in
+ * hot code paths. The {@link RowKind} is inherited from the mutable row.
+ */
+@PublicEvolving
+public class ExtendedRowData implements RowData {
+
+ private final RowData fixedRow;
+ // The index mapping is built as follows: positive indexes are indexes refer to mutable row
+ // positions,
+ // while negative indexes (with -1 offset) refer to fixed row positions.
+ // For example an index mapping [0, 1, -1, -2, 2] means:
+ // * Index 0 -> mutable row index 0
+ // * Index 1 -> mutable row index 1
+ // * Index -1 -> fixed row index 0
+ // * Index -2 -> fixed row index 1
+ // * Index 2 -> mutable row index 2
+ private final int[] indexMapping;
Review comment:
But note that this is completely transparent to the user of this class. The user access the class using the normal indexing. The reason to use this mapping is to avoid a math operation on one of the two cases (no need to do an increment), and the reason to not create a new pojo is to allocate less.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353219
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit cb72fd0b0045437e947bfde4e729d0abb1df5b22 (Fri Oct 22 07:13:34 UTC 2021)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] twalthr commented on a change in pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
twalthr commented on a change in pull request #17544:
URL: https://github.com/apache/flink/pull/17544#discussion_r734367759
##########
File path: flink-table/flink-table-common/src/main/java/org/apache/flink/table/data/utils/ExtendedRowData.java
##########
@@ -0,0 +1,312 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.data.utils;
+
+import org.apache.flink.annotation.PublicEvolving;
+import org.apache.flink.table.data.ArrayData;
+import org.apache.flink.table.data.DecimalData;
+import org.apache.flink.table.data.MapData;
+import org.apache.flink.table.data.RawValueData;
+import org.apache.flink.table.data.RowData;
+import org.apache.flink.table.data.StringData;
+import org.apache.flink.table.data.TimestampData;
+import org.apache.flink.types.RowKind;
+
+import java.util.List;
+import java.util.Objects;
+
+/**
+ * An implementation of {@link RowData} which is backed by two {@link RowData} with a well-defined
+ * index mapping, One of the rows is fixed, while the other can be swapped for performant changes in
+ * hot code paths. The {@link RowKind} is inherited from the mutable row.
+ */
+@PublicEvolving
+public class ExtendedRowData implements RowData {
Review comment:
maybe `EnrichedRowData`? When I read "extended" it sounds like we add fields at the end.
##########
File path: flink-table/flink-table-common/src/main/java/org/apache/flink/table/data/utils/ExtendedRowData.java
##########
@@ -0,0 +1,312 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.data.utils;
+
+import org.apache.flink.annotation.PublicEvolving;
+import org.apache.flink.table.data.ArrayData;
+import org.apache.flink.table.data.DecimalData;
+import org.apache.flink.table.data.MapData;
+import org.apache.flink.table.data.RawValueData;
+import org.apache.flink.table.data.RowData;
+import org.apache.flink.table.data.StringData;
+import org.apache.flink.table.data.TimestampData;
+import org.apache.flink.types.RowKind;
+
+import java.util.List;
+import java.util.Objects;
+
+/**
+ * An implementation of {@link RowData} which is backed by two {@link RowData} with a well-defined
+ * index mapping, One of the rows is fixed, while the other can be swapped for performant changes in
+ * hot code paths. The {@link RowKind} is inherited from the mutable row.
+ */
+@PublicEvolving
+public class ExtendedRowData implements RowData {
+
+ private final RowData fixedRow;
+ // The index mapping is built as follows: positive indexes are indexes refer to mutable row
+ // positions,
+ // while negative indexes (with -1 offset) refer to fixed row positions.
+ // For example an index mapping [0, 1, -1, -2, 2] means:
+ // * Index 0 -> mutable row index 0
+ // * Index 1 -> mutable row index 1
+ // * Index -1 -> fixed row index 0
+ // * Index -2 -> fixed row index 1
+ // * Index 2 -> mutable row index 2
+ private final int[] indexMapping;
+
+ private RowData mutableRow;
+
+ public ExtendedRowData(RowData fixedRow, int[] indexMapping) {
+ this.fixedRow = fixedRow;
+ this.indexMapping = indexMapping;
+ }
+
+ /**
+ * Replaces the mutable {@link RowData} backing this {@link ExtendedRowData}.
+ *
+ * <p>This method replaces the mutable row data in place and does not return a new object. This
+ * is done for performance reasons.
+ */
+ public ExtendedRowData replaceMutableRow(RowData mutableRow) {
+ this.mutableRow = mutableRow;
+ return this;
+ }
+
+ // ---------------------------------------------------------------------------------------------
+
+ @Override
+ public int getArity() {
+ return indexMapping.length;
+ }
+
+ @Override
+ public RowKind getRowKind() {
+ return this.mutableRow.getRowKind();
Review comment:
nit: `this.` can be removed
##########
File path: flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/runtime/stream/sql/StreamFileSystemITCaseBase.scala
##########
@@ -56,6 +56,28 @@ abstract class StreamFileSystemITCaseBase extends StreamingTestBase with FileSys
sink.getAppendResults.sorted)
}
+ override def checkPredicate(sqlQuery: String, checkFunc: Row => Unit): Unit = {
+ val result = tEnv.sqlQuery(sqlQuery).toAppendStream[Row]
Review comment:
`toAppendStream` is deprecated use `toDataStream` instead
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -330,4 +386,89 @@ private DataType getProjectedDataType() {
}
return DataType.projectFields(physicalDataType, projectedFields);
}
+
+ @Override
+ DataType getPhysicalDataTypeWithoutPartitionColumns() {
+ if (this.producedDataType != null) {
+ return DataTypes.ROW(
+ DataType.getFields(this.producedDataType).stream()
+ .filter(field -> !usedMetadataKeys.contains(field.getName()))
+ .filter(field -> !partitionKeys.contains(field.getName()))
+ .toArray(DataTypes.Field[]::new));
+ }
+ return super.getPhysicalDataTypeWithoutPartitionColumns();
+ }
+
+ // --------------------------------------------------------------------------------------------
+ // Metadata handling
+ // --------------------------------------------------------------------------------------------
+
+ @Override
+ public Map<String, DataType> listReadableMetadata() {
+ return Arrays.stream(ReadableFileInfo.values())
+ .collect(Collectors.toMap(ReadableFileInfo::getKey, ReadableFileInfo::getDataType));
+ }
+
+ @Override
+ public void applyReadableMetadata(List<String> metadataKeys, DataType producedDataType) {
+ if (metadataKeys.isEmpty()) {
+ // This method should be idempotent
+ this.usedMetadataKeys = null;
+ this.usedMetadata = null;
+ this.producedDataType = null;
+ return;
+ }
+
+ this.usedMetadataKeys = metadataKeys;
+ this.usedMetadata =
+ metadataKeys.stream().map(ReadableFileInfo::resolve).collect(Collectors.toList());
+ this.producedDataType = producedDataType;
+ }
+
+ interface FileInfoAccessor extends Serializable {
+ /**
+ * Access the information from the {@link org.apache.flink.core.fs.FileInputSplit}. The
+ * return value type must be an internal type.
+ */
+ Object getValue(FileSourceSplit split);
+ }
+
+ enum ReadableFileInfo implements Serializable {
+ FILENAME(
+ "filename",
Review comment:
+1 for filepath
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileInfoExtractorBulkFormat.java
##########
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.filesystem;
+
+import org.apache.flink.api.common.typeinfo.TypeInformation;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.connector.file.src.FileSourceSplit;
+import org.apache.flink.connector.file.src.reader.BulkFormat;
+import org.apache.flink.connector.file.src.util.RecordMapperWrapperRecordIterator;
+import org.apache.flink.table.data.GenericRowData;
+import org.apache.flink.table.data.RowData;
+import org.apache.flink.table.data.utils.ExtendedRowData;
+import org.apache.flink.table.runtime.typeutils.InternalTypeInfo;
+import org.apache.flink.table.types.DataType;
+
+import javax.annotation.Nullable;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * This {@link BulkFormat} is a wrapper that attaches file information columns to the output
+ * records.
+ */
+class FileInfoExtractorBulkFormat implements BulkFormat<RowData, FileSourceSplit> {
+
+ private final BulkFormat<RowData, FileSourceSplit> wrapped;
+ private final TypeInformation<RowData> producedType;
+
+ private final List<FileSystemTableSource.FileInfoAccessor> metadataColumnsFunctions;
+ private final int[] extendedRowIndexMapping;
+
+ public FileInfoExtractorBulkFormat(
+ BulkFormat<RowData, FileSourceSplit> wrapped,
+ DataType fullDataType,
+ Map<String, FileSystemTableSource.FileInfoAccessor> metadataColumns) {
+ this.wrapped = wrapped;
+ this.producedType = InternalTypeInfo.of(fullDataType.getLogicalType());
+
+ // Compute index mapping for the extended row and the functions to compute metadata
+ List<String> completeRowFields = DataType.getFieldNames(fullDataType);
+ List<String> mutableRowFields =
+ completeRowFields.stream()
+ .filter(key -> !metadataColumns.containsKey(key))
+ .collect(Collectors.toList());
+ List<String> fixedRowFields = new ArrayList<>(metadataColumns.keySet());
+ this.extendedRowIndexMapping =
+ ExtendedRowData.computeIndexMapping(
+ completeRowFields, mutableRowFields, fixedRowFields);
+ this.metadataColumnsFunctions =
+ fixedRowFields.stream().map(metadataColumns::get).collect(Collectors.toList());
+ }
+
+ @Override
+ public Reader<RowData> createReader(Configuration config, FileSourceSplit split)
+ throws IOException {
+ return wrapReader(wrapped.createReader(config, split), split);
+ }
+
+ @Override
+ public Reader<RowData> restoreReader(Configuration config, FileSourceSplit split)
+ throws IOException {
+ return wrapReader(wrapped.restoreReader(config, split), split);
+ }
+
+ @Override
+ public boolean isSplittable() {
+ return wrapped.isSplittable();
+ }
+
+ @Override
+ public TypeInformation<RowData> getProducedType() {
+ return producedType;
+ }
+
+ private Reader<RowData> wrapReader(Reader<RowData> superReader, FileSourceSplit split) {
+ // Fill the metadata row
+ final GenericRowData metadataRowData = new GenericRowData(metadataColumnsFunctions.size());
+ for (int i = 0; i < metadataColumnsFunctions.size(); i++) {
+ metadataRowData.setField(i, metadataColumnsFunctions.get(i).getValue(split));
+ }
+
+ // This row is going to be reused for every record
+ final ExtendedRowData extendedRowData =
+ new ExtendedRowData(metadataRowData, this.extendedRowIndexMapping);
+
+ return new ReaderWrapper(
+ superReader,
+ physicalRowData -> {
Review comment:
Let's move this into ReaderWrapper directly. The reusable record should be visible as a member variable of `ReaderWrapper` instead of implicitly accessing a local variable in the lambda.
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -315,13 +340,44 @@ public String asSummaryString() {
}
private int[] readFields() {
+ if (this.producedDataType != null) {
+ return IntStream.range(
+ 0,
+ (int)
+ DataType.getFields(this.producedDataType).stream()
+ .filter(
+ field ->
+ !usedMetadataKeys.contains(
+ field.getName()))
+ .count())
+ .toArray();
+ }
return projectedFields == null
? IntStream.range(0, DataType.getFieldCount(getPhysicalDataType())).toArray()
: Arrays.stream(projectedFields).mapToInt(array -> array[0]).toArray();
}
+ @Override
+ DataType getPhysicalDataType() {
+ if (this.usedMetadataKeys != null) {
+ return DataTypes.ROW(
+ DataType.getFields(super.getPhysicalDataType()).stream()
+ .filter(field -> !usedMetadataKeys.contains(field.getName()))
+ .toArray(DataTypes.Field[]::new));
+ }
+ return super.getPhysicalDataType();
+ }
+
private DataType getProjectedDataType() {
- final DataType physicalDataType = super.getPhysicalDataType();
+ final DataType physicalDataType =
+ this.producedDataType != null
Review comment:
can't we just store `physicalDataType` in a separate variable instead of recomputing it multiple times? both `physialDataType` and `producedDataType` should be member variables. Please take `KafkaDynamicSource` as a reference implementation, also for structuring the member variables.
##########
File path: flink-table/flink-table-common/src/main/java/org/apache/flink/table/data/utils/ExtendedRowData.java
##########
@@ -0,0 +1,312 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.data.utils;
+
+import org.apache.flink.annotation.PublicEvolving;
+import org.apache.flink.table.data.ArrayData;
+import org.apache.flink.table.data.DecimalData;
+import org.apache.flink.table.data.MapData;
+import org.apache.flink.table.data.RawValueData;
+import org.apache.flink.table.data.RowData;
+import org.apache.flink.table.data.StringData;
+import org.apache.flink.table.data.TimestampData;
+import org.apache.flink.types.RowKind;
+
+import java.util.List;
+import java.util.Objects;
+
+/**
+ * An implementation of {@link RowData} which is backed by two {@link RowData} with a well-defined
+ * index mapping, One of the rows is fixed, while the other can be swapped for performant changes in
+ * hot code paths. The {@link RowKind} is inherited from the mutable row.
+ */
+@PublicEvolving
+public class ExtendedRowData implements RowData {
+
+ private final RowData fixedRow;
+ // The index mapping is built as follows: positive indexes are indexes refer to mutable row
+ // positions,
+ // while negative indexes (with -1 offset) refer to fixed row positions.
+ // For example an index mapping [0, 1, -1, -2, 2] means:
+ // * Index 0 -> mutable row index 0
+ // * Index 1 -> mutable row index 1
+ // * Index -1 -> fixed row index 0
+ // * Index -2 -> fixed row index 1
+ // * Index 2 -> mutable row index 2
+ private final int[] indexMapping;
Review comment:
I feel this can be quite difficult to use due to `-1 -> fixed row index 0` and `Index 0 -> mutable row index 0`. How about we introduce a helper POJO `RowMapping<RowType.LEFT, 2>` and `RowMapping<RowType.RIGHT, 3>`. Or we simply make both indexes 1 based and skip the 0.
##########
File path: flink-table/flink-table-common/src/main/java/org/apache/flink/table/data/utils/ExtendedRowData.java
##########
@@ -0,0 +1,312 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.data.utils;
+
+import org.apache.flink.annotation.PublicEvolving;
+import org.apache.flink.table.data.ArrayData;
+import org.apache.flink.table.data.DecimalData;
+import org.apache.flink.table.data.MapData;
+import org.apache.flink.table.data.RawValueData;
+import org.apache.flink.table.data.RowData;
+import org.apache.flink.table.data.StringData;
+import org.apache.flink.table.data.TimestampData;
+import org.apache.flink.types.RowKind;
+
+import java.util.List;
+import java.util.Objects;
+
+/**
+ * An implementation of {@link RowData} which is backed by two {@link RowData} with a well-defined
+ * index mapping, One of the rows is fixed, while the other can be swapped for performant changes in
+ * hot code paths. The {@link RowKind} is inherited from the mutable row.
+ */
+@PublicEvolving
+public class ExtendedRowData implements RowData {
+
+ private final RowData fixedRow;
+ // The index mapping is built as follows: positive indexes are indexes refer to mutable row
+ // positions,
+ // while negative indexes (with -1 offset) refer to fixed row positions.
+ // For example an index mapping [0, 1, -1, -2, 2] means:
+ // * Index 0 -> mutable row index 0
+ // * Index 1 -> mutable row index 1
+ // * Index -1 -> fixed row index 0
+ // * Index -2 -> fixed row index 1
+ // * Index 2 -> mutable row index 2
+ private final int[] indexMapping;
+
+ private RowData mutableRow;
+
+ public ExtendedRowData(RowData fixedRow, int[] indexMapping) {
+ this.fixedRow = fixedRow;
+ this.indexMapping = indexMapping;
+ }
+
+ /**
+ * Replaces the mutable {@link RowData} backing this {@link ExtendedRowData}.
+ *
+ * <p>This method replaces the mutable row data in place and does not return a new object. This
+ * is done for performance reasons.
+ */
+ public ExtendedRowData replaceMutableRow(RowData mutableRow) {
+ this.mutableRow = mutableRow;
+ return this;
+ }
+
+ // ---------------------------------------------------------------------------------------------
+
+ @Override
+ public int getArity() {
+ return indexMapping.length;
+ }
+
+ @Override
+ public RowKind getRowKind() {
+ return this.mutableRow.getRowKind();
+ }
+
+ @Override
+ public void setRowKind(RowKind kind) {
+ this.mutableRow.setRowKind(kind);
+ }
+
+ @Override
+ public boolean isNullAt(int pos) {
+ int index = indexMapping[pos];
+ if (index >= 0) {
+ return mutableRow.isNullAt(index);
+ } else {
+ return fixedRow.isNullAt(-(index + 1));
+ }
+ }
+
+ @Override
+ public boolean getBoolean(int pos) {
+ int index = indexMapping[pos];
+ if (index >= 0) {
+ return mutableRow.getBoolean(index);
+ } else {
+ return fixedRow.getBoolean(-(index + 1));
+ }
+ }
+
+ @Override
+ public byte getByte(int pos) {
+ int index = indexMapping[pos];
+ if (index >= 0) {
+ return mutableRow.getByte(index);
+ } else {
+ return fixedRow.getByte(-(index + 1));
+ }
+ }
+
+ @Override
+ public short getShort(int pos) {
+ int index = indexMapping[pos];
+ if (index >= 0) {
+ return mutableRow.getShort(index);
+ } else {
+ return fixedRow.getShort(-(index + 1));
+ }
+ }
+
+ @Override
+ public int getInt(int pos) {
+ int index = indexMapping[pos];
+ if (index >= 0) {
+ return mutableRow.getInt(index);
+ } else {
+ return fixedRow.getInt(-(index + 1));
+ }
+ }
+
+ @Override
+ public long getLong(int pos) {
+ int index = indexMapping[pos];
+ if (index >= 0) {
+ return mutableRow.getLong(index);
+ } else {
+ return fixedRow.getLong(-(index + 1));
+ }
+ }
+
+ @Override
+ public float getFloat(int pos) {
+ int index = indexMapping[pos];
+ if (index >= 0) {
+ return mutableRow.getFloat(index);
+ } else {
+ return fixedRow.getFloat(-(index + 1));
+ }
+ }
+
+ @Override
+ public double getDouble(int pos) {
+ int index = indexMapping[pos];
+ if (index >= 0) {
+ return mutableRow.getDouble(index);
+ } else {
+ return fixedRow.getDouble(-(index + 1));
+ }
+ }
+
+ @Override
+ public StringData getString(int pos) {
+ int index = indexMapping[pos];
+ if (index >= 0) {
+ return mutableRow.getString(index);
+ } else {
+ return fixedRow.getString(-(index + 1));
+ }
+ }
+
+ @Override
+ public DecimalData getDecimal(int pos, int precision, int scale) {
+ int index = indexMapping[pos];
+ if (index >= 0) {
+ return mutableRow.getDecimal(index, precision, scale);
+ } else {
+ return fixedRow.getDecimal(-(index + 1), precision, scale);
+ }
+ }
+
+ @Override
+ public TimestampData getTimestamp(int pos, int precision) {
+ int index = indexMapping[pos];
+ if (index >= 0) {
+ return mutableRow.getTimestamp(index, precision);
+ } else {
+ return fixedRow.getTimestamp(-(index + 1), precision);
+ }
+ }
+
+ @Override
+ public <T> RawValueData<T> getRawValue(int pos) {
+ int index = indexMapping[pos];
+ if (index >= 0) {
+ return mutableRow.getRawValue(index);
+ } else {
+ return fixedRow.getRawValue(-(index + 1));
+ }
+ }
+
+ @Override
+ public byte[] getBinary(int pos) {
+ int index = indexMapping[pos];
+ if (index >= 0) {
+ return mutableRow.getBinary(index);
+ } else {
+ return fixedRow.getBinary(-(index + 1));
+ }
+ }
+
+ @Override
+ public ArrayData getArray(int pos) {
+ int index = indexMapping[pos];
+ if (index >= 0) {
+ return mutableRow.getArray(index);
+ } else {
+ return fixedRow.getArray(-(index + 1));
+ }
+ }
+
+ @Override
+ public MapData getMap(int pos) {
+ int index = indexMapping[pos];
+ if (index >= 0) {
+ return mutableRow.getMap(index);
+ } else {
+ return fixedRow.getMap(-(index + 1));
+ }
+ }
+
+ @Override
+ public RowData getRow(int pos, int numFields) {
+ int index = indexMapping[pos];
+ if (index >= 0) {
+ return mutableRow.getRow(index, numFields);
+ } else {
+ return fixedRow.getRow(-(index + 1), numFields);
+ }
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (o == null || getClass() != o.getClass()) {
+ return false;
+ }
+ ExtendedRowData that = (ExtendedRowData) o;
+ return Objects.equals(this.fixedRow, that.fixedRow)
+ && Objects.equals(this.mutableRow, that.mutableRow);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(fixedRow, mutableRow);
+ }
+
+ @Override
+ public String toString() {
+ return mutableRow.getRowKind().shortString()
+ + "{"
+ + "fixedRow="
+ + fixedRow
+ + ", mutableRow="
+ + mutableRow
+ + '}';
+ }
+
+ /**
+ * Creates a new {@link ExtendedRowData} with the provided {@code fixedRow} as the immutable
+ * static row, and uses the {@code completeRowFields}, {@code fixedRowFields} and {@code
+ * mutableRowFields} arguments to compute the indexes mapping.
+ */
+ public static ExtendedRowData from(
+ RowData fixedRow,
+ List<String> completeRowFields,
Review comment:
can you add more docs for the paramters or an example? at first glance it is might not be obvious what `completeRowFields` means.
##########
File path: flink-table/flink-table-common/src/test/java/org/apache/flink/table/data/utils/ExtendedRowDataTest.java
##########
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.data.utils;
+
+import org.apache.flink.table.data.GenericRowData;
+import org.apache.flink.table.data.RowData;
+import org.apache.flink.types.RowKind;
+
+import org.junit.jupiter.api.Test;
+
+import java.util.Arrays;
+import java.util.List;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+/** Tests for {@link JoinedRowData}. */
+public class ExtendedRowDataTest {
+
+ @Test
+ public void testJoinedRows() {
+ List<String> completeRowFields =
Review comment:
nit: can also be final
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -330,4 +386,89 @@ private DataType getProjectedDataType() {
}
return DataType.projectFields(physicalDataType, projectedFields);
}
+
+ @Override
+ DataType getPhysicalDataTypeWithoutPartitionColumns() {
+ if (this.producedDataType != null) {
+ return DataTypes.ROW(
+ DataType.getFields(this.producedDataType).stream()
+ .filter(field -> !usedMetadataKeys.contains(field.getName()))
+ .filter(field -> !partitionKeys.contains(field.getName()))
+ .toArray(DataTypes.Field[]::new));
+ }
+ return super.getPhysicalDataTypeWithoutPartitionColumns();
+ }
+
+ // --------------------------------------------------------------------------------------------
+ // Metadata handling
+ // --------------------------------------------------------------------------------------------
+
+ @Override
+ public Map<String, DataType> listReadableMetadata() {
+ return Arrays.stream(ReadableFileInfo.values())
+ .collect(Collectors.toMap(ReadableFileInfo::getKey, ReadableFileInfo::getDataType));
+ }
+
+ @Override
+ public void applyReadableMetadata(List<String> metadataKeys, DataType producedDataType) {
+ if (metadataKeys.isEmpty()) {
+ // This method should be idempotent
+ this.usedMetadataKeys = null;
+ this.usedMetadata = null;
+ this.producedDataType = null;
+ return;
+ }
+
+ this.usedMetadataKeys = metadataKeys;
+ this.usedMetadata =
+ metadataKeys.stream().map(ReadableFileInfo::resolve).collect(Collectors.toList());
+ this.producedDataType = producedDataType;
+ }
+
+ interface FileInfoAccessor extends Serializable {
+ /**
+ * Access the information from the {@link org.apache.flink.core.fs.FileInputSplit}. The
+ * return value type must be an internal type.
+ */
+ Object getValue(FileSourceSplit split);
+ }
+
+ enum ReadableFileInfo implements Serializable {
+ FILENAME(
+ "filename",
+ DataTypes.STRING().notNull(),
+ split -> StringData.fromString(split.path().getPath()));
Review comment:
use an anonymous inner class with a serialVersionID (see Kafka source). we had issue in the past because lambdas are not proper serializable when Maven shading is involved.
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileInfoExtractorBulkFormat.java
##########
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.filesystem;
+
+import org.apache.flink.api.common.typeinfo.TypeInformation;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.connector.file.src.FileSourceSplit;
+import org.apache.flink.connector.file.src.reader.BulkFormat;
+import org.apache.flink.connector.file.src.util.RecordMapperWrapperRecordIterator;
+import org.apache.flink.table.data.GenericRowData;
+import org.apache.flink.table.data.RowData;
+import org.apache.flink.table.data.utils.ExtendedRowData;
+import org.apache.flink.table.runtime.typeutils.InternalTypeInfo;
+import org.apache.flink.table.types.DataType;
+
+import javax.annotation.Nullable;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+/**
+ * This {@link BulkFormat} is a wrapper that attaches file information columns to the output
+ * records.
+ */
+class FileInfoExtractorBulkFormat implements BulkFormat<RowData, FileSourceSplit> {
+
+ private final BulkFormat<RowData, FileSourceSplit> wrapped;
+ private final TypeInformation<RowData> producedType;
+
+ private final List<FileSystemTableSource.FileInfoAccessor> metadataColumnsFunctions;
+ private final int[] extendedRowIndexMapping;
+
+ public FileInfoExtractorBulkFormat(
+ BulkFormat<RowData, FileSourceSplit> wrapped,
+ DataType fullDataType,
+ Map<String, FileSystemTableSource.FileInfoAccessor> metadataColumns) {
+ this.wrapped = wrapped;
+ this.producedType = InternalTypeInfo.of(fullDataType.getLogicalType());
Review comment:
at some point we should move this into a connector module, it doesn't belong into `table-runtime`. Is it easily possible to let the type information be created before and just passed into this class?
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -315,13 +340,44 @@ public String asSummaryString() {
}
private int[] readFields() {
+ if (this.producedDataType != null) {
+ return IntStream.range(
+ 0,
+ (int)
+ DataType.getFields(this.producedDataType).stream()
+ .filter(
+ field ->
+ !usedMetadataKeys.contains(
+ field.getName()))
+ .count())
+ .toArray();
+ }
return projectedFields == null
? IntStream.range(0, DataType.getFieldCount(getPhysicalDataType())).toArray()
: Arrays.stream(projectedFields).mapToInt(array -> array[0]).toArray();
}
+ @Override
+ DataType getPhysicalDataType() {
+ if (this.usedMetadataKeys != null) {
+ return DataTypes.ROW(
+ DataType.getFields(super.getPhysicalDataType()).stream()
+ .filter(field -> !usedMetadataKeys.contains(field.getName()))
+ .toArray(DataTypes.Field[]::new));
+ }
+ return super.getPhysicalDataType();
+ }
+
private DataType getProjectedDataType() {
- final DataType physicalDataType = super.getPhysicalDataType();
+ final DataType physicalDataType =
+ this.producedDataType != null
+ ? DataTypes.ROW(
+ DataType.getFields(this.producedDataType).stream()
+ .filter(
+ field ->
+ !usedMetadataKeys.contains(field.getName()))
+ .toArray(DataTypes.Field[]::new))
+ : super.getPhysicalDataType();
Review comment:
What do you mean with `physical columns + partition columns`? Partition columns are always part of the physical columns, no?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e9b8cb2a13f66a727ce22c9b99dea47801701e76 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376)
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
* 7e8209d6c05454c4ad642a2f3e71d044a2bb65b6 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
* 7e8209d6c05454c4ad642a2f3e71d044a2bb65b6 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
* 716a79d6c46b930226baf5fa4ed38ee83d859987 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
}, {
"hash" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784",
"triggerID" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0cd3dcf68c5e470489388cacad95ec40fde6f74d Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634)
* a40d8242fcdc183cada6687f62722c356ad4e36e Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24615][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
}, {
"hash" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25600",
"triggerID" : "716a79d6c46b930226baf5fa4ed38ee83d859987",
"triggerType" : "PUSH"
}, {
"hash" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25602",
"triggerID" : "7e8209d6c05454c4ad642a2f3e71d044a2bb65b6",
"triggerType" : "PUSH"
}, {
"hash" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25634",
"triggerID" : "0cd3dcf68c5e470489388cacad95ec40fde6f74d",
"triggerType" : "PUSH"
}, {
"hash" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25784",
"triggerID" : "a40d8242fcdc183cada6687f62722c356ad4e36e",
"triggerType" : "PUSH"
}, {
"hash" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26053",
"triggerID" : "d98738ad4d70cc734983a0dc46da41f2c057b059",
"triggerType" : "PUSH"
}, {
"hash" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26061",
"triggerID" : "8ed8594acccabd5105cb3cb6bb046e6b1394fb7d",
"triggerType" : "PUSH"
}, {
"hash" : "89b9e48aacedba0ff4d47dc8f7fdb3e5315945be",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26166",
"triggerID" : "89b9e48aacedba0ff4d47dc8f7fdb3e5315945be",
"triggerType" : "PUSH"
}, {
"hash" : "50b077bbd46a5a0e7c99dcdd1a252558df2dc526",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "50b077bbd46a5a0e7c99dcdd1a252558df2dc526",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 89b9e48aacedba0ff4d47dc8f7fdb3e5315945be Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=26166)
* 50b077bbd46a5a0e7c99dcdd1a252558df2dc526 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] twalthr commented on a change in pull request #17544: [FLINK-24615][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
twalthr commented on a change in pull request #17544:
URL: https://github.com/apache/flink/pull/17544#discussion_r745388320
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -314,20 +394,86 @@ public String asSummaryString() {
return map;
}
- private int[] readFields() {
- return projectedFields == null
- ? IntStream.range(0, DataType.getFieldCount(getPhysicalDataType())).toArray()
- : Arrays.stream(projectedFields).mapToInt(array -> array[0]).toArray();
+ // --------------------------------------------------------------------------------------------
+ // Methods to apply projections and metadata,
+ // will influence the final output and physical type used by formats
+ // --------------------------------------------------------------------------------------------
+
+ @Override
+ public void applyProjection(int[][] projectedFields, DataType producedDataType) {
+ this.projectFields = projectedFields;
+ this.producedDataType = producedDataType;
+ }
+
+ @Override
+ public void applyReadableMetadata(List<String> metadataKeys, DataType producedDataType) {
+ if (!metadataKeys.isEmpty()) {
Review comment:
this could be problematic. the optimizer might call this method twice. once with the declared metadata columns and once with the actual used ones (maybe empty). It should be safe to remove this if branch?
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -101,54 +109,123 @@ public FileSystemTableSource(
this.bulkReaderFormat = bulkReaderFormat;
this.deserializationFormat = deserializationFormat;
this.formatFactory = formatFactory;
+
+ this.producedDataType = context.getPhysicalRowDataType();
}
@Override
public ScanRuntimeProvider getScanRuntimeProvider(ScanContext scanContext) {
+ // When this table has no partition, just return a empty source.
if (!partitionKeys.isEmpty() && getOrFetchPartitions().isEmpty()) {
- // When this table has no partition, just return a empty source.
return InputFormatProvider.of(new CollectionInputFormat<>(new ArrayList<>(), null));
- } else if (bulkReaderFormat != null) {
- if (bulkReaderFormat instanceof BulkDecodingFormat
- && filters != null
- && filters.size() > 0) {
- ((BulkDecodingFormat<RowData>) bulkReaderFormat).applyFilters(filters);
+ }
+
+ // Physical type is computed from the full data type, filtering out partition and
+ // metadata columns. This type is going to be used by formats to parse the input.
+ List<DataTypes.Field> producedDataTypeFields = DataType.getFields(producedDataType);
+ if (metadataKeys != null) {
+ // If metadata keys are present, then by SupportsReadingMetadata contract all the
+ // metadata columns will be at the end of the producedDataType, so we can just remove
+ // from the list the last metadataKeys.size() fields.
+ producedDataTypeFields =
+ producedDataTypeFields.subList(
+ 0, producedDataTypeFields.size() - metadataKeys.size());
+ }
+ DataType physicalDataType =
+ producedDataTypeFields.stream()
Review comment:
do we have an issue already for introducing `Context.getPartitionKeys: int[]` and use `DataType.excludeFields`?
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -314,20 +394,86 @@ public String asSummaryString() {
return map;
}
- private int[] readFields() {
- return projectedFields == null
- ? IntStream.range(0, DataType.getFieldCount(getPhysicalDataType())).toArray()
- : Arrays.stream(projectedFields).mapToInt(array -> array[0]).toArray();
+ // --------------------------------------------------------------------------------------------
+ // Methods to apply projections and metadata,
+ // will influence the final output and physical type used by formats
+ // --------------------------------------------------------------------------------------------
+
+ @Override
+ public void applyProjection(int[][] projectedFields, DataType producedDataType) {
+ this.projectFields = projectedFields;
+ this.producedDataType = producedDataType;
+ }
+
+ @Override
+ public void applyReadableMetadata(List<String> metadataKeys, DataType producedDataType) {
+ if (!metadataKeys.isEmpty()) {
+ this.metadataKeys = metadataKeys;
+ this.producedDataType = producedDataType;
+ }
+ }
+
+ // --------------------------------------------------------------------------------------------
+ // Metadata handling
+ // --------------------------------------------------------------------------------------------
+
+ @Override
+ public Map<String, DataType> listReadableMetadata() {
+ return Arrays.stream(ReadableFileInfo.values())
+ .collect(Collectors.toMap(ReadableFileInfo::getKey, ReadableFileInfo::getDataType));
+ }
+
+ interface FileInfoAccessor extends Serializable {
+ /**
+ * Access the information from the {@link org.apache.flink.core.fs.FileInputSplit}. The
+ * return value type must be an internal type.
+ */
+ Object getValue(FileSourceSplit split);
}
- private DataType getProjectedDataType() {
- final DataType physicalDataType = super.getPhysicalDataType();
+ enum ReadableFileInfo implements Serializable {
+ FILEPATH(
+ "filepath",
+ DataTypes.STRING().notNull(),
+ new FileInfoAccessor() {
+ private static final long serialVersionUID = 1L;
+
+ @Override
+ public Object getValue(FileSourceSplit split) {
+ return StringData.fromString(split.path().getPath());
+ }
+ });
+
+ final String key;
+ final DataType dataType;
+ final FileInfoAccessor converter;
+
+ ReadableFileInfo(String key, DataType dataType, FileInfoAccessor converter) {
+ this.key = key;
+ this.dataType = dataType;
+ this.converter = converter;
+ }
- // If we haven't projected fields, we just return the original physical data type,
- // otherwise we need to compute the physical data type depending on the projected fields.
- if (projectedFields == null) {
- return physicalDataType;
+ public String getKey() {
+ return key;
+ }
+
+ public DataType getDataType() {
+ return dataType;
+ }
+
+ public FileInfoAccessor getAccessor() {
+ return converter;
+ }
+
+ public static ReadableFileInfo resolve(String key) {
+ switch (key) {
+ case "filepath":
Review comment:
why having the constant two times? we can simply iterate through ReadableFileInfo values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 237353b8cef1a7fd7747a6f62bd1910595d40d5c Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379)
* bbe23062517c6ecd1ff3dd55a221da63fe00b3d3 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d23020004351008490b234654859363929e2cdac Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521)
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] slinkydeveloper commented on a change in pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
slinkydeveloper commented on a change in pull request #17544:
URL: https://github.com/apache/flink/pull/17544#discussion_r737726465
##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/filesystem/FileSystemTableSource.java
##########
@@ -315,13 +340,44 @@ public String asSummaryString() {
}
private int[] readFields() {
+ if (this.producedDataType != null) {
+ return IntStream.range(
+ 0,
+ (int)
+ DataType.getFields(this.producedDataType).stream()
+ .filter(
+ field ->
+ !usedMetadataKeys.contains(
+ field.getName()))
+ .count())
+ .toArray();
+ }
return projectedFields == null
? IntStream.range(0, DataType.getFieldCount(getPhysicalDataType())).toArray()
: Arrays.stream(projectedFields).mapToInt(array -> array[0]).toArray();
}
+ @Override
+ DataType getPhysicalDataType() {
+ if (this.usedMetadataKeys != null) {
+ return DataTypes.ROW(
+ DataType.getFields(super.getPhysicalDataType()).stream()
+ .filter(field -> !usedMetadataKeys.contains(field.getName()))
+ .toArray(DataTypes.Field[]::new));
+ }
+ return super.getPhysicalDataType();
+ }
+
private DataType getProjectedDataType() {
- final DataType physicalDataType = super.getPhysicalDataType();
+ final DataType physicalDataType =
+ this.producedDataType != null
Review comment:
This one can be closed as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17544: [FLINK-24165][table] Add infrastructure to support metadata for filesystem connector
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17544:
URL: https://github.com/apache/flink/pull/17544#issuecomment-949353495
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25355",
"triggerID" : "cb72fd0b0045437e947bfde4e729d0abb1df5b22",
"triggerType" : "PUSH"
}, {
"hash" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25362",
"triggerID" : "ba7ae1015a95b39276744f6cd0aaf19638e61509",
"triggerType" : "PUSH"
}, {
"hash" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25376",
"triggerID" : "e9b8cb2a13f66a727ce22c9b99dea47801701e76",
"triggerType" : "PUSH"
}, {
"hash" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25379",
"triggerID" : "237353b8cef1a7fd7747a6f62bd1910595d40d5c",
"triggerType" : "PUSH"
}, {
"hash" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25509",
"triggerID" : "bbe23062517c6ecd1ff3dd55a221da63fe00b3d3",
"triggerType" : "PUSH"
}, {
"hash" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25513",
"triggerID" : "efc206d83ea51d690d045cb3dc0529dcb9002534",
"triggerType" : "PUSH"
}, {
"hash" : "d23020004351008490b234654859363929e2cdac",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25521",
"triggerID" : "d23020004351008490b234654859363929e2cdac",
"triggerType" : "PUSH"
}, {
"hash" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538",
"triggerID" : "a20fe2427c03ceb71b8d4b5f30bdc282aba530fe",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a20fe2427c03ceb71b8d4b5f30bdc282aba530fe Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25538)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org