You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/07/01 00:23:09 UTC
[GitHub] [hudi] alexeykudinkin opened a new pull request, #6016: [WIP] Optimizing file-listing sequence of Metadata Table
alexeykudinkin opened a new pull request, #6016:
URL: https://github.com/apache/hudi/pull/6016
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*
## What is the purpose of the pull request
This PR optimizes file-listing sequence of the Metadata Table to make sure it's on par or better than FS-based file-listing
Change log:
- Cleaned up avoidable instantiations of Hadoop's `Path`
- Replaced `new Path` w/ `createUnsafePath` where possible
- Cached `TimestampFormatter`, `DateFormatter` for timezone
- Avoid loading defaults in Hadoop conf when init-ing HFile reader
-
## Brief change log
*(for example:)*
- *Modify AnnotationLocation checkstyle rule in checkstyle.xml*
## Verify this pull request
*(Please pick either of the following options)*
This pull request is a trivial rework / code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end.*
- *Added HoodieClientWriteTest to verify the change.*
- *Manually verified the change by running a job locally.*
- Avoid re-instantiating `BaseTableMetadata` twice w/in `BaseHoodieTableFileIndex`
- Avoid looking up FileSystem for every partition when listing partitioned table, instead do it just once
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1238794170
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
}, {
"hash" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440",
"triggerID" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"triggerType" : "PUSH"
}, {
"hash" : "95ce817e050387177ba9620d33868eae1d04306c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11127",
"triggerID" : "95ce817e050387177ba9620d33868eae1d04306c",
"triggerType" : "PUSH"
}, {
"hash" : "ea2c947722271521c860ee1244586654b20cead0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11129",
"triggerID" : "ea2c947722271521c860ee1244586654b20cead0",
"triggerType" : "PUSH"
}, {
"hash" : "8be611a24503d2bd26a924b815b0b92aac4e787a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11131",
"triggerID" : "8be611a24503d2bd26a924b815b0b92aac4e787a",
"triggerType" : "PUSH"
}, {
"hash" : "be9c51945dc2d60fff7f99956567d99e9d5143ea",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11197",
"triggerID" : "be9c51945dc2d60fff7f99956567d99e9d5143ea",
"triggerType" : "PUSH"
}, {
"hash" : "46e53b5182ffdf6fa43b5a93921222e869e4e535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "46e53b5182ffdf6fa43b5a93921222e869e4e535",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* be9c51945dc2d60fff7f99956567d99e9d5143ea Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11197)
* 46e53b5182ffdf6fa43b5a93921222e869e4e535 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1194386494
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ecf41141256ad1dd839398ee0e7097f67f9b9341 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1236054584
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
}, {
"hash" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440",
"triggerID" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"triggerType" : "PUSH"
}, {
"hash" : "95ce817e050387177ba9620d33868eae1d04306c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11127",
"triggerID" : "95ce817e050387177ba9620d33868eae1d04306c",
"triggerType" : "PUSH"
}, {
"hash" : "ea2c947722271521c860ee1244586654b20cead0",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11129",
"triggerID" : "ea2c947722271521c860ee1244586654b20cead0",
"triggerType" : "PUSH"
}, {
"hash" : "8be611a24503d2bd26a924b815b0b92aac4e787a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11131",
"triggerID" : "8be611a24503d2bd26a924b815b0b92aac4e787a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ea2c947722271521c860ee1244586654b20cead0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11129)
* 8be611a24503d2bd26a924b815b0b92aac4e787a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11131)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1235853711
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
}, {
"hash" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440",
"triggerID" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"triggerType" : "PUSH"
}, {
"hash" : "95ce817e050387177ba9620d33868eae1d04306c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "95ce817e050387177ba9620d33868eae1d04306c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 705660efda3e17a13071c7ab3550daceefa9d3b8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440)
* 95ce817e050387177ba9620d33868eae1d04306c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on a diff in pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
codope commented on code in PR #6016:
URL: https://github.com/apache/hudi/pull/6016#discussion_r966582484
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/keygen/SimpleKeyGenerator.java:
##########
@@ -46,6 +47,12 @@ public SimpleKeyGenerator(TypedProperties props) {
SimpleKeyGenerator(TypedProperties props, String recordKeyField, String partitionPathField) {
super(props);
+ // Make sure key-generator is configured properly
+ ValidationUtils.checkArgument(recordKeyField == null || !recordKeyField.isEmpty(),
+ "Record key field has to be non-empty!");
+ ValidationUtils.checkArgument(partitionPathField == null || !partitionPathField.isEmpty(),
Review Comment:
Discussed offline. It will be taken up separately. @alexeykudinkin in case if you have a JIRA, please link it here. For simple keygen we need the validation because of misconfiguration of some tests that were passing “” as partition fields.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1198441226
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9e57627ffddf0eedcd613b54e0f5be5d54462b0b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416)
* b038d97682d74420f906dffabf439eee036553dd UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1238796768
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
}, {
"hash" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440",
"triggerID" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"triggerType" : "PUSH"
}, {
"hash" : "95ce817e050387177ba9620d33868eae1d04306c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11127",
"triggerID" : "95ce817e050387177ba9620d33868eae1d04306c",
"triggerType" : "PUSH"
}, {
"hash" : "ea2c947722271521c860ee1244586654b20cead0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11129",
"triggerID" : "ea2c947722271521c860ee1244586654b20cead0",
"triggerType" : "PUSH"
}, {
"hash" : "8be611a24503d2bd26a924b815b0b92aac4e787a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11131",
"triggerID" : "8be611a24503d2bd26a924b815b0b92aac4e787a",
"triggerType" : "PUSH"
}, {
"hash" : "be9c51945dc2d60fff7f99956567d99e9d5143ea",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11197",
"triggerID" : "be9c51945dc2d60fff7f99956567d99e9d5143ea",
"triggerType" : "PUSH"
}, {
"hash" : "46e53b5182ffdf6fa43b5a93921222e869e4e535",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11200",
"triggerID" : "46e53b5182ffdf6fa43b5a93921222e869e4e535",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* be9c51945dc2d60fff7f99956567d99e9d5143ea Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11197)
* 46e53b5182ffdf6fa43b5a93921222e869e4e535 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11200)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on a diff in pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
codope commented on code in PR #6016:
URL: https://github.com/apache/hudi/pull/6016#discussion_r962568570
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/keygen/SimpleKeyGenerator.java:
##########
@@ -46,6 +47,12 @@ public SimpleKeyGenerator(TypedProperties props) {
SimpleKeyGenerator(TypedProperties props, String recordKeyField, String partitionPathField) {
super(props);
+ // Make sure key-generator is configured properly
+ ValidationUtils.checkArgument(recordKeyField == null || !recordKeyField.isEmpty(),
+ "Record key field has to be non-empty!");
+ ValidationUtils.checkArgument(partitionPathField == null || !partitionPathField.isEmpty(),
Review Comment:
Should the validation message be more user-friendly? Let's say
`"Partition path field has to be non-empty! For non-partitioned table, set key generator class to NonPartitionedKeyGenerator"`.
Also, why are these validations only added for SimpleKeyGenerator? Why not other keygens as well?
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestTimeTravelQuery.scala:
##########
@@ -238,6 +238,7 @@ class TestTimeTravelQuery extends HoodieClientTestBase {
.options(commonOpts)
.option(DataSourceWriteOptions.TABLE_TYPE.key, tableType.name())
.option(PARTITIONPATH_FIELD.key, "")
+ .option(KEYGENERATOR_CLASS_NAME.key, classOf[NonpartitionedKeyGenerator].getName)
Review Comment:
I'd rather remove`.option(PARTITIONPATH_FIELD.key, "")` instead of setting keygen. That way the default behavior gets tested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1196040002
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 619409148a5fd0369e1ae17564972518e7341e86 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368)
* fe477a848ec801ed01dc8fea53a3a8237f9b8034 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1236045116
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
}, {
"hash" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440",
"triggerID" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"triggerType" : "PUSH"
}, {
"hash" : "95ce817e050387177ba9620d33868eae1d04306c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11127",
"triggerID" : "95ce817e050387177ba9620d33868eae1d04306c",
"triggerType" : "PUSH"
}, {
"hash" : "ea2c947722271521c860ee1244586654b20cead0",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11129",
"triggerID" : "ea2c947722271521c860ee1244586654b20cead0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 95ce817e050387177ba9620d33868eae1d04306c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11127)
* ea2c947722271521c860ee1244586654b20cead0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11129)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1196051578
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fe477a848ec801ed01dc8fea53a3a8237f9b8034 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1197377856
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 40e76396c15d5934c90f8163b670c95ef79fa3fa Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1196043233
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 619409148a5fd0369e1ae17564972518e7341e86 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368)
* fe477a848ec801ed01dc8fea53a3a8237f9b8034 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on a diff in pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
xushiyan commented on code in PR #6016:
URL: https://github.com/apache/hudi/pull/6016#discussion_r1005459420
##########
hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java:
##########
@@ -172,14 +174,19 @@ public int getFileSlicesCount() {
.mapToInt(List::size).sum();
}
+ @Override
+ public void close() throws Exception {
+ resetTableMetadata(null);
+ }
+
protected List<PartitionPath> getAllQueryPartitionPaths() {
List<String> queryRelativePartitionPaths = queryPaths.stream()
- .map(path -> FSUtils.getRelativePartitionPath(new Path(basePath), path))
+ .map(path -> FSUtils.getRelativePartitionPath(basePath, path))
.collect(Collectors.toList());
// Load all the partition path from the basePath, and filter by the query partition path.
// TODO load files from the queryRelativePartitionPaths directly.
- List<String> matchedPartitionPaths = FSUtils.getAllPartitionPaths(engineContext, metadataConfig, basePath)
+ List<String> matchedPartitionPaths = getAllPartitionPathsUnchecked()
Review Comment:
this change affects the tables which do not have both 1) `hoodie.table.partition.fields` in table config, and 2) metadata disabled. A partitioned table under these 2 conditions will be deemed as non-partitioned table, and resulted in not loading any records.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #6016:
URL: https://github.com/apache/hudi/pull/6016#discussion_r963885326
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/keygen/SimpleKeyGenerator.java:
##########
@@ -46,6 +47,12 @@ public SimpleKeyGenerator(TypedProperties props) {
SimpleKeyGenerator(TypedProperties props, String recordKeyField, String partitionPathField) {
super(props);
+ // Make sure key-generator is configured properly
+ ValidationUtils.checkArgument(recordKeyField == null || !recordKeyField.isEmpty(),
+ "Record key field has to be non-empty!");
+ ValidationUtils.checkArgument(partitionPathField == null || !partitionPathField.isEmpty(),
Review Comment:
I don't think it makes sense to put suggestions into exception messages -- exception messages should be focused on the problem triggering it, rather than on potential to remedy it (empty partition-path field is usually a sign of misconfiguration, since there's no default value, meaning that user passes "" explicitly)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1197586258
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 178c9f74162b9a420bf42e08a7609cedf1db7104 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412)
* a04e54da5a30fad61de9db4350ed4c9b18f148dd Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1197595779
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 178c9f74162b9a420bf42e08a7609cedf1db7104 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412)
* a04e54da5a30fad61de9db4350ed4c9b18f148dd Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415)
* 9e57627ffddf0eedcd613b54e0f5be5d54462b0b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1198717956
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
}, {
"hash" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440",
"triggerID" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b038d97682d74420f906dffabf439eee036553dd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432)
* 705660efda3e17a13071c7ab3550daceefa9d3b8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1235857674
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
}, {
"hash" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440",
"triggerID" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"triggerType" : "PUSH"
}, {
"hash" : "95ce817e050387177ba9620d33868eae1d04306c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11127",
"triggerID" : "95ce817e050387177ba9620d33868eae1d04306c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 705660efda3e17a13071c7ab3550daceefa9d3b8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440)
* 95ce817e050387177ba9620d33868eae1d04306c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11127)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1236053887
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
}, {
"hash" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440",
"triggerID" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"triggerType" : "PUSH"
}, {
"hash" : "95ce817e050387177ba9620d33868eae1d04306c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11127",
"triggerID" : "95ce817e050387177ba9620d33868eae1d04306c",
"triggerType" : "PUSH"
}, {
"hash" : "ea2c947722271521c860ee1244586654b20cead0",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11129",
"triggerID" : "ea2c947722271521c860ee1244586654b20cead0",
"triggerType" : "PUSH"
}, {
"hash" : "8be611a24503d2bd26a924b815b0b92aac4e787a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8be611a24503d2bd26a924b815b0b92aac4e787a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ea2c947722271521c860ee1244586654b20cead0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11129)
* 8be611a24503d2bd26a924b815b0b92aac4e787a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1197559880
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 178c9f74162b9a420bf42e08a7609cedf1db7104 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412)
* a04e54da5a30fad61de9db4350ed4c9b18f148dd UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1197588724
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 178c9f74162b9a420bf42e08a7609cedf1db7104 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412)
* a04e54da5a30fad61de9db4350ed4c9b18f148dd Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415)
* 9e57627ffddf0eedcd613b54e0f5be5d54462b0b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1236038010
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
}, {
"hash" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440",
"triggerID" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"triggerType" : "PUSH"
}, {
"hash" : "95ce817e050387177ba9620d33868eae1d04306c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11127",
"triggerID" : "95ce817e050387177ba9620d33868eae1d04306c",
"triggerType" : "PUSH"
}, {
"hash" : "ea2c947722271521c860ee1244586654b20cead0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ea2c947722271521c860ee1244586654b20cead0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 95ce817e050387177ba9620d33868eae1d04306c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11127)
* ea2c947722271521c860ee1244586654b20cead0 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on a diff in pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
codope commented on code in PR #6016:
URL: https://github.com/apache/hudi/pull/6016#discussion_r965462152
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/keygen/SimpleKeyGenerator.java:
##########
@@ -46,6 +47,12 @@ public SimpleKeyGenerator(TypedProperties props) {
SimpleKeyGenerator(TypedProperties props, String recordKeyField, String partitionPathField) {
super(props);
+ // Make sure key-generator is configured properly
+ ValidationUtils.checkArgument(recordKeyField == null || !recordKeyField.isEmpty(),
+ "Record key field has to be non-empty!");
+ ValidationUtils.checkArgument(partitionPathField == null || !partitionPathField.isEmpty(),
Review Comment:
Fair enough.
Should we add these validations to other keygens as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on a diff in pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
xushiyan commented on code in PR #6016:
URL: https://github.com/apache/hudi/pull/6016#discussion_r1005459420
##########
hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java:
##########
@@ -172,14 +174,19 @@ public int getFileSlicesCount() {
.mapToInt(List::size).sum();
}
+ @Override
+ public void close() throws Exception {
+ resetTableMetadata(null);
+ }
+
protected List<PartitionPath> getAllQueryPartitionPaths() {
List<String> queryRelativePartitionPaths = queryPaths.stream()
- .map(path -> FSUtils.getRelativePartitionPath(new Path(basePath), path))
+ .map(path -> FSUtils.getRelativePartitionPath(basePath, path))
.collect(Collectors.toList());
// Load all the partition path from the basePath, and filter by the query partition path.
// TODO load files from the queryRelativePartitionPaths directly.
- List<String> matchedPartitionPaths = FSUtils.getAllPartitionPaths(engineContext, metadataConfig, basePath)
+ List<String> matchedPartitionPaths = getAllPartitionPathsUnchecked()
Review Comment:
this change affects partitioned tables that meet both of these conditions 1) `hoodie.table.partition.fields` not present in table config, and 2) metadata disabled. `getAllPartitionPathsUnchecked()` treats them as non-partitioned table, and resulted in not loading any records.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1198708367
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b038d97682d74420f906dffabf439eee036553dd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1197663152
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9e57627ffddf0eedcd613b54e0f5be5d54462b0b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1236071582
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
}, {
"hash" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440",
"triggerID" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"triggerType" : "PUSH"
}, {
"hash" : "95ce817e050387177ba9620d33868eae1d04306c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11127",
"triggerID" : "95ce817e050387177ba9620d33868eae1d04306c",
"triggerType" : "PUSH"
}, {
"hash" : "ea2c947722271521c860ee1244586654b20cead0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11129",
"triggerID" : "ea2c947722271521c860ee1244586654b20cead0",
"triggerType" : "PUSH"
}, {
"hash" : "8be611a24503d2bd26a924b815b0b92aac4e787a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11131",
"triggerID" : "8be611a24503d2bd26a924b815b0b92aac4e787a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8be611a24503d2bd26a924b815b0b92aac4e787a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11131)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1236047005
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
}, {
"hash" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440",
"triggerID" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"triggerType" : "PUSH"
}, {
"hash" : "95ce817e050387177ba9620d33868eae1d04306c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11127",
"triggerID" : "95ce817e050387177ba9620d33868eae1d04306c",
"triggerType" : "PUSH"
}, {
"hash" : "ea2c947722271521c860ee1244586654b20cead0",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11129",
"triggerID" : "ea2c947722271521c860ee1244586654b20cead0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ea2c947722271521c860ee1244586654b20cead0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11129)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1194435700
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ecf41141256ad1dd839398ee0e7097f67f9b9341 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1195995444
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325)
* 619409148a5fd0369e1ae17564972518e7341e86 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1238898719
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
}, {
"hash" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440",
"triggerID" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"triggerType" : "PUSH"
}, {
"hash" : "95ce817e050387177ba9620d33868eae1d04306c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11127",
"triggerID" : "95ce817e050387177ba9620d33868eae1d04306c",
"triggerType" : "PUSH"
}, {
"hash" : "ea2c947722271521c860ee1244586654b20cead0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11129",
"triggerID" : "ea2c947722271521c860ee1244586654b20cead0",
"triggerType" : "PUSH"
}, {
"hash" : "8be611a24503d2bd26a924b815b0b92aac4e787a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11131",
"triggerID" : "8be611a24503d2bd26a924b815b0b92aac4e787a",
"triggerType" : "PUSH"
}, {
"hash" : "be9c51945dc2d60fff7f99956567d99e9d5143ea",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11197",
"triggerID" : "be9c51945dc2d60fff7f99956567d99e9d5143ea",
"triggerType" : "PUSH"
}, {
"hash" : "46e53b5182ffdf6fa43b5a93921222e869e4e535",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11200",
"triggerID" : "46e53b5182ffdf6fa43b5a93921222e869e4e535",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 46e53b5182ffdf6fa43b5a93921222e869e4e535 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11200)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #6016:
URL: https://github.com/apache/hudi/pull/6016#discussion_r1006210059
##########
hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java:
##########
@@ -172,14 +174,19 @@ public int getFileSlicesCount() {
.mapToInt(List::size).sum();
}
+ @Override
+ public void close() throws Exception {
+ resetTableMetadata(null);
+ }
+
protected List<PartitionPath> getAllQueryPartitionPaths() {
List<String> queryRelativePartitionPaths = queryPaths.stream()
- .map(path -> FSUtils.getRelativePartitionPath(new Path(basePath), path))
+ .map(path -> FSUtils.getRelativePartitionPath(basePath, path))
.collect(Collectors.toList());
// Load all the partition path from the basePath, and filter by the query partition path.
// TODO load files from the queryRelativePartitionPaths directly.
- List<String> matchedPartitionPaths = FSUtils.getAllPartitionPaths(engineContext, metadataConfig, basePath)
+ List<String> matchedPartitionPaths = getAllPartitionPathsUnchecked()
Review Comment:
The core of the problem is that `hoodie.table.partition.fields` has to be properly configured -- the table would be considered non-partitioned by some parts of the code (outside of this one) so we need to make sure this is set properly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1197547958
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 178c9f74162b9a420bf42e08a7609cedf1db7104 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on a diff in pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
codope commented on code in PR #6016:
URL: https://github.com/apache/hudi/pull/6016#discussion_r931683314
##########
hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java:
##########
@@ -244,12 +255,7 @@ private Map<PartitionPath, FileStatus[]> loadPartitionPathFiles() {
);
fetchedPartitionToFiles =
- FSUtils.getFilesInPartitions(
- engineContext,
- metadataConfig,
- basePath,
- fullPartitionPathsMapToFetch.keySet().toArray(new String[0]),
- fileSystemStorageConfig.getSpillableDir())
Review Comment:
So if this is not needed then let's remove and avoid instantiating `fileSystemStorageConfig` in the constructor.
##########
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/table/functional/TestHoodieSparkMergeOnReadTableCompaction.java:
##########
@@ -98,8 +99,13 @@ public void testWriteDuringCompaction() throws IOException {
.withLayoutConfig(HoodieLayoutConfig.newBuilder()
.withLayoutType(HoodieStorageLayout.LayoutType.BUCKET.name())
.withLayoutPartitioner(SparkBucketIndexPartitioner.class.getName()).build())
- .withIndexConfig(HoodieIndexConfig.newBuilder().withIndexType(HoodieIndex.IndexType.BUCKET).withBucketNum("1").build()).build();
- metaClient = getHoodieMetaClient(HoodieTableType.MERGE_ON_READ, config.getProps());
+ .withIndexConfig(HoodieIndexConfig.newBuilder().withIndexType(HoodieIndex.IndexType.BUCKET).withBucketNum("1").build())
+ .build();
+
+ Properties props = getPropertiesForKeyGen(true);
Review Comment:
pass `HoodieTableConfig.POPULATE_META_FIELDS.defaultValue()` instead of hard-coding true?
##########
hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java:
##########
@@ -353,12 +395,14 @@ public String getPath() {
return path;
}
- Path fullPartitionPath(String basePath) {
+ Path fullPartitionPath(Path basePath) {
if (!path.isEmpty()) {
- return new CachingPath(basePath, path);
+ // NOTE: Since we now that the path is a proper relative path that doesn't require
Review Comment:
```suggestion
// NOTE: Since we know that the path is a proper relative path that doesn't require
```
##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java:
##########
@@ -398,7 +398,7 @@ public HoodieArchivedTimeline getArchivedTimeline(String startTs) {
public void validateTableProperties(Properties properties) {
// Once meta fields are disabled, it cant be re-enabled for a given table.
if (!getTableConfig().populateMetaFields()
- && Boolean.parseBoolean((String) properties.getOrDefault(HoodieTableConfig.POPULATE_META_FIELDS.key(), HoodieTableConfig.POPULATE_META_FIELDS.defaultValue()))) {
+ && Boolean.parseBoolean((String) properties.getOrDefault(HoodieTableConfig.POPULATE_META_FIELDS.key(), HoodieTableConfig.POPULATE_META_FIELDS.defaultValue().toString()))) {
Review Comment:
is it necessary? it's already being type cast to String
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieHFileDataBlock.java:
##########
@@ -167,9 +164,8 @@ protected ClosableIterator<IndexedRecord> deserializeRecords(byte[] content) thr
// Get schema from the header
Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
- FileSystem fs = FSUtils.getFs(pathForReader.toString(), new Configuration());
// Read the content
- HoodieHFileReader<IndexedRecord> reader = new HoodieHFileReader<>(fs, pathForReader, content, Option.of(writerSchema));
+ HoodieHFileReader<IndexedRecord> reader = new HoodieHFileReader<>(null, pathForReader, content, Option.of(writerSchema));
Review Comment:
This could affect HFile reading. I believe there is some validation in HFile system or HFile's reader context for fs to be non-null. I think we should still pass `fs` and still keep this line in `HoodieHFileUtils#createHFileReader`:
```
Configuration conf = new Configuration(false);
```
##########
hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieTestUtils.java:
##########
@@ -124,28 +125,28 @@ public static HoodieTableMetaClient init(Configuration hadoopConf, String basePa
}
public static HoodieTableMetaClient init(Configuration hadoopConf, String basePath, HoodieTableType tableType,
- Properties properties)
- throws IOException {
- properties = HoodieTableMetaClient.withPropertyBuilder()
- .setTableName(RAW_TRIPS_TEST_NAME)
- .setTableType(tableType)
- .setPayloadClass(HoodieAvroPayload.class)
- .fromProperties(properties)
- .build();
- return HoodieTableMetaClient.initTableAndGetMetaClient(hadoopConf, basePath, properties);
+ Properties properties) throws IOException {
+ return init(hadoopConf, basePath, tableType, properties, null);
}
public static HoodieTableMetaClient init(Configuration hadoopConf, String basePath, HoodieTableType tableType,
Properties properties, String databaseName)
throws IOException {
- properties = HoodieTableMetaClient.withPropertyBuilder()
- .setDatabaseName(databaseName)
- .setTableName(RAW_TRIPS_TEST_NAME)
- .setTableType(tableType)
- .setPayloadClass(HoodieAvroPayload.class)
- .fromProperties(properties)
- .build();
- return HoodieTableMetaClient.initTableAndGetMetaClient(hadoopConf, basePath, properties);
+ HoodieTableMetaClient.PropertyBuilder builder =
+ HoodieTableMetaClient.withPropertyBuilder()
+ .setDatabaseName(databaseName)
+ .setTableName(RAW_TRIPS_TEST_NAME)
+ .setTableType(tableType)
+ .setPayloadClass(HoodieAvroPayload.class);
+
+ String keyGen = properties.getProperty("hoodie.datasource.write.keygenerator.class");
+ if (!Objects.equals(keyGen, "org.apache.hudi.keygen.NonpartitionedKeyGenerator")) {
+ builder.setPartitionFields("some_nonexistent_field");
Review Comment:
extract to constant to standardize across tests?
##########
hudi-common/src/main/java/org/apache/hudi/metadata/BaseTableMetadata.java:
##########
@@ -138,13 +144,17 @@ public FileStatus[] getAllFilesInPartition(Path partitionPath)
}
}
- return new FileSystemBackedTableMetadata(getEngineContext(), hadoopConf, dataBasePath, metadataConfig.shouldAssumeDatePartitioning())
+ return new FileSystemBackedTableMetadata(getEngineContext(), hadoopConf, dataBasePath.toString(), metadataConfig.shouldAssumeDatePartitioning())
.getAllFilesInPartition(partitionPath);
}
@Override
public Map<String, FileStatus[]> getAllFilesInPartitions(List<String> partitions)
throws IOException {
+ if (partitions.isEmpty()) {
+ return Collections.emptyMap();
Review Comment:
If `BaseHoodieTableFileIndex#getAllPartitionPathsUnchecked` returns `Collections.singletonList("")` then should we add an entry for `""` in the map, or rather make `getAllPartitionPathsUnchecked` return `Collections.emptyList()`
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/SparkHoodieTableFileIndex.scala:
##########
@@ -341,4 +340,9 @@ object SparkHoodieTableFileIndex {
override def invalidate(): Unit = cache.invalidateAll()
}
}
+
+ private def shouldValidatePartitionColumns(spark: SparkSession): Boolean = {
+ // NOTE: We can't use helper, method nor the config-entry to stay compatible w/ Spark 2.4
+ spark.sessionState.conf.getConfString("spark.sql.sources.validatePartitionColumns", "true").toBoolean
Review Comment:
Should this go into Spark2Adapter or Spark2ParsePartitionUtil?
##########
hudi-common/src/main/java/org/apache/hudi/hadoop/CachingPath.java:
##########
@@ -83,4 +96,41 @@ public String toString() {
}
return fullPathStr;
}
+
+ public CachingPath subPath(String relativePath) {
+ return new CachingPath(this, createPathUnsafe(relativePath));
+ }
+
+ public static CachingPath wrap(Path path) {
+ if (path instanceof CachingPath) {
+ return (CachingPath) path;
+ }
+
+ return new CachingPath(path.toUri());
+ }
+
+ /**
+ * TODO elaborate
+ */
+ public static CachingPath createPathUnsafe(String relativePath) {
+ try {
+ // NOTE: {@code normalize} is going to be invoked by {@code Path} ctor, so there's no
+ // point in invoking it here
+ URI uri = new URI(null, null, relativePath, null, null);
+ return new CachingPath(uri);
+ } catch (URISyntaxException e) {
+ throw new HoodieException("Failed to instantiate relative path", e);
+ }
+ }
+
+ /**
+ * TODO elaborate
Review Comment:
todo? javadoc only right?
##########
hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java:
##########
@@ -324,6 +335,26 @@ private void doRefresh() {
LOG.info(String.format("Refresh table %s, spent: %d ms", metaClient.getTableConfig().getTableName(), duration));
}
+ private Map<String, FileStatus[]> getAllFilesInPartitionsUnchecked(Collection<String> fullPartitionPathsMapToFetch) {
+ try {
+ return tableMetadata.getAllFilesInPartitions(new ArrayList<>(fullPartitionPathsMapToFetch));
+ } catch (IOException e) {
+ throw new HoodieIOException("Failed to list partition paths for a table", e);
+ }
+ }
+
+ private List<String> getAllPartitionPathsUnchecked() {
+ try {
+ if (partitionColumns.length == 0) {
+ return Collections.singletonList("");
Review Comment:
Should it be `Collections.emptyList()`?
##########
hudi-common/src/main/java/org/apache/hudi/hadoop/CachingPath.java:
##########
@@ -32,11 +33,12 @@
* NOTE: This class is thread-safe
*/
@ThreadSafe
-public class CachingPath extends Path implements Serializable {
Review Comment:
`Path` is not serializable right? Have we vetted all paths that this does not get sent to executors (to avoid TaskNotSerializable execption)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #6016:
URL: https://github.com/apache/hudi/pull/6016#discussion_r931715112
##########
hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java:
##########
@@ -244,12 +255,7 @@ private Map<PartitionPath, FileStatus[]> loadPartitionPathFiles() {
);
fetchedPartitionToFiles =
- FSUtils.getFilesInPartitions(
- engineContext,
- metadataConfig,
- basePath,
- fullPartitionPathsMapToFetch.keySet().toArray(new String[0]),
- fileSystemStorageConfig.getSpillableDir())
Review Comment:
Good call
##########
hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieTestUtils.java:
##########
@@ -124,28 +125,28 @@ public static HoodieTableMetaClient init(Configuration hadoopConf, String basePa
}
public static HoodieTableMetaClient init(Configuration hadoopConf, String basePath, HoodieTableType tableType,
- Properties properties)
- throws IOException {
- properties = HoodieTableMetaClient.withPropertyBuilder()
- .setTableName(RAW_TRIPS_TEST_NAME)
- .setTableType(tableType)
- .setPayloadClass(HoodieAvroPayload.class)
- .fromProperties(properties)
- .build();
- return HoodieTableMetaClient.initTableAndGetMetaClient(hadoopConf, basePath, properties);
+ Properties properties) throws IOException {
+ return init(hadoopConf, basePath, tableType, properties, null);
}
public static HoodieTableMetaClient init(Configuration hadoopConf, String basePath, HoodieTableType tableType,
Properties properties, String databaseName)
throws IOException {
- properties = HoodieTableMetaClient.withPropertyBuilder()
- .setDatabaseName(databaseName)
- .setTableName(RAW_TRIPS_TEST_NAME)
- .setTableType(tableType)
- .setPayloadClass(HoodieAvroPayload.class)
- .fromProperties(properties)
- .build();
- return HoodieTableMetaClient.initTableAndGetMetaClient(hadoopConf, basePath, properties);
+ HoodieTableMetaClient.PropertyBuilder builder =
+ HoodieTableMetaClient.withPropertyBuilder()
+ .setDatabaseName(databaseName)
+ .setTableName(RAW_TRIPS_TEST_NAME)
+ .setTableType(tableType)
+ .setPayloadClass(HoodieAvroPayload.class);
+
+ String keyGen = properties.getProperty("hoodie.datasource.write.keygenerator.class");
+ if (!Objects.equals(keyGen, "org.apache.hudi.keygen.NonpartitionedKeyGenerator")) {
+ builder.setPartitionFields("some_nonexistent_field");
Review Comment:
I don't think we should actually standardize on this one, it's just to stop the bleeding in misconfigured tests
##########
hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java:
##########
@@ -324,6 +335,26 @@ private void doRefresh() {
LOG.info(String.format("Refresh table %s, spent: %d ms", metaClient.getTableConfig().getTableName(), duration));
}
+ private Map<String, FileStatus[]> getAllFilesInPartitionsUnchecked(Collection<String> fullPartitionPathsMapToFetch) {
+ try {
+ return tableMetadata.getAllFilesInPartitions(new ArrayList<>(fullPartitionPathsMapToFetch));
+ } catch (IOException e) {
+ throw new HoodieIOException("Failed to list partition paths for a table", e);
+ }
+ }
+
+ private List<String> getAllPartitionPathsUnchecked() {
+ try {
+ if (partitionColumns.length == 0) {
+ return Collections.singletonList("");
Review Comment:
Non-partitioned table has exactly one partition, which we designate w/ ""
##########
hudi-common/src/main/java/org/apache/hudi/hadoop/CachingPath.java:
##########
@@ -32,11 +33,12 @@
* NOTE: This class is thread-safe
*/
@ThreadSafe
-public class CachingPath extends Path implements Serializable {
Review Comment:
Yep
##########
hudi-common/src/main/java/org/apache/hudi/metadata/BaseTableMetadata.java:
##########
@@ -138,13 +144,17 @@ public FileStatus[] getAllFilesInPartition(Path partitionPath)
}
}
- return new FileSystemBackedTableMetadata(getEngineContext(), hadoopConf, dataBasePath, metadataConfig.shouldAssumeDatePartitioning())
+ return new FileSystemBackedTableMetadata(getEngineContext(), hadoopConf, dataBasePath.toString(), metadataConfig.shouldAssumeDatePartitioning())
.getAllFilesInPartition(partitionPath);
}
@Override
public Map<String, FileStatus[]> getAllFilesInPartitions(List<String> partitions)
throws IOException {
+ if (partitions.isEmpty()) {
+ return Collections.emptyMap();
Review Comment:
Agree, this is somewhat dissonant, but that's just the way things are -- for non-partitioned tables it's assumed that the only partition that is there has to be identified by ""
##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java:
##########
@@ -398,7 +398,7 @@ public HoodieArchivedTimeline getArchivedTimeline(String startTs) {
public void validateTableProperties(Properties properties) {
// Once meta fields are disabled, it cant be re-enabled for a given table.
if (!getTableConfig().populateMetaFields()
- && Boolean.parseBoolean((String) properties.getOrDefault(HoodieTableConfig.POPULATE_META_FIELDS.key(), HoodieTableConfig.POPULATE_META_FIELDS.defaultValue()))) {
+ && Boolean.parseBoolean((String) properties.getOrDefault(HoodieTableConfig.POPULATE_META_FIELDS.key(), HoodieTableConfig.POPULATE_META_FIELDS.defaultValue().toString()))) {
Review Comment:
It's not
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieHFileDataBlock.java:
##########
@@ -167,9 +164,8 @@ protected ClosableIterator<IndexedRecord> deserializeRecords(byte[] content) thr
// Get schema from the header
Schema writerSchema = new Schema.Parser().parse(super.getLogBlockHeader().get(HeaderMetadataType.SCHEMA));
- FileSystem fs = FSUtils.getFs(pathForReader.toString(), new Configuration());
// Read the content
- HoodieHFileReader<IndexedRecord> reader = new HoodieHFileReader<>(fs, pathForReader, content, Option.of(writerSchema));
+ HoodieHFileReader<IndexedRecord> reader = new HoodieHFileReader<>(null, pathForReader, content, Option.of(writerSchema));
Review Comment:
Yeah, i checked it and it actually doesn't use `fs` at all
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/SparkHoodieTableFileIndex.scala:
##########
@@ -341,4 +340,9 @@ object SparkHoodieTableFileIndex {
override def invalidate(): Unit = cache.invalidateAll()
}
}
+
+ private def shouldValidatePartitionColumns(spark: SparkSession): Boolean = {
+ // NOTE: We can't use helper, method nor the config-entry to stay compatible w/ Spark 2.4
+ spark.sessionState.conf.getConfString("spark.sql.sources.validatePartitionColumns", "true").toBoolean
Review Comment:
We don't need to
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1198816105
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
}, {
"hash" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440",
"triggerID" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 705660efda3e17a13071c7ab3550daceefa9d3b8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1235900840
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
}, {
"hash" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440",
"triggerID" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"triggerType" : "PUSH"
}, {
"hash" : "95ce817e050387177ba9620d33868eae1d04306c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11127",
"triggerID" : "95ce817e050387177ba9620d33868eae1d04306c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 95ce817e050387177ba9620d33868eae1d04306c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11127)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1238761398
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
}, {
"hash" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440",
"triggerID" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"triggerType" : "PUSH"
}, {
"hash" : "95ce817e050387177ba9620d33868eae1d04306c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11127",
"triggerID" : "95ce817e050387177ba9620d33868eae1d04306c",
"triggerType" : "PUSH"
}, {
"hash" : "ea2c947722271521c860ee1244586654b20cead0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11129",
"triggerID" : "ea2c947722271521c860ee1244586654b20cead0",
"triggerType" : "PUSH"
}, {
"hash" : "8be611a24503d2bd26a924b815b0b92aac4e787a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11131",
"triggerID" : "8be611a24503d2bd26a924b815b0b92aac4e787a",
"triggerType" : "PUSH"
}, {
"hash" : "be9c51945dc2d60fff7f99956567d99e9d5143ea",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11197",
"triggerID" : "be9c51945dc2d60fff7f99956567d99e9d5143ea",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8be611a24503d2bd26a924b815b0b92aac4e787a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11131)
* be9c51945dc2d60fff7f99956567d99e9d5143ea Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11197)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1238758890
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
}, {
"hash" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10440",
"triggerID" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"triggerType" : "PUSH"
}, {
"hash" : "95ce817e050387177ba9620d33868eae1d04306c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11127",
"triggerID" : "95ce817e050387177ba9620d33868eae1d04306c",
"triggerType" : "PUSH"
}, {
"hash" : "ea2c947722271521c860ee1244586654b20cead0",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11129",
"triggerID" : "ea2c947722271521c860ee1244586654b20cead0",
"triggerType" : "PUSH"
}, {
"hash" : "8be611a24503d2bd26a924b815b0b92aac4e787a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11131",
"triggerID" : "8be611a24503d2bd26a924b815b0b92aac4e787a",
"triggerType" : "PUSH"
}, {
"hash" : "be9c51945dc2d60fff7f99956567d99e9d5143ea",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "be9c51945dc2d60fff7f99956567d99e9d5143ea",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8be611a24503d2bd26a924b815b0b92aac4e787a Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11131)
* be9c51945dc2d60fff7f99956567d99e9d5143ea UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1197365120
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fe477a848ec801ed01dc8fea53a3a8237f9b8034 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371)
* 40e76396c15d5934c90f8163b670c95ef79fa3fa UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1197487511
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 40e76396c15d5934c90f8163b670c95ef79fa3fa Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402)
* 178c9f74162b9a420bf42e08a7609cedf1db7104 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1194457717
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1198715352
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
}, {
"hash" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "705660efda3e17a13071c7ab3550daceefa9d3b8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b038d97682d74420f906dffabf439eee036553dd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432)
* 705660efda3e17a13071c7ab3550daceefa9d3b8 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1197622320
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a04e54da5a30fad61de9db4350ed4c9b18f148dd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415)
* 9e57627ffddf0eedcd613b54e0f5be5d54462b0b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1198445061
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
}, {
"hash" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10415",
"triggerID" : "a04e54da5a30fad61de9db4350ed4c9b18f148dd",
"triggerType" : "PUSH"
}, {
"hash" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416",
"triggerID" : "9e57627ffddf0eedcd613b54e0f5be5d54462b0b",
"triggerType" : "PUSH"
}, {
"hash" : "b038d97682d74420f906dffabf439eee036553dd",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432",
"triggerID" : "b038d97682d74420f906dffabf439eee036553dd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9e57627ffddf0eedcd613b54e0f5be5d54462b0b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10416)
* b038d97682d74420f906dffabf439eee036553dd Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10432)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1197513714
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
}, {
"hash" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412",
"triggerID" : "178c9f74162b9a420bf42e08a7609cedf1db7104",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 40e76396c15d5934c90f8163b670c95ef79fa3fa Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402)
* 178c9f74162b9a420bf42e08a7609cedf1db7104 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10412)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1197369475
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
}, {
"hash" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371",
"triggerID" : "fe477a848ec801ed01dc8fea53a3a8237f9b8034",
"triggerType" : "PUSH"
}, {
"hash" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402",
"triggerID" : "40e76396c15d5934c90f8163b670c95ef79fa3fa",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fe477a848ec801ed01dc8fea53a3a8237f9b8034 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10371)
* 40e76396c15d5934c90f8163b670c95ef79fa3fa Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10402)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #6016: [WIP] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1188815206
hey @alexeykudinkin : can you link the right jira for the patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope merged pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
codope merged PR #6016:
URL: https://github.com/apache/hudi/pull/6016
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1194441477
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ecf41141256ad1dd839398ee0e7097f67f9b9341 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323)
* 0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6016: [HUDI-4465] Optimizing file-listing sequence of Metadata Table
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6016:
URL: https://github.com/apache/hudi/pull/6016#issuecomment-1195999079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10323",
"triggerID" : "ecf41141256ad1dd839398ee0e7097f67f9b9341",
"triggerType" : "PUSH"
}, {
"hash" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325",
"triggerID" : "0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79",
"triggerType" : "PUSH"
}, {
"hash" : "619409148a5fd0369e1ae17564972518e7341e86",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368",
"triggerID" : "619409148a5fd0369e1ae17564972518e7341e86",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0ae8d0eb6e8a7573835bf3096f0e86f7fa6b9e79 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10325)
* 619409148a5fd0369e1ae17564972518e7341e86 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10368)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org