You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/05/10 02:03:45 UTC
[GitHub] [hudi] cuibo01 opened a new pull request, #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
cuibo01 opened a new pull request, #5545:
URL: https://github.com/apache/hudi/pull/5545
…ion files
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*
## What is the purpose of the pull request
*(For example: This pull request adds quick-start document.)*
## Brief change log
*(for example:)*
- *Modify AnnotationLocation checkstyle rule in checkstyle.xml*
## Verify this pull request
*(Please pick either of the following options)*
This pull request is a trivial rework / code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end.*
- *Added HoodieClientWriteTest to verify the change.*
- *Manually verified the change by running a job locally.*
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on a diff in pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on code in PR #5545:
URL: https://github.com/apache/hudi/pull/5545#discussion_r869860559
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -644,14 +644,17 @@ public final Stream<FileSlice> getLatestUnCompactedFileSlices(String partitionSt
@Override
public final Stream<FileSlice> getLatestFileSlicesBeforeOrOn(String partitionStr, String maxCommitTime,
- boolean includeFileSlicesInPendingCompaction) {
+ boolean includeFileSlicesInPendingCompaction, boolean includeFilesInPendingCompaction) {
try {
readLock.lock();
String partitionPath = formatPartitionKey(partitionStr);
ensurePartitionLoadedCorrectly(partitionPath);
Stream<FileSlice> fileSliceStream = fetchLatestFileSlicesBeforeOrOn(partitionPath, maxCommitTime)
.filter(slice -> !isFileGroupReplacedBeforeOrOn(slice.getFileGroupId(), maxCommitTime));
if (includeFileSlicesInPendingCompaction) {
+ if (includeFilesInPendingCompaction) {
+ return fileSliceStream.map(this::addBootstrapBaseFileIfPresent);
Review Comment:
and u can run TestBoostrapOperator locally.
` .getLatestFileSlicesBeforeOrOn("par1", latestCommitTime.get().getTimestamp(), true, true)`
or
` .getLatestFileSlicesBeforeOrOn("par1", latestCommitTime.get().getTimestamp(), true, false)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 merged pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
danny0405 merged PR #5545:
URL: https://github.com/apache/hudi/pull/5545
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5545:
URL: https://github.com/apache/hudi/pull/5545#discussion_r868778594
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -644,14 +644,17 @@ public final Stream<FileSlice> getLatestUnCompactedFileSlices(String partitionSt
@Override
public final Stream<FileSlice> getLatestFileSlicesBeforeOrOn(String partitionStr, String maxCommitTime,
- boolean includeFileSlicesInPendingCompaction) {
+ boolean includeFileSlicesInPendingCompaction, boolean includeFilesInPendingCompaction) {
try {
readLock.lock();
String partitionPath = formatPartitionKey(partitionStr);
ensurePartitionLoadedCorrectly(partitionPath);
Stream<FileSlice> fileSliceStream = fetchLatestFileSlicesBeforeOrOn(partitionPath, maxCommitTime)
.filter(slice -> !isFileGroupReplacedBeforeOrOn(slice.getFileGroupId(), maxCommitTime));
if (includeFileSlicesInPendingCompaction) {
+ if (includeFilesInPendingCompaction) {
+ return fileSliceStream.map(this::addBootstrapBaseFileIfPresent);
Review Comment:
Can you explain a little more the details ? BTW, we should not modify the interfaces of fs view in most of the cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1123657822
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* 9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on a diff in pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on code in PR #5545:
URL: https://github.com/apache/hudi/pull/5545#discussion_r869860559
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -644,14 +644,17 @@ public final Stream<FileSlice> getLatestUnCompactedFileSlices(String partitionSt
@Override
public final Stream<FileSlice> getLatestFileSlicesBeforeOrOn(String partitionStr, String maxCommitTime,
- boolean includeFileSlicesInPendingCompaction) {
+ boolean includeFileSlicesInPendingCompaction, boolean includeFilesInPendingCompaction) {
try {
readLock.lock();
String partitionPath = formatPartitionKey(partitionStr);
ensurePartitionLoadedCorrectly(partitionPath);
Stream<FileSlice> fileSliceStream = fetchLatestFileSlicesBeforeOrOn(partitionPath, maxCommitTime)
.filter(slice -> !isFileGroupReplacedBeforeOrOn(slice.getFileGroupId(), maxCommitTime));
if (includeFileSlicesInPendingCompaction) {
+ if (includeFilesInPendingCompaction) {
+ return fileSliceStream.map(this::addBootstrapBaseFileIfPresent);
Review Comment:
and then u can run TestBoostrapOperator locally.
` .getLatestFileSlicesBeforeOrOn("par1", latestCommitTime.get().getTimestamp(), true, true)`
or
` .getLatestFileSlicesBeforeOrOn("par1", latestCommitTime.get().getTimestamp(), true, false)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1123824917
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1123824503",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* 9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1123830236
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123824503",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* 9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1122323473
@danny0405 hi I reproduced it on 0.11. and add UT
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1121884853
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cd51f10efdbb4209a81b1cf85da03aabc3ab06f1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1125626848
Hello @cuibo01 , there are some test failures, i have fired a fix here:
[fix.patch.zip](https://github.com/apache/hudi/files/8683928/fix.patch.zip)
, can you apply the patch and fix it then ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1123575458
> Hi, @cuibo01 , thanks for the contribution, instead of adding new interface, using `getLatestMergedFileSlicesBeforeOrOn` instead would solve the problem.
👍 update
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1123824503
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1123151841
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1123577248
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560)
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* 9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1124732913
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123824503",
"triggerType" : "MANUAL"
}, {
"hash" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"triggerType" : "PUSH"
}, {
"hash" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588",
"triggerID" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"triggerType" : "PUSH"
}, {
"hash" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8600",
"triggerID" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"triggerType" : "PUSH"
}, {
"hash" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602",
"triggerID" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* f08364dbd036051b0cf8cc0e018deaa6d353fe44 UNKNOWN
* ce582122dde22dc64c8528cb229a9eb76ab630d6 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588)
* 995e1b573368430392ecd18eff9d6809eab2cf15 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8600)
* 3f3faa0ac0b8d41bc070b085869dce6472b1a649 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1125633913
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123824503",
"triggerType" : "MANUAL"
}, {
"hash" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"triggerType" : "PUSH"
}, {
"hash" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588",
"triggerID" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"triggerType" : "PUSH"
}, {
"hash" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8600",
"triggerID" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"triggerType" : "PUSH"
}, {
"hash" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602",
"triggerID" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"triggerType" : "PUSH"
}, {
"hash" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602",
"triggerID" : "1125551328",
"triggerType" : "MANUAL"
}, {
"hash" : "61f5b613e7d59e61474bb7f2be313dca31cf13e3",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "61f5b613e7d59e61474bb7f2be313dca31cf13e3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* f08364dbd036051b0cf8cc0e018deaa6d353fe44 UNKNOWN
* 3f3faa0ac0b8d41bc070b085869dce6472b1a649 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602)
* 61f5b613e7d59e61474bb7f2be313dca31cf13e3 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1124453926
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123824503",
"triggerType" : "MANUAL"
}, {
"hash" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"triggerType" : "PUSH"
}, {
"hash" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588",
"triggerID" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* 9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573)
* f08364dbd036051b0cf8cc0e018deaa6d353fe44 UNKNOWN
* ce582122dde22dc64c8528cb229a9eb76ab630d6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1124450667
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123824503",
"triggerType" : "MANUAL"
}, {
"hash" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* 9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573)
* f08364dbd036051b0cf8cc0e018deaa6d353fe44 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1124514364
Thanks for the contribution, i have created a patch for testing:
[4078.patch.zip](https://github.com/apache/hudi/files/8674933/4078.patch.zip)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1121996361
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* cd51f10efdbb4209a81b1cf85da03aabc3ab06f1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1125557479
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123824503",
"triggerType" : "MANUAL"
}, {
"hash" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"triggerType" : "PUSH"
}, {
"hash" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588",
"triggerID" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"triggerType" : "PUSH"
}, {
"hash" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8600",
"triggerID" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"triggerType" : "PUSH"
}, {
"hash" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602",
"triggerID" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"triggerType" : "PUSH"
}, {
"hash" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602",
"triggerID" : "1125551328",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* f08364dbd036051b0cf8cc0e018deaa6d353fe44 UNKNOWN
* 3f3faa0ac0b8d41bc070b085869dce6472b1a649 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on a diff in pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on code in PR #5545:
URL: https://github.com/apache/hudi/pull/5545#discussion_r868868607
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -644,14 +644,17 @@ public final Stream<FileSlice> getLatestUnCompactedFileSlices(String partitionSt
@Override
public final Stream<FileSlice> getLatestFileSlicesBeforeOrOn(String partitionStr, String maxCommitTime,
- boolean includeFileSlicesInPendingCompaction) {
+ boolean includeFileSlicesInPendingCompaction, boolean includeFilesInPendingCompaction) {
try {
readLock.lock();
String partitionPath = formatPartitionKey(partitionStr);
ensurePartitionLoadedCorrectly(partitionPath);
Stream<FileSlice> fileSliceStream = fetchLatestFileSlicesBeforeOrOn(partitionPath, maxCommitTime)
.filter(slice -> !isFileGroupReplacedBeforeOrOn(slice.getFileGroupId(), maxCommitTime));
if (includeFileSlicesInPendingCompaction) {
+ if (includeFilesInPendingCompaction) {
+ return fileSliceStream.map(this::addBootstrapBaseFileIfPresent);
Review Comment:
thx for your review, @danny0405
from the TM log:
2022-04-25 14:21:25,198 INFO org.apache.hudi.common.table.view.AbstractTableFileSystemView - Pending Compaction instant for (FileSlice {fileGroupId=HoodieFileGroupId{partitionPath='', fileId='02d71f2b-bb2f-4c44-96f2-78cb791da28e'}, baseCommitTime=20220425131449, baseFile='null', logFiles='[HoodieLogFile{pathStr='hdfs://hacluster/tmp/hudi/hudi_sink/.02d71f2b-bb2f-4c44-96f2-78cb791da28e_20220425131449.log.1_4-50-0', fileLen=20884490}]'}) **is :Option**{val=(20220425131449,CompactionOperation{baseInstantTime='20220425131251', dataFileCommitTime=Option{val=20220425131251}, deltaFileNames=[.02d71f2b-bb2f-4c44-96f2-78cb791da28e_20220425131251.log.1_4-50-0], dataFileName=Option{val=02d71f2b-bb2f-4c44-96f2-78cb791da28e_8-10-0_20220425131251.parquet}, id='HoodieFileGroupId{partitionPath='', fileId='02d71f2b-bb2f-4c44-96f2-78cb791da28e'}', metrics={TOTAL_LOG_FILES=1.0, TOTAL_IO_READ_MB=84.0, TOTAL_LOG_FILES_SIZE=2.2786803E7, TOTAL_IO_WRITE_MB=62.0, TOTAL_IO_MB=146.0}, bootstrapFilePath=Opt
ional.empty})}
2022-04-25 14:21:25,198 INFO org.apache.hudi.common.table.view.AbstractTableFileSystemView - File Slice (FileSlice {fileGroupId=HoodieFileGroupId{partitionPath='', fileId='02d71f2b-bb2f-4c44-96f2-78cb791da28e'}, baseCommitTime=20220425131449, baseFile='null', logFiles='[HoodieLogFile{pathStr='hdfs://hacluster/tmp/hudi/hudi_sink/.02d71f2b-bb2f-4c44-96f2-78cb791da28e_20220425131449.log.1_4-50-0', fileLen=20884490}]'}) is in pending compaction
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1123092703
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 38b4297a96d673590ec31ae099e989cbf345827a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548)
* 8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1121792331
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cd51f10efdbb4209a81b1cf85da03aabc3ab06f1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1121861555
100000001 records:
Before the repair
After the repair
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1122444873
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 38b4297a96d673590ec31ae099e989cbf345827a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1123103141
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 38b4297a96d673590ec31ae099e989cbf345827a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548)
* 8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5545:
URL: https://github.com/apache/hudi/pull/5545#discussion_r869973432
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -644,14 +644,17 @@ public final Stream<FileSlice> getLatestUnCompactedFileSlices(String partitionSt
@Override
public final Stream<FileSlice> getLatestFileSlicesBeforeOrOn(String partitionStr, String maxCommitTime,
- boolean includeFileSlicesInPendingCompaction) {
+ boolean includeFileSlicesInPendingCompaction, boolean includeFilesInPendingCompaction) {
try {
readLock.lock();
String partitionPath = formatPartitionKey(partitionStr);
ensurePartitionLoadedCorrectly(partitionPath);
Stream<FileSlice> fileSliceStream = fetchLatestFileSlicesBeforeOrOn(partitionPath, maxCommitTime)
.filter(slice -> !isFileGroupReplacedBeforeOrOn(slice.getFileGroupId(), maxCommitTime));
if (includeFileSlicesInPendingCompaction) {
+ if (includeFilesInPendingCompaction) {
+ return fileSliceStream.map(this::addBootstrapBaseFileIfPresent);
Review Comment:
Yes, you are right, then using `getLatestMergedFileSlicesBeforeOrOn` instead should generates correct result.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -644,14 +644,17 @@ public final Stream<FileSlice> getLatestUnCompactedFileSlices(String partitionSt
@Override
public final Stream<FileSlice> getLatestFileSlicesBeforeOrOn(String partitionStr, String maxCommitTime,
- boolean includeFileSlicesInPendingCompaction) {
+ boolean includeFileSlicesInPendingCompaction, boolean includeFilesInPendingCompaction) {
try {
readLock.lock();
String partitionPath = formatPartitionKey(partitionStr);
ensurePartitionLoadedCorrectly(partitionPath);
Stream<FileSlice> fileSliceStream = fetchLatestFileSlicesBeforeOrOn(partitionPath, maxCommitTime)
.filter(slice -> !isFileGroupReplacedBeforeOrOn(slice.getFileGroupId(), maxCommitTime));
if (includeFileSlicesInPendingCompaction) {
+ if (includeFilesInPendingCompaction) {
+ return fileSliceStream.map(this::addBootstrapBaseFileIfPresent);
Review Comment:
Yes, you are right, then using `getLatestMergedFileSlicesBeforeOrOn` instead should generate correct result.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1124920766
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123824503",
"triggerType" : "MANUAL"
}, {
"hash" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"triggerType" : "PUSH"
}, {
"hash" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588",
"triggerID" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"triggerType" : "PUSH"
}, {
"hash" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8600",
"triggerID" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"triggerType" : "PUSH"
}, {
"hash" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602",
"triggerID" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* f08364dbd036051b0cf8cc0e018deaa6d353fe44 UNKNOWN
* 3f3faa0ac0b8d41bc070b085869dce6472b1a649 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1124675058
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123824503",
"triggerType" : "MANUAL"
}, {
"hash" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"triggerType" : "PUSH"
}, {
"hash" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588",
"triggerID" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"triggerType" : "PUSH"
}, {
"hash" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8600",
"triggerID" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* f08364dbd036051b0cf8cc0e018deaa6d353fe44 UNKNOWN
* ce582122dde22dc64c8528cb229a9eb76ab630d6 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588)
* 995e1b573368430392ecd18eff9d6809eab2cf15 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8600)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1123504608
Hi, @cuibo01 , thanks for the contribution, instead of adding new interface, using `getLatestMergedFileSlicesBeforeOrOn` instead would solve the problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1124451998
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123824503",
"triggerType" : "MANUAL"
}, {
"hash" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"triggerType" : "PUSH"
}, {
"hash" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* 9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573)
* f08364dbd036051b0cf8cc0e018deaa6d353fe44 UNKNOWN
* ce582122dde22dc64c8528cb229a9eb76ab630d6 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on a diff in pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on code in PR #5545:
URL: https://github.com/apache/hudi/pull/5545#discussion_r869830519
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -644,14 +644,17 @@ public final Stream<FileSlice> getLatestUnCompactedFileSlices(String partitionSt
@Override
public final Stream<FileSlice> getLatestFileSlicesBeforeOrOn(String partitionStr, String maxCommitTime,
- boolean includeFileSlicesInPendingCompaction) {
+ boolean includeFileSlicesInPendingCompaction, boolean includeFilesInPendingCompaction) {
try {
readLock.lock();
String partitionPath = formatPartitionKey(partitionStr);
ensurePartitionLoadedCorrectly(partitionPath);
Stream<FileSlice> fileSliceStream = fetchLatestFileSlicesBeforeOrOn(partitionPath, maxCommitTime)
.filter(slice -> !isFileGroupReplacedBeforeOrOn(slice.getFileGroupId(), maxCommitTime));
if (includeFileSlicesInPendingCompaction) {
+ if (includeFilesInPendingCompaction) {
+ return fileSliceStream.map(this::addBootstrapBaseFileIfPresent);
Review Comment:
When job is restarted, compaction operator and bootstrap operator are initialized at the same time. The bootstrapOperator#initializeState is executed before the compactionOperator#open. So bootstrapOperator#loadRecords may have got the wrong fileSlices, because the logic (https://github.com/apache/hudi/blob/4258a715174e0a97271112148ab20ef9307e2bd8/hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java#L416) filters out pendingCompaction files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1124729324
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123824503",
"triggerType" : "MANUAL"
}, {
"hash" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"triggerType" : "PUSH"
}, {
"hash" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588",
"triggerID" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"triggerType" : "PUSH"
}, {
"hash" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8600",
"triggerID" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"triggerType" : "PUSH"
}, {
"hash" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* f08364dbd036051b0cf8cc0e018deaa6d353fe44 UNKNOWN
* ce582122dde22dc64c8528cb229a9eb76ab630d6 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588)
* 995e1b573368430392ecd18eff9d6809eab2cf15 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8600)
* 3f3faa0ac0b8d41bc070b085869dce6472b1a649 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1125555832
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123824503",
"triggerType" : "MANUAL"
}, {
"hash" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"triggerType" : "PUSH"
}, {
"hash" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588",
"triggerID" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"triggerType" : "PUSH"
}, {
"hash" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8600",
"triggerID" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"triggerType" : "PUSH"
}, {
"hash" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602",
"triggerID" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"triggerType" : "PUSH"
}, {
"hash" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1125551328",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* f08364dbd036051b0cf8cc0e018deaa6d353fe44 UNKNOWN
* 3f3faa0ac0b8d41bc070b085869dce6472b1a649 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1124503372
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123824503",
"triggerType" : "MANUAL"
}, {
"hash" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"triggerType" : "PUSH"
}, {
"hash" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588",
"triggerID" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* f08364dbd036051b0cf8cc0e018deaa6d353fe44 UNKNOWN
* ce582122dde22dc64c8528cb229a9eb76ab630d6 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1122360398
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 81dd014157036cf607987a8d989b8412a5f84dce Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1122369762
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 81dd014157036cf607987a8d989b8412a5f84dce Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546)
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on a diff in pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on code in PR #5545:
URL: https://github.com/apache/hudi/pull/5545#discussion_r869830519
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -644,14 +644,17 @@ public final Stream<FileSlice> getLatestUnCompactedFileSlices(String partitionSt
@Override
public final Stream<FileSlice> getLatestFileSlicesBeforeOrOn(String partitionStr, String maxCommitTime,
- boolean includeFileSlicesInPendingCompaction) {
+ boolean includeFileSlicesInPendingCompaction, boolean includeFilesInPendingCompaction) {
try {
readLock.lock();
String partitionPath = formatPartitionKey(partitionStr);
ensurePartitionLoadedCorrectly(partitionPath);
Stream<FileSlice> fileSliceStream = fetchLatestFileSlicesBeforeOrOn(partitionPath, maxCommitTime)
.filter(slice -> !isFileGroupReplacedBeforeOrOn(slice.getFileGroupId(), maxCommitTime));
if (includeFileSlicesInPendingCompaction) {
+ if (includeFilesInPendingCompaction) {
+ return fileSliceStream.map(this::addBootstrapBaseFileIfPresent);
Review Comment:
When job is restarted, compaction operator and bootstrap operator are initialized at the same time. The bootstrapOperator#initializeState is executed before the compactionOperator#open. So bootstrapOperator#loadRecords may have got the wrong fileSlices, the logic (https://github.com/apache/hudi/blob/4258a715174e0a97271112148ab20ef9307e2bd8/hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java#L416) filters out pendingCompaction files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5545:
URL: https://github.com/apache/hudi/pull/5545#discussion_r869843707
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -644,14 +644,17 @@ public final Stream<FileSlice> getLatestUnCompactedFileSlices(String partitionSt
@Override
public final Stream<FileSlice> getLatestFileSlicesBeforeOrOn(String partitionStr, String maxCommitTime,
- boolean includeFileSlicesInPendingCompaction) {
+ boolean includeFileSlicesInPendingCompaction, boolean includeFilesInPendingCompaction) {
try {
readLock.lock();
String partitionPath = formatPartitionKey(partitionStr);
ensurePartitionLoadedCorrectly(partitionPath);
Stream<FileSlice> fileSliceStream = fetchLatestFileSlicesBeforeOrOn(partitionPath, maxCommitTime)
.filter(slice -> !isFileGroupReplacedBeforeOrOn(slice.getFileGroupId(), maxCommitTime));
if (includeFileSlicesInPendingCompaction) {
+ if (includeFilesInPendingCompaction) {
+ return fileSliceStream.map(this::addBootstrapBaseFileIfPresent);
Review Comment:
I think the `filters out pendingCompaction files` is right behavior here ? If the file slice is in pending state, we would read the log files and resolve the index info there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on a diff in pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on code in PR #5545:
URL: https://github.com/apache/hudi/pull/5545#discussion_r869859141
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -644,14 +644,17 @@ public final Stream<FileSlice> getLatestUnCompactedFileSlices(String partitionSt
@Override
public final Stream<FileSlice> getLatestFileSlicesBeforeOrOn(String partitionStr, String maxCommitTime,
- boolean includeFileSlicesInPendingCompaction) {
+ boolean includeFileSlicesInPendingCompaction, boolean includeFilesInPendingCompaction) {
try {
readLock.lock();
String partitionPath = formatPartitionKey(partitionStr);
ensurePartitionLoadedCorrectly(partitionPath);
Stream<FileSlice> fileSliceStream = fetchLatestFileSlicesBeforeOrOn(partitionPath, maxCommitTime)
.filter(slice -> !isFileGroupReplacedBeforeOrOn(slice.getFileGroupId(), maxCommitTime));
if (includeFileSlicesInPendingCompaction) {
+ if (includeFilesInPendingCompaction) {
+ return fileSliceStream.map(this::addBootstrapBaseFileIfPresent);
Review Comment:
but from log，pendingCompaction contains not only logs, but also parquet file(02d71f2b-bb2f-4c44-96f2-78cb791da28e_8-10-0_20220425131251.parquet), and then the logic filters the parquet,
https://github.com/apache/hudi/blob/4258a715174e0a97271112148ab20ef9307e2bd8/hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java#L422
so bootstrapOperator cannot obtain the data in the parquet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5545:
URL: https://github.com/apache/hudi/pull/5545#discussion_r869913467
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -644,14 +644,17 @@ public final Stream<FileSlice> getLatestUnCompactedFileSlices(String partitionSt
@Override
public final Stream<FileSlice> getLatestFileSlicesBeforeOrOn(String partitionStr, String maxCommitTime,
- boolean includeFileSlicesInPendingCompaction) {
+ boolean includeFileSlicesInPendingCompaction, boolean includeFilesInPendingCompaction) {
try {
readLock.lock();
String partitionPath = formatPartitionKey(partitionStr);
ensurePartitionLoadedCorrectly(partitionPath);
Stream<FileSlice> fileSliceStream = fetchLatestFileSlicesBeforeOrOn(partitionPath, maxCommitTime)
.filter(slice -> !isFileGroupReplacedBeforeOrOn(slice.getFileGroupId(), maxCommitTime));
if (includeFileSlicesInPendingCompaction) {
+ if (includeFilesInPendingCompaction) {
+ return fileSliceStream.map(this::addBootstrapBaseFileIfPresent);
Review Comment:
> and then the logic filters the parquet
We should filter the parquet here, the compaction parquet and log have same content.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on a diff in pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on code in PR #5545:
URL: https://github.com/apache/hudi/pull/5545#discussion_r869918605
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -644,14 +644,17 @@ public final Stream<FileSlice> getLatestUnCompactedFileSlices(String partitionSt
@Override
public final Stream<FileSlice> getLatestFileSlicesBeforeOrOn(String partitionStr, String maxCommitTime,
- boolean includeFileSlicesInPendingCompaction) {
+ boolean includeFileSlicesInPendingCompaction, boolean includeFilesInPendingCompaction) {
try {
readLock.lock();
String partitionPath = formatPartitionKey(partitionStr);
ensurePartitionLoadedCorrectly(partitionPath);
Stream<FileSlice> fileSliceStream = fetchLatestFileSlicesBeforeOrOn(partitionPath, maxCommitTime)
.filter(slice -> !isFileGroupReplacedBeforeOrOn(slice.getFileGroupId(), maxCommitTime));
if (includeFileSlicesInPendingCompaction) {
+ if (includeFilesInPendingCompaction) {
+ return fileSliceStream.map(this::addBootstrapBaseFileIfPresent);
Review Comment:
However, the log file is not obtained from UT
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1123661444
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* 9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1124671575
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123824503",
"triggerType" : "MANUAL"
}, {
"hash" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"triggerType" : "PUSH"
}, {
"hash" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588",
"triggerID" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"triggerType" : "PUSH"
}, {
"hash" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* f08364dbd036051b0cf8cc0e018deaa6d353fe44 UNKNOWN
* ce582122dde22dc64c8528cb229a9eb76ab630d6 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588)
* 995e1b573368430392ecd18eff9d6809eab2cf15 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1121993995
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* cd51f10efdbb4209a81b1cf85da03aabc3ab06f1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1123654939
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on a diff in pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on code in PR #5545:
URL: https://github.com/apache/hudi/pull/5545#discussion_r869918605
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -644,14 +644,17 @@ public final Stream<FileSlice> getLatestUnCompactedFileSlices(String partitionSt
@Override
public final Stream<FileSlice> getLatestFileSlicesBeforeOrOn(String partitionStr, String maxCommitTime,
- boolean includeFileSlicesInPendingCompaction) {
+ boolean includeFileSlicesInPendingCompaction, boolean includeFilesInPendingCompaction) {
try {
readLock.lock();
String partitionPath = formatPartitionKey(partitionStr);
ensurePartitionLoadedCorrectly(partitionPath);
Stream<FileSlice> fileSliceStream = fetchLatestFileSlicesBeforeOrOn(partitionPath, maxCommitTime)
.filter(slice -> !isFileGroupReplacedBeforeOrOn(slice.getFileGroupId(), maxCommitTime));
if (includeFileSlicesInPendingCompaction) {
+ if (includeFilesInPendingCompaction) {
+ return fileSliceStream.map(this::addBootstrapBaseFileIfPresent);
Review Comment:
However, the some data are not obtained from UT
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on a diff in pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on code in PR #5545:
URL: https://github.com/apache/hudi/pull/5545#discussion_r869918605
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -644,14 +644,17 @@ public final Stream<FileSlice> getLatestUnCompactedFileSlices(String partitionSt
@Override
public final Stream<FileSlice> getLatestFileSlicesBeforeOrOn(String partitionStr, String maxCommitTime,
- boolean includeFileSlicesInPendingCompaction) {
+ boolean includeFileSlicesInPendingCompaction, boolean includeFilesInPendingCompaction) {
try {
readLock.lock();
String partitionPath = formatPartitionKey(partitionStr);
ensurePartitionLoadedCorrectly(partitionPath);
Stream<FileSlice> fileSliceStream = fetchLatestFileSlicesBeforeOrOn(partitionPath, maxCommitTime)
.filter(slice -> !isFileGroupReplacedBeforeOrOn(slice.getFileGroupId(), maxCommitTime));
if (includeFileSlicesInPendingCompaction) {
+ if (includeFilesInPendingCompaction) {
+ return fileSliceStream.map(this::addBootstrapBaseFileIfPresent);
Review Comment:
However, the log file is not obtained. and
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1123580545
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560)
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* 9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5545:
URL: https://github.com/apache/hudi/pull/5545#discussion_r869824080
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -644,14 +644,17 @@ public final Stream<FileSlice> getLatestUnCompactedFileSlices(String partitionSt
@Override
public final Stream<FileSlice> getLatestFileSlicesBeforeOrOn(String partitionStr, String maxCommitTime,
- boolean includeFileSlicesInPendingCompaction) {
+ boolean includeFileSlicesInPendingCompaction, boolean includeFilesInPendingCompaction) {
try {
readLock.lock();
String partitionPath = formatPartitionKey(partitionStr);
ensurePartitionLoadedCorrectly(partitionPath);
Stream<FileSlice> fileSliceStream = fetchLatestFileSlicesBeforeOrOn(partitionPath, maxCommitTime)
.filter(slice -> !isFileGroupReplacedBeforeOrOn(slice.getFileGroupId(), maxCommitTime));
if (includeFileSlicesInPendingCompaction) {
+ if (includeFilesInPendingCompaction) {
+ return fileSliceStream.map(this::addBootstrapBaseFileIfPresent);
Review Comment:
You mean the `includeFileSlicesInPendingCompaction` does not work for pending compaction files ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1121990938
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1122374306
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 81dd014157036cf607987a8d989b8412a5f84dce Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546)
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 38b4297a96d673590ec31ae099e989cbf345827a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1125685938
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123824503",
"triggerType" : "MANUAL"
}, {
"hash" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"triggerType" : "PUSH"
}, {
"hash" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588",
"triggerID" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"triggerType" : "PUSH"
}, {
"hash" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8600",
"triggerID" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"triggerType" : "PUSH"
}, {
"hash" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602",
"triggerID" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"triggerType" : "PUSH"
}, {
"hash" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602",
"triggerID" : "1125551328",
"triggerType" : "MANUAL"
}, {
"hash" : "61f5b613e7d59e61474bb7f2be313dca31cf13e3",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8629",
"triggerID" : "61f5b613e7d59e61474bb7f2be313dca31cf13e3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* f08364dbd036051b0cf8cc0e018deaa6d353fe44 UNKNOWN
* 61f5b613e7d59e61474bb7f2be313dca31cf13e3 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8629)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1123644126
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* 9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1121987331
@danny0405 hi now i don't have a hudi 0.10.1 and flink cluster
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1122431093
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 81dd014157036cf607987a8d989b8412a5f84dce Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546)
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 38b4297a96d673590ec31ae099e989cbf345827a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1123572483
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560)
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] cuibo01 commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
cuibo01 commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1125551328
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1121827852
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cd51f10efdbb4209a81b1cf85da03aabc3ab06f1 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1122351869
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cd51f10efdbb4209a81b1cf85da03aabc3ab06f1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537)
* 81dd014157036cf607987a8d989b8412a5f84dce Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1122347509
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cd51f10efdbb4209a81b1cf85da03aabc3ab06f1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537)
* 81dd014157036cf607987a8d989b8412a5f84dce UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1124862113
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123824503",
"triggerType" : "MANUAL"
}, {
"hash" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"triggerType" : "PUSH"
}, {
"hash" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588",
"triggerID" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"triggerType" : "PUSH"
}, {
"hash" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8600",
"triggerID" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"triggerType" : "PUSH"
}, {
"hash" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602",
"triggerID" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* f08364dbd036051b0cf8cc0e018deaa6d353fe44 UNKNOWN
* 995e1b573368430392ecd18eff9d6809eab2cf15 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8600)
* 3f3faa0ac0b8d41bc070b085869dce6472b1a649 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #5545:
URL: https://github.com/apache/hudi/pull/5545#discussion_r871130851
##########
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/TestBoostrapOperator.java:
##########
@@ -0,0 +1,196 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.sink;
+
+import org.apache.hudi.client.common.HoodieFlinkEngineContext;
+import org.apache.hudi.common.model.FileSlice;
+import org.apache.hudi.common.model.HoodieKey;
+import org.apache.hudi.common.model.HoodieRecord;
+import org.apache.hudi.common.model.HoodieTableType;
+import org.apache.hudi.common.table.TableSchemaResolver;
+import org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.util.BaseFileUtils;
+import org.apache.hudi.common.util.ClosableIterator;
+import org.apache.hudi.common.util.Option;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.configuration.FlinkOptions;
+import org.apache.hudi.exception.HoodieException;
+import org.apache.hudi.sink.utils.Pipelines;
+import org.apache.hudi.table.HoodieFlinkTable;
+import org.apache.hudi.table.format.FormatUtils;
+import org.apache.hudi.util.AvroSchemaConverter;
+import org.apache.hudi.util.FlinkClientUtil;
+import org.apache.hudi.util.StreamerUtil;
+import org.apache.hudi.utils.TestConfigurations;
+import org.apache.hudi.utils.source.ContinuousFileSource;
+
+import org.apache.avro.Schema;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.core.execution.JobClient;
+import org.apache.flink.core.fs.Path;
+import org.apache.flink.formats.common.TimestampFormat;
+import org.apache.flink.formats.json.JsonRowDataDeserializationSchema;
+import org.apache.flink.streaming.api.CheckpointingMode;
+import org.apache.flink.streaming.api.datastream.DataStream;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+import org.apache.flink.table.data.RowData;
+import org.apache.flink.table.runtime.typeutils.InternalTypeInfo;
+import org.apache.flink.table.types.logical.RowType;
+import org.apache.flink.util.TestLogger;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+import java.io.File;
+import java.io.FilenameFilter;
+import java.nio.charset.StandardCharsets;
+import java.util.List;
+import java.util.Objects;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static java.util.stream.Collectors.toList;
+import static org.apache.hudi.util.StreamerUtil.isValidFile;
+import static org.junit.jupiter.api.Assertions.assertEquals;
Review Comment:
The test can be removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5545: [HUDI-4078][HUDI-FLINK]BootstrapOperator contains the pending compact…
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5545:
URL: https://github.com/apache/hudi/pull/5545#issuecomment-1125636934
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"triggerType" : "PUSH"
}, {
"hash" : "cd51f10efdbb4209a81b1cf85da03aabc3ab06f1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8537",
"triggerID" : "1121990938",
"triggerType" : "MANUAL"
}, {
"hash" : "81dd014157036cf607987a8d989b8412a5f84dce",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8546",
"triggerID" : "81dd014157036cf607987a8d989b8412a5f84dce",
"triggerType" : "PUSH"
}, {
"hash" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b",
"triggerType" : "PUSH"
}, {
"hash" : "38b4297a96d673590ec31ae099e989cbf345827a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8548",
"triggerID" : "38b4297a96d673590ec31ae099e989cbf345827a",
"triggerType" : "PUSH"
}, {
"hash" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8560",
"triggerID" : "8e9e250a4e6f0b5c92f62f93f6e6e67837359fdb",
"triggerType" : "PUSH"
}, {
"hash" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0075890d5520071b5a4c7c2acea0410ad8f3a9d1",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"triggerType" : "PUSH"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123654939",
"triggerType" : "MANUAL"
}, {
"hash" : "9fe53ab3a4ec9ecda3b8a3e3b83be37a474f5e73",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8573",
"triggerID" : "1123824503",
"triggerType" : "MANUAL"
}, {
"hash" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f08364dbd036051b0cf8cc0e018deaa6d353fe44",
"triggerType" : "PUSH"
}, {
"hash" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8588",
"triggerID" : "ce582122dde22dc64c8528cb229a9eb76ab630d6",
"triggerType" : "PUSH"
}, {
"hash" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8600",
"triggerID" : "995e1b573368430392ecd18eff9d6809eab2cf15",
"triggerType" : "PUSH"
}, {
"hash" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602",
"triggerID" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"triggerType" : "PUSH"
}, {
"hash" : "3f3faa0ac0b8d41bc070b085869dce6472b1a649",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602",
"triggerID" : "1125551328",
"triggerType" : "MANUAL"
}, {
"hash" : "61f5b613e7d59e61474bb7f2be313dca31cf13e3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8629",
"triggerID" : "61f5b613e7d59e61474bb7f2be313dca31cf13e3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9fa78959cc7e37d47fe6756ced4cc6eb21e0c92b UNKNOWN
* 0075890d5520071b5a4c7c2acea0410ad8f3a9d1 UNKNOWN
* f08364dbd036051b0cf8cc0e018deaa6d353fe44 UNKNOWN
* 3f3faa0ac0b8d41bc070b085869dce6472b1a649 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8602)
* 61f5b613e7d59e61474bb7f2be313dca31cf13e3 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8629)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org