You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/04/18 08:26:48 UTC
[GitHub] [hudi] zhangyue19921010 opened a new pull request, #5344: [HUDI-3879]Suppress exceptions that are not fatal in HoodieMetadataTableValidator
zhangyue19921010 opened a new pull request, #5344:
URL: https://github.com/apache/hudi/pull/5344
https://issues.apache.org/jira/browse/HUDI-3879
Several improvements for HoodieMetadataTableValidator.
## What is the purpose of the pull request
*(For example: This pull request adds quick-start document.)*
## Brief change log
*(for example:)*
- *Modify AnnotationLocation checkstyle rule in checkstyle.xml*
## Verify this pull request
*(Please pick either of the following options)*
This pull request is a trivial rework / code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end.*
- *Added HoodieClientWriteTest to verify the change.*
- *Manually verified the change by running a job locally.*
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5344: [HUDI-3879] Suppress exceptions that are not fatal in HoodieMetadataTableValidator
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5344:
URL: https://github.com/apache/hudi/pull/5344#issuecomment-1244947832
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8098",
"triggerID" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"triggerType" : "PUSH"
}, {
"hash" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8102",
"triggerID" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"triggerType" : "PUSH"
}, {
"hash" : "6da5a939cc985fc21be40168503c43a98d39e566",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8124",
"triggerID" : "6da5a939cc985fc21be40168503c43a98d39e566",
"triggerType" : "PUSH"
}, {
"hash" : "e4facd5876ed2c8b6918d29bf07410e97966a9f9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11318",
"triggerID" : "e4facd5876ed2c8b6918d29bf07410e97966a9f9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6da5a939cc985fc21be40168503c43a98d39e566 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8124)
* e4facd5876ed2c8b6918d29bf07410e97966a9f9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11318)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a diff in pull request #5344: [HUDI-3879]Suppress exceptions that are not fatal in HoodieMetadataTableValidator
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on code in PR #5344:
URL: https://github.com/apache/hudi/pull/5344#discussion_r852372132
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java:
##########
@@ -399,7 +400,9 @@ public void doMetadataTableValidation() {
Set<String> baseFilesForCleaning = Collections.emptySet();
// check metadata table is available to read.
- checkMetadataTableIsAvailable();
+ if (!checkMetadataTableIsAvailable()) {
+ return;
+ }
Review Comment:
even if Metadata table does not have any commits, do we need to add an else block here and check that there are no completed commits in data table?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #5344: [HUDI-3879]Suppress exceptions that are not fatal in HoodieMetadataTableValidator
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #5344:
URL: https://github.com/apache/hudi/pull/5344#discussion_r852603749
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java:
##########
@@ -399,7 +400,9 @@ public void doMetadataTableValidation() {
Set<String> baseFilesForCleaning = Collections.emptySet();
// check metadata table is available to read.
- checkMetadataTableIsAvailable();
+ if (!checkMetadataTableIsAvailable()) {
+ return;
+ }
Review Comment:
Sure, changed. add another else block in checkMetadataTableIsAvailable()
If there is no completed commits neither in data table nor MDT, HoodieMetadataTableValidator will skip current loop and log an info message.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on pull request #5344: [HUDI-3879]Suppress exceptions that are not fatal in HoodieMetadataTableValidator
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on PR #5344:
URL: https://github.com/apache/hudi/pull/5344#issuecomment-1102064611
> when there are no commits in metadata table.
Yeap, after this patch, when there are no commits in metadata table, HoodieMetadataTableValidator will skip current loop and log a warn message.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5344: [HUDI-3879]Suppress exceptions that are not fatal in HoodieMetadataTableValidator
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5344:
URL: https://github.com/apache/hudi/pull/5344#issuecomment-1101238386
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8098",
"triggerID" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"triggerType" : "PUSH"
}, {
"hash" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8098)
* e2200282573a65abbae99f5bc0fb81f903996ffe UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5344: [HUDI-3879]Suppress exceptions that are not fatal in HoodieMetadataTableValidator
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5344:
URL: https://github.com/apache/hudi/pull/5344#issuecomment-1102046824
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8098",
"triggerID" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"triggerType" : "PUSH"
}, {
"hash" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8102",
"triggerID" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"triggerType" : "PUSH"
}, {
"hash" : "6da5a939cc985fc21be40168503c43a98d39e566",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8124",
"triggerID" : "6da5a939cc985fc21be40168503c43a98d39e566",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6da5a939cc985fc21be40168503c43a98d39e566 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8124)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5344: [HUDI-3879]Suppress exceptions that are not fatal in HoodieMetadataTableValidator
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5344:
URL: https://github.com/apache/hudi/pull/5344#issuecomment-1101218095
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8098",
"triggerID" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8098)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5344: [HUDI-3879]Suppress exceptions that are not fatal in HoodieMetadataTableValidator
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5344:
URL: https://github.com/apache/hudi/pull/5344#issuecomment-1101348147
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8098",
"triggerID" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"triggerType" : "PUSH"
}, {
"hash" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8102",
"triggerID" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e2200282573a65abbae99f5bc0fb81f903996ffe Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8102)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5344: [HUDI-3879]Suppress exceptions that are not fatal in HoodieMetadataTableValidator
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5344:
URL: https://github.com/apache/hudi/pull/5344#issuecomment-1101930886
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8098",
"triggerID" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"triggerType" : "PUSH"
}, {
"hash" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8102",
"triggerID" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"triggerType" : "PUSH"
}, {
"hash" : "6da5a939cc985fc21be40168503c43a98d39e566",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8124",
"triggerID" : "6da5a939cc985fc21be40168503c43a98d39e566",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e2200282573a65abbae99f5bc0fb81f903996ffe Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8102)
* 6da5a939cc985fc21be40168503c43a98d39e566 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8124)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5344: [HUDI-3879]Suppress exceptions that are not fatal in HoodieMetadataTableValidator
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5344:
URL: https://github.com/apache/hudi/pull/5344#issuecomment-1101929476
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8098",
"triggerID" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"triggerType" : "PUSH"
}, {
"hash" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8102",
"triggerID" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"triggerType" : "PUSH"
}, {
"hash" : "6da5a939cc985fc21be40168503c43a98d39e566",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6da5a939cc985fc21be40168503c43a98d39e566",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e2200282573a65abbae99f5bc0fb81f903996ffe Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8102)
* 6da5a939cc985fc21be40168503c43a98d39e566 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on a diff in pull request #5344: [HUDI-3879] Suppress exceptions that are not fatal in HoodieMetadataTableValidator
Posted by GitBox <gi...@apache.org>.
yihua commented on code in PR #5344:
URL: https://github.com/apache/hudi/pull/5344#discussion_r969160839
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/HoodieMetadataTableValidator.java:
##########
@@ -465,18 +474,29 @@ public void doMetadataTableValidation() {
* Check metadata is initialized and available to ready.
* If not we will log.warn and skip current validation.
*/
- private void checkMetadataTableIsAvailable() {
+ private boolean checkMetadataTableIsAvailable() {
try {
HoodieTableMetaClient mdtMetaClient = HoodieTableMetaClient.builder()
.setConf(jsc.hadoopConfiguration()).setBasePath(new Path(cfg.basePath, HoodieTableMetaClient.METADATA_TABLE_FOLDER_PATH).toString())
.setLoadActiveTimelineOnLoad(true)
.build();
int finishedInstants = mdtMetaClient.getActiveTimeline().filterCompletedInstants().countInstants();
if (finishedInstants == 0) {
- throw new HoodieValidationException("There is no completed instant for metadata table.");
+ if (metaClient.getActiveTimeline().filterCompletedInstants().countInstants() == 0) {
+ LOG.info("There is no completed instant both in metadata table and corresponding data table.");
+ return false;
+ } else {
+ throw new HoodieValidationException("There is no completed instant for metadata table.");
+ }
}
+ return true;
+ } catch (TableNotFoundException tbe) {
+ // Suppress the TableNotFound exception if Metadata table is not available to read for now
+ LOG.warn("Meta data table is not found. Skip current validation.");
Review Comment:
nit: `Meta data` -> `Metadata`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5344: [HUDI-3879] Suppress exceptions that are not fatal in HoodieMetadataTableValidator
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5344:
URL: https://github.com/apache/hudi/pull/5344#issuecomment-1244944075
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8098",
"triggerID" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"triggerType" : "PUSH"
}, {
"hash" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8102",
"triggerID" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"triggerType" : "PUSH"
}, {
"hash" : "6da5a939cc985fc21be40168503c43a98d39e566",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8124",
"triggerID" : "6da5a939cc985fc21be40168503c43a98d39e566",
"triggerType" : "PUSH"
}, {
"hash" : "e4facd5876ed2c8b6918d29bf07410e97966a9f9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e4facd5876ed2c8b6918d29bf07410e97966a9f9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6da5a939cc985fc21be40168503c43a98d39e566 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8124)
* e4facd5876ed2c8b6918d29bf07410e97966a9f9 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #5344: [HUDI-3879]Suppress exceptions that are not fatal in HoodieMetadataTableValidator
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on PR #5344:
URL: https://github.com/apache/hudi/pull/5344#issuecomment-1101710498
@zhangyue19921010 : One of my error injection test run failed w/ below exception with metadata table validation.
```
Run Number : 0
├── Deltastreamer Job
│ ├── Status: Failed
│ ├── Exceptions
│ └── Killed
│ └── Killing the jvm at /tmp/fail2_mt_write.txt Reason: Fail metadata
│ table writing before commit 00000000000000
└── Validation Job for Latest files
├── Status: Failed
├── Exceptions
│ ├── org.apache.hudi.exception.HoodieValidationException: There is no
│ │ completed instant for metadata table.
│ ├── org.apache.hudi.exception.HoodieException: Unable to do hoodie
│ │ metadata table validation in $BASE_PATH/COWDeltastreamerRun/base_path//hoodie_Table
│ ├── Caused by: org.apache.hudi.exception.HoodieException: Error fetching
│ │ partition paths from metadata table
│ ├── Caused by: org.apache.hudi.exception.HoodieMetadataException: Failed
│ │ to retrieve list of partition from metadata
│ └── Caused by: java.util.NoSuchElementException: No value present in
│ Option
└── Killed: False
Uploading logs to s3 $BASE_PATH/COWDeltastreamerRun/
Completed 1.2 KiB/435.1 KiB (17.4 KiB/s) with 4 file(s) remaining
upload: job_runs/18-04-2022T00-37-10/hoodie_table_report.log to $BASE_PATH/COWDeltastreamerRun/hoodie_table_report.log
```
Do you think, w/ above fix, this failure will go away. This failure happened when there are no commits in metadata table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5344: [HUDI-3879]Suppress exceptions that are not fatal in HoodieMetadataTableValidator
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5344:
URL: https://github.com/apache/hudi/pull/5344#issuecomment-1101215459
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua merged pull request #5344: [HUDI-3879] Suppress exceptions that are not fatal in HoodieMetadataTableValidator
Posted by GitBox <gi...@apache.org>.
yihua merged PR #5344:
URL: https://github.com/apache/hudi/pull/5344
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5344: [HUDI-3879]Suppress exceptions that are not fatal in HoodieMetadataTableValidator
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5344:
URL: https://github.com/apache/hudi/pull/5344#issuecomment-1101240952
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8098",
"triggerID" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"triggerType" : "PUSH"
}, {
"hash" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8098)
* e2200282573a65abbae99f5bc0fb81f903996ffe UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5344: [HUDI-3879]Suppress exceptions that are not fatal in HoodieMetadataTableValidator
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5344:
URL: https://github.com/apache/hudi/pull/5344#issuecomment-1101261803
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8098",
"triggerID" : "ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2",
"triggerType" : "PUSH"
}, {
"hash" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8102",
"triggerID" : "e2200282573a65abbae99f5bc0fb81f903996ffe",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ce61b19c10d4b4ad1f51ccd428d1aa5f8273f8f2 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8098)
* e2200282573a65abbae99f5bc0fb81f903996ffe Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8102)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org