You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "jonvex (via GitHub)" <gi...@apache.org> on 2023/04/27 20:28:04 UTC
[GitHub] [hudi] jonvex opened a new pull request, #8589: [6147] Deltastreamer finish failed compaction before ingestion
jonvex opened a new pull request, #8589:
URL: https://github.com/apache/hudi/pull/8589
### Change Logs
Deltastreamer rolls back and does compaction after the deltacommit if it failed previously. Now, if the config `--retry-last-pending-inline-compaction` is set it will do the compaction before the commit.
### Impact
Improvement
### Risk level (write none, low medium or high below)
None
### Documentation Update
N/A
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8589: [HUDI-6147] Deltastreamer finish failed compaction before ingestion
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8589:
URL: https://github.com/apache/hudi/pull/8589#issuecomment-1537341010
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9ee4505209c15331987695061f05d0b8e6d06848",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16712",
"triggerID" : "9ee4505209c15331987695061f05d0b8e6d06848",
"triggerType" : "PUSH"
}, {
"hash" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16783",
"triggerID" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"triggerType" : "PUSH"
}, {
"hash" : "02d5fb851ae5c5936e04c8b760bab57e4ec0ad33",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16836",
"triggerID" : "02d5fb851ae5c5936e04c8b760bab57e4ec0ad33",
"triggerType" : "PUSH"
}, {
"hash" : "ade7b317412042c529febfe491469ef855b376b9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16837",
"triggerID" : "ade7b317412042c529febfe491469ef855b376b9",
"triggerType" : "PUSH"
}, {
"hash" : "d31955ef5b729029897444b0aa5c30c242cfadb9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d31955ef5b729029897444b0aa5c30c242cfadb9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ade7b317412042c529febfe491469ef855b376b9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16837)
* d31955ef5b729029897444b0aa5c30c242cfadb9 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8589: [HUDI-6147] Deltastreamer finish failed compaction before ingestion
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8589:
URL: https://github.com/apache/hudi/pull/8589#issuecomment-1535462312
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9ee4505209c15331987695061f05d0b8e6d06848",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16712",
"triggerID" : "9ee4505209c15331987695061f05d0b8e6d06848",
"triggerType" : "PUSH"
}, {
"hash" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16783",
"triggerID" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"triggerType" : "PUSH"
}, {
"hash" : "02d5fb851ae5c5936e04c8b760bab57e4ec0ad33",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16836",
"triggerID" : "02d5fb851ae5c5936e04c8b760bab57e4ec0ad33",
"triggerType" : "PUSH"
}, {
"hash" : "ade7b317412042c529febfe491469ef855b376b9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16837",
"triggerID" : "ade7b317412042c529febfe491469ef855b376b9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ade7b317412042c529febfe491469ef855b376b9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16837)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8589: [HUDI-6147] Deltastreamer finish failed compaction before ingestion
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8589:
URL: https://github.com/apache/hudi/pull/8589#issuecomment-1535279077
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9ee4505209c15331987695061f05d0b8e6d06848",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16712",
"triggerID" : "9ee4505209c15331987695061f05d0b8e6d06848",
"triggerType" : "PUSH"
}, {
"hash" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16783",
"triggerID" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"triggerType" : "PUSH"
}, {
"hash" : "02d5fb851ae5c5936e04c8b760bab57e4ec0ad33",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16836",
"triggerID" : "02d5fb851ae5c5936e04c8b760bab57e4ec0ad33",
"triggerType" : "PUSH"
}, {
"hash" : "ade7b317412042c529febfe491469ef855b376b9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ade7b317412042c529febfe491469ef855b376b9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 02d5fb851ae5c5936e04c8b760bab57e4ec0ad33 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16836)
* ade7b317412042c529febfe491469ef855b376b9 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8589: [HUDI-6147] Deltastreamer finish failed compaction before ingestion
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8589:
URL: https://github.com/apache/hudi/pull/8589#issuecomment-1537412498
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9ee4505209c15331987695061f05d0b8e6d06848",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16712",
"triggerID" : "9ee4505209c15331987695061f05d0b8e6d06848",
"triggerType" : "PUSH"
}, {
"hash" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16783",
"triggerID" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"triggerType" : "PUSH"
}, {
"hash" : "02d5fb851ae5c5936e04c8b760bab57e4ec0ad33",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16836",
"triggerID" : "02d5fb851ae5c5936e04c8b760bab57e4ec0ad33",
"triggerType" : "PUSH"
}, {
"hash" : "ade7b317412042c529febfe491469ef855b376b9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16837",
"triggerID" : "ade7b317412042c529febfe491469ef855b376b9",
"triggerType" : "PUSH"
}, {
"hash" : "d31955ef5b729029897444b0aa5c30c242cfadb9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16913",
"triggerID" : "d31955ef5b729029897444b0aa5c30c242cfadb9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d31955ef5b729029897444b0aa5c30c242cfadb9 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16913)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan merged pull request #8589: [HUDI-6147] Deltastreamer finish failed compaction before ingestion
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan merged PR #8589:
URL: https://github.com/apache/hudi/pull/8589
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8589: [6147] Deltastreamer finish failed compaction before ingestion
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8589:
URL: https://github.com/apache/hudi/pull/8589#issuecomment-1526758337
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9ee4505209c15331987695061f05d0b8e6d06848",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16712",
"triggerID" : "9ee4505209c15331987695061f05d0b8e6d06848",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9ee4505209c15331987695061f05d0b8e6d06848 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16712)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8589: [HUDI-6147] Deltastreamer finish failed compaction before ingestion
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8589:
URL: https://github.com/apache/hudi/pull/8589#issuecomment-1535198591
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9ee4505209c15331987695061f05d0b8e6d06848",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16712",
"triggerID" : "9ee4505209c15331987695061f05d0b8e6d06848",
"triggerType" : "PUSH"
}, {
"hash" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16783",
"triggerID" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"triggerType" : "PUSH"
}, {
"hash" : "02d5fb851ae5c5936e04c8b760bab57e4ec0ad33",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "02d5fb851ae5c5936e04c8b760bab57e4ec0ad33",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbcf0601680fcc5e298b0d36b4e297147f806043 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16783)
* 02d5fb851ae5c5936e04c8b760bab57e4ec0ad33 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8589: [HUDI-6147] Deltastreamer finish failed compaction before ingestion
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8589:
URL: https://github.com/apache/hudi/pull/8589#issuecomment-1537343026
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9ee4505209c15331987695061f05d0b8e6d06848",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16712",
"triggerID" : "9ee4505209c15331987695061f05d0b8e6d06848",
"triggerType" : "PUSH"
}, {
"hash" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16783",
"triggerID" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"triggerType" : "PUSH"
}, {
"hash" : "02d5fb851ae5c5936e04c8b760bab57e4ec0ad33",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16836",
"triggerID" : "02d5fb851ae5c5936e04c8b760bab57e4ec0ad33",
"triggerType" : "PUSH"
}, {
"hash" : "ade7b317412042c529febfe491469ef855b376b9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16837",
"triggerID" : "ade7b317412042c529febfe491469ef855b376b9",
"triggerType" : "PUSH"
}, {
"hash" : "d31955ef5b729029897444b0aa5c30c242cfadb9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16913",
"triggerID" : "d31955ef5b729029897444b0aa5c30c242cfadb9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ade7b317412042c529febfe491469ef855b376b9 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16837)
* d31955ef5b729029897444b0aa5c30c242cfadb9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16913)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #8589: [HUDI-6147] Deltastreamer finish failed compaction before ingestion
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on PR #8589:
URL: https://github.com/apache/hudi/pull/8589#issuecomment-1537334544
rebased w/ latest master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8589: [6147] Deltastreamer finish failed compaction before ingestion
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8589:
URL: https://github.com/apache/hudi/pull/8589#issuecomment-1526415903
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9ee4505209c15331987695061f05d0b8e6d06848",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16712",
"triggerID" : "9ee4505209c15331987695061f05d0b8e6d06848",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9ee4505209c15331987695061f05d0b8e6d06848 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16712)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8589: [6147] Deltastreamer finish failed compaction before ingestion
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8589:
URL: https://github.com/apache/hudi/pull/8589#issuecomment-1530495687
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9ee4505209c15331987695061f05d0b8e6d06848",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16712",
"triggerID" : "9ee4505209c15331987695061f05d0b8e6d06848",
"triggerType" : "PUSH"
}, {
"hash" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9ee4505209c15331987695061f05d0b8e6d06848 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16712)
* dbcf0601680fcc5e298b0d36b4e297147f806043 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8589: [HUDI-6147] Deltastreamer finish failed compaction before ingestion
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8589:
URL: https://github.com/apache/hudi/pull/8589#issuecomment-1535208050
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9ee4505209c15331987695061f05d0b8e6d06848",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16712",
"triggerID" : "9ee4505209c15331987695061f05d0b8e6d06848",
"triggerType" : "PUSH"
}, {
"hash" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16783",
"triggerID" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"triggerType" : "PUSH"
}, {
"hash" : "02d5fb851ae5c5936e04c8b760bab57e4ec0ad33",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16836",
"triggerID" : "02d5fb851ae5c5936e04c8b760bab57e4ec0ad33",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 02d5fb851ae5c5936e04c8b760bab57e4ec0ad33 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16836)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8589: [6147] Deltastreamer finish failed compaction before ingestion
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8589:
URL: https://github.com/apache/hudi/pull/8589#issuecomment-1530733293
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9ee4505209c15331987695061f05d0b8e6d06848",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16712",
"triggerID" : "9ee4505209c15331987695061f05d0b8e6d06848",
"triggerType" : "PUSH"
}, {
"hash" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16783",
"triggerID" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbcf0601680fcc5e298b0d36b4e297147f806043 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16783)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8589: [6147] Deltastreamer finish failed compaction before ingestion
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8589:
URL: https://github.com/apache/hudi/pull/8589#issuecomment-1530512552
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9ee4505209c15331987695061f05d0b8e6d06848",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16712",
"triggerID" : "9ee4505209c15331987695061f05d0b8e6d06848",
"triggerType" : "PUSH"
}, {
"hash" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16783",
"triggerID" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9ee4505209c15331987695061f05d0b8e6d06848 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16712)
* dbcf0601680fcc5e298b0d36b4e297147f806043 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16783)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a diff in pull request #8589: [6147] Deltastreamer finish failed compaction before ingestion
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #8589:
URL: https://github.com/apache/hudi/pull/8589#discussion_r1181807316
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java:
##########
@@ -421,8 +422,17 @@ public Pair<Option<String>, JavaRDD<WriteStatus>> syncOnce() throws IOException
}
}
- // complete the pending clustering before writing to sink
- if (cfg.retryLastPendingInlineClusteringJob && getHoodieClientConfig(this.schemaProvider).inlineClusteringEnabled()) {
+ // complete the pending compaction before writing to sink
+ if (cfg.retryLastPendingInlineCompactionJob && getHoodieClientConfig(this.schemaProvider).inlineCompactionEnabled()) {
+ Option<String> pendingCompactionInstant = getLastPendingCompactionInstant(allCommitsTimelineOpt);
+ if (pendingCompactionInstant.isPresent()) {
+ HoodieWriteMetadata<JavaRDD<WriteStatus>> writeMetadata = writeClient.compact(pendingCompactionInstant.get());
+ writeClient.commitCompaction(pendingCompactionInstant.get(), writeMetadata.getCommitMetadata().get(), Option.empty());
+ //refreshTimeline();
Review Comment:
is refresh timeline not required? Its better to refresh the timeline.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8589: [HUDI-6147] Deltastreamer finish failed compaction before ingestion
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8589:
URL: https://github.com/apache/hudi/pull/8589#issuecomment-1535337647
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9ee4505209c15331987695061f05d0b8e6d06848",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16712",
"triggerID" : "9ee4505209c15331987695061f05d0b8e6d06848",
"triggerType" : "PUSH"
}, {
"hash" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16783",
"triggerID" : "dbcf0601680fcc5e298b0d36b4e297147f806043",
"triggerType" : "PUSH"
}, {
"hash" : "02d5fb851ae5c5936e04c8b760bab57e4ec0ad33",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16836",
"triggerID" : "02d5fb851ae5c5936e04c8b760bab57e4ec0ad33",
"triggerType" : "PUSH"
}, {
"hash" : "ade7b317412042c529febfe491469ef855b376b9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16837",
"triggerID" : "ade7b317412042c529febfe491469ef855b376b9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 02d5fb851ae5c5936e04c8b760bab57e4ec0ad33 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16836)
* ade7b317412042c529febfe491469ef855b376b9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16837)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8589: [6147] Deltastreamer finish failed compaction before ingestion
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8589:
URL: https://github.com/apache/hudi/pull/8589#issuecomment-1526399416
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9ee4505209c15331987695061f05d0b8e6d06848",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9ee4505209c15331987695061f05d0b8e6d06848",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9ee4505209c15331987695061f05d0b8e6d06848 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org