You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "prashantwason (via GitHub)" <gi...@apache.org> on 2023/04/18 10:11:45 UTC
[GitHub] [hudi] prashantwason opened a new pull request, #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
prashantwason opened a new pull request, #8487:
URL: https://github.com/apache/hudi/pull/8487
[HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
### Change Logs
Specify the correct partitionToReplacedFileIds while creating a commit in DeltaStreamer.
### Impact
Fixes the HoodieCommitMetadata written as part of a commit.
### Risk level (write none, low medium or high below)
None
### Documentation Update
None
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1592739409
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422",
"triggerID" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"triggerType" : "PUSH"
}, {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495",
"triggerID" : "1516049764",
"triggerType" : "MANUAL"
}, {
"hash" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17314",
"triggerID" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"triggerType" : "PUSH"
}, {
"hash" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17553",
"triggerID" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"triggerType" : "PUSH"
}, {
"hash" : "95348db09687663867016251a82359f45d5cb6cd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17869",
"triggerID" : "95348db09687663867016251a82359f45d5cb6cd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 95348db09687663867016251a82359f45d5cb6cd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17869)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1592407548
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422",
"triggerID" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"triggerType" : "PUSH"
}, {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495",
"triggerID" : "1516049764",
"triggerType" : "MANUAL"
}, {
"hash" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17314",
"triggerID" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"triggerType" : "PUSH"
}, {
"hash" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17553",
"triggerID" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"triggerType" : "PUSH"
}, {
"hash" : "95348db09687663867016251a82359f45d5cb6cd",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "95348db09687663867016251a82359f45d5cb6cd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 280515ea1c939f0afa7a4cd8a5593e55bd394648 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17553)
* 95348db09687663867016251a82359f45d5cb6cd UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1516049764
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1516987859
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422",
"triggerID" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"triggerType" : "PUSH"
}, {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495",
"triggerID" : "1516049764",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 0db49c70e22e0f50b93390786f6a877295f680ec Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1570322345
hey @prashantwason :
LGTM. Ci is failing. and you may need to rebase w/ latest master. Once both these are done, we are good to go.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1596616259
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422",
"triggerID" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"triggerType" : "PUSH"
}, {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495",
"triggerID" : "1516049764",
"triggerType" : "MANUAL"
}, {
"hash" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17314",
"triggerID" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"triggerType" : "PUSH"
}, {
"hash" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17553",
"triggerID" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"triggerType" : "PUSH"
}, {
"hash" : "95348db09687663867016251a82359f45d5cb6cd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17869",
"triggerID" : "95348db09687663867016251a82359f45d5cb6cd",
"triggerType" : "PUSH"
}, {
"hash" : "a9193e4fdd49a06e869bf9463076ce2ef6e74dbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a9193e4fdd49a06e869bf9463076ce2ef6e74dbf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 95348db09687663867016251a82359f45d5cb6cd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17869)
* a9193e4fdd49a06e869bf9463076ce2ef6e74dbf UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1598010303
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a9193e4fdd49a06e869bf9463076ce2ef6e74dbf",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17968",
"triggerID" : "a9193e4fdd49a06e869bf9463076ce2ef6e74dbf",
"triggerType" : "PUSH"
}, {
"hash" : "a9193e4fdd49a06e869bf9463076ce2ef6e74dbf",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17988",
"triggerID" : "1516049764",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* a9193e4fdd49a06e869bf9463076ce2ef6e74dbf Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17968) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17988)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1560467979
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422",
"triggerID" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"triggerType" : "PUSH"
}, {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495",
"triggerID" : "1516049764",
"triggerType" : "MANUAL"
}, {
"hash" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17314",
"triggerID" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 195a90f292a61ec35698ab1d2472c56316c4a611 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17314)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1539199344
ping me once the feedback is addressed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1597895394
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a9193e4fdd49a06e869bf9463076ce2ef6e74dbf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a9193e4fdd49a06e869bf9463076ce2ef6e74dbf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a9193e4fdd49a06e869bf9463076ce2ef6e74dbf UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] prashantwason commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "prashantwason (via GitHub)" <gi...@apache.org>.
prashantwason commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1515805437
Says all checks have passed. Probably those tests are flaky and not due to this commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1571614677
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422",
"triggerID" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"triggerType" : "PUSH"
}, {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495",
"triggerID" : "1516049764",
"triggerType" : "MANUAL"
}, {
"hash" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17314",
"triggerID" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"triggerType" : "PUSH"
}, {
"hash" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 195a90f292a61ec35698ab1d2472c56316c4a611 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17314)
* 280515ea1c939f0afa7a4cd8a5593e55bd394648 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1571625646
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422",
"triggerID" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"triggerType" : "PUSH"
}, {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495",
"triggerID" : "1516049764",
"triggerType" : "MANUAL"
}, {
"hash" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17314",
"triggerID" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"triggerType" : "PUSH"
}, {
"hash" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17553",
"triggerID" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 195a90f292a61ec35698ab1d2472c56316c4a611 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17314)
* 280515ea1c939f0afa7a4cd8a5593e55bd394648 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17553)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1592413897
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422",
"triggerID" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"triggerType" : "PUSH"
}, {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495",
"triggerID" : "1516049764",
"triggerType" : "MANUAL"
}, {
"hash" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17314",
"triggerID" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"triggerType" : "PUSH"
}, {
"hash" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17553",
"triggerID" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"triggerType" : "PUSH"
}, {
"hash" : "95348db09687663867016251a82359f45d5cb6cd",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17869",
"triggerID" : "95348db09687663867016251a82359f45d5cb6cd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 280515ea1c939f0afa7a4cd8a5593e55bd394648 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17553)
* 95348db09687663867016251a82359f45d5cb6cd Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17869)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1517164253
> Says all checks have passed. Probably those tests are flaky and not due to this commit.
These tests are failing continuously, you may need to check them again:
```java
[ERROR] Failures:
[ERROR] TestHoodieDeltaStreamer.testInsertOverwriteTable:2450->testDeltaStreamerWithSpecifiedOperation:2519 expected: <2> but was: <1>
[ERROR] TestHoodieDeltaStreamer.testInsertOverwriteTable:2450->testDeltaStreamerWithSpecifiedOperation:2501 expected: <1000> but was: <2000>
[ERROR] TestHoodieDeltaStreamer.testInsertOverwrite:2444->testDeltaStreamerWithSpecifiedOperation:2525 expected: <1950> but was: <950>
[ERROR] TestHoodieDeltaStreamer.testInsertOverwrite:2444->testDeltaStreamerWithSpecifiedOperation:2501 expected: <1000> but was: <1950>
[ERROR] Errors:
[ERROR] TestHoodieDeltaStreamer.testHoodieIndexer:1201->deltaStreamerTestRunner:974->deltaStreamerTestRunner:929->deltaStreamerTestRunner:946 ยป Execution
[INFO]
[ERROR] Tests run: 342, Failures: 4, Errors: 1, Skipped: 7
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1560072002
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422",
"triggerID" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"triggerType" : "PUSH"
}, {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495",
"triggerID" : "1516049764",
"triggerType" : "MANUAL"
}, {
"hash" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0db49c70e22e0f50b93390786f6a877295f680ec Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495)
* 195a90f292a61ec35698ab1d2472c56316c4a611 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1596629264
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422",
"triggerID" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"triggerType" : "PUSH"
}, {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495",
"triggerID" : "1516049764",
"triggerType" : "MANUAL"
}, {
"hash" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17314",
"triggerID" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"triggerType" : "PUSH"
}, {
"hash" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17553",
"triggerID" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"triggerType" : "PUSH"
}, {
"hash" : "95348db09687663867016251a82359f45d5cb6cd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17869",
"triggerID" : "95348db09687663867016251a82359f45d5cb6cd",
"triggerType" : "PUSH"
}, {
"hash" : "a9193e4fdd49a06e869bf9463076ce2ef6e74dbf",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17968",
"triggerID" : "a9193e4fdd49a06e869bf9463076ce2ef6e74dbf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 95348db09687663867016251a82359f45d5cb6cd Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17869)
* a9193e4fdd49a06e869bf9463076ce2ef6e74dbf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17968)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope merged pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope merged PR #8487:
URL: https://github.com/apache/hudi/pull/8487
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] prashantwason commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "prashantwason (via GitHub)" <gi...@apache.org>.
prashantwason commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1572447665
@nsivabalan I fixed the conflict and all tests are passing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1577976567
<img width="1204" alt="image" src="https://github.com/apache/hudi/assets/513218/0ebe4060-b8c3-4772-9d6e-28e1b39fd156">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1513392553
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422",
"triggerID" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0db49c70e22e0f50b93390786f6a877295f680ec Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1516055222
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422",
"triggerID" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"triggerType" : "PUSH"
}, {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495",
"triggerID" : "1516049764",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 0db49c70e22e0f50b93390786f6a877295f680ec Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1514178982
Hi, @prashantwason , can you check the test failures: https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=16422&view=logs&j=dcedfe73-9485-5cc5-817a-73b61fc5dcb0&t=746585d8-b50a-55c3-26c5-517d93af9934&l=33079
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1597901091
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a9193e4fdd49a06e869bf9463076ce2ef6e74dbf",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17968",
"triggerID" : "a9193e4fdd49a06e869bf9463076ce2ef6e74dbf",
"triggerType" : "PUSH"
}, {
"hash" : "a9193e4fdd49a06e869bf9463076ce2ef6e74dbf",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17988",
"triggerID" : "1516049764",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* a9193e4fdd49a06e869bf9463076ce2ef6e74dbf Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17968) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17988)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1571996647
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422",
"triggerID" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"triggerType" : "PUSH"
}, {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495",
"triggerID" : "1516049764",
"triggerType" : "MANUAL"
}, {
"hash" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17314",
"triggerID" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"triggerType" : "PUSH"
}, {
"hash" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17553",
"triggerID" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 280515ea1c939f0afa7a4cd8a5593e55bd394648 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17553)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a diff in pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #8487:
URL: https://github.com/apache/hudi/pull/8487#discussion_r1185531564
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/deltastreamer/TestHoodieDeltaStreamer.java:
##########
@@ -2506,13 +2507,21 @@ void testDeltaStreamerWithSpecifiedOperation(final String tableBasePath, WriteOp
// No new data => no commits.
cfg.sourceLimit = 0;
new HoodieDeltaStreamer(cfg, jsc).sync();
- TestHelpers.assertRecordCount(1000, tableBasePath, sqlContext);
- TestHelpers.assertDistanceCount(1000, tableBasePath, sqlContext);
- TestHelpers.assertCommitMetadata("00000", tableBasePath, fs, 1);
+
+ if (operationType == WriteOperationType.INSERT_OVERWRITE) {
+ TestHelpers.assertRecordCount(1000, tableBasePath, sqlContext);
+ TestHelpers.assertDistanceCount(1000, tableBasePath, sqlContext);
+ TestHelpers.assertCommitMetadata("00000", tableBasePath, fs, 1);
+ } else if (operationType == WriteOperationType.INSERT_OVERWRITE_TABLE) {
+ HoodieTableMetaClient metaClient = HoodieTableMetaClient.builder().setConf(jsc.hadoopConfiguration()).setBasePath(tableBasePath).build();
+ final HoodieTableFileSystemView fsView = new HoodieTableFileSystemView(metaClient, metaClient.getCommitsAndCompactionTimeline());
Review Comment:
can we fetch the partitionToReplaceFileId and validate those are not part of latest FS view calls.
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/deltastreamer/TestHoodieDeltaStreamer.java:
##########
@@ -2506,13 +2507,21 @@ void testDeltaStreamerWithSpecifiedOperation(final String tableBasePath, WriteOp
// No new data => no commits.
cfg.sourceLimit = 0;
new HoodieDeltaStreamer(cfg, jsc).sync();
- TestHelpers.assertRecordCount(1000, tableBasePath, sqlContext);
- TestHelpers.assertDistanceCount(1000, tableBasePath, sqlContext);
- TestHelpers.assertCommitMetadata("00000", tableBasePath, fs, 1);
+
+ if (operationType == WriteOperationType.INSERT_OVERWRITE) {
+ TestHelpers.assertRecordCount(1000, tableBasePath, sqlContext);
+ TestHelpers.assertDistanceCount(1000, tableBasePath, sqlContext);
+ TestHelpers.assertCommitMetadata("00000", tableBasePath, fs, 1);
+ } else if (operationType == WriteOperationType.INSERT_OVERWRITE_TABLE) {
+ HoodieTableMetaClient metaClient = HoodieTableMetaClient.builder().setConf(jsc.hadoopConfiguration()).setBasePath(tableBasePath).build();
+ final HoodieTableFileSystemView fsView = new HoodieTableFileSystemView(metaClient, metaClient.getCommitsAndCompactionTimeline());
+ assertEquals(0, fsView.getLatestFileSlices("").count());
+ TestHelpers.assertCommitMetadata("00000", tableBasePath, fs, 2);
+ }
Review Comment:
lets see if we have any tests for DeletePartition and add assertion there.
testDeletePartitions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1512875671
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422",
"triggerID" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0db49c70e22e0f50b93390786f6a877295f680ec Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1512865163
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0db49c70e22e0f50b93390786f6a877295f680ec UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on a diff in pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #8487:
URL: https://github.com/apache/hudi/pull/8487#discussion_r1230534707
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/deltastreamer/TestHoodieDeltaStreamer.java:
##########
@@ -2519,19 +2529,35 @@ void testDeltaStreamerWithSpecifiedOperation(final String tableBasePath, WriteOp
TestHelpers.assertDistanceCount(1000, tableBasePath, sqlContext);
TestHelpers.assertCommitMetadata("00000", tableBasePath, fs, 1);
+ // Collect the fileIds before running HoodieDeltaStreamer
+ Set<String> beforeFileIDs = getAllFileIDsInTable(tableBasePath, Option.empty());
+
// setting the operationType
cfg.operation = operationType;
// No new data => no commits.
cfg.sourceLimit = 0;
new HoodieDeltaStreamer(cfg, jsc).sync();
- TestHelpers.assertRecordCount(1000, tableBasePath, sqlContext);
- TestHelpers.assertDistanceCount(1000, tableBasePath, sqlContext);
- TestHelpers.assertCommitMetadata("00000", tableBasePath, fs, 1);
+
+ if (operationType == WriteOperationType.INSERT_OVERWRITE) {
+ TestHelpers.assertRecordCount(1000, tableBasePath, sqlContext);
+ TestHelpers.assertDistanceCount(1000, tableBasePath, sqlContext);
+ TestHelpers.assertCommitMetadata("00000", tableBasePath, fs, 1);
+ } else if (operationType == WriteOperationType.INSERT_OVERWRITE_TABLE) {
+ HoodieTableMetaClient metaClient = HoodieTableMetaClient.builder().setConf(jsc.hadoopConfiguration()).setBasePath(tableBasePath).build();
+ final HoodieTableFileSystemView fsView = new HoodieTableFileSystemView(metaClient, metaClient.getCommitsAndCompactionTimeline());
+ assertEquals(0, fsView.getLatestFileSlices("").count());
+ TestHelpers.assertCommitMetadata("00000", tableBasePath, fs, 2);
Review Comment:
This is still failing. There is just one commit in the data timeline after insert_overwrite_table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] prashantwason commented on a diff in pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "prashantwason (via GitHub)" <gi...@apache.org>.
prashantwason commented on code in PR #8487:
URL: https://github.com/apache/hudi/pull/8487#discussion_r1202930477
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/deltastreamer/TestHoodieDeltaStreamer.java:
##########
@@ -2506,13 +2507,21 @@ void testDeltaStreamerWithSpecifiedOperation(final String tableBasePath, WriteOp
// No new data => no commits.
cfg.sourceLimit = 0;
new HoodieDeltaStreamer(cfg, jsc).sync();
- TestHelpers.assertRecordCount(1000, tableBasePath, sqlContext);
- TestHelpers.assertDistanceCount(1000, tableBasePath, sqlContext);
- TestHelpers.assertCommitMetadata("00000", tableBasePath, fs, 1);
+
+ if (operationType == WriteOperationType.INSERT_OVERWRITE) {
+ TestHelpers.assertRecordCount(1000, tableBasePath, sqlContext);
+ TestHelpers.assertDistanceCount(1000, tableBasePath, sqlContext);
+ TestHelpers.assertCommitMetadata("00000", tableBasePath, fs, 1);
+ } else if (operationType == WriteOperationType.INSERT_OVERWRITE_TABLE) {
+ HoodieTableMetaClient metaClient = HoodieTableMetaClient.builder().setConf(jsc.hadoopConfiguration()).setBasePath(tableBasePath).build();
+ final HoodieTableFileSystemView fsView = new HoodieTableFileSystemView(metaClient, metaClient.getCommitsAndCompactionTimeline());
Review Comment:
Done
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/deltastreamer/TestHoodieDeltaStreamer.java:
##########
@@ -2506,13 +2507,21 @@ void testDeltaStreamerWithSpecifiedOperation(final String tableBasePath, WriteOp
// No new data => no commits.
cfg.sourceLimit = 0;
new HoodieDeltaStreamer(cfg, jsc).sync();
- TestHelpers.assertRecordCount(1000, tableBasePath, sqlContext);
- TestHelpers.assertDistanceCount(1000, tableBasePath, sqlContext);
- TestHelpers.assertCommitMetadata("00000", tableBasePath, fs, 1);
+
+ if (operationType == WriteOperationType.INSERT_OVERWRITE) {
+ TestHelpers.assertRecordCount(1000, tableBasePath, sqlContext);
+ TestHelpers.assertDistanceCount(1000, tableBasePath, sqlContext);
+ TestHelpers.assertCommitMetadata("00000", tableBasePath, fs, 1);
+ } else if (operationType == WriteOperationType.INSERT_OVERWRITE_TABLE) {
+ HoodieTableMetaClient metaClient = HoodieTableMetaClient.builder().setConf(jsc.hadoopConfiguration()).setBasePath(tableBasePath).build();
+ final HoodieTableFileSystemView fsView = new HoodieTableFileSystemView(metaClient, metaClient.getCommitsAndCompactionTimeline());
+ assertEquals(0, fsView.getLatestFileSlices("").count());
+ TestHelpers.assertCommitMetadata("00000", tableBasePath, fs, 2);
+ }
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] prashantwason commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "prashantwason (via GitHub)" <gi...@apache.org>.
prashantwason commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1560048780
@nsivabalan Addressed all your feedback. PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1560083261
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422",
"triggerID" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"triggerType" : "PUSH"
}, {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495",
"triggerID" : "1516049764",
"triggerType" : "MANUAL"
}, {
"hash" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17314",
"triggerID" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0db49c70e22e0f50b93390786f6a877295f680ec Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495)
* 195a90f292a61ec35698ab1d2472c56316c4a611 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17314)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8487: [HUDI-6093] Use the correct partitionToReplacedFileIds during commit.
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8487:
URL: https://github.com/apache/hudi/pull/8487#issuecomment-1597171814
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16422",
"triggerID" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"triggerType" : "PUSH"
}, {
"hash" : "0db49c70e22e0f50b93390786f6a877295f680ec",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16495",
"triggerID" : "1516049764",
"triggerType" : "MANUAL"
}, {
"hash" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17314",
"triggerID" : "195a90f292a61ec35698ab1d2472c56316c4a611",
"triggerType" : "PUSH"
}, {
"hash" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17553",
"triggerID" : "280515ea1c939f0afa7a4cd8a5593e55bd394648",
"triggerType" : "PUSH"
}, {
"hash" : "95348db09687663867016251a82359f45d5cb6cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17869",
"triggerID" : "95348db09687663867016251a82359f45d5cb6cd",
"triggerType" : "PUSH"
}, {
"hash" : "a9193e4fdd49a06e869bf9463076ce2ef6e74dbf",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17968",
"triggerID" : "a9193e4fdd49a06e869bf9463076ce2ef6e74dbf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a9193e4fdd49a06e869bf9463076ce2ef6e74dbf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=17968)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org