You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/10/23 15:45:01 UTC
[GitHub] [hudi] hussein-awala opened a new pull request, #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
hussein-awala opened a new pull request, #7041:
URL: https://github.com/apache/hudi/pull/7041
### Change Logs
When the clean planner lists the files in the partitions and it doesn't find any file to delete, the clean operation is skipped without any commit, then in the next clean, if the incremental cleaning mode is enabled, the clean planner doesn't find any information about the checked commits, and it will recheck all the files a second time. This PR creates a clean commit contains the `earliestCommitToRetain` regardless the deleted files list, in this case the clean planner will check only the partitions that have been changed since the `earliestCommitToRetain` in the last clean commit.
### Impact
A new clean commit will be added to the timeline even if there was not a real clean operation. For the benefits, a big performance improvement (and cost reduction of S3 listing) in cleaning operation for table where old partitions are seldom changed.
### Risk level (write none, low medium or high below)
low:
The risk level is low because these changes affects only the clean plans without files to delete, and I kept the checks on the empty commit files to avoid Avro empty file exception, and I improved the method which clean this empty files. If for some reason we have an empty Avro file, a brute force will be performed to prepare the clean plan.
I will test these changes on our project within the week to make sure everything is fine
### Documentation Update
_Describe any necessary documentation update if there is any new feature, config, or user-facing change_
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Change Logs and Impact were stated clearly
- [x] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1299516581
hey @hussein-awala : thanks for the patch. we can definitely take up this patch. But would prefer to guard it using a new flag. reason is, for those who are running clean after every commit, it could keep producing empty clean commit files in the timeline which could impact the query latency for large scale datasets. So, we can let interested folks enable it if need be.
let me know wdyt.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hussein-awala commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hussein-awala commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1291019081
I tested the PR in our project, it works fine as expected. For each clean we have the 3 states requested, inflight and completed, and the clean planner checks only the partitions that have been modified since `earliestCommitToRetain`.
Recently, we incremented `CLEAN_MAX_COMMITS` to 24 as @nsivabalan [proposed](https://github.com/apache/hudi/issues/6953#issuecomment-1283143573) in order to clean the tables every 24 hours (we have a commit per hour) and avoid listing S3 partitions in the tables with with infrequently changed partitions, but the config doesn't work as expected, because after 24 commits, if the list of files to delete is empty, the cleaner will be executed at each next commit until delete something, because for the clean planner, the last clean was when the were some files to delete, and all the next clean operations are not considered because they write nothing to the timeline.
In brief, we need this patch ASAP, can you please add it to 0.13.0?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hussein-awala commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hussein-awala commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1340978945
@codope yes I will rebase it by tomorrow
> Is this really necessary? Also, can you please rebase?
yes, if we use the incremental cleaning, we need to store `earliestCommitToRetain` and use it in the next clean operation to avoid rechecking the partitions we already checked when there is no files to delete.
Also as @nsivabalan proposed, I created a new config `hoodie.cleaner.allow.empty.commits` which is false by default, if it's false, the clean commit will be added only if there is some deleted files. And if it's true, I apply the proposed logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1341811187
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
}, {
"hash" : "a78cf3612346492a082bf211940c46c9e82752a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650",
"triggerID" : "a78cf3612346492a082bf211940c46c9e82752a1",
"triggerType" : "PUSH"
}, {
"hash" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12812",
"triggerID" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"triggerType" : "PUSH"
}, {
"hash" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12813",
"triggerID" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"triggerType" : "PUSH"
}, {
"hash" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12814",
"triggerID" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"triggerType" : "PUSH"
}, {
"hash" : "40d125a000fad7439d6c21fa3e20f7128b151dad",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13520",
"triggerID" : "40d125a000fad7439d6c21fa3e20f7128b151dad",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 40d125a000fad7439d6c21fa3e20f7128b151dad Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13520)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1304850461
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
}, {
"hash" : "a78cf3612346492a082bf211940c46c9e82752a1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650",
"triggerID" : "a78cf3612346492a082bf211940c46c9e82752a1",
"triggerType" : "PUSH"
}, {
"hash" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a78cf3612346492a082bf211940c46c9e82752a1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650)
* d18514b58b5fa732f55bd3ad6212484b7766f50e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1304866940
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
}, {
"hash" : "a78cf3612346492a082bf211940c46c9e82752a1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650",
"triggerID" : "a78cf3612346492a082bf211940c46c9e82752a1",
"triggerType" : "PUSH"
}, {
"hash" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12812",
"triggerID" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"triggerType" : "PUSH"
}, {
"hash" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a78cf3612346492a082bf211940c46c9e82752a1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650)
* d18514b58b5fa732f55bd3ad6212484b7766f50e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12812)
* 1177f60208a2d97c0fce5b8c9aca309a0494cd18 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1294717454
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* acd416d779132b9fd7a7b1fe58eaaeebcf1b821f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498)
* 00f351db8eab5af32063e4a41c4e451ca9838ee4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1295283476
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
}, {
"hash" : "a78cf3612346492a082bf211940c46c9e82752a1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650",
"triggerID" : "a78cf3612346492a082bf211940c46c9e82752a1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a78cf3612346492a082bf211940c46c9e82752a1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hussein-awala commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hussein-awala commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1301440409
Perfect! I was going to suggest that because this is useless when incremental cleaning mode is not activated. I will add a new config for the empty clean commits, and I will duplicate the tests I already fixed (the old version without empty clean commit and the new one with empty clean commit enabled).
> for those who are running clean after every commit, it could keep producing empty clean commit files in the timeline which could impact the query latency for large scale datasets.
If they are running clean after every commit with incremental cleaning, it's better to add an empty clean commit to check only the changed partitions since the last commit instead of checked all the changed partitions since the last no empty clean. Based on our tests, I confirm that this improve the query latency and not the opposite. But I will create a new separate config for this patch and not activating it when incremental cleaning is enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1288185718
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3300f7bdbf9d1cb178390d36523db2ec0279448c Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496)
* acd416d779132b9fd7a7b1fe58eaaeebcf1b821f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1304867810
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
}, {
"hash" : "a78cf3612346492a082bf211940c46c9e82752a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650",
"triggerID" : "a78cf3612346492a082bf211940c46c9e82752a1",
"triggerType" : "PUSH"
}, {
"hash" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12812",
"triggerID" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"triggerType" : "PUSH"
}, {
"hash" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12813",
"triggerID" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d18514b58b5fa732f55bd3ad6212484b7766f50e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12812)
* 1177f60208a2d97c0fce5b8c9aca309a0494cd18 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12813)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1871330623
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
}, {
"hash" : "a78cf3612346492a082bf211940c46c9e82752a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650",
"triggerID" : "a78cf3612346492a082bf211940c46c9e82752a1",
"triggerType" : "PUSH"
}, {
"hash" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12812",
"triggerID" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"triggerType" : "PUSH"
}, {
"hash" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12813",
"triggerID" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"triggerType" : "PUSH"
}, {
"hash" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12814",
"triggerID" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"triggerType" : "PUSH"
}, {
"hash" : "40d125a000fad7439d6c21fa3e20f7128b151dad",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13520",
"triggerID" : "40d125a000fad7439d6c21fa3e20f7128b151dad",
"triggerType" : "PUSH"
}, {
"hash" : "6ab2a114214e2d961abb8f80f535dc02cbae548e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21746",
"triggerID" : "6ab2a114214e2d961abb8f80f535dc02cbae548e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6ab2a114214e2d961abb8f80f535dc02cbae548e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21746)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1288228417
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* acd416d779132b9fd7a7b1fe58eaaeebcf1b821f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1301502928
sg
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1304882995
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
}, {
"hash" : "a78cf3612346492a082bf211940c46c9e82752a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650",
"triggerID" : "a78cf3612346492a082bf211940c46c9e82752a1",
"triggerType" : "PUSH"
}, {
"hash" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12812",
"triggerID" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"triggerType" : "PUSH"
}, {
"hash" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12813",
"triggerID" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"triggerType" : "PUSH"
}, {
"hash" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12814",
"triggerID" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1177f60208a2d97c0fce5b8c9aca309a0494cd18 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12813)
* 36e1e84c05112409ae4d1d4e8b2f13eadc69237b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12814)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1294872589
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 00f351db8eab5af32063e4a41c4e451ca9838ee4 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641)
* 74e7a278867b56d1b6a6bbee29fb477469246487 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1304881559
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
}, {
"hash" : "a78cf3612346492a082bf211940c46c9e82752a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650",
"triggerID" : "a78cf3612346492a082bf211940c46c9e82752a1",
"triggerType" : "PUSH"
}, {
"hash" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12812",
"triggerID" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"triggerType" : "PUSH"
}, {
"hash" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12813",
"triggerID" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"triggerType" : "PUSH"
}, {
"hash" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d18514b58b5fa732f55bd3ad6212484b7766f50e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12812)
* 1177f60208a2d97c0fce5b8c9aca309a0494cd18 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12813)
* 36e1e84c05112409ae4d1d4e8b2f13eadc69237b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1341247642
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
}, {
"hash" : "a78cf3612346492a082bf211940c46c9e82752a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650",
"triggerID" : "a78cf3612346492a082bf211940c46c9e82752a1",
"triggerType" : "PUSH"
}, {
"hash" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12812",
"triggerID" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"triggerType" : "PUSH"
}, {
"hash" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12813",
"triggerID" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"triggerType" : "PUSH"
}, {
"hash" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12814",
"triggerID" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"triggerType" : "PUSH"
}, {
"hash" : "40d125a000fad7439d6c21fa3e20f7128b151dad",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13520",
"triggerID" : "40d125a000fad7439d6c21fa3e20f7128b151dad",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 36e1e84c05112409ae4d1d4e8b2f13eadc69237b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12814)
* 40d125a000fad7439d6c21fa3e20f7128b151dad Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13520)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1294711267
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* acd416d779132b9fd7a7b1fe58eaaeebcf1b821f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498)
* 00f351db8eab5af32063e4a41c4e451ca9838ee4 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1294816115
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* acd416d779132b9fd7a7b1fe58eaaeebcf1b821f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498)
* 00f351db8eab5af32063e4a41c4e451ca9838ee4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641)
* 74e7a278867b56d1b6a6bbee29fb477469246487 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hussein-awala commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hussein-awala commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1341451515
@codope I rebased it
Is there any chance to have this feature in 0.12.2? We've already been using it in production for 1 month without any problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1341238433
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
}, {
"hash" : "a78cf3612346492a082bf211940c46c9e82752a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650",
"triggerID" : "a78cf3612346492a082bf211940c46c9e82752a1",
"triggerType" : "PUSH"
}, {
"hash" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12812",
"triggerID" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"triggerType" : "PUSH"
}, {
"hash" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12813",
"triggerID" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"triggerType" : "PUSH"
}, {
"hash" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12814",
"triggerID" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"triggerType" : "PUSH"
}, {
"hash" : "40d125a000fad7439d6c21fa3e20f7128b151dad",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "40d125a000fad7439d6c21fa3e20f7128b151dad",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 36e1e84c05112409ae4d1d4e8b2f13eadc69237b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12814)
* 40d125a000fad7439d6c21fa3e20f7128b151dad UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hussein-awala commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hussein-awala commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1304909796
Hey @nsivabalan, can you review the PR please?
Feel free to change the name of the new configuration if you have a better suggestion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1871237465
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
}, {
"hash" : "a78cf3612346492a082bf211940c46c9e82752a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650",
"triggerID" : "a78cf3612346492a082bf211940c46c9e82752a1",
"triggerType" : "PUSH"
}, {
"hash" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12812",
"triggerID" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"triggerType" : "PUSH"
}, {
"hash" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12813",
"triggerID" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"triggerType" : "PUSH"
}, {
"hash" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12814",
"triggerID" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"triggerType" : "PUSH"
}, {
"hash" : "40d125a000fad7439d6c21fa3e20f7128b151dad",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13520",
"triggerID" : "40d125a000fad7439d6c21fa3e20f7128b151dad",
"triggerType" : "PUSH"
}, {
"hash" : "6ab2a114214e2d961abb8f80f535dc02cbae548e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21746",
"triggerID" : "6ab2a114214e2d961abb8f80f535dc02cbae548e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 40d125a000fad7439d6c21fa3e20f7128b151dad Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13520)
* 6ab2a114214e2d961abb8f80f535dc02cbae548e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21746)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1871230778
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
}, {
"hash" : "a78cf3612346492a082bf211940c46c9e82752a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650",
"triggerID" : "a78cf3612346492a082bf211940c46c9e82752a1",
"triggerType" : "PUSH"
}, {
"hash" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12812",
"triggerID" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"triggerType" : "PUSH"
}, {
"hash" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12813",
"triggerID" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"triggerType" : "PUSH"
}, {
"hash" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12814",
"triggerID" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"triggerType" : "PUSH"
}, {
"hash" : "40d125a000fad7439d6c21fa3e20f7128b151dad",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13520",
"triggerID" : "40d125a000fad7439d6c21fa3e20f7128b151dad",
"triggerType" : "PUSH"
}, {
"hash" : "6ab2a114214e2d961abb8f80f535dc02cbae548e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6ab2a114214e2d961abb8f80f535dc02cbae548e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 40d125a000fad7439d6c21fa3e20f7128b151dad Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13520)
* 6ab2a114214e2d961abb8f80f535dc02cbae548e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] pratyakshsharma commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
pratyakshsharma commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1293864073
Thank you for raising this PR @hussein-awala .
I will take a look at it by tomorrow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1294883514
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
}, {
"hash" : "a78cf3612346492a082bf211940c46c9e82752a1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a78cf3612346492a082bf211940c46c9e82752a1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 00f351db8eab5af32063e4a41c4e451ca9838ee4 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641)
* 74e7a278867b56d1b6a6bbee29fb477469246487 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646)
* a78cf3612346492a082bf211940c46c9e82752a1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1304851516
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
}, {
"hash" : "a78cf3612346492a082bf211940c46c9e82752a1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650",
"triggerID" : "a78cf3612346492a082bf211940c46c9e82752a1",
"triggerType" : "PUSH"
}, {
"hash" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12812",
"triggerID" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a78cf3612346492a082bf211940c46c9e82752a1 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650)
* d18514b58b5fa732f55bd3ad6212484b7766f50e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12812)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1304912867
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
}, {
"hash" : "a78cf3612346492a082bf211940c46c9e82752a1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650",
"triggerID" : "a78cf3612346492a082bf211940c46c9e82752a1",
"triggerType" : "PUSH"
}, {
"hash" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12812",
"triggerID" : "d18514b58b5fa732f55bd3ad6212484b7766f50e",
"triggerType" : "PUSH"
}, {
"hash" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12813",
"triggerID" : "1177f60208a2d97c0fce5b8c9aca309a0494cd18",
"triggerType" : "PUSH"
}, {
"hash" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12814",
"triggerID" : "36e1e84c05112409ae4d1d4e8b2f13eadc69237b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 36e1e84c05112409ae4d1d4e8b2f13eadc69237b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12814)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1288143002
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3300f7bdbf9d1cb178390d36523db2ec0279448c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1288142113
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3300f7bdbf9d1cb178390d36523db2ec0279448c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1294888771
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12498",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
}, {
"hash" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12641",
"triggerID" : "00f351db8eab5af32063e4a41c4e451ca9838ee4",
"triggerType" : "PUSH"
}, {
"hash" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646",
"triggerID" : "74e7a278867b56d1b6a6bbee29fb477469246487",
"triggerType" : "PUSH"
}, {
"hash" : "a78cf3612346492a082bf211940c46c9e82752a1",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650",
"triggerID" : "a78cf3612346492a082bf211940c46c9e82752a1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 74e7a278867b56d1b6a6bbee29fb477469246487 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12646)
* a78cf3612346492a082bf211940c46c9e82752a1 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12650)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7041:
URL: https://github.com/apache/hudi/pull/7041#issuecomment-1288184587
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496",
"triggerID" : "3300f7bdbf9d1cb178390d36523db2ec0279448c",
"triggerType" : "PUSH"
}, {
"hash" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "acd416d779132b9fd7a7b1fe58eaaeebcf1b821f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3300f7bdbf9d1cb178390d36523db2ec0279448c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12496)
* acd416d779132b9fd7a7b1fe58eaaeebcf1b821f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] pratyakshsharma commented on a diff in pull request #7041: [HUDI-5053] Create clean complete commit when there is none to clean in order to leverage incremental cleaning
Posted by GitBox <gi...@apache.org>.
pratyakshsharma commented on code in PR #7041:
URL: https://github.com/apache/hudi/pull/7041#discussion_r1061583585
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanner.java:
##########
@@ -488,13 +497,14 @@ public Option<HoodieInstant> getEarliestCommitToRetain() {
int hoursRetained = config.getCleanerHoursRetained();
if (config.getCleanerPolicy() == HoodieCleaningPolicy.KEEP_LATEST_COMMITS
&& commitTimeline.countInstants() > commitsRetained) {
- earliestCommitToRetain = commitTimeline.nthInstant(commitTimeline.countInstants() - commitsRetained); //15 instants total, 10 commits to retain, this gives 6th instant in the list
+ earliestCommitToRetain =
+ commitTimeline.nthInstant(commitTimeline.countInstants() - commitsRetained); //15 instants total, 10 commits to retain, this gives 6th instant in the list- commitsRetained, 0));
Review Comment:
Can you remove the changes in this comment here? Are they needed?
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanActionExecutor.java:
##########
@@ -146,10 +179,12 @@ HoodieCleanerPlan requestClean(HoodieEngineContext context) {
*/
protected Option<HoodieCleanerPlan> requestClean(String startCleanTime) {
final HoodieCleanerPlan cleanerPlan = requestClean(context);
- if ((cleanerPlan.getFilePathsToBeDeletedPerPartition() != null)
+ // Create a clean request contains the cleaner plan if:
+ // - ALLOW_EMPTY_CLEAN_COMMITS is true
+ // - or the list of the file paths to be deleted is not empty
+ if (config.allowEmptyCleanCommits() || (cleanerPlan.getFilePathsToBeDeletedPerPartition() != null
Review Comment:
we should probably add another check to validate that ALLOW_EMPTY_CLEAN_COMMITS is set to true only if incremental cleaning is enabled.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanActionExecutor.java:
##########
@@ -38,10 +39,15 @@
import org.apache.hudi.exception.HoodieIOException;
import org.apache.hudi.table.HoodieTable;
import org.apache.hudi.table.action.BaseActionExecutor;
+
Review Comment:
nit: remove the extra line.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanner.java:
##########
@@ -187,9 +187,18 @@ private List<String> getPartitionPathsForIncrementalCleaning(HoodieCleanMetadata
LOG.info("Incremental Cleaning mode is enabled. Looking up partition-paths that have since changed "
+ "since last cleaned at " + cleanMetadata.getEarliestCommitToRetain()
+ ". New Instant to retain : " + newInstantToRetain);
+ String commitJustBeforeEarliestCommitToRetain = null;
Review Comment:
Just wondering if these changes in this method are actually needed. Even if we keep the original logic, I guess we are not missing out on any scenario. Please correct me if I am missing anything here. Ultimately you are only getting the lastCheckedCommit and that is already taken care of in the original logic.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanActionExecutor.java:
##########
@@ -86,7 +92,29 @@ private boolean needsCleaning(CleaningTriggerStrategy strategy) {
if (strategy == CleaningTriggerStrategy.NUM_COMMITS) {
int numberOfCommits = getCommitsSinceLastCleaning();
int maxInlineCommitsForNextClean = config.getCleaningMaxCommits();
- return numberOfCommits >= maxInlineCommitsForNextClean;
+ if (numberOfCommits >= maxInlineCommitsForNextClean) {
+ // check if the number of commits created after the last clean is greater than clean.max.commits
+ int commitsRetained = config.getCleanerCommitsRetained();
+ int hoursRetained = config.getCleanerHoursRetained();
+ if (config.getCleanerPolicy() == HoodieCleaningPolicy.KEEP_LATEST_COMMITS) {
+ // if cleaner policy is KEEP_LATEST_COMMITS then
+ // check if the number of completed commits in the timeline is greater than cleaner.commits.retained
+ return table.getCompletedCommitsTimeline().countInstants() > commitsRetained;
Review Comment:
These checks are already present in CleanPlanner here - https://github.com/apache/hudi/blob/a0df6ecfb47ac9e393b5eb977850cb8fbe7d3f72/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanner.java#L313.
The new changes are redundant and we should remove the checks from any one place. I will leave this to you to decide. I feel we should remove the check from the place I mentioned since this makes more sense to keep them in this method here.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanActionExecutor.java:
##########
@@ -86,7 +92,29 @@ private boolean needsCleaning(CleaningTriggerStrategy strategy) {
if (strategy == CleaningTriggerStrategy.NUM_COMMITS) {
int numberOfCommits = getCommitsSinceLastCleaning();
int maxInlineCommitsForNextClean = config.getCleaningMaxCommits();
- return numberOfCommits >= maxInlineCommitsForNextClean;
+ if (numberOfCommits >= maxInlineCommitsForNextClean) {
+ // check if the number of commits created after the last clean is greater than clean.max.commits
+ int commitsRetained = config.getCleanerCommitsRetained();
+ int hoursRetained = config.getCleanerHoursRetained();
+ if (config.getCleanerPolicy() == HoodieCleaningPolicy.KEEP_LATEST_COMMITS) {
+ // if cleaner policy is KEEP_LATEST_COMMITS then
+ // check if the number of completed commits in the timeline is greater than cleaner.commits.retained
+ return table.getCompletedCommitsTimeline().countInstants() > commitsRetained;
+ } else if (config.getCleanerPolicy() == HoodieCleaningPolicy.KEEP_LATEST_BY_HOURS) {
+ // if cleaner policy is KEEP_LATEST_BY_HOURS then
+ // check if there is a commit with timestamp older than current instant - cleaner.hours.retained
+ Instant instant = Instant.now();
+ ZonedDateTime currentDateTime = ZonedDateTime.ofInstant(instant, ZoneId.systemDefault());
+ String earliestTimeToRetain = HoodieActiveTimeline.formatDate(Date.from(currentDateTime.minusHours(hoursRetained).toInstant()));
+ return table.getCompletedCommitsTimeline().getInstantsAsStream().filter(i -> HoodieTimeline.compareTimestamps(i.getTimestamp(),
+ HoodieTimeline.LESSER_THAN, earliestTimeToRetain)).count() > 0;
+ } else if (config.getCleanerPolicy() == HoodieCleaningPolicy.KEEP_LATEST_FILE_VERSIONS) {
+ // if cleaner policy is KEEP_LATEST_BY_HOURS then
Review Comment:
nit: change the cleaner policy to KEEP_LATEST_FILE_VERSIONS.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanActionExecutor.java:
##########
@@ -146,10 +179,12 @@ HoodieCleanerPlan requestClean(HoodieEngineContext context) {
*/
protected Option<HoodieCleanerPlan> requestClean(String startCleanTime) {
final HoodieCleanerPlan cleanerPlan = requestClean(context);
- if ((cleanerPlan.getFilePathsToBeDeletedPerPartition() != null)
+ // Create a clean request contains the cleaner plan if:
Review Comment:
nit: Create a clean request -> New clean request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org