You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/07/18 09:30:31 UTC
[GitHub] [hudi] zhangyue19921010 opened a new pull request, #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
zhangyue19921010 opened a new pull request, #6133:
URL: https://github.com/apache/hudi/pull/6133
Please take a look at https://github.com/apache/hudi/pull/6003 for more details.
replaced https://github.com/apache/hudi/pull/6059
## What is the purpose of the pull request
*(For example: This pull request adds quick-start document.)*
## Brief change log
*(for example:)*
- *Modify AnnotationLocation checkstyle rule in checkstyle.xml*
## Verify this pull request
*(Please pick either of the following options)*
This pull request is a trivial rework / code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end.*
- *Added HoodieClientWriteTest to verify the change.*
- *Manually verified the change by running a job locally.*
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1187194606
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fcaaf9d6dc063aed48319e335c181e913ac9106f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031)
* 553fb00b590e9586ade21e86569ccb30d8dec7a9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033)
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 66b7d1b75c7a201b5d108cd48a70363dd00a1795 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1396336655
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214",
"triggerID" : "1376756472",
"triggerType" : "MANUAL"
}, {
"hash" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422",
"triggerID" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 1b837ec65943331f6c805f723b088240ae66dfaf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207) Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214)
* 67b3892f4bafe64ecf203a53bb86db22e61ae587 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1396331260
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214",
"triggerID" : "1376756472",
"triggerType" : "MANUAL"
}, {
"hash" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 1b837ec65943331f6c805f723b088240ae66dfaf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207) Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214)
* 67b3892f4bafe64ecf203a53bb86db22e61ae587 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1399850160
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214",
"triggerID" : "1376756472",
"triggerType" : "MANUAL"
}, {
"hash" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422",
"triggerID" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"triggerType" : "PUSH"
}, {
"hash" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14480",
"triggerID" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"triggerType" : "PUSH"
}, {
"hash" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14484",
"triggerID" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"triggerType" : "PUSH"
}, {
"hash" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14518",
"triggerID" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"triggerType" : "PUSH"
}, {
"hash" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14549",
"triggerID" : "1399842193",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 501e47fb5c554a32acd7d2b410b5cdcf880f6057 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14518) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14549)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1312472305
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1283832608
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 645766d09d376bc46a1e0b6af770d9e84f778b24 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1397948658
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214",
"triggerID" : "1376756472",
"triggerType" : "MANUAL"
}, {
"hash" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422",
"triggerID" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"triggerType" : "PUSH"
}, {
"hash" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14480",
"triggerID" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"triggerType" : "PUSH"
}, {
"hash" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 67b3892f4bafe64ecf203a53bb86db22e61ae587 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422)
* a2980b73d4ad32976360804059fcd6df969b9f89 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14480)
* 7344fabacee437adb4b55ca922df0f5fb14ae372 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1312474551
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 465536ff2afcf217c19ca00b4a90a5e04d372bc3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1192553211
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 64819e4ab1ed12788e126072fa50e964fc62c257 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1188066164
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 66b7d1b75c7a201b5d108cd48a70363dd00a1795 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1322348048
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 71e0d1e91d2f540b441838263362c31251fa4ab5 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149)
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1322459358
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 316e5aefa89ec2dc35928af7c7da4b11a23aab20 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155)
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* ffd8315ca6dc2c771e90b5fa983c0b41010605eb Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1322617042
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* ffd8315ca6dc2c771e90b5fa983c0b41010605eb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028232202
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieLockConfig.java:
##########
@@ -189,6 +190,33 @@ public class HoodieLockConfig extends HoodieConfig {
.withDocumentation("Lock provider class name, this should be subclass of "
+ "org.apache.hudi.client.transaction.ConflictResolutionStrategy");
+ // Pluggable strategies to use when early conflict detection
+ public static final ConfigProperty<String> EARLY_CONFLICT_DETECTION_STRATEGY_CLASS_NAME = ConfigProperty
+ .key(LOCK_PREFIX + "early.conflict.detection.strategy")
+ .defaultValue(AsyncTimelineMarkerEarlyConflictDetectionStrategy.class.getName())
+ .sinceVersion("0.12.0")
+ .withDocumentation("Early conflict detection class name, this should be subclass of "
+ + "org.apache.hudi.common.conflict.detection.HoodieEarlyConflictDetectionStrategy");
+
+ public static final ConfigProperty<Boolean> EARLY_CONFLICT_DETECTION_ENABLE = ConfigProperty
+ .key(LOCK_PREFIX + "early.conflict.detection.enable")
+ .defaultValue(false)
+ .sinceVersion("0.12.0")
Review Comment:
changed.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieLockConfig.java:
##########
@@ -189,6 +190,33 @@ public class HoodieLockConfig extends HoodieConfig {
.withDocumentation("Lock provider class name, this should be subclass of "
+ "org.apache.hudi.client.transaction.ConflictResolutionStrategy");
+ // Pluggable strategies to use when early conflict detection
+ public static final ConfigProperty<String> EARLY_CONFLICT_DETECTION_STRATEGY_CLASS_NAME = ConfigProperty
+ .key(LOCK_PREFIX + "early.conflict.detection.strategy")
+ .defaultValue(AsyncTimelineMarkerEarlyConflictDetectionStrategy.class.getName())
Review Comment:
changed.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieWriteHandle.java:
##########
@@ -187,8 +194,54 @@ protected Path makeNewFilePath(String partitionPath, String fileName) {
* @param partitionPath Partition path
*/
protected void createMarkerFile(String partitionPath, String dataFileName) {
- WriteMarkersFactory.get(config.getMarkersType(), hoodieTable, instantTime)
- .create(partitionPath, dataFileName, getIOType());
+ WriteMarkers writeMarkers = WriteMarkersFactory.get(config.getMarkersType(), hoodieTable, instantTime);
+ // do early conflict detection before create markers.
+ if (config.getWriteConcurrencyMode().supportsOptimisticConcurrencyControl()
+ && config.isEarlyConflictDetectionEnable()) {
+ HoodieEarlyConflictDetectionStrategy earlyConflictDetectionStrategy = config.getEarlyConflictDetectionStrategy();
+ if (earlyConflictDetectionStrategy instanceof HoodieTransactionDirectMarkerBasedEarlyConflictDetectionStrategy) {
+ createMarkerWithTransaction(earlyConflictDetectionStrategy, writeMarkers, partitionPath, dataFileName);
+ } else {
+ createMarkerWithEarlyConflictDetection(earlyConflictDetectionStrategy, writeMarkers, partitionPath, dataFileName);
+ }
+ } else {
+ // create marker directly
+ writeMarkers.create(partitionPath, dataFileName, getIOType());
+ }
+ }
+
+ private Option<Path> createMarkerWithEarlyConflictDetection(HoodieEarlyConflictDetectionStrategy resolutionStrategy,
+ WriteMarkers writeMarkers,
+ String partitionPath,
+ String dataFileName) {
+ Set<HoodieInstant> completedCommitInstants = hoodieTable.getMetaClient().getActiveTimeline()
Review Comment:
changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1373312323
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 0447a7135e0cf136cd894291a1efd4374c3efd66 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206)
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* c973c81cc138a320e562f293f1dc72828ed8fac6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
yihua commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1064210793
##########
hudi-common/src/main/java/org/apache/hudi/common/conflict/detection/HoodieTimelineServerBasedEarlyConflictDetectionStrategy.java:
##########
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.conflict.detection;
+
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+
+import java.util.Set;
+
+public abstract class HoodieTimelineServerBasedEarlyConflictDetectionStrategy implements HoodieEarlyConflictDetectionStrategy {
Review Comment:
Implementation-wise, `HoodieEarlyConflictDetectionStrategy` is used by the write client. I suggest that the timeline-server-related logic, i.e., the `fresh` method containing the scheduling of the marker checker, be moved to the timeline server.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1290868262
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* e23ab613d4aad3844374b21d987f95ca6f5183e5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1291957547
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* e23ab613d4aad3844374b21d987f95ca6f5183e5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1192465084
Hi @yanghua and @yihua Sorry for the late response.
Resolved conflict! PTAL :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1292749008
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* e23ab613d4aad3844374b21d987f95ca6f5183e5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584) Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1312417316
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028215118
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/LockManager.java:
##########
@@ -62,6 +63,33 @@ public LockManager(HoodieWriteConfig writeConfig, FileSystem fs) {
metrics = new HoodieLockMetrics(writeConfig);
}
+ /**
+ * Try to have a lock at partitionPath + fileID level for different write handler.
+ * @param writeConfig
+ * @param fs
+ * @param partitionPath
+ * @param fileId
+ */
+ public LockManager(HoodieWriteConfig writeConfig, FileSystem fs, String partitionPath, String fileId) {
Review Comment:
Added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028232906
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieWriteHandle.java:
##########
@@ -187,8 +194,54 @@ protected Path makeNewFilePath(String partitionPath, String fileName) {
* @param partitionPath Partition path
*/
protected void createMarkerFile(String partitionPath, String dataFileName) {
- WriteMarkersFactory.get(config.getMarkersType(), hoodieTable, instantTime)
- .create(partitionPath, dataFileName, getIOType());
+ WriteMarkers writeMarkers = WriteMarkersFactory.get(config.getMarkersType(), hoodieTable, instantTime);
+ // do early conflict detection before create markers.
+ if (config.getWriteConcurrencyMode().supportsOptimisticConcurrencyControl()
+ && config.isEarlyConflictDetectionEnable()) {
+ HoodieEarlyConflictDetectionStrategy earlyConflictDetectionStrategy = config.getEarlyConflictDetectionStrategy();
+ if (earlyConflictDetectionStrategy instanceof HoodieTransactionDirectMarkerBasedEarlyConflictDetectionStrategy) {
+ createMarkerWithTransaction(earlyConflictDetectionStrategy, writeMarkers, partitionPath, dataFileName);
+ } else {
+ createMarkerWithEarlyConflictDetection(earlyConflictDetectionStrategy, writeMarkers, partitionPath, dataFileName);
+ }
+ } else {
+ // create marker directly
+ writeMarkers.create(partitionPath, dataFileName, getIOType());
+ }
Review Comment:
Nice catch here. Changed.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/SimpleTransactionDirectMarkerBasedEarlyConflictDetectionStrategy.java:
##########
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.table.marker;
+
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hudi.common.conflict.detection.HoodieTransactionDirectMarkerBasedEarlyConflictDetectionStrategy;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.exception.HoodieEarlyConflictDetectionException;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.ConcurrentModificationException;
+import java.util.Set;
+
+/**
+ * This strategy is used for direct marker writers, trying to do early conflict detection.
+ * It will use fileSystem api like list and exist directly to check if there is any marker file conflict.
+ */
+public class SimpleTransactionDirectMarkerBasedEarlyConflictDetectionStrategy extends HoodieTransactionDirectMarkerBasedEarlyConflictDetectionStrategy {
+ private static final Logger LOG = LogManager.getLogger(SimpleTransactionDirectMarkerBasedEarlyConflictDetectionStrategy.class);
+
+ @Override
+ public boolean hasMarkerConflict(String basePath, FileSystem fs, String partitionPath, String fileId, String instantTime,
Review Comment:
Nice catch here. Changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028240012
##########
hudi-common/src/main/java/org/apache/hudi/common/util/MarkerUtils.java:
##########
@@ -208,10 +216,56 @@ public static Set<String> readMarkersFromFile(Path markersFilePath, Serializable
fsDataInputStream = fs.open(markersFilePath);
markers = new HashSet<>(FileIOUtils.readAsUTFStringLines(fsDataInputStream));
} catch (IOException e) {
- throw new HoodieIOException("Failed to read MARKERS file " + markersFilePath, e);
+ if (ignoreException) {
+ LOG.warn("IOException occurs during read MARKERS file, ", e);
+ } else {
+ throw new HoodieIOException("Failed to read MARKERS file " + markersFilePath, e);
+ }
} finally {
closeQuietly(fsDataInputStream);
}
return markers;
}
+
+ /**
+ * Reads files containing the markers written by timeline-server-based marker mechanism locally instead of using cluster Context.
+ *
+ * @param markerDir marker directory.
+ * @param fileSystem file system to use.
+ * @return A {@code Map} of file name to the set of markers stored in the file.
+ */
+ public static Set<String> readTimelineServerBasedMarkersFromFileSystemLocally(String markerDir, FileSystem fileSystem) {
Review Comment:
Nice catch. Changed.
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/MarkerHandler.java:
##########
@@ -109,6 +126,7 @@ public void stop() {
}
dispatchingExecutorService.shutdown();
batchingExecutorService.shutdown();
+ checkers.values().forEach(ExecutorService::shutdown);
Review Comment:
Removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1322354911
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 71e0d1e91d2f540b441838263362c31251fa4ab5 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149)
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 316e5aefa89ec2dc35928af7c7da4b11a23aab20 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028247148
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/MarkerHandler.java:
##########
@@ -150,7 +168,50 @@ public boolean doesMarkerDirExist(String markerDir) {
* @param markerName marker name
* @return the {@code CompletableFuture} instance for the request
*/
- public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName) {
+ public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName,
+ String batchInterval, String period, String maxAllowableHeartbeatIntervalInMs,
+ String basePath, String earlyConflictDetectionEnable,
+ String earlyConflictDetectionClassName) {
+ // Step1 do early conflict detection if enable
Review Comment:
Yeap, for now we init strategy in the first time call this create marker API, and only do fresh (which will start checker thread) when the instant time changed or create at first time.
We can also init this thread in constructor, but for now, we don't pass hoodieWriteConfig in MarkerHandler..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1323548601
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* ffd8315ca6dc2c771e90b5fa983c0b41010605eb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157)
* 8a402c4c8574cca66047e614038a002cb24e8e94 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1029446767
##########
hudi-common/src/main/java/org/apache/hudi/common/util/MarkerUtils.java:
##########
@@ -208,10 +219,116 @@ public static Set<String> readMarkersFromFile(Path markersFilePath, Serializable
fsDataInputStream = fs.open(markersFilePath);
markers = new HashSet<>(FileIOUtils.readAsUTFStringLines(fsDataInputStream));
} catch (IOException e) {
- throw new HoodieIOException("Failed to read MARKERS file " + markersFilePath, e);
+ if (ignoreException) {
+ LOG.warn("IOException occurs during read MARKERS file, ", e);
+ } else {
+ throw new HoodieIOException("Failed to read MARKERS file " + markersFilePath, e);
+ }
} finally {
closeQuietly(fsDataInputStream);
}
return markers;
}
+
+ public static List<Path> getAllMarkerDir(Path tempPath, FileSystem fs) throws IOException {
+ return Arrays.stream(fs.listStatus(tempPath)).map(FileStatus::getPath).collect(Collectors.toList());
+ }
+
+ public static boolean hasCommitConflict(HoodieActiveTimeline activeTimeline, Set<String> currentFileIDs, Set<HoodieInstant> completedCommitInstants) {
+
+ Set<HoodieInstant> currentInstants = activeTimeline.reload().getCommitsTimeline().filterCompletedInstants().getInstants().collect(Collectors.toSet());
+
+ currentInstants.removeAll(completedCommitInstants);
+ Set<String> missingFileIDs = currentInstants.stream().flatMap(instant -> {
+ try {
+ return HoodieCommitMetadata.fromBytes(activeTimeline.getInstantDetails(instant).get(), HoodieCommitMetadata.class)
+ .getFileIdAndRelativePaths().keySet().stream();
+ } catch (Exception e) {
+ return Stream.empty();
+ }
+ }).collect(Collectors.toSet());
+ currentFileIDs.retainAll(missingFileIDs);
+ return !currentFileIDs.isEmpty();
+ }
+
+ /**
+ * Get Candidate Instant to do conflict checking:
+ * 1. Skip current writer related instant(currentInstantTime)
+ * 2. Skip all instants after currentInstantTime
+ * 3. Skip dead writers related instants based on heart-beat
+ * 4. Skip pending compaction instant (For now we don' do early conflict check with compact action)
+ * Because we don't want to let pending compaction block common writer.
+ * @param instants
+ * @return
+ */
+ public static List<String> getCandidateInstants(HoodieActiveTimeline activeTimeline, List<Path> instants, String currentInstantTime,
+ long maxAllowableHeartbeatIntervalInMs, FileSystem fs, String basePath) {
+
+ HoodieActiveTimeline reloadActive = activeTimeline.reload();
+
+ return instants.stream().map(Path::toString).filter(instantPath -> {
+ String instantTime = markerDirToInstantTime(instantPath);
+ return instantTime.compareToIgnoreCase(currentInstantTime) < 0
+ && !reloadActive.filterPendingCompactionTimeline().containsInstant(instantTime)
Review Comment:
in other words, this early conflict detection is focus on common writers' conflict detection.
We can pay attention to writers and table services as next step if necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1396759475
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214",
"triggerID" : "1376756472",
"triggerType" : "MANUAL"
}, {
"hash" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422",
"triggerID" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 67b3892f4bafe64ecf203a53bb86db22e61ae587 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
yihua commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1063209840
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -549,6 +553,50 @@ public class HoodieWriteConfig extends HoodieConfig {
.withDocumentation("When table is upgraded from pre 0.12 to 0.12, we check for \"default\" partition and fail if found one. "
+ "Users are expected to rewrite the data in those partitions. Enabling this config will bypass this validation");
+ // Pluggable strategies to use when early conflict detection
+ public static final ConfigProperty<String> EARLY_CONFLICT_DETECTION_STRATEGY_CLASS_NAME = ConfigProperty
+ .key(CONCURRENCY_PREFIX + "early.conflict.detection.strategy")
+ .noDefaultValue()
+ .sinceVersion("0.13.0")
+ .withInferFunction(cfg -> {
+ MarkerType markerType = MarkerType.valueOf(cfg.getString(MARKERS_TYPE).toUpperCase());
+ switch (markerType) {
+ case DIRECT:
+ return Option.of(SimpleDirectMarkerBasedEarlyConflictDetectionStrategy.class.getName());
+ case TIMELINE_SERVER_BASED:
+ default:
+ return Option.of(AsyncTimelineMarkerEarlyConflictDetectionStrategy.class.getName());
+ }
+ })
+ .withDocumentation("Early conflict detection class name, this should be subclass of "
+ + "org.apache.hudi.common.conflict.detection.HoodieEarlyConflictDetectionStrategy");
+
+ public static final ConfigProperty<Boolean> EARLY_CONFLICT_DETECTION_ENABLE = ConfigProperty
+ .key(CONCURRENCY_PREFIX + "early.conflict.detection.enable")
+ .defaultValue(false)
+ .sinceVersion("0.13.0")
+ .withDocumentation("Enable early conflict detection based on markers. It will try to detect writing conflict before create markers and fast fail"
+ + " which will release cluster resources as soon as possible.");
+
+ public static final ConfigProperty<Long> MARKER_CONFLICT_CHECKER_BATCH_INTERVAL = ConfigProperty
Review Comment:
Add unit for the time configs
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/TimelineServerBasedWriteMarkers.java:
##########
@@ -148,13 +185,30 @@ protected Option<Path> create(String partitionPath, String dataFileName, IOType
} catch (IOException e) {
throw new HoodieRemoteException("Failed to create marker file " + partitionPath + "/" + markerFileName, e);
}
- LOG.info("[timeline-server-based] Created marker file " + partitionPath + "/" + markerFileName
- + " in " + timer.endTimer() + " ms");
- if (success) {
- return Option.of(new Path(FSUtils.getPartitionPath(markerDirPath, partitionPath), markerFileName));
+ return success;
+ }
+
+ /**
+ * init create marker related config maps.
+ * @param partitionPath
+ * @param markerFileName
+ * @return
+ */
+ private Map<String, String> initConfigMap(String partitionPath, String markerFileName, boolean initEarlyConflictConfigs) {
Review Comment:
nit: `initEarlyConflictConfigs` -> `initConflictDetectionConfigs`
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -549,6 +553,50 @@ public class HoodieWriteConfig extends HoodieConfig {
.withDocumentation("When table is upgraded from pre 0.12 to 0.12, we check for \"default\" partition and fail if found one. "
+ "Users are expected to rewrite the data in those partitions. Enabling this config will bypass this validation");
+ // Pluggable strategies to use when early conflict detection
+ public static final ConfigProperty<String> EARLY_CONFLICT_DETECTION_STRATEGY_CLASS_NAME = ConfigProperty
+ .key(CONCURRENCY_PREFIX + "early.conflict.detection.strategy")
+ .noDefaultValue()
+ .sinceVersion("0.13.0")
+ .withInferFunction(cfg -> {
+ MarkerType markerType = MarkerType.valueOf(cfg.getString(MARKERS_TYPE).toUpperCase());
+ switch (markerType) {
+ case DIRECT:
+ return Option.of(SimpleDirectMarkerBasedEarlyConflictDetectionStrategy.class.getName());
+ case TIMELINE_SERVER_BASED:
+ default:
+ return Option.of(AsyncTimelineMarkerEarlyConflictDetectionStrategy.class.getName());
+ }
+ })
+ .withDocumentation("Early conflict detection class name, this should be subclass of "
+ + "org.apache.hudi.common.conflict.detection.HoodieEarlyConflictDetectionStrategy");
+
+ public static final ConfigProperty<Boolean> EARLY_CONFLICT_DETECTION_ENABLE = ConfigProperty
+ .key(CONCURRENCY_PREFIX + "early.conflict.detection.enable")
+ .defaultValue(false)
+ .sinceVersion("0.13.0")
+ .withDocumentation("Enable early conflict detection based on markers. It will try to detect writing conflict before create markers and fast fail"
+ + " which will release cluster resources as soon as possible.");
+
+ public static final ConfigProperty<Long> MARKER_CONFLICT_CHECKER_BATCH_INTERVAL = ConfigProperty
Review Comment:
naming: `checker` -> `detector` in all places.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/LockManager.java:
##########
@@ -43,23 +47,52 @@
public class LockManager implements Serializable, AutoCloseable {
private static final Logger LOG = LogManager.getLogger(LockManager.class);
- private final HoodieWriteConfig writeConfig;
- private final LockConfiguration lockConfiguration;
- private final SerializableConfiguration hadoopConf;
- private final int maxRetries;
- private final long maxWaitTimeInMs;
+ private HoodieWriteConfig writeConfig;
+ private LockConfiguration lockConfiguration;
+ private SerializableConfiguration hadoopConf;
+ private int maxRetries;
+ private long maxWaitTimeInMs;
Review Comment:
These should still be final.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/WriteMarkers.java:
##########
@@ -52,16 +58,46 @@ public WriteMarkers(String basePath, String markerFolderPath, String instantTime
this.instantTime = instantTime;
}
+ public Option<Path> create(String partitionPath, String dataFileName, IOType type) {
+ return create(partitionPath, dataFileName, type, Option.empty());
+ }
+
/**
* Creates a marker without checking if the marker already exists.
*
* @param partitionPath partition path in the table
* @param dataFileName data file name
* @param type write IO type
+ * @param handler could be empty
* @return the marker path
*/
- public Option<Path> create(String partitionPath, String dataFileName, IOType type) {
- return create(partitionPath, dataFileName, type, false);
+ public Option<Path> create(String partitionPath, String dataFileName, IOType type, Option<HoodieWriteHandle> handler) {
+ boolean checkIfExists = false;
+
+ if (handler.isPresent()
+ && handler.get().getConfig().getWriteConcurrencyMode().supportsOptimisticConcurrencyControl()
+ && handler.get().getConfig().isEarlyConflictDetectionEnable()) {
+
+ HoodieTableMetaClient metaClient = handler.get().getHoodieTableMetaClient();
+ HoodieActiveTimeline activeTimeline = metaClient.getActiveTimeline();
+
+ HoodieTimeline pendingCompactionTimeline = activeTimeline.filterPendingCompactionTimeline();
+ HoodieTimeline pendingReplaceTimeline = activeTimeline.filterPendingReplaceTimeline();
+ // TODO if current is compact or clustering then create marker directly without early conflict detection.
+ // Need to support early conflict detection between table service and common writers.
+ if (pendingCompactionTimeline.containsInstant(instantTime) || pendingReplaceTimeline.containsInstant(instantTime)) {
Review Comment:
Sg. @zhangyue19921010 could you add the details around this implementation decision to the RFC?
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/embedded/EmbeddedTimelineService.java:
##########
@@ -86,7 +86,17 @@ public void startServer() throws IOException {
.enableMarkerRequests(true)
.markerBatchNumThreads(writeConfig.getMarkersTimelineServerBasedBatchNumThreads())
.markerBatchIntervalMs(writeConfig.getMarkersTimelineServerBasedBatchIntervalMs())
- .markerParallelism(writeConfig.getMarkersDeleteParallelism());
+ .markerParallelism(writeConfig.getMarkersDeleteParallelism())
+ ;
Review Comment:
nit: should be in the same line.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/DirectWriteMarkers.java:
##########
@@ -155,6 +160,20 @@ protected Option<Path> create(String partitionPath, String dataFileName, IOType
return create(getMarkerPath(partitionPath, dataFileName, type), checkIfExists);
}
+ @Override
+ public Option<Path> createWithEarlyConflictDetection(String partitionPath, String dataFileName, IOType type, boolean checkIfExists, Set<HoodieInstant> completedCommitInstants,
Review Comment:
Could `completedCommitInstants` be derived from `activeTimeline`? If so, there is no need to pass in `completedCommitInstants`.
##########
hudi-common/src/main/java/org/apache/hudi/common/conflict/detection/HoodieTimelineServerBasedEarlyConflictDetectionStrategy.java:
##########
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.conflict.detection;
+
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+
+import java.util.Set;
+
+public abstract class HoodieTimelineServerBasedEarlyConflictDetectionStrategy implements HoodieEarlyConflictDetectionStrategy {
Review Comment:
Implementation-wise, `HoodieEarlyConflictDetectionStrategy` is used by the write client. I suggest that the timeline-server-related logic, i.e., the `fresh` method containing the scheduling of the marker checker, be moved to the timeline server.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/WriteMarkers.java:
##########
@@ -52,16 +58,47 @@ public WriteMarkers(String basePath, String markerFolderPath, String instantTime
this.instantTime = instantTime;
}
+ public Option<Path> create(String partitionPath, String dataFileName, IOType type) {
+ return create(partitionPath, dataFileName, type, Option.empty());
+ }
+
/**
* Creates a marker without checking if the marker already exists.
*
* @param partitionPath partition path in the table
* @param dataFileName data file name
* @param type write IO type
+ * @param handler could be empty
* @return the marker path
*/
- public Option<Path> create(String partitionPath, String dataFileName, IOType type) {
- return create(partitionPath, dataFileName, type, false);
+ public Option<Path> create(String partitionPath, String dataFileName, IOType type, Option<HoodieWriteHandle> handler) {
Review Comment:
nit: `HoodieWriteHandle` instance should not be directly passed in here. Instead, only pass in what's needed, i.e., relevant write config, the timeline, and the file ID, to avoid exposure of `HoodieWriteHandle`.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/LockManager.java:
##########
@@ -62,6 +63,33 @@ public LockManager(HoodieWriteConfig writeConfig, FileSystem fs) {
metrics = new HoodieLockMetrics(writeConfig);
}
+ /**
+ * Try to have a lock at partitionPath + fileID level for different write handler.
+ * @param writeConfig
+ * @param fs
+ * @param partitionPath
+ * @param fileId
+ */
+ public LockManager(HoodieWriteConfig writeConfig, FileSystem fs, String partitionPath, String fileId) {
+ this.writeConfig = writeConfig;
+ this.hadoopConf = new SerializableConfiguration(fs.getConf());
+ TypedProperties props = refreshLockConfig(writeConfig, partitionPath + "/" + fileId);
+ this.lockConfiguration = new LockConfiguration(props);
+ maxRetries = lockConfiguration.getConfig().getInteger(LOCK_ACQUIRE_CLIENT_NUM_RETRIES_PROP_KEY,
+ Integer.parseInt(HoodieLockConfig.LOCK_ACQUIRE_CLIENT_NUM_RETRIES.defaultValue()));
+ maxWaitTimeInMs = lockConfiguration.getConfig().getLong(LOCK_ACQUIRE_CLIENT_RETRY_WAIT_TIME_IN_MILLIS_PROP_KEY,
+ Long.parseLong(HoodieLockConfig.LOCK_ACQUIRE_CLIENT_RETRY_WAIT_TIME_IN_MILLIS.defaultValue()));
+ }
+
+ /**
+ * rebuild lock related configs, only support ZK related lock for now.
+ */
+ private TypedProperties refreshLockConfig(HoodieWriteConfig writeConfig, String key) {
+ TypedProperties props = new TypedProperties(writeConfig.getProps());
+ props.setProperty(LockConfiguration.ZK_LOCK_KEY_PROP_KEY, key);
Review Comment:
Sg. Let's use `LOCK_PROVIDER_CLASS_NAME` instead of `ZK_BASE_PATH_PROP_KEY` for checking whether ZK-based lock is configured.
@zhangyue19921010 could you file a JIRA ticket besides the TODO because this requires more work?
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/TransactionManager.java:
##########
@@ -57,6 +62,14 @@ public void beginTransaction(Option<HoodieInstant> newTxnOwnerInstant,
}
}
+ public void beginTransaction(String partitionPath, String fileId) {
Review Comment:
This can be simplified without taking the arguments because the constructor takes the partition path and file ID. The transaction manager for the direct makers should be implemented in a separate class to isolate the logic.
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/TimelineService.java:
##########
@@ -124,6 +124,29 @@ public static class Config implements Serializable {
@Parameter(names = {"--marker-parallelism", "-mdp"}, description = "Parallelism to use for reading and deleting marker files")
public int markerParallelism = 100;
+ @Parameter(names = {"--early-conflict-detection-strategy"}, description = "Early conflict detection class name, this should be subclass of "
+ + "org.apache.hudi.common.conflict.detection.HoodieEarlyConflictDetectionStrategy")
+ public String earlyConflictDetectStrategy = "org.apache.hudi.timeline.service.handlers.marker.AsyncTimelineMarkerEarlyConflictDetectionStrategy";
+
+ @Parameter(names = {"--early-conflict-detection-check-commit-conflict"}, description = "Enable check commit conflict or not during early conflict detect")
+ public Boolean checkCommitConflict = false;
+
+ @Parameter(names = {"--early-conflict-detection-enable"}, description = "Enable early conflict detection based on markers. It will try to detect writing conflict "
+ + "before create markers and fast fail which will release cluster resources as soon as possible.")
+ public Boolean earlyConflictDetectionEnable = false;
+
+ @Parameter(names = {"--early-conflict-async-checker-batch-interval"}, description = "Used for timeline based marker AsyncTimelineMarkerConflictResolutionStrategy. "
Review Comment:
nit: rename the timeline server configs to be aligned with `HoodieWriteConfig`
##########
hudi-common/src/main/java/org/apache/hudi/common/conflict/detection/HoodieDirectMarkerBasedEarlyConflictDetectionStrategy.java:
##########
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.conflict.detection;
+
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.common.config.HoodieConfig;
+import org.apache.hudi.common.fs.HoodieWrapperFileSystem;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieActiveTimeline;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.MarkerUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+public abstract class HoodieDirectMarkerBasedEarlyConflictDetectionStrategy implements HoodieEarlyConflictDetectionStrategy {
+
+ private static final Logger LOG = LogManager.getLogger(HoodieDirectMarkerBasedEarlyConflictDetectionStrategy.class);
+ protected final String basePath;
+ protected final FileSystem fs;
+ protected final String partitionPath;
+ protected final String fileId;
+ protected final String instantTime;
+ protected final HoodieActiveTimeline activeTimeline;
+ protected final HoodieConfig config;
+ protected Set<HoodieInstant> completedCommitInstants;
+ protected final Boolean checkCommitConflict;
+ protected final Long maxAllowableHeartbeatIntervalInMs;
+
+ public HoodieDirectMarkerBasedEarlyConflictDetectionStrategy(String basePath, HoodieWrapperFileSystem fs, String partitionPath, String fileId, String instantTime,
+ HoodieActiveTimeline activeTimeline, HoodieConfig config, Boolean checkCommitConflict, Long maxAllowableHeartbeatIntervalInMs,
+ HashSet<HoodieInstant> completedCommitInstants) {
+ this.basePath = basePath;
+ this.fs = fs;
+ this.partitionPath = partitionPath;
+ this.fileId = fileId;
+ this.instantTime = instantTime;
+ this.completedCommitInstants = completedCommitInstants;
+ this.activeTimeline = activeTimeline;
+ this.config = config;
+ this.checkCommitConflict = checkCommitConflict;
+ this.maxAllowableHeartbeatIntervalInMs = maxAllowableHeartbeatIntervalInMs;
+ }
+
+ /**
+ * We need to do list operation here.
+ * In order to reduce the list pressure as much as possible, first we build path prefix in advance: '$base_path/.temp/instant_time/partition_path',
+ * and only list these specific partition_paths we need instead of list all the '$base_path/.temp/'
+ * @param basePath
+ * @param partitionPath
+ * @param fileId 162b13d7-9530-48cf-88a4-02241817ae0c-0_1-74-100_003.parquet
+ * @return true if current fileID is already existed under .temp/instant_time/partition_path/..
+ * @throws IOException
+ */
+ public boolean checkMarkerConflict(HoodieActiveTimeline activeTimeline, String basePath, String partitionPath, String fileId,
Review Comment:
Similarly, the arguments should be simplified since the constructor takes most of the information already.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/LockManager.java:
##########
@@ -43,23 +47,52 @@
public class LockManager implements Serializable, AutoCloseable {
private static final Logger LOG = LogManager.getLogger(LockManager.class);
- private final HoodieWriteConfig writeConfig;
- private final LockConfiguration lockConfiguration;
- private final SerializableConfiguration hadoopConf;
- private final int maxRetries;
- private final long maxWaitTimeInMs;
+ private HoodieWriteConfig writeConfig;
+ private LockConfiguration lockConfiguration;
+ private SerializableConfiguration hadoopConf;
+ private int maxRetries;
+ private long maxWaitTimeInMs;
private transient HoodieLockMetrics metrics;
private volatile LockProvider lockProvider;
public LockManager(HoodieWriteConfig writeConfig, FileSystem fs) {
+ init(writeConfig, fs.getConf(), writeConfig.getProps());
+ }
+
+ /**
+ * Try to have a lock at partitionPath + fileID level for different write handler.
+ * @param writeConfig
+ * @param fs
+ * @param partitionPath
+ * @param fileId
+ */
+ public LockManager(HoodieWriteConfig writeConfig, FileSystem fs, String partitionPath, String fileId) {
+ TypedProperties props = refreshLockConfig(writeConfig, partitionPath + "/" + fileId);
+ init(writeConfig, fs.getConf(), props);
+ }
+
+ private void init(HoodieWriteConfig writeConfig, Configuration conf, TypedProperties lockProps) {
+ this.lockConfiguration = new LockConfiguration(lockProps);
this.writeConfig = writeConfig;
- this.hadoopConf = new SerializableConfiguration(fs.getConf());
- this.lockConfiguration = new LockConfiguration(writeConfig.getProps());
- maxRetries = lockConfiguration.getConfig().getInteger(LOCK_ACQUIRE_CLIENT_NUM_RETRIES_PROP_KEY,
+ this.hadoopConf = new SerializableConfiguration(conf);
+ this.maxRetries = lockConfiguration.getConfig().getInteger(LOCK_ACQUIRE_CLIENT_NUM_RETRIES_PROP_KEY,
Integer.parseInt(HoodieLockConfig.LOCK_ACQUIRE_CLIENT_NUM_RETRIES.defaultValue()));
- maxWaitTimeInMs = lockConfiguration.getConfig().getLong(LOCK_ACQUIRE_CLIENT_RETRY_WAIT_TIME_IN_MILLIS_PROP_KEY,
+ this.maxWaitTimeInMs = lockConfiguration.getConfig().getLong(LOCK_ACQUIRE_CLIENT_RETRY_WAIT_TIME_IN_MILLIS_PROP_KEY,
Long.parseLong(HoodieLockConfig.LOCK_ACQUIRE_CLIENT_RETRY_WAIT_TIME_IN_MILLIS.defaultValue()));
- metrics = new HoodieLockMetrics(writeConfig);
+ this.metrics = new HoodieLockMetrics(writeConfig);
+ }
+
+ /**
+ * rebuild lock related configs, only support ZK related lock for now.
+ */
+ private TypedProperties refreshLockConfig(HoodieWriteConfig writeConfig, String key) {
Review Comment:
It is better to move this logic of changing lock configs to the transaction manager for the direct makers, as the lock manager itself should not be aware of any higher-layer logic.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/SimpleDirectMarkerBasedEarlyConflictDetectionStrategy.java:
##########
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.table.marker;
+
+import org.apache.hudi.common.conflict.detection.HoodieDirectMarkerBasedEarlyConflictDetectionStrategy;
+import org.apache.hudi.common.fs.HoodieWrapperFileSystem;
+import org.apache.hudi.common.table.timeline.HoodieActiveTimeline;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.MarkerUtils;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.exception.HoodieEarlyConflictDetectionException;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.ConcurrentModificationException;
+import java.util.HashSet;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+/**
+ * This strategy is used for direct marker writers, trying to do early conflict detection.
+ * It will use fileSystem api like list and exist directly to check if there is any marker file conflict.
+ */
+public class SimpleDirectMarkerBasedEarlyConflictDetectionStrategy extends HoodieDirectMarkerBasedEarlyConflictDetectionStrategy {
+
+ private static final Logger LOG = LogManager.getLogger(SimpleDirectMarkerBasedEarlyConflictDetectionStrategy.class);
+
+ public SimpleDirectMarkerBasedEarlyConflictDetectionStrategy(String basePath, HoodieWrapperFileSystem fs, String partitionPath, String fileId, String instantTime,
Review Comment:
nit: the full write config is not needed. We should simply the list of arguments passed in here.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/SimpleDirectMarkerBasedEarlyConflictDetectionStrategy.java:
##########
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.table.marker;
+
+import org.apache.hudi.common.conflict.detection.HoodieDirectMarkerBasedEarlyConflictDetectionStrategy;
+import org.apache.hudi.common.fs.HoodieWrapperFileSystem;
+import org.apache.hudi.common.table.timeline.HoodieActiveTimeline;
+import org.apache.hudi.common.util.MarkerUtils;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.exception.HoodieEarlyConflictDetectionException;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.ConcurrentModificationException;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+/**
+ * This strategy is used for direct marker writers, trying to do early conflict detection.
+ * It will use fileSystem api like list and exist directly to check if there is any marker file conflict.
+ */
+public class SimpleDirectMarkerBasedEarlyConflictDetectionStrategy extends HoodieDirectMarkerBasedEarlyConflictDetectionStrategy {
+
+ private static final Logger LOG = LogManager.getLogger(SimpleDirectMarkerBasedEarlyConflictDetectionStrategy.class);
+
+ public SimpleDirectMarkerBasedEarlyConflictDetectionStrategy(String basePath, HoodieWrapperFileSystem fs, String partitionPath, String fileId, String instantTime,
+ HoodieActiveTimeline activeTimeline, HoodieWriteConfig config, Boolean checkCommitConflict) {
+ super(basePath, fs, partitionPath, fileId, instantTime, activeTimeline, config, checkCommitConflict);
+ }
+
+ @Override
+ public boolean hasMarkerConflict() {
+ try {
+ return checkMarkerConflict(basePath, partitionPath, fileId, fs, instantTime)
+ || (checkCommitConflict && MarkerUtils.hasCommitConflict(Stream.of(fileId).collect(Collectors.toSet()), basePath, completedCommitInstants));
+ } catch (IOException e) {
+ LOG.warn("Exception occurs during create marker file in eager conflict detection mode.");
+ throw new HoodieIOException("Exception occurs during create marker file in eager conflict detection mode.", e);
+ }
+ }
+
+ @Override
+ public void resolveMarkerConflict(String basePath, String partitionPath, String dataFileName) {
+ throw new HoodieEarlyConflictDetectionException(new ConcurrentModificationException("Early conflict detected but cannot resolve conflicts for overlapping writes"));
Review Comment:
As @zhangyue19921010 explained, this is unnecessary because compaction and clustering instants do not go through the early conflict detection logic.
##########
hudi-common/src/main/java/org/apache/hudi/common/util/MarkerUtils.java:
##########
@@ -208,10 +219,116 @@ public static Set<String> readMarkersFromFile(Path markersFilePath, Serializable
fsDataInputStream = fs.open(markersFilePath);
markers = new HashSet<>(FileIOUtils.readAsUTFStringLines(fsDataInputStream));
} catch (IOException e) {
- throw new HoodieIOException("Failed to read MARKERS file " + markersFilePath, e);
+ if (ignoreException) {
+ LOG.warn("IOException occurs during read MARKERS file, ", e);
+ } else {
+ throw new HoodieIOException("Failed to read MARKERS file " + markersFilePath, e);
+ }
} finally {
closeQuietly(fsDataInputStream);
}
return markers;
}
+
+ public static List<Path> getAllMarkerDir(Path tempPath, FileSystem fs) throws IOException {
+ return Arrays.stream(fs.listStatus(tempPath)).map(FileStatus::getPath).collect(Collectors.toList());
+ }
+
+ public static boolean hasCommitConflict(HoodieActiveTimeline activeTimeline, Set<String> currentFileIDs, Set<HoodieInstant> completedCommitInstants) {
+
+ Set<HoodieInstant> currentInstants = activeTimeline.reload().getCommitsTimeline().filterCompletedInstants().getInstants().collect(Collectors.toSet());
+
+ currentInstants.removeAll(completedCommitInstants);
+ Set<String> missingFileIDs = currentInstants.stream().flatMap(instant -> {
+ try {
+ return HoodieCommitMetadata.fromBytes(activeTimeline.getInstantDetails(instant).get(), HoodieCommitMetadata.class)
+ .getFileIdAndRelativePaths().keySet().stream();
+ } catch (Exception e) {
+ return Stream.empty();
+ }
+ }).collect(Collectors.toSet());
+ currentFileIDs.retainAll(missingFileIDs);
+ return !currentFileIDs.isEmpty();
+ }
+
+ /**
+ * Get Candidate Instant to do conflict checking:
+ * 1. Skip current writer related instant(currentInstantTime)
+ * 2. Skip all instants after currentInstantTime
+ * 3. Skip dead writers related instants based on heart-beat
+ * 4. Skip pending compaction instant (For now we don' do early conflict check with compact action)
+ * Because we don't want to let pending compaction block common writer.
+ * @param instants
+ * @return
+ */
+ public static List<String> getCandidateInstants(HoodieActiveTimeline activeTimeline, List<Path> instants, String currentInstantTime,
+ long maxAllowableHeartbeatIntervalInMs, FileSystem fs, String basePath) {
+
+ HoodieActiveTimeline reloadActive = activeTimeline.reload();
+
+ return instants.stream().map(Path::toString).filter(instantPath -> {
+ String instantTime = markerDirToInstantTime(instantPath);
+ return instantTime.compareToIgnoreCase(currentInstantTime) < 0
+ && !reloadActive.filterPendingCompactionTimeline().containsInstant(instantTime)
+ && !reloadActive.filterPendingReplaceTimeline().containsInstant(instantTime);
+ }).filter(instantPath -> {
+ try {
+ return !isHeartbeatExpired(markerDirToInstantTime(instantPath), maxAllowableHeartbeatIntervalInMs, fs, basePath);
+ } catch (IOException e) {
+ return false;
+ }
+ }).collect(Collectors.toList());
+ }
+
+ /**
+ * Get fileID from full marker path, for example:
+ * 20210623/0/20210825/932a86d9-5c1d-44c7-ac99-cb88b8ef8478-0_85-15-1390_20220620181735781.parquet.marker.MERGE
+ * ==> get 20210623/0/20210825/932a86d9-5c1d-44c7-ac99-cb88b8ef8478-0
+ * @param marker
+ * @return
+ */
+ public static String makerToPartitionAndFileID(String marker) {
+ String[] ele = marker.split("_");
+ return ele[0];
+ }
+
+ /**
+ * Get instantTime from full marker path, for example:
+ * /var/folders/t3/th1dw75d0yz2x2k2qt6ys9zh0000gp/T/junit6502909693741900820/dataset/.hoodie/.temp/003
+ * ==> 003
+ * @param marker
+ * @return
+ */
+ public static String markerDirToInstantTime(String marker) {
+ String[] ele = marker.split("/");
+ return ele[ele.length - 1];
+ }
+
+ /**
+ * Use modification time as last heart beat time
+ * @param fs
+ * @param basePath
+ * @param instantTime
+ * @return
+ * @throws IOException
+ */
+ public static Long getLastHeartbeatTime(FileSystem fs, String basePath, String instantTime) throws IOException {
+ Path heartbeatFilePath = new Path(HoodieTableMetaClient.getHeartbeatFolderPath(basePath) + Path.SEPARATOR + instantTime);
+ if (fs.exists(heartbeatFilePath)) {
+ return fs.getFileStatus(heartbeatFilePath).getModificationTime();
+ } else {
+ // NOTE : This can happen when a writer is upgraded to use lazy cleaning and the last write had failed
+ return 0L;
+ }
+ }
+
+ public static boolean isHeartbeatExpired(String instantTime, long maxAllowableHeartbeatIntervalInMs, FileSystem fs, String basePath) throws IOException {
+ Long currentTime = System.currentTimeMillis();
+ Long lastHeartbeatTime = getLastHeartbeatTime(fs, basePath, instantTime);
+ if (currentTime - lastHeartbeatTime > maxAllowableHeartbeatIntervalInMs) {
+ LOG.warn("Heartbeat expired, for instant: " + instantTime);
+ return true;
+ }
+ return false;
+ }
Review Comment:
Given the limitation, now it makes sense to move the `HeartbeatUtils` to `hudi-common`, so common utils can be shared instead of code duplication.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1399075405
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214",
"triggerID" : "1376756472",
"triggerType" : "MANUAL"
}, {
"hash" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422",
"triggerID" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"triggerType" : "PUSH"
}, {
"hash" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14480",
"triggerID" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"triggerType" : "PUSH"
}, {
"hash" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14484",
"triggerID" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"triggerType" : "PUSH"
}, {
"hash" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14518",
"triggerID" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 7344fabacee437adb4b55ca922df0f5fb14ae372 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14484)
* 501e47fb5c554a32acd7d2b410b5cdcf880f6057 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14518)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1399152911
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214",
"triggerID" : "1376756472",
"triggerType" : "MANUAL"
}, {
"hash" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422",
"triggerID" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"triggerType" : "PUSH"
}, {
"hash" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14480",
"triggerID" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"triggerType" : "PUSH"
}, {
"hash" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14484",
"triggerID" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"triggerType" : "PUSH"
}, {
"hash" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14518",
"triggerID" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 501e47fb5c554a32acd7d2b410b5cdcf880f6057 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14518)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1376626838
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* be0d5b4e5a211b1ec9f3bdad65177e4667f9a222 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200)
* 1b837ec65943331f6c805f723b088240ae66dfaf Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1284262865
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 645766d09d376bc46a1e0b6af770d9e84f778b24 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1397907106
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214",
"triggerID" : "1376756472",
"triggerType" : "MANUAL"
}, {
"hash" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422",
"triggerID" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"triggerType" : "PUSH"
}, {
"hash" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 67b3892f4bafe64ecf203a53bb86db22e61ae587 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422)
* a2980b73d4ad32976360804059fcd6df969b9f89 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1107668532
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/WriteMarkers.java:
##########
@@ -52,16 +58,46 @@ public WriteMarkers(String basePath, String markerFolderPath, String instantTime
this.instantTime = instantTime;
}
+ public Option<Path> create(String partitionPath, String dataFileName, IOType type) {
+ return create(partitionPath, dataFileName, type, Option.empty());
+ }
+
/**
* Creates a marker without checking if the marker already exists.
*
* @param partitionPath partition path in the table
* @param dataFileName data file name
* @param type write IO type
+ * @param handler could be empty
* @return the marker path
*/
- public Option<Path> create(String partitionPath, String dataFileName, IOType type) {
- return create(partitionPath, dataFileName, type, false);
+ public Option<Path> create(String partitionPath, String dataFileName, IOType type, Option<HoodieWriteHandle> handler) {
+ boolean checkIfExists = false;
+
+ if (handler.isPresent()
+ && handler.get().getConfig().getWriteConcurrencyMode().supportsOptimisticConcurrencyControl()
+ && handler.get().getConfig().isEarlyConflictDetectionEnable()) {
+
+ HoodieTableMetaClient metaClient = handler.get().getHoodieTableMetaClient();
+ HoodieActiveTimeline activeTimeline = metaClient.getActiveTimeline();
+
+ HoodieTimeline pendingCompactionTimeline = activeTimeline.filterPendingCompactionTimeline();
+ HoodieTimeline pendingReplaceTimeline = activeTimeline.filterPendingReplaceTimeline();
+ // TODO if current is compact or clustering then create marker directly without early conflict detection.
+ // Need to support early conflict detection between table service and common writers.
+ if (pendingCompactionTimeline.containsInstant(instantTime) || pendingReplaceTimeline.containsInstant(instantTime)) {
Review Comment:
The details will be captured by HUDI-5809.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1064931339
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/MarkerHandler.java:
##########
@@ -150,7 +166,52 @@ public boolean doesMarkerDirExist(String markerDir) {
* @param markerName marker name
* @return the {@code CompletableFuture} instance for the request
*/
- public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName) {
+ public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName, String basePath) {
+ // Step1 do early conflict detection if enable
+ if (timelineServiceConfig.earlyConflictDetectionEnable) {
+ try {
+ synchronized (earlyConflictDetectionLock) {
+ if (earlyConflictDetectionStrategy == null) {
+ earlyConflictDetectionStrategy = (HoodieTimelineServerBasedEarlyConflictDetectionStrategy) ReflectionUtils.loadClass(timelineServiceConfig.earlyConflictDetectionStrategy,
+ basePath, markerDir, markerName, timelineServiceConfig.checkCommitConflict);
+ }
+
+ // markerDir => $base_path/.hoodie/.temp/$instant_time
+ // If markerDir is changed like move to the next instant action, we need to fresh this earlyConflictDetectionStrategy.
+ // For specific instant related create marker action, we only call this check/fresh once
+ // instead of starting the conflict detector for every request
+ if (!markerDir.equalsIgnoreCase(currentMarkerDir)) {
+ this.currentMarkerDir = markerDir;
+ Set<String> actions = CollectionUtils.createSet(COMMIT_ACTION, DELTA_COMMIT_ACTION, REPLACE_COMMIT_ACTION);
+ Set<HoodieInstant> oldInstants = new HashSet<>(
+ viewManager.getFileSystemView(basePath)
+ .getTimeline()
+ .filterCompletedInstants()
+ .filter(instant -> actions.contains(instant.getAction()))
+ .getInstants());
+
+ earlyConflictDetectionStrategy.fresh(timelineServiceConfig.asyncConflictDetectorBatchIntervalMs,
+ timelineServiceConfig.asyncConflictDetectorBatchPeriodMs, markerDir, basePath, timelineServiceConfig.maxAllowableHeartbeatIntervalInMs, fileSystem,
+ this, oldInstants);
+ }
+ }
+
+ earlyConflictDetectionStrategy.detectAndResolveConflictIfNecessary();
+
+ } catch (Exception ex) {
Review Comment:
as for exception during early conflict detection, maybe any Exception need to catch and return false as marker creation result.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1064928603
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/SimpleDirectMarkerBasedEarlyConflictDetectionStrategy.java:
##########
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.table.marker;
+
+import org.apache.hudi.common.conflict.detection.HoodieDirectMarkerBasedEarlyConflictDetectionStrategy;
+import org.apache.hudi.common.fs.HoodieWrapperFileSystem;
+import org.apache.hudi.common.table.timeline.HoodieActiveTimeline;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.MarkerUtils;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.exception.HoodieEarlyConflictDetectionException;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.ConcurrentModificationException;
+import java.util.HashSet;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+/**
+ * This strategy is used for direct marker writers, trying to do early conflict detection.
+ * It will use fileSystem api like list and exist directly to check if there is any marker file conflict.
+ */
+public class SimpleDirectMarkerBasedEarlyConflictDetectionStrategy extends HoodieDirectMarkerBasedEarlyConflictDetectionStrategy {
+
+ private static final Logger LOG = LogManager.getLogger(SimpleDirectMarkerBasedEarlyConflictDetectionStrategy.class);
+
+ public SimpleDirectMarkerBasedEarlyConflictDetectionStrategy(String basePath, HoodieWrapperFileSystem fs, String partitionPath, String fileId, String instantTime,
Review Comment:
simplify the args. Removed basePath, checkCommitConflict, maxAllowableHeartbeatIntervalInMs,and completedCommitInstants. We can get all these info from config or timeline.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/WriteMarkers.java:
##########
@@ -52,16 +58,47 @@ public WriteMarkers(String basePath, String markerFolderPath, String instantTime
this.instantTime = instantTime;
}
+ public Option<Path> create(String partitionPath, String dataFileName, IOType type) {
+ return create(partitionPath, dataFileName, type, Option.empty());
+ }
+
/**
* Creates a marker without checking if the marker already exists.
*
* @param partitionPath partition path in the table
* @param dataFileName data file name
* @param type write IO type
+ * @param handler could be empty
* @return the marker path
*/
- public Option<Path> create(String partitionPath, String dataFileName, IOType type) {
- return create(partitionPath, dataFileName, type, false);
+ public Option<Path> create(String partitionPath, String dataFileName, IOType type, Option<HoodieWriteHandle> handler) {
Review Comment:
changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1283538105
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 64819e4ab1ed12788e126072fa50e964fc62c257 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203)
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028233418
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/TimelineServerBasedWriteMarkers.java:
##########
@@ -132,14 +143,54 @@ protected Option<Path> create(String partitionPath, String dataFileName, IOType
HoodieTimer timer = HoodieTimer.start();
String markerFileName = getMarkerFileName(dataFileName, type);
- Map<String, String> paramsMap = new HashMap<>();
+ Map<String, String> paramsMap = initConfigMap(partitionPath, markerFileName);
+ boolean success = executeCreateMarkerRequest(paramsMap, partitionPath, markerFileName);
+ LOG.info("[timeline-server-based] Created marker file " + partitionPath + "/" + markerFileName
+ + " in " + timer.endTimer() + " ms");
+ if (success) {
+ return Option.of(new Path(FSUtils.getPartitionPath(markerDirPath, partitionPath), markerFileName));
+ } else {
+ return Option.empty();
+ }
+ }
+
+ @Override
+ public Option<Path> createWithEarlyConflictDetection(String partitionPath, String dataFileName, IOType type, boolean checkIfExists,
+ HoodieEarlyConflictDetectionStrategy resolutionStrategy,
+ Set<HoodieInstant> completedCommitInstants, HoodieWriteConfig config, String fileId) {
+ HoodieTimer timer = new HoodieTimer().startTimer();
+ String markerFileName = getMarkerFileName(dataFileName, type);
+ Map<String, String> paramsMap = initConfigMap(partitionPath, markerFileName);
+
+ paramsMap.put(MARKER_CONFLICT_CHECKER_BATCH_INTERVAL, config.getMarkerConflictCheckerBatchInterval());
+ paramsMap.put(MARKER_CONFLICT_CHECKER_PERIOD, config.getMarkerConflictCheckerPeriod());
Review Comment:
Changed.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/TimelineServerBasedWriteMarkers.java:
##########
@@ -132,14 +143,54 @@ protected Option<Path> create(String partitionPath, String dataFileName, IOType
HoodieTimer timer = HoodieTimer.start();
String markerFileName = getMarkerFileName(dataFileName, type);
- Map<String, String> paramsMap = new HashMap<>();
+ Map<String, String> paramsMap = initConfigMap(partitionPath, markerFileName);
+ boolean success = executeCreateMarkerRequest(paramsMap, partitionPath, markerFileName);
+ LOG.info("[timeline-server-based] Created marker file " + partitionPath + "/" + markerFileName
+ + " in " + timer.endTimer() + " ms");
+ if (success) {
+ return Option.of(new Path(FSUtils.getPartitionPath(markerDirPath, partitionPath), markerFileName));
+ } else {
+ return Option.empty();
+ }
+ }
+
+ @Override
+ public Option<Path> createWithEarlyConflictDetection(String partitionPath, String dataFileName, IOType type, boolean checkIfExists,
+ HoodieEarlyConflictDetectionStrategy resolutionStrategy,
+ Set<HoodieInstant> completedCommitInstants, HoodieWriteConfig config, String fileId) {
+ HoodieTimer timer = new HoodieTimer().startTimer();
+ String markerFileName = getMarkerFileName(dataFileName, type);
+ Map<String, String> paramsMap = initConfigMap(partitionPath, markerFileName);
+
+ paramsMap.put(MARKER_CONFLICT_CHECKER_BATCH_INTERVAL, config.getMarkerConflictCheckerBatchInterval());
+ paramsMap.put(MARKER_CONFLICT_CHECKER_PERIOD, config.getMarkerConflictCheckerPeriod());
paramsMap.put(MARKER_DIR_PATH_PARAM, markerDirPath.toString());
Review Comment:
Changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028231466
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/TransactionManager.java:
##########
@@ -67,6 +80,14 @@ public void endTransaction(Option<HoodieInstant> currentTxnOwnerInstant) {
}
}
+ public void endTransaction(String filePath) {
Review Comment:
Sure, changed.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieLockConfig.java:
##########
@@ -189,6 +190,33 @@ public class HoodieLockConfig extends HoodieConfig {
.withDocumentation("Lock provider class name, this should be subclass of "
+ "org.apache.hudi.client.transaction.ConflictResolutionStrategy");
+ // Pluggable strategies to use when early conflict detection
+ public static final ConfigProperty<String> EARLY_CONFLICT_DETECTION_STRATEGY_CLASS_NAME = ConfigProperty
+ .key(LOCK_PREFIX + "early.conflict.detection.strategy")
+ .defaultValue(AsyncTimelineMarkerEarlyConflictDetectionStrategy.class.getName())
+ .sinceVersion("0.12.0")
+ .withDocumentation("Early conflict detection class name, this should be subclass of "
+ + "org.apache.hudi.common.conflict.detection.HoodieEarlyConflictDetectionStrategy");
+
+ public static final ConfigProperty<Boolean> EARLY_CONFLICT_DETECTION_ENABLE = ConfigProperty
+ .key(LOCK_PREFIX + "early.conflict.detection.enable")
+ .defaultValue(false)
+ .sinceVersion("0.12.0")
+ .withDocumentation("Enable early conflict detection based on markers. It will try to detect writing conflict before create markers and fast fail"
+ + " which will release cluster resources as soon as possible.");
+
+ public static final ConfigProperty<Long> MARKER_CONFLICT_CHECKER_BATCH_INTERVAL = ConfigProperty
+ .key(LOCK_PREFIX + "early.conflict.async.checker.batch.interval")
+ .defaultValue(30000L)
+ .sinceVersion("0.12.0")
+ .withDocumentation("Used for timeline based marker AsyncTimelineMarkerConflictResolutionStrategy. The time to delay first async marker conflict checking.");
+
+ public static final ConfigProperty<Long> MARKER_CONFLICT_CHECKER_PERIOD = ConfigProperty
+ .key(LOCK_PREFIX + "early.conflict.async.checker.period")
+ .defaultValue(30000L)
+ .sinceVersion("0.12.0")
+ .withDocumentation("Used for timeline based marker AsyncTimelineMarkerConflictResolutionStrategy. The period between each marker conflict checking.");
Review Comment:
all changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1323660324
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 8a402c4c8574cca66047e614038a002cb24e8e94 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182)
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* b97bb164efb5264476ebdf3673396583ec90d1a6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028255753
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/MarkerHandler.java:
##########
@@ -150,7 +168,50 @@ public boolean doesMarkerDirExist(String markerDir) {
* @param markerName marker name
* @return the {@code CompletableFuture} instance for the request
*/
- public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName) {
+ public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName,
+ String batchInterval, String period, String maxAllowableHeartbeatIntervalInMs,
+ String basePath, String earlyConflictDetectionEnable,
+ String earlyConflictDetectionClassName) {
+ // Step1 do early conflict detection if enable
+ if (Boolean.parseBoolean(earlyConflictDetectionEnable)) {
+ try {
+ synchronized (earlyConflictDetectionLock) {
+ if (earlyConflictDetectionStrategy == null) {
+ earlyConflictDetectionStrategy = ReflectionUtils.loadClass(earlyConflictDetectionClassName);
+ }
+
+ if (!markerDir.equalsIgnoreCase(currentMarkerDir)) {
+ this.currentMarkerDir = markerDir;
+ Set<String> actions = CollectionUtils.createSet(COMMIT_ACTION, DELTA_COMMIT_ACTION, REPLACE_COMMIT_ACTION);
+ Set<HoodieInstant> oldInstants = viewManager.getFileSystemView(basePath)
+ .getTimeline()
+ .filterCompletedInstants()
+ .filter(instant -> actions.contains(instant.getAction()))
+ .getInstants()
+ .collect(Collectors.toSet());
+
+ earlyConflictDetectionStrategy.fresh(batchInterval, period, markerDir, basePath, maxAllowableHeartbeatIntervalInMs, fileSystem,
+ this, oldInstants);
+ }
+ }
+
+ if (earlyConflictDetectionStrategy.hasMarkerConflict()) {
+ earlyConflictDetectionStrategy.resolveMarkerConflict(basePath, markerDir, markerName);
+ }
Review Comment:
As we discuss before, checker is doing the detection in a batching and async way. So that it's not related to specific batches requests. For example timeline service is deal with marker1, marker2 and marker3. But this checker find a conflict on marker0.
So could we do this check and get check result at api level? what do you think.
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/MarkerHandler.java:
##########
@@ -150,7 +168,50 @@ public boolean doesMarkerDirExist(String markerDir) {
* @param markerName marker name
* @return the {@code CompletableFuture} instance for the request
*/
- public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName) {
+ public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName,
+ String batchInterval, String period, String maxAllowableHeartbeatIntervalInMs,
+ String basePath, String earlyConflictDetectionEnable,
+ String earlyConflictDetectionClassName) {
+ // Step1 do early conflict detection if enable
+ if (Boolean.parseBoolean(earlyConflictDetectionEnable)) {
+ try {
+ synchronized (earlyConflictDetectionLock) {
+ if (earlyConflictDetectionStrategy == null) {
+ earlyConflictDetectionStrategy = ReflectionUtils.loadClass(earlyConflictDetectionClassName);
+ }
+
+ if (!markerDir.equalsIgnoreCase(currentMarkerDir)) {
+ this.currentMarkerDir = markerDir;
+ Set<String> actions = CollectionUtils.createSet(COMMIT_ACTION, DELTA_COMMIT_ACTION, REPLACE_COMMIT_ACTION);
+ Set<HoodieInstant> oldInstants = viewManager.getFileSystemView(basePath)
+ .getTimeline()
+ .filterCompletedInstants()
+ .filter(instant -> actions.contains(instant.getAction()))
+ .getInstants()
+ .collect(Collectors.toSet());
+
+ earlyConflictDetectionStrategy.fresh(batchInterval, period, markerDir, basePath, maxAllowableHeartbeatIntervalInMs, fileSystem,
+ this, oldInstants);
+ }
+ }
+
+ if (earlyConflictDetectionStrategy.hasMarkerConflict()) {
+ earlyConflictDetectionStrategy.resolveMarkerConflict(basePath, markerDir, markerName);
+ }
Review Comment:
As we discuss before, checker is doing the detection in a batching and async way. So that it's not related to specific batches requests. For example timeline service is deal with marker1, marker2 and marker3. But this checker find a conflict on marker0.
So could we do this check and get check result at api level? what do you think :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1322362124
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 71e0d1e91d2f540b441838263362c31251fa4ab5 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149)
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 316e5aefa89ec2dc35928af7c7da4b11a23aab20 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028281267
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/marker/MarkerCheckerRunnable.java:
##########
@@ -0,0 +1,193 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.timeline.service.handlers.marker;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.common.model.HoodieCommitMetadata;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.HoodieTimer;
+import org.apache.hudi.common.util.MarkerUtils;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.hudi.timeline.service.handlers.MarkerHandler;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+public class MarkerCheckerRunnable implements Runnable {
+ private static final Logger LOG = LogManager.getLogger(MarkerCheckerRunnable.class);
+
+ private MarkerHandler markerHandler;
+ private String markerDir;
+ private String basePath;
+ private FileSystem fs;
+ private AtomicBoolean hasConflict;
+ private long maxAllowableHeartbeatIntervalInMs;
+ private Set<HoodieInstant> oldInstants;
+
+ public MarkerCheckerRunnable(AtomicBoolean hasConflict, MarkerHandler markerHandler, String markerDir,
Review Comment:
Added. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1029454364
##########
hudi-common/src/main/java/org/apache/hudi/common/util/MarkerUtils.java:
##########
@@ -208,10 +219,116 @@ public static Set<String> readMarkersFromFile(Path markersFilePath, Serializable
fsDataInputStream = fs.open(markersFilePath);
markers = new HashSet<>(FileIOUtils.readAsUTFStringLines(fsDataInputStream));
} catch (IOException e) {
- throw new HoodieIOException("Failed to read MARKERS file " + markersFilePath, e);
+ if (ignoreException) {
+ LOG.warn("IOException occurs during read MARKERS file, ", e);
+ } else {
+ throw new HoodieIOException("Failed to read MARKERS file " + markersFilePath, e);
+ }
} finally {
closeQuietly(fsDataInputStream);
}
return markers;
}
+
+ public static List<Path> getAllMarkerDir(Path tempPath, FileSystem fs) throws IOException {
+ return Arrays.stream(fs.listStatus(tempPath)).map(FileStatus::getPath).collect(Collectors.toList());
+ }
+
+ public static boolean hasCommitConflict(HoodieActiveTimeline activeTimeline, Set<String> currentFileIDs, Set<HoodieInstant> completedCommitInstants) {
+
+ Set<HoodieInstant> currentInstants = activeTimeline.reload().getCommitsTimeline().filterCompletedInstants().getInstants().collect(Collectors.toSet());
+
+ currentInstants.removeAll(completedCommitInstants);
+ Set<String> missingFileIDs = currentInstants.stream().flatMap(instant -> {
+ try {
+ return HoodieCommitMetadata.fromBytes(activeTimeline.getInstantDetails(instant).get(), HoodieCommitMetadata.class)
+ .getFileIdAndRelativePaths().keySet().stream();
+ } catch (Exception e) {
+ return Stream.empty();
+ }
+ }).collect(Collectors.toSet());
+ currentFileIDs.retainAll(missingFileIDs);
+ return !currentFileIDs.isEmpty();
+ }
+
+ /**
+ * Get Candidate Instant to do conflict checking:
+ * 1. Skip current writer related instant(currentInstantTime)
+ * 2. Skip all instants after currentInstantTime
+ * 3. Skip dead writers related instants based on heart-beat
+ * 4. Skip pending compaction instant (For now we don' do early conflict check with compact action)
+ * Because we don't want to let pending compaction block common writer.
+ * @param instants
+ * @return
+ */
+ public static List<String> getCandidateInstants(HoodieActiveTimeline activeTimeline, List<Path> instants, String currentInstantTime,
+ long maxAllowableHeartbeatIntervalInMs, FileSystem fs, String basePath) {
+
+ HoodieActiveTimeline reloadActive = activeTimeline.reload();
+
+ return instants.stream().map(Path::toString).filter(instantPath -> {
+ String instantTime = markerDirToInstantTime(instantPath);
+ return instantTime.compareToIgnoreCase(currentInstantTime) < 0
+ && !reloadActive.filterPendingCompactionTimeline().containsInstant(instantTime)
+ && !reloadActive.filterPendingReplaceTimeline().containsInstant(instantTime);
+ }).filter(instantPath -> {
+ try {
+ return !isHeartbeatExpired(markerDirToInstantTime(instantPath), maxAllowableHeartbeatIntervalInMs, fs, basePath);
+ } catch (IOException e) {
+ return false;
+ }
+ }).collect(Collectors.toList());
+ }
+
+ /**
+ * Get fileID from full marker path, for example:
+ * 20210623/0/20210825/932a86d9-5c1d-44c7-ac99-cb88b8ef8478-0_85-15-1390_20220620181735781.parquet.marker.MERGE
+ * ==> get 20210623/0/20210825/932a86d9-5c1d-44c7-ac99-cb88b8ef8478-0
+ * @param marker
+ * @return
+ */
+ public static String makerToPartitionAndFileID(String marker) {
+ String[] ele = marker.split("_");
+ return ele[0];
+ }
+
+ /**
+ * Get instantTime from full marker path, for example:
+ * /var/folders/t3/th1dw75d0yz2x2k2qt6ys9zh0000gp/T/junit6502909693741900820/dataset/.hoodie/.temp/003
+ * ==> 003
+ * @param marker
+ * @return
+ */
+ public static String markerDirToInstantTime(String marker) {
+ String[] ele = marker.split("/");
+ return ele[ele.length - 1];
+ }
+
+ /**
+ * Use modification time as last heart beat time
+ * @param fs
+ * @param basePath
+ * @param instantTime
+ * @return
+ * @throws IOException
+ */
+ public static Long getLastHeartbeatTime(FileSystem fs, String basePath, String instantTime) throws IOException {
+ Path heartbeatFilePath = new Path(HoodieTableMetaClient.getHeartbeatFolderPath(basePath) + Path.SEPARATOR + instantTime);
+ if (fs.exists(heartbeatFilePath)) {
+ return fs.getFileStatus(heartbeatFilePath).getModificationTime();
+ } else {
+ // NOTE : This can happen when a writer is upgraded to use lazy cleaning and the last write had failed
+ return 0L;
+ }
+ }
+
+ public static boolean isHeartbeatExpired(String instantTime, long maxAllowableHeartbeatIntervalInMs, FileSystem fs, String basePath) throws IOException {
+ Long currentTime = System.currentTimeMillis();
+ Long lastHeartbeatTime = getLastHeartbeatTime(fs, basePath, instantTime);
+ if (currentTime - lastHeartbeatTime > maxAllowableHeartbeatIntervalInMs) {
+ LOG.warn("Heartbeat expired, for instant: " + instantTime);
+ return true;
+ }
+ return false;
+ }
Review Comment:
These methods in hudi-common are the same as org.apache.hudi.client.heartbeat.xxx under hudi-client-common
We can not add hudi-client-common as a dependency in hudi-common. So just copy necessary methods here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1291798345
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* e23ab613d4aad3844374b21d987f95ca6f5183e5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1235086904
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 64819e4ab1ed12788e126072fa50e964fc62c257 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] XuQianJin-Stars commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
XuQianJin-Stars commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1235062563
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1376737665
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 1b837ec65943331f6c805f723b088240ae66dfaf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1376620316
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* be0d5b4e5a211b1ec9f3bdad65177e4667f9a222 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200)
* 1b837ec65943331f6c805f723b088240ae66dfaf UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1377058777
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214",
"triggerID" : "1376756472",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 1b837ec65943331f6c805f723b088240ae66dfaf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207) Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1064930126
##########
hudi-common/src/main/java/org/apache/hudi/common/util/MarkerUtils.java:
##########
@@ -208,10 +220,89 @@ public static Set<String> readMarkersFromFile(Path markersFilePath, Serializable
fsDataInputStream = fs.open(markersFilePath);
markers = new HashSet<>(FileIOUtils.readAsUTFStringLines(fsDataInputStream));
} catch (IOException e) {
- throw new HoodieIOException("Failed to read MARKERS file " + markersFilePath, e);
+ if (ignoreException) {
+ LOG.warn("IOException occurs during read MARKERS file, ", e);
+ } else {
+ throw new HoodieIOException("Failed to read MARKERS file " + markersFilePath, e);
+ }
} finally {
closeQuietly(fsDataInputStream);
}
return markers;
}
+
+ public static List<Path> getAllMarkerDir(Path tempPath, FileSystem fs) throws IOException {
+ return Arrays.stream(fs.listStatus(tempPath)).map(FileStatus::getPath).collect(Collectors.toList());
+ }
+
+ public static boolean hasCommitConflict(HoodieActiveTimeline activeTimeline, Set<String> currentFileIDs, Set<HoodieInstant> completedCommitInstants) {
+
+ Set<HoodieInstant> currentInstants = new HashSet<>(
+ activeTimeline.reload().getCommitsTimeline().filterCompletedInstants().getInstants());
+
+ currentInstants.removeAll(completedCommitInstants);
+ Set<String> missingFileIDs = currentInstants.stream().flatMap(instant -> {
+ try {
+ return HoodieCommitMetadata.fromBytes(activeTimeline.getInstantDetails(instant).get(), HoodieCommitMetadata.class)
+ .getFileIdAndRelativePaths().keySet().stream();
+ } catch (Exception e) {
+ return Stream.empty();
+ }
+ }).collect(Collectors.toSet());
+ currentFileIDs.retainAll(missingFileIDs);
+ return !currentFileIDs.isEmpty();
+ }
+
+ /**
+ * Get Candidate Instant to do conflict checking:
+ * 1. Skip current writer related instant(currentInstantTime)
+ * 2. Skip all instants after currentInstantTime
+ * 3. Skip dead writers related instants based on heart-beat
+ * 4. Skip pending compaction instant (For now we don' do early conflict check with compact action)
+ * Because we don't want to let pending compaction block common writer.
+ * @param instants
+ * @return
+ */
+ public static List<String> getCandidateInstants(HoodieActiveTimeline activeTimeline, List<Path> instants, String currentInstantTime,
+ long maxAllowableHeartbeatIntervalInMs, FileSystem fs, String basePath) {
+
+ HoodieActiveTimeline reloadActive = activeTimeline.reload();
Review Comment:
Nice catch here. No need actually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1399862691
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214",
"triggerID" : "1376756472",
"triggerType" : "MANUAL"
}, {
"hash" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422",
"triggerID" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"triggerType" : "PUSH"
}, {
"hash" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14480",
"triggerID" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"triggerType" : "PUSH"
}, {
"hash" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14484",
"triggerID" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"triggerType" : "PUSH"
}, {
"hash" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14518",
"triggerID" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"triggerType" : "PUSH"
}, {
"hash" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14549",
"triggerID" : "1399842193",
"triggerType" : "MANUAL"
}, {
"hash" : "0a77616b5bd7b7d20259b3eb12d31e571f2af60e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0a77616b5bd7b7d20259b3eb12d31e571f2af60e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 501e47fb5c554a32acd7d2b410b5cdcf880f6057 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14518) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14549)
* 0a77616b5bd7b7d20259b3eb12d31e571f2af60e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1284215076
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1192488902
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 66b7d1b75c7a201b5d108cd48a70363dd00a1795 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034)
* 64819e4ab1ed12788e126072fa50e964fc62c257 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1192485236
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 66b7d1b75c7a201b5d108cd48a70363dd00a1795 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034)
* 64819e4ab1ed12788e126072fa50e964fc62c257 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1312147303
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* e23ab613d4aad3844374b21d987f95ca6f5183e5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584) Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600)
* 465536ff2afcf217c19ca00b4a90a5e04d372bc3 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1321906282
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 465536ff2afcf217c19ca00b4a90a5e04d372bc3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975) Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976)
* 844b10a447845b27056b2520b67a624a0d1bd71a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028259582
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/marker/MarkerCheckerRunnable.java:
##########
@@ -0,0 +1,193 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.timeline.service.handlers.marker;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.common.model.HoodieCommitMetadata;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.HoodieTimer;
+import org.apache.hudi.common.util.MarkerUtils;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.hudi.timeline.service.handlers.MarkerHandler;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+public class MarkerCheckerRunnable implements Runnable {
Review Comment:
Added!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1322452327
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 316e5aefa89ec2dc35928af7c7da4b11a23aab20 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155)
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* ffd8315ca6dc2c771e90b5fa983c0b41010605eb UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1312267303
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 465536ff2afcf217c19ca00b4a90a5e04d372bc3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1312509736
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 465536ff2afcf217c19ca00b4a90a5e04d372bc3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975) Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1187182864
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fcaaf9d6dc063aed48319e335c181e913ac9106f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031)
* 553fb00b590e9586ade21e86569ccb30d8dec7a9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033)
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1187205118
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fcaaf9d6dc063aed48319e335c181e913ac9106f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031)
* 553fb00b590e9586ade21e86569ccb30d8dec7a9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033)
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 66b7d1b75c7a201b5d108cd48a70363dd00a1795 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1187050041
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fcaaf9d6dc063aed48319e335c181e913ac9106f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031)
* 553fb00b590e9586ade21e86569ccb30d8dec7a9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028265894
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/marker/MarkerCheckerRunnable.java:
##########
@@ -0,0 +1,193 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.timeline.service.handlers.marker;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.common.model.HoodieCommitMetadata;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.HoodieTimer;
+import org.apache.hudi.common.util.MarkerUtils;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.hudi.timeline.service.handlers.MarkerHandler;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+public class MarkerCheckerRunnable implements Runnable {
+ private static final Logger LOG = LogManager.getLogger(MarkerCheckerRunnable.class);
+
+ private MarkerHandler markerHandler;
+ private String markerDir;
+ private String basePath;
+ private FileSystem fs;
+ private AtomicBoolean hasConflict;
+ private long maxAllowableHeartbeatIntervalInMs;
+ private Set<HoodieInstant> oldInstants;
+
+ public MarkerCheckerRunnable(AtomicBoolean hasConflict, MarkerHandler markerHandler, String markerDir,
+ String basePath, FileSystem fileSystem, long maxAllowableHeartbeatIntervalInMs,
+ Set<HoodieInstant> oldInstants) {
+ this.markerHandler = markerHandler;
+ this.markerDir = markerDir;
+ this.basePath = basePath;
+ this.fs = fileSystem;
+ this.hasConflict = hasConflict;
+ this.maxAllowableHeartbeatIntervalInMs = maxAllowableHeartbeatIntervalInMs;
+ this.oldInstants = oldInstants;
+ }
+
+ @Override
+ public void run() {
+ try {
+ if (!fs.exists(new Path(markerDir))) {
+ return;
+ }
+
+ HoodieTimer timer = new HoodieTimer().startTimer();
+ Set<String> currentInstantAllMarkers = markerHandler.getAllMarkers(markerDir);
+ Path tempPath = new Path(basePath + Path.SEPARATOR + HoodieTableMetaClient.TEMPFOLDER_NAME);
+
+ List<Path> instants = MarkerUtils.getAllMarkerDir(tempPath, fs);
+ List<String> candidate = getCandidateInstants(instants, markerDirToInstantTime(markerDir));
+ Set<String> tableMarkers = candidate.stream().flatMap(instant -> {
+ return MarkerUtils.readTimelineServerBasedMarkersFromFileSystemLocally(instant, fs).stream();
+ }).collect(Collectors.toSet());
+
+ Set<String> currentFileIDs = currentInstantAllMarkers.stream().map(this::makerToPartitionAndFileID).collect(Collectors.toSet());
+ Set<String> tableFilesIDs = tableMarkers.stream().map(this::makerToPartitionAndFileID).collect(Collectors.toSet());
+
+ currentFileIDs.retainAll(tableFilesIDs);
+
+ if (!currentFileIDs.isEmpty() || hasCommitConflict(currentInstantAllMarkers, basePath)) {
+ LOG.warn("Conflict writing detected based on markers!\n"
+ + "Conflict markers: " + currentInstantAllMarkers + "\n"
+ + "Table markers: " + tableMarkers);
+ hasConflict.compareAndSet(false, true);
+ }
+ LOG.info("Finish batch marker checker in " + timer.endTimer() + " ms");
+
+ } catch (IOException e) {
+ throw new HoodieIOException("IOException occurs during checking marker conflict");
+ }
+ }
+
+ /**
+ * Get Candidate Instant to do conflict checking:
+ * 1. Skip current writer related instant(currentInstantTime)
+ * 2. Skip all instants after currentInstantTime
+ * 3. Skip dead writers related instants based on heart-beat
+ * @param instants
+ * @return
+ */
+ private List<String> getCandidateInstants(List<Path> instants, String currentInstantTime) {
Review Comment:
Also Thanks for your reviewing here! Really appreciate!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028247739
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/MarkerHandler.java:
##########
@@ -150,7 +168,50 @@ public boolean doesMarkerDirExist(String markerDir) {
* @param markerName marker name
* @return the {@code CompletableFuture} instance for the request
*/
- public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName) {
+ public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName,
+ String batchInterval, String period, String maxAllowableHeartbeatIntervalInMs,
+ String basePath, String earlyConflictDetectionEnable,
+ String earlyConflictDetectionClassName) {
+ // Step1 do early conflict detection if enable
+ if (Boolean.parseBoolean(earlyConflictDetectionEnable)) {
+ try {
+ synchronized (earlyConflictDetectionLock) {
+ if (earlyConflictDetectionStrategy == null) {
+ earlyConflictDetectionStrategy = ReflectionUtils.loadClass(earlyConflictDetectionClassName);
+ }
Review Comment:
Yeap, for now we init detection thread in the first time call this create marker API, and only fresh this thread when the instant time changed.
We can also init this thread in constructor, but for now, we don't pass hoodieWriteConfig in MarkerHandler..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1323779890
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 8a402c4c8574cca66047e614038a002cb24e8e94 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182)
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* b97bb164efb5264476ebdf3673396583ec90d1a6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185)
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1322215255
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 71e0d1e91d2f540b441838263362c31251fa4ab5 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1284760321
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 645766d09d376bc46a1e0b6af770d9e84f778b24 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1029439999
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/WriteMarkers.java:
##########
@@ -52,16 +58,46 @@ public WriteMarkers(String basePath, String markerFolderPath, String instantTime
this.instantTime = instantTime;
}
+ public Option<Path> create(String partitionPath, String dataFileName, IOType type) {
+ return create(partitionPath, dataFileName, type, Option.empty());
+ }
+
/**
* Creates a marker without checking if the marker already exists.
*
* @param partitionPath partition path in the table
* @param dataFileName data file name
* @param type write IO type
+ * @param handler could be empty
* @return the marker path
*/
- public Option<Path> create(String partitionPath, String dataFileName, IOType type) {
- return create(partitionPath, dataFileName, type, false);
+ public Option<Path> create(String partitionPath, String dataFileName, IOType type, Option<HoodieWriteHandle> handler) {
+ boolean checkIfExists = false;
+
+ if (handler.isPresent()
+ && handler.get().getConfig().getWriteConcurrencyMode().supportsOptimisticConcurrencyControl()
+ && handler.get().getConfig().isEarlyConflictDetectionEnable()) {
+
+ HoodieTableMetaClient metaClient = handler.get().getHoodieTableMetaClient();
+ HoodieActiveTimeline activeTimeline = metaClient.getActiveTimeline();
+
+ HoodieTimeline pendingCompactionTimeline = activeTimeline.filterPendingCompactionTimeline();
+ HoodieTimeline pendingReplaceTimeline = activeTimeline.filterPendingReplaceTimeline();
+ // TODO if current is compact or clustering then create marker directly without early conflict detection.
+ // Need to support early conflict detection between table service and common writers.
+ if (pendingCompactionTimeline.containsInstant(instantTime) || pendingReplaceTimeline.containsInstant(instantTime)) {
Review Comment:
Compaction and Clustering will create markers directly without any early conflict detection.
Bcz these hoodie table service can be async and not depend on multi-writer concurrency control.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1323885753
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* b97bb164efb5264476ebdf3673396583ec90d1a6 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185)
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 869baf7be4f626d10ca20606f7816b941203ec3a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1324475418
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 869baf7be4f626d10ca20606f7816b941203ec3a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189)
* 0447a7135e0cf136cd894291a1efd4374c3efd66 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1400733512
The Azure CI run with the feature flag turned on by default (#7703) has succeeded. The CI failure of this PR is due to flaky tests. Merging this PR.
<img width="2273" alt="Screen Shot 2023-01-23 at 09 37 28" src="https://user-images.githubusercontent.com/2497195/214110457-0595a20b-45d1-4b65-8ba8-c52fb3dea7eb.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua merged pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua merged PR #6133:
URL: https://github.com/apache/hudi/pull/6133
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1312460508
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 465536ff2afcf217c19ca00b4a90a5e04d372bc3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1323643652
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* ffd8315ca6dc2c771e90b5fa983c0b41010605eb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157)
* 8a402c4c8574cca66047e614038a002cb24e8e94 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182)
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028238749
##########
hudi-common/src/main/java/org/apache/hudi/common/conflict/detection/HoodieDirectMarkerBasedEarlyConflictDetectionStrategy.java:
##########
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.conflict.detection;
+
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.common.model.HoodieCommitMetadata;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+public abstract class HoodieDirectMarkerBasedEarlyConflictDetectionStrategy implements HoodieEarlyConflictDetectionStrategy {
+ private static final Logger LOG = LogManager.getLogger(HoodieDirectMarkerBasedEarlyConflictDetectionStrategy.class);
+
+ public abstract boolean hasMarkerConflict(String basePath, FileSystem fs, String partitionPath, String dataFileName, String instantTime,
+ Set<HoodieInstant> completedCommitInstants, HoodieTableMetaClient metaClient);
+
+ public abstract void resolveMarkerConflict(String basePath, String partitionPath, String dataFileName);
+
+ /**
+ * We need to do list operation here.
+ * In order to reduce the list pressure as much as possible, first we build path prefix in advance: '$base_path/.temp/instant_time/partition_path',
+ * and only list these specific partition_paths we need instead of list all the '$base_path/.temp/'
+ * @param basePath
+ * @param partitionPath
+ * @param fileId 162b13d7-9530-48cf-88a4-02241817ae0c-0_1-74-100_003.parquet
+ * @return true if current fileID is already existed under .temp/instant_time/partition_path/..
+ * @throws IOException
+ */
+ public boolean checkMarkerConflict(String basePath, String partitionPath, String fileId,
+ FileSystem fs, String instantTime) throws IOException {
+ String tempFolderPath = getTempFolderPath(basePath);
+ long res = Arrays.stream(fs.listStatus(new Path(tempFolderPath)))
+ .parallel()
+ .map(FileStatus::getPath)
+ .filter(markerPath -> {
+ return !markerPath.getName().equalsIgnoreCase(instantTime);
+ })
Review Comment:
oh, maybe the list action here is necessary. Because we need to get all the inflight instant under .temp. But HoodieTableMetaClient::getMarkerFolderPath(String instantTs) can only get the path related current instantTs.
##########
hudi-common/src/main/java/org/apache/hudi/common/conflict/detection/HoodieEarlyConflictDetectionStrategy.java:
##########
@@ -0,0 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.conflict.detection;
+
+public interface HoodieEarlyConflictDetectionStrategy {
Review Comment:
Sure thing. Changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1323542158
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* ffd8315ca6dc2c771e90b5fa983c0b41010605eb Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157)
* 8a402c4c8574cca66047e614038a002cb24e8e94 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1322006760
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 465536ff2afcf217c19ca00b4a90a5e04d372bc3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975) Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976)
* 844b10a447845b27056b2520b67a624a0d1bd71a Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148)
* 71e0d1e91d2f540b441838263362c31251fa4ab5 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028264254
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/marker/MarkerCheckerRunnable.java:
##########
@@ -0,0 +1,193 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.timeline.service.handlers.marker;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.common.model.HoodieCommitMetadata;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.HoodieTimer;
+import org.apache.hudi.common.util.MarkerUtils;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.hudi.timeline.service.handlers.MarkerHandler;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+public class MarkerCheckerRunnable implements Runnable {
+ private static final Logger LOG = LogManager.getLogger(MarkerCheckerRunnable.class);
+
+ private MarkerHandler markerHandler;
+ private String markerDir;
+ private String basePath;
+ private FileSystem fs;
+ private AtomicBoolean hasConflict;
+ private long maxAllowableHeartbeatIntervalInMs;
+ private Set<HoodieInstant> oldInstants;
+
+ public MarkerCheckerRunnable(AtomicBoolean hasConflict, MarkerHandler markerHandler, String markerDir,
+ String basePath, FileSystem fileSystem, long maxAllowableHeartbeatIntervalInMs,
+ Set<HoodieInstant> oldInstants) {
+ this.markerHandler = markerHandler;
+ this.markerDir = markerDir;
+ this.basePath = basePath;
+ this.fs = fileSystem;
+ this.hasConflict = hasConflict;
+ this.maxAllowableHeartbeatIntervalInMs = maxAllowableHeartbeatIntervalInMs;
+ this.oldInstants = oldInstants;
+ }
+
+ @Override
+ public void run() {
+ try {
+ if (!fs.exists(new Path(markerDir))) {
+ return;
+ }
+
+ HoodieTimer timer = new HoodieTimer().startTimer();
+ Set<String> currentInstantAllMarkers = markerHandler.getAllMarkers(markerDir);
+ Path tempPath = new Path(basePath + Path.SEPARATOR + HoodieTableMetaClient.TEMPFOLDER_NAME);
+
+ List<Path> instants = MarkerUtils.getAllMarkerDir(tempPath, fs);
+ List<String> candidate = getCandidateInstants(instants, markerDirToInstantTime(markerDir));
+ Set<String> tableMarkers = candidate.stream().flatMap(instant -> {
+ return MarkerUtils.readTimelineServerBasedMarkersFromFileSystemLocally(instant, fs).stream();
+ }).collect(Collectors.toSet());
+
+ Set<String> currentFileIDs = currentInstantAllMarkers.stream().map(this::makerToPartitionAndFileID).collect(Collectors.toSet());
+ Set<String> tableFilesIDs = tableMarkers.stream().map(this::makerToPartitionAndFileID).collect(Collectors.toSet());
+
+ currentFileIDs.retainAll(tableFilesIDs);
+
+ if (!currentFileIDs.isEmpty() || hasCommitConflict(currentInstantAllMarkers, basePath)) {
+ LOG.warn("Conflict writing detected based on markers!\n"
+ + "Conflict markers: " + currentInstantAllMarkers + "\n"
+ + "Table markers: " + tableMarkers);
+ hasConflict.compareAndSet(false, true);
+ }
+ LOG.info("Finish batch marker checker in " + timer.endTimer() + " ms");
+
+ } catch (IOException e) {
+ throw new HoodieIOException("IOException occurs during checking marker conflict");
+ }
+ }
+
+ /**
+ * Get Candidate Instant to do conflict checking:
+ * 1. Skip current writer related instant(currentInstantTime)
+ * 2. Skip all instants after currentInstantTime
+ * 3. Skip dead writers related instants based on heart-beat
+ * @param instants
+ * @return
+ */
+ private List<String> getCandidateInstants(List<Path> instants, String currentInstantTime) {
Review Comment:
Yeap, actually there are some diff here using the same name :)
for occ getCandidateInstants which depends on a state:
```
// To find which instants are conflicting, we apply the following logic
// 1. Get completed instants timeline only for commits that have happened since the last successful write.
// 2. Get any scheduled or completed compaction or clustering operations that have started and/or finished
// after the current instant. We need to check for write conflicts since they may have mutated the same files
// that are being newly created by the current write.
```
For current early conflict detection getCandidateInstants:
```
/**
* Get Candidate Instant to do conflict checking:
* 1. Skip current writer related instant(currentInstantTime)
* 2. Skip all instants after currentInstantTime
* 3. Skip dead writers related instants based on heart-beat
* @param instants
* @return
*/
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1322369715
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 71e0d1e91d2f540b441838263362c31251fa4ab5 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149)
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 316e5aefa89ec2dc35928af7c7da4b11a23aab20 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155)
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028313523
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieWriteHandle.java:
##########
@@ -187,8 +194,54 @@ protected Path makeNewFilePath(String partitionPath, String fileName) {
* @param partitionPath Partition path
*/
protected void createMarkerFile(String partitionPath, String dataFileName) {
- WriteMarkersFactory.get(config.getMarkersType(), hoodieTable, instantTime)
- .create(partitionPath, dataFileName, getIOType());
+ WriteMarkers writeMarkers = WriteMarkersFactory.get(config.getMarkersType(), hoodieTable, instantTime);
+ // do early conflict detection before create markers.
+ if (config.getWriteConcurrencyMode().supportsOptimisticConcurrencyControl()
+ && config.isEarlyConflictDetectionEnable()) {
+ HoodieEarlyConflictDetectionStrategy earlyConflictDetectionStrategy = config.getEarlyConflictDetectionStrategy();
+ if (earlyConflictDetectionStrategy instanceof HoodieTransactionDirectMarkerBasedEarlyConflictDetectionStrategy) {
+ createMarkerWithTransaction(earlyConflictDetectionStrategy, writeMarkers, partitionPath, dataFileName);
+ } else {
+ createMarkerWithEarlyConflictDetection(earlyConflictDetectionStrategy, writeMarkers, partitionPath, dataFileName);
+ }
+ } else {
+ // create marker directly
+ writeMarkers.create(partitionPath, dataFileName, getIOType());
+ }
+ }
+
+ private Option<Path> createMarkerWithEarlyConflictDetection(HoodieEarlyConflictDetectionStrategy resolutionStrategy,
+ WriteMarkers writeMarkers,
+ String partitionPath,
+ String dataFileName) {
+ Set<HoodieInstant> completedCommitInstants = hoodieTable.getMetaClient().getActiveTimeline()
Review Comment:
For transaction `hasCommitConflict()` checking, we are now reloading active timeline to get these infos at the beginning of the transaction
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1283544259
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 64819e4ab1ed12788e126072fa50e964fc62c257 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203)
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 645766d09d376bc46a1e0b6af770d9e84f778b24 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1291441877
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* e23ab613d4aad3844374b21d987f95ca6f5183e5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1186988905
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fcaaf9d6dc063aed48319e335c181e913ac9106f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1064924276
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/DirectWriteMarkers.java:
##########
@@ -155,6 +160,20 @@ protected Option<Path> create(String partitionPath, String dataFileName, IOType
return create(getMarkerPath(partitionPath, dataFileName, type), checkIfExists);
}
+ @Override
+ public Option<Path> createWithEarlyConflictDetection(String partitionPath, String dataFileName, IOType type, boolean checkIfExists, Set<HoodieInstant> completedCommitInstants,
Review Comment:
removed completedCommitInstants.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1376126666
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 6bb19747ad7841c458fd0f1adbbfb99d6be13a14 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179)
* be0d5b4e5a211b1ec9f3bdad65177e4667f9a222 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1374995553
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* c973c81cc138a320e562f293f1dc72828ed8fac6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138)
* c412635478d07b2bb37ad6738c06882f84b54001 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
yihua commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1064236073
##########
hudi-common/src/main/java/org/apache/hudi/common/conflict/detection/HoodieDirectMarkerBasedEarlyConflictDetectionStrategy.java:
##########
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.conflict.detection;
+
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.common.model.HoodieCommitMetadata;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+public abstract class HoodieDirectMarkerBasedEarlyConflictDetectionStrategy implements HoodieEarlyConflictDetectionStrategy {
+ private static final Logger LOG = LogManager.getLogger(HoodieDirectMarkerBasedEarlyConflictDetectionStrategy.class);
+
+ public abstract boolean hasMarkerConflict(String basePath, FileSystem fs, String partitionPath, String dataFileName, String instantTime,
+ Set<HoodieInstant> completedCommitInstants, HoodieTableMetaClient metaClient);
+
+ public abstract void resolveMarkerConflict(String basePath, String partitionPath, String dataFileName);
+
+ /**
+ * We need to do list operation here.
+ * In order to reduce the list pressure as much as possible, first we build path prefix in advance: '$base_path/.temp/instant_time/partition_path',
+ * and only list these specific partition_paths we need instead of list all the '$base_path/.temp/'
+ * @param basePath
+ * @param partitionPath
+ * @param fileId 162b13d7-9530-48cf-88a4-02241817ae0c-0_1-74-100_003.parquet
+ * @return true if current fileID is already existed under .temp/instant_time/partition_path/..
+ * @throws IOException
+ */
+ public boolean checkMarkerConflict(String basePath, String partitionPath, String fileId,
+ FileSystem fs, String instantTime) throws IOException {
+ String tempFolderPath = getTempFolderPath(basePath);
+ long res = Arrays.stream(fs.listStatus(new Path(tempFolderPath)))
+ .parallel()
+ .map(FileStatus::getPath)
+ .filter(markerPath -> {
+ return !markerPath.getName().equalsIgnoreCase(instantTime);
+ })
Review Comment:
I think there's a gap here. We should use the active timeline loaded at the beginning of one commit to determine which inflight instants to check against, instead of checking and filtering all marker folders under `.temp/` which is inefficient.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
yihua commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1064249219
##########
hudi-common/src/main/java/org/apache/hudi/common/util/MarkerUtils.java:
##########
@@ -208,10 +220,89 @@ public static Set<String> readMarkersFromFile(Path markersFilePath, Serializable
fsDataInputStream = fs.open(markersFilePath);
markers = new HashSet<>(FileIOUtils.readAsUTFStringLines(fsDataInputStream));
} catch (IOException e) {
- throw new HoodieIOException("Failed to read MARKERS file " + markersFilePath, e);
+ if (ignoreException) {
+ LOG.warn("IOException occurs during read MARKERS file, ", e);
+ } else {
+ throw new HoodieIOException("Failed to read MARKERS file " + markersFilePath, e);
+ }
} finally {
closeQuietly(fsDataInputStream);
}
return markers;
}
+
+ public static List<Path> getAllMarkerDir(Path tempPath, FileSystem fs) throws IOException {
+ return Arrays.stream(fs.listStatus(tempPath)).map(FileStatus::getPath).collect(Collectors.toList());
+ }
+
+ public static boolean hasCommitConflict(HoodieActiveTimeline activeTimeline, Set<String> currentFileIDs, Set<HoodieInstant> completedCommitInstants) {
+
+ Set<HoodieInstant> currentInstants = new HashSet<>(
+ activeTimeline.reload().getCommitsTimeline().filterCompletedInstants().getInstants());
+
+ currentInstants.removeAll(completedCommitInstants);
+ Set<String> missingFileIDs = currentInstants.stream().flatMap(instant -> {
+ try {
+ return HoodieCommitMetadata.fromBytes(activeTimeline.getInstantDetails(instant).get(), HoodieCommitMetadata.class)
+ .getFileIdAndRelativePaths().keySet().stream();
+ } catch (Exception e) {
+ return Stream.empty();
+ }
+ }).collect(Collectors.toSet());
+ currentFileIDs.retainAll(missingFileIDs);
+ return !currentFileIDs.isEmpty();
+ }
+
+ /**
+ * Get Candidate Instant to do conflict checking:
+ * 1. Skip current writer related instant(currentInstantTime)
+ * 2. Skip all instants after currentInstantTime
+ * 3. Skip dead writers related instants based on heart-beat
+ * 4. Skip pending compaction instant (For now we don' do early conflict check with compact action)
+ * Because we don't want to let pending compaction block common writer.
+ * @param instants
+ * @return
+ */
+ public static List<String> getCandidateInstants(HoodieActiveTimeline activeTimeline, List<Path> instants, String currentInstantTime,
+ long maxAllowableHeartbeatIntervalInMs, FileSystem fs, String basePath) {
+
+ HoodieActiveTimeline reloadActive = activeTimeline.reload();
Review Comment:
Do we need to reload the active timeline when creating a marker file? This can introduce non-trivial latency overhead.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/DirectWriteMarkers.java:
##########
@@ -155,6 +160,20 @@ protected Option<Path> create(String partitionPath, String dataFileName, IOType
return create(getMarkerPath(partitionPath, dataFileName, type), checkIfExists);
}
+ @Override
+ public Option<Path> createWithEarlyConflictDetection(String partitionPath, String dataFileName, IOType type, boolean checkIfExists, Set<HoodieInstant> completedCommitInstants,
+ HoodieWriteConfig config, String fileId, HoodieActiveTimeline activeTimeline) {
+
+ long maxAllowableHeartbeatIntervalInMs = config.getHoodieClientHeartbeatIntervalInMs() * config.getHoodieClientHeartbeatTolerableMisses();
+
+ HoodieDirectMarkerBasedEarlyConflictDetectionStrategy strategy =
+ (HoodieDirectMarkerBasedEarlyConflictDetectionStrategy) ReflectionUtils.loadClass(config.getEarlyConflictDetectionStrategyClassName(),
Review Comment:
nit: we can think about loading the strategy class through reflection in a common place for reuse, instead of loading for every marker creation.
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/MarkerHandler.java:
##########
@@ -150,7 +166,52 @@ public boolean doesMarkerDirExist(String markerDir) {
* @param markerName marker name
* @return the {@code CompletableFuture} instance for the request
*/
- public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName) {
+ public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName, String basePath) {
+ // Step1 do early conflict detection if enable
+ if (timelineServiceConfig.earlyConflictDetectionEnable) {
+ try {
+ synchronized (earlyConflictDetectionLock) {
+ if (earlyConflictDetectionStrategy == null) {
+ earlyConflictDetectionStrategy = (HoodieTimelineServerBasedEarlyConflictDetectionStrategy) ReflectionUtils.loadClass(timelineServiceConfig.earlyConflictDetectionStrategy,
+ basePath, markerDir, markerName, timelineServiceConfig.checkCommitConflict);
+ }
+
+ // markerDir => $base_path/.hoodie/.temp/$instant_time
+ // If markerDir is changed like move to the next instant action, we need to fresh this earlyConflictDetectionStrategy.
+ // For specific instant related create marker action, we only call this check/fresh once
+ // instead of starting the conflict detector for every request
+ if (!markerDir.equalsIgnoreCase(currentMarkerDir)) {
+ this.currentMarkerDir = markerDir;
+ Set<String> actions = CollectionUtils.createSet(COMMIT_ACTION, DELTA_COMMIT_ACTION, REPLACE_COMMIT_ACTION);
+ Set<HoodieInstant> oldInstants = new HashSet<>(
+ viewManager.getFileSystemView(basePath)
+ .getTimeline()
+ .filterCompletedInstants()
+ .filter(instant -> actions.contains(instant.getAction()))
+ .getInstants());
+
+ earlyConflictDetectionStrategy.fresh(timelineServiceConfig.asyncConflictDetectorBatchIntervalMs,
+ timelineServiceConfig.asyncConflictDetectorBatchPeriodMs, markerDir, basePath, timelineServiceConfig.maxAllowableHeartbeatIntervalInMs, fileSystem,
+ this, oldInstants);
+ }
+ }
+
+ earlyConflictDetectionStrategy.detectAndResolveConflictIfNecessary();
+
+ } catch (Exception ex) {
Review Comment:
Let's catch `HoodieEarlyConflictDetectionException` separately for detected conflict and then `Exception` for other errors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1187045375
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fcaaf9d6dc063aed48319e335c181e913ac9106f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031)
* 553fb00b590e9586ade21e86569ccb30d8dec7a9 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1187714085
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fcaaf9d6dc063aed48319e335c181e913ac9106f Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031)
* 553fb00b590e9586ade21e86569ccb30d8dec7a9 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033)
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 66b7d1b75c7a201b5d108cd48a70363dd00a1795 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1399004655
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214",
"triggerID" : "1376756472",
"triggerType" : "MANUAL"
}, {
"hash" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422",
"triggerID" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"triggerType" : "PUSH"
}, {
"hash" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14480",
"triggerID" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"triggerType" : "PUSH"
}, {
"hash" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14484",
"triggerID" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"triggerType" : "PUSH"
}, {
"hash" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 7344fabacee437adb4b55ca922df0f5fb14ae372 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14484)
* 501e47fb5c554a32acd7d2b410b5cdcf880f6057 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1397953606
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214",
"triggerID" : "1376756472",
"triggerType" : "MANUAL"
}, {
"hash" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422",
"triggerID" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"triggerType" : "PUSH"
}, {
"hash" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14480",
"triggerID" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"triggerType" : "PUSH"
}, {
"hash" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14484",
"triggerID" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* a2980b73d4ad32976360804059fcd6df969b9f89 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14480)
* 7344fabacee437adb4b55ca922df0f5fb14ae372 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14484)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1398174461
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214",
"triggerID" : "1376756472",
"triggerType" : "MANUAL"
}, {
"hash" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422",
"triggerID" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"triggerType" : "PUSH"
}, {
"hash" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14480",
"triggerID" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"triggerType" : "PUSH"
}, {
"hash" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14484",
"triggerID" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 7344fabacee437adb4b55ca922df0f5fb14ae372 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14484)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1291894521
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1290002376
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 645766d09d376bc46a1e0b6af770d9e84f778b24 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360)
* e23ab613d4aad3844374b21d987f95ca6f5183e5 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1289996934
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 645766d09d376bc46a1e0b6af770d9e84f778b24 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360)
* e23ab613d4aad3844374b21d987f95ca6f5183e5 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1235084038
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 64819e4ab1ed12788e126072fa50e964fc62c257 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1324478500
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 869baf7be4f626d10ca20606f7816b941203ec3a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189)
* 0447a7135e0cf136cd894291a1efd4374c3efd66 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1324690823
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 0447a7135e0cf136cd894291a1efd4374c3efd66 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028283595
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/MarkerHandler.java:
##########
@@ -150,7 +168,50 @@ public boolean doesMarkerDirExist(String markerDir) {
* @param markerName marker name
* @return the {@code CompletableFuture} instance for the request
*/
- public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName) {
+ public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName,
+ String batchInterval, String period, String maxAllowableHeartbeatIntervalInMs,
+ String basePath, String earlyConflictDetectionEnable,
+ String earlyConflictDetectionClassName) {
+ // Step1 do early conflict detection if enable
+ if (Boolean.parseBoolean(earlyConflictDetectionEnable)) {
+ try {
+ synchronized (earlyConflictDetectionLock) {
+ if (earlyConflictDetectionStrategy == null) {
+ earlyConflictDetectionStrategy = ReflectionUtils.loadClass(earlyConflictDetectionClassName);
+ }
+
+ if (!markerDir.equalsIgnoreCase(currentMarkerDir)) {
+ this.currentMarkerDir = markerDir;
+ Set<String> actions = CollectionUtils.createSet(COMMIT_ACTION, DELTA_COMMIT_ACTION, REPLACE_COMMIT_ACTION);
+ Set<HoodieInstant> oldInstants = viewManager.getFileSystemView(basePath)
+ .getTimeline()
+ .filterCompletedInstants()
+ .filter(instant -> actions.contains(instant.getAction()))
+ .getInstants()
+ .collect(Collectors.toSet());
+
+ earlyConflictDetectionStrategy.fresh(batchInterval, period, markerDir, basePath, maxAllowableHeartbeatIntervalInMs, fileSystem,
+ this, oldInstants);
+ }
+ }
+
+ if (earlyConflictDetectionStrategy.hasMarkerConflict()) {
+ earlyConflictDetectionStrategy.resolveMarkerConflict(basePath, markerDir, markerName);
Review Comment:
> The timeline server should simply return false for the marker creation request
Totally agree with it.
For now timeline server will return false for executor request and and executor will
```
if (success) {
return Option.of(new Path(FSUtils.getPartitionPath(markerDirPath, partitionPath), markerFileName));
} else {
// this failed may due to early conflict detection, so we need to throw out.
throw new HoodieEarlyConflictDetectionException(new ConcurrentModificationException("Early conflict detected but cannot resolve conflicts for overlapping writes"));
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028247148
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/MarkerHandler.java:
##########
@@ -150,7 +168,50 @@ public boolean doesMarkerDirExist(String markerDir) {
* @param markerName marker name
* @return the {@code CompletableFuture} instance for the request
*/
- public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName) {
+ public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName,
+ String batchInterval, String period, String maxAllowableHeartbeatIntervalInMs,
+ String basePath, String earlyConflictDetectionEnable,
+ String earlyConflictDetectionClassName) {
+ // Step1 do early conflict detection if enable
Review Comment:
Yeap, for now we init detection thread in the first time call this create marker API, and only fresh this thread when the instant time changed.
We can also init this thread in constructor, but for now, we don't pass hoodieWriteConfig in MarkerHandler..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1323876222
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* b97bb164efb5264476ebdf3673396583ec90d1a6 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185)
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 869baf7be4f626d10ca20606f7816b941203ec3a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1064929707
##########
hudi-common/src/main/java/org/apache/hudi/common/conflict/detection/HoodieDirectMarkerBasedEarlyConflictDetectionStrategy.java:
##########
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.conflict.detection;
+
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.common.config.HoodieConfig;
+import org.apache.hudi.common.fs.HoodieWrapperFileSystem;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieActiveTimeline;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.MarkerUtils;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+public abstract class HoodieDirectMarkerBasedEarlyConflictDetectionStrategy implements HoodieEarlyConflictDetectionStrategy {
+
+ private static final Logger LOG = LogManager.getLogger(HoodieDirectMarkerBasedEarlyConflictDetectionStrategy.class);
+ protected final String basePath;
+ protected final FileSystem fs;
+ protected final String partitionPath;
+ protected final String fileId;
+ protected final String instantTime;
+ protected final HoodieActiveTimeline activeTimeline;
+ protected final HoodieConfig config;
+ protected Set<HoodieInstant> completedCommitInstants;
+ protected final Boolean checkCommitConflict;
+ protected final Long maxAllowableHeartbeatIntervalInMs;
+
+ public HoodieDirectMarkerBasedEarlyConflictDetectionStrategy(String basePath, HoodieWrapperFileSystem fs, String partitionPath, String fileId, String instantTime,
+ HoodieActiveTimeline activeTimeline, HoodieConfig config, Boolean checkCommitConflict, Long maxAllowableHeartbeatIntervalInMs,
+ HashSet<HoodieInstant> completedCommitInstants) {
+ this.basePath = basePath;
+ this.fs = fs;
+ this.partitionPath = partitionPath;
+ this.fileId = fileId;
+ this.instantTime = instantTime;
+ this.completedCommitInstants = completedCommitInstants;
+ this.activeTimeline = activeTimeline;
+ this.config = config;
+ this.checkCommitConflict = checkCommitConflict;
+ this.maxAllowableHeartbeatIntervalInMs = maxAllowableHeartbeatIntervalInMs;
+ }
+
+ /**
+ * We need to do list operation here.
+ * In order to reduce the list pressure as much as possible, first we build path prefix in advance: '$base_path/.temp/instant_time/partition_path',
+ * and only list these specific partition_paths we need instead of list all the '$base_path/.temp/'
+ * @param basePath
+ * @param partitionPath
+ * @param fileId 162b13d7-9530-48cf-88a4-02241817ae0c-0_1-74-100_003.parquet
+ * @return true if current fileID is already existed under .temp/instant_time/partition_path/..
+ * @throws IOException
+ */
+ public boolean checkMarkerConflict(HoodieActiveTimeline activeTimeline, String basePath, String partitionPath, String fileId,
Review Comment:
simplify
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1375041550
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* c412635478d07b2bb37ad6738c06882f84b54001 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176)
* 6bb19747ad7841c458fd0f1adbbfb99d6be13a14 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1375185705
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 6bb19747ad7841c458fd0f1adbbfb99d6be13a14 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1186984628
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fcaaf9d6dc063aed48319e335c181e913ac9106f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
yihua commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1023177121
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/LockManager.java:
##########
@@ -62,6 +63,33 @@ public LockManager(HoodieWriteConfig writeConfig, FileSystem fs) {
metrics = new HoodieLockMetrics(writeConfig);
}
+ /**
+ * Try to have a lock at partitionPath + fileID level for different write handler.
+ * @param writeConfig
+ * @param fs
+ * @param partitionPath
+ * @param fileId
+ */
+ public LockManager(HoodieWriteConfig writeConfig, FileSystem fs, String partitionPath, String fileId) {
Review Comment:
Does this constructor also need to initialize `metrics`?
`metrics = new HoodieLockMetrics(writeConfig);`
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/LockManager.java:
##########
@@ -62,6 +63,33 @@ public LockManager(HoodieWriteConfig writeConfig, FileSystem fs) {
metrics = new HoodieLockMetrics(writeConfig);
}
+ /**
+ * Try to have a lock at partitionPath + fileID level for different write handler.
+ * @param writeConfig
+ * @param fs
+ * @param partitionPath
+ * @param fileId
+ */
+ public LockManager(HoodieWriteConfig writeConfig, FileSystem fs, String partitionPath, String fileId) {
+ this.writeConfig = writeConfig;
+ this.hadoopConf = new SerializableConfiguration(fs.getConf());
+ TypedProperties props = refreshLockConfig(writeConfig, partitionPath + "/" + fileId);
+ this.lockConfiguration = new LockConfiguration(props);
+ maxRetries = lockConfiguration.getConfig().getInteger(LOCK_ACQUIRE_CLIENT_NUM_RETRIES_PROP_KEY,
+ Integer.parseInt(HoodieLockConfig.LOCK_ACQUIRE_CLIENT_NUM_RETRIES.defaultValue()));
+ maxWaitTimeInMs = lockConfiguration.getConfig().getLong(LOCK_ACQUIRE_CLIENT_RETRY_WAIT_TIME_IN_MILLIS_PROP_KEY,
+ Long.parseLong(HoodieLockConfig.LOCK_ACQUIRE_CLIENT_RETRY_WAIT_TIME_IN_MILLIS.defaultValue()));
Review Comment:
nit: extract the common init logic into a method instead of copying the code?
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/LockManager.java:
##########
@@ -62,6 +63,33 @@ public LockManager(HoodieWriteConfig writeConfig, FileSystem fs) {
metrics = new HoodieLockMetrics(writeConfig);
}
+ /**
+ * Try to have a lock at partitionPath + fileID level for different write handler.
+ * @param writeConfig
+ * @param fs
+ * @param partitionPath
+ * @param fileId
+ */
+ public LockManager(HoodieWriteConfig writeConfig, FileSystem fs, String partitionPath, String fileId) {
+ this.writeConfig = writeConfig;
+ this.hadoopConf = new SerializableConfiguration(fs.getConf());
+ TypedProperties props = refreshLockConfig(writeConfig, partitionPath + "/" + fileId);
+ this.lockConfiguration = new LockConfiguration(props);
+ maxRetries = lockConfiguration.getConfig().getInteger(LOCK_ACQUIRE_CLIENT_NUM_RETRIES_PROP_KEY,
+ Integer.parseInt(HoodieLockConfig.LOCK_ACQUIRE_CLIENT_NUM_RETRIES.defaultValue()));
+ maxWaitTimeInMs = lockConfiguration.getConfig().getLong(LOCK_ACQUIRE_CLIENT_RETRY_WAIT_TIME_IN_MILLIS_PROP_KEY,
+ Long.parseLong(HoodieLockConfig.LOCK_ACQUIRE_CLIENT_RETRY_WAIT_TIME_IN_MILLIS.defaultValue()));
+ }
+
+ /**
+ * rebuild lock related configs, only support ZK related lock for now.
+ */
+ private TypedProperties refreshLockConfig(HoodieWriteConfig writeConfig, String key) {
+ TypedProperties props = new TypedProperties(writeConfig.getProps());
+ props.setProperty(LockConfiguration.ZK_LOCK_KEY_PROP_KEY, key);
Review Comment:
Here it should check if the ZK-based lock is configured. Otherwise, it should throw an exception.
Generally, we should think about how to support different lock provider implementations. For the first cut, it may be okay to have this specific logic here.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieLockConfig.java:
##########
@@ -189,6 +190,33 @@ public class HoodieLockConfig extends HoodieConfig {
.withDocumentation("Lock provider class name, this should be subclass of "
+ "org.apache.hudi.client.transaction.ConflictResolutionStrategy");
+ // Pluggable strategies to use when early conflict detection
+ public static final ConfigProperty<String> EARLY_CONFLICT_DETECTION_STRATEGY_CLASS_NAME = ConfigProperty
+ .key(LOCK_PREFIX + "early.conflict.detection.strategy")
+ .defaultValue(AsyncTimelineMarkerEarlyConflictDetectionStrategy.class.getName())
+ .sinceVersion("0.12.0")
+ .withDocumentation("Early conflict detection class name, this should be subclass of "
+ + "org.apache.hudi.common.conflict.detection.HoodieEarlyConflictDetectionStrategy");
+
+ public static final ConfigProperty<Boolean> EARLY_CONFLICT_DETECTION_ENABLE = ConfigProperty
+ .key(LOCK_PREFIX + "early.conflict.detection.enable")
+ .defaultValue(false)
+ .sinceVersion("0.12.0")
Review Comment:
nit: now this should be planned for `0.13.0` :)
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/TransactionManager.java:
##########
@@ -67,6 +80,14 @@ public void endTransaction(Option<HoodieInstant> currentTxnOwnerInstant) {
}
}
+ public void endTransaction(String filePath) {
Review Comment:
Align the argument with `beginTransaction()`, using the same set of arguments?
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieLockConfig.java:
##########
@@ -189,6 +190,33 @@ public class HoodieLockConfig extends HoodieConfig {
.withDocumentation("Lock provider class name, this should be subclass of "
+ "org.apache.hudi.client.transaction.ConflictResolutionStrategy");
+ // Pluggable strategies to use when early conflict detection
+ public static final ConfigProperty<String> EARLY_CONFLICT_DETECTION_STRATEGY_CLASS_NAME = ConfigProperty
+ .key(LOCK_PREFIX + "early.conflict.detection.strategy")
+ .defaultValue(AsyncTimelineMarkerEarlyConflictDetectionStrategy.class.getName())
Review Comment:
Should this depend on the marker type used with `.withInferFunction()`? An example of `withInferFunction()`:
```
public static final ConfigProperty<String> DYNAMODB_LOCK_PARTITION_KEY = ConfigProperty
.key(DYNAMODB_BASED_LOCK_PROPERTY_PREFIX + "partition_key")
.noDefaultValue()
.sinceVersion("0.10.0")
.withInferFunction(cfg -> {
if (cfg.contains(HoodieTableConfig.NAME)) {
return Option.of(cfg.getString(HoodieTableConfig.NAME));
}
return Option.empty();
})
.withDocumentation("For DynamoDB based lock provider, the partition key for the DynamoDB lock table. "
+ "Each Hudi dataset should has it's unique key so concurrent writers could refer to the same partition key."
+ " By default we use the Hudi table name specified to be the partition key");
```
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieWriteHandle.java:
##########
@@ -187,8 +194,54 @@ protected Path makeNewFilePath(String partitionPath, String fileName) {
* @param partitionPath Partition path
*/
protected void createMarkerFile(String partitionPath, String dataFileName) {
- WriteMarkersFactory.get(config.getMarkersType(), hoodieTable, instantTime)
- .create(partitionPath, dataFileName, getIOType());
+ WriteMarkers writeMarkers = WriteMarkersFactory.get(config.getMarkersType(), hoodieTable, instantTime);
+ // do early conflict detection before create markers.
+ if (config.getWriteConcurrencyMode().supportsOptimisticConcurrencyControl()
+ && config.isEarlyConflictDetectionEnable()) {
+ HoodieEarlyConflictDetectionStrategy earlyConflictDetectionStrategy = config.getEarlyConflictDetectionStrategy();
+ if (earlyConflictDetectionStrategy instanceof HoodieTransactionDirectMarkerBasedEarlyConflictDetectionStrategy) {
+ createMarkerWithTransaction(earlyConflictDetectionStrategy, writeMarkers, partitionPath, dataFileName);
+ } else {
+ createMarkerWithEarlyConflictDetection(earlyConflictDetectionStrategy, writeMarkers, partitionPath, dataFileName);
+ }
+ } else {
+ // create marker directly
+ writeMarkers.create(partitionPath, dataFileName, getIOType());
+ }
+ }
+
+ private Option<Path> createMarkerWithEarlyConflictDetection(HoodieEarlyConflictDetectionStrategy resolutionStrategy,
+ WriteMarkers writeMarkers,
+ String partitionPath,
+ String dataFileName) {
+ Set<HoodieInstant> completedCommitInstants = hoodieTable.getMetaClient().getActiveTimeline()
Review Comment:
I think the `hoodieTable` here should be initialized at the beginning of the transaction and used throughout the transaction. Let's make sure that's the case, to make sure it always returns the snapshot of the timeline at the beginning of the transaction (later concurrent writes should not leak in), so to guarantee the correct behavior for conflict detection.
##########
hudi-common/src/main/java/org/apache/hudi/common/conflict/detection/HoodieDirectMarkerBasedEarlyConflictDetectionStrategy.java:
##########
@@ -0,0 +1,109 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.conflict.detection;
+
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.common.model.HoodieCommitMetadata;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.StringUtils;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+public abstract class HoodieDirectMarkerBasedEarlyConflictDetectionStrategy implements HoodieEarlyConflictDetectionStrategy {
+ private static final Logger LOG = LogManager.getLogger(HoodieDirectMarkerBasedEarlyConflictDetectionStrategy.class);
+
+ public abstract boolean hasMarkerConflict(String basePath, FileSystem fs, String partitionPath, String dataFileName, String instantTime,
+ Set<HoodieInstant> completedCommitInstants, HoodieTableMetaClient metaClient);
+
+ public abstract void resolveMarkerConflict(String basePath, String partitionPath, String dataFileName);
+
+ /**
+ * We need to do list operation here.
+ * In order to reduce the list pressure as much as possible, first we build path prefix in advance: '$base_path/.temp/instant_time/partition_path',
+ * and only list these specific partition_paths we need instead of list all the '$base_path/.temp/'
+ * @param basePath
+ * @param partitionPath
+ * @param fileId 162b13d7-9530-48cf-88a4-02241817ae0c-0_1-74-100_003.parquet
+ * @return true if current fileID is already existed under .temp/instant_time/partition_path/..
+ * @throws IOException
+ */
+ public boolean checkMarkerConflict(String basePath, String partitionPath, String fileId,
+ FileSystem fs, String instantTime) throws IOException {
+ String tempFolderPath = getTempFolderPath(basePath);
+ long res = Arrays.stream(fs.listStatus(new Path(tempFolderPath)))
+ .parallel()
+ .map(FileStatus::getPath)
+ .filter(markerPath -> {
+ return !markerPath.getName().equalsIgnoreCase(instantTime);
+ })
Review Comment:
Could we somehow reuse `HoodieTableMetaClient::getMarkerFolderPath(String instantTs)` to avoid one list call?
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieWriteHandle.java:
##########
@@ -187,8 +194,54 @@ protected Path makeNewFilePath(String partitionPath, String fileName) {
* @param partitionPath Partition path
*/
protected void createMarkerFile(String partitionPath, String dataFileName) {
- WriteMarkersFactory.get(config.getMarkersType(), hoodieTable, instantTime)
- .create(partitionPath, dataFileName, getIOType());
+ WriteMarkers writeMarkers = WriteMarkersFactory.get(config.getMarkersType(), hoodieTable, instantTime);
+ // do early conflict detection before create markers.
+ if (config.getWriteConcurrencyMode().supportsOptimisticConcurrencyControl()
+ && config.isEarlyConflictDetectionEnable()) {
+ HoodieEarlyConflictDetectionStrategy earlyConflictDetectionStrategy = config.getEarlyConflictDetectionStrategy();
+ if (earlyConflictDetectionStrategy instanceof HoodieTransactionDirectMarkerBasedEarlyConflictDetectionStrategy) {
+ createMarkerWithTransaction(earlyConflictDetectionStrategy, writeMarkers, partitionPath, dataFileName);
+ } else {
+ createMarkerWithEarlyConflictDetection(earlyConflictDetectionStrategy, writeMarkers, partitionPath, dataFileName);
+ }
+ } else {
+ // create marker directly
+ writeMarkers.create(partitionPath, dataFileName, getIOType());
+ }
Review Comment:
Should we move this logic into specific `WriteMarkers` implementations, i.e., direct marker-based strategy into DirectWriteMarkers, and async timeline server-based strategy into TimelineServerBasedWriteMarkers? Because early conflict detection is tightly coupled with the marker creation process and the type of markers.
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/MarkerHandler.java:
##########
@@ -109,6 +126,7 @@ public void stop() {
}
dispatchingExecutorService.shutdown();
batchingExecutorService.shutdown();
+ checkers.values().forEach(ExecutorService::shutdown);
Review Comment:
Clear the checkers map?
##########
hudi-common/src/main/java/org/apache/hudi/common/util/MarkerUtils.java:
##########
@@ -208,10 +216,56 @@ public static Set<String> readMarkersFromFile(Path markersFilePath, Serializable
fsDataInputStream = fs.open(markersFilePath);
markers = new HashSet<>(FileIOUtils.readAsUTFStringLines(fsDataInputStream));
} catch (IOException e) {
- throw new HoodieIOException("Failed to read MARKERS file " + markersFilePath, e);
+ if (ignoreException) {
+ LOG.warn("IOException occurs during read MARKERS file, ", e);
+ } else {
+ throw new HoodieIOException("Failed to read MARKERS file " + markersFilePath, e);
+ }
} finally {
closeQuietly(fsDataInputStream);
}
return markers;
}
+
+ /**
+ * Reads files containing the markers written by timeline-server-based marker mechanism locally instead of using cluster Context.
+ *
+ * @param markerDir marker directory.
+ * @param fileSystem file system to use.
+ * @return A {@code Map} of file name to the set of markers stored in the file.
+ */
+ public static Set<String> readTimelineServerBasedMarkersFromFileSystemLocally(String markerDir, FileSystem fileSystem) {
Review Comment:
You avoid adding this method by reusing the following method with local engine context.
```
public static Map<String, Set<String>> readTimelineServerBasedMarkersFromFileSystem(
String markerDir, FileSystem fileSystem, HoodieEngineContext context, int parallelism)
```
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/MarkerHandler.java:
##########
@@ -150,7 +168,50 @@ public boolean doesMarkerDirExist(String markerDir) {
* @param markerName marker name
* @return the {@code CompletableFuture} instance for the request
*/
- public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName) {
+ public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName,
+ String batchInterval, String period, String maxAllowableHeartbeatIntervalInMs,
+ String basePath, String earlyConflictDetectionEnable,
+ String earlyConflictDetectionClassName) {
+ // Step1 do early conflict detection if enable
+ if (Boolean.parseBoolean(earlyConflictDetectionEnable)) {
+ try {
+ synchronized (earlyConflictDetectionLock) {
+ if (earlyConflictDetectionStrategy == null) {
+ earlyConflictDetectionStrategy = ReflectionUtils.loadClass(earlyConflictDetectionClassName);
+ }
+
+ if (!markerDir.equalsIgnoreCase(currentMarkerDir)) {
+ this.currentMarkerDir = markerDir;
+ Set<String> actions = CollectionUtils.createSet(COMMIT_ACTION, DELTA_COMMIT_ACTION, REPLACE_COMMIT_ACTION);
+ Set<HoodieInstant> oldInstants = viewManager.getFileSystemView(basePath)
+ .getTimeline()
+ .filterCompletedInstants()
+ .filter(instant -> actions.contains(instant.getAction()))
+ .getInstants()
+ .collect(Collectors.toSet());
+
+ earlyConflictDetectionStrategy.fresh(batchInterval, period, markerDir, basePath, maxAllowableHeartbeatIntervalInMs, fileSystem,
+ this, oldInstants);
+ }
+ }
+
+ if (earlyConflictDetectionStrategy.hasMarkerConflict()) {
+ earlyConflictDetectionStrategy.resolveMarkerConflict(basePath, markerDir, markerName);
Review Comment:
No exception should be thrown here at the timeline server if there is detected conflict. The timeline server should simply return false for the marker creation request and let the executor/write handle resolve the marker conflict (throw the exception).
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/TimelineServerBasedWriteMarkers.java:
##########
@@ -132,14 +143,54 @@ protected Option<Path> create(String partitionPath, String dataFileName, IOType
HoodieTimer timer = HoodieTimer.start();
String markerFileName = getMarkerFileName(dataFileName, type);
- Map<String, String> paramsMap = new HashMap<>();
+ Map<String, String> paramsMap = initConfigMap(partitionPath, markerFileName);
+ boolean success = executeCreateMarkerRequest(paramsMap, partitionPath, markerFileName);
+ LOG.info("[timeline-server-based] Created marker file " + partitionPath + "/" + markerFileName
+ + " in " + timer.endTimer() + " ms");
+ if (success) {
+ return Option.of(new Path(FSUtils.getPartitionPath(markerDirPath, partitionPath), markerFileName));
+ } else {
+ return Option.empty();
+ }
+ }
+
+ @Override
+ public Option<Path> createWithEarlyConflictDetection(String partitionPath, String dataFileName, IOType type, boolean checkIfExists,
+ HoodieEarlyConflictDetectionStrategy resolutionStrategy,
+ Set<HoodieInstant> completedCommitInstants, HoodieWriteConfig config, String fileId) {
+ HoodieTimer timer = new HoodieTimer().startTimer();
+ String markerFileName = getMarkerFileName(dataFileName, type);
+ Map<String, String> paramsMap = initConfigMap(partitionPath, markerFileName);
+
+ paramsMap.put(MARKER_CONFLICT_CHECKER_BATCH_INTERVAL, config.getMarkerConflictCheckerBatchInterval());
+ paramsMap.put(MARKER_CONFLICT_CHECKER_PERIOD, config.getMarkerConflictCheckerPeriod());
Review Comment:
nit: group the `MARKER_CONFLICT_CHECKER_*` params together
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieLockConfig.java:
##########
@@ -189,6 +190,33 @@ public class HoodieLockConfig extends HoodieConfig {
.withDocumentation("Lock provider class name, this should be subclass of "
+ "org.apache.hudi.client.transaction.ConflictResolutionStrategy");
+ // Pluggable strategies to use when early conflict detection
+ public static final ConfigProperty<String> EARLY_CONFLICT_DETECTION_STRATEGY_CLASS_NAME = ConfigProperty
+ .key(LOCK_PREFIX + "early.conflict.detection.strategy")
+ .defaultValue(AsyncTimelineMarkerEarlyConflictDetectionStrategy.class.getName())
+ .sinceVersion("0.12.0")
+ .withDocumentation("Early conflict detection class name, this should be subclass of "
+ + "org.apache.hudi.common.conflict.detection.HoodieEarlyConflictDetectionStrategy");
+
+ public static final ConfigProperty<Boolean> EARLY_CONFLICT_DETECTION_ENABLE = ConfigProperty
+ .key(LOCK_PREFIX + "early.conflict.detection.enable")
+ .defaultValue(false)
+ .sinceVersion("0.12.0")
+ .withDocumentation("Enable early conflict detection based on markers. It will try to detect writing conflict before create markers and fast fail"
+ + " which will release cluster resources as soon as possible.");
+
+ public static final ConfigProperty<Long> MARKER_CONFLICT_CHECKER_BATCH_INTERVAL = ConfigProperty
+ .key(LOCK_PREFIX + "early.conflict.async.checker.batch.interval")
+ .defaultValue(30000L)
+ .sinceVersion("0.12.0")
+ .withDocumentation("Used for timeline based marker AsyncTimelineMarkerConflictResolutionStrategy. The time to delay first async marker conflict checking.");
+
+ public static final ConfigProperty<Long> MARKER_CONFLICT_CHECKER_PERIOD = ConfigProperty
+ .key(LOCK_PREFIX + "early.conflict.async.checker.period")
+ .defaultValue(30000L)
+ .sinceVersion("0.12.0")
+ .withDocumentation("Used for timeline based marker AsyncTimelineMarkerConflictResolutionStrategy. The period between each marker conflict checking.");
Review Comment:
Instead of using the prefix of `hoodie.write.lock.` (`LOCK_PREFIX`), we should use `hoodie.write.concurrency.` (creating a new constant String for that) and put these configs into `HoodieWriteConfig`, since the early conflict detection falls under the concurrency control.
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/marker/MarkerCheckerRunnable.java:
##########
@@ -0,0 +1,193 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.timeline.service.handlers.marker;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.common.model.HoodieCommitMetadata;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.HoodieTimer;
+import org.apache.hudi.common.util.MarkerUtils;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.hudi.timeline.service.handlers.MarkerHandler;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+public class MarkerCheckerRunnable implements Runnable {
+ private static final Logger LOG = LogManager.getLogger(MarkerCheckerRunnable.class);
+
+ private MarkerHandler markerHandler;
+ private String markerDir;
+ private String basePath;
+ private FileSystem fs;
+ private AtomicBoolean hasConflict;
+ private long maxAllowableHeartbeatIntervalInMs;
+ private Set<HoodieInstant> oldInstants;
+
+ public MarkerCheckerRunnable(AtomicBoolean hasConflict, MarkerHandler markerHandler, String markerDir,
+ String basePath, FileSystem fileSystem, long maxAllowableHeartbeatIntervalInMs,
+ Set<HoodieInstant> oldInstants) {
+ this.markerHandler = markerHandler;
+ this.markerDir = markerDir;
+ this.basePath = basePath;
+ this.fs = fileSystem;
+ this.hasConflict = hasConflict;
+ this.maxAllowableHeartbeatIntervalInMs = maxAllowableHeartbeatIntervalInMs;
+ this.oldInstants = oldInstants;
+ }
+
+ @Override
+ public void run() {
+ try {
+ if (!fs.exists(new Path(markerDir))) {
+ return;
+ }
+
+ HoodieTimer timer = new HoodieTimer().startTimer();
+ Set<String> currentInstantAllMarkers = markerHandler.getAllMarkers(markerDir);
+ Path tempPath = new Path(basePath + Path.SEPARATOR + HoodieTableMetaClient.TEMPFOLDER_NAME);
+
+ List<Path> instants = MarkerUtils.getAllMarkerDir(tempPath, fs);
+ List<String> candidate = getCandidateInstants(instants, markerDirToInstantTime(markerDir));
+ Set<String> tableMarkers = candidate.stream().flatMap(instant -> {
+ return MarkerUtils.readTimelineServerBasedMarkersFromFileSystemLocally(instant, fs).stream();
+ }).collect(Collectors.toSet());
+
+ Set<String> currentFileIDs = currentInstantAllMarkers.stream().map(this::makerToPartitionAndFileID).collect(Collectors.toSet());
+ Set<String> tableFilesIDs = tableMarkers.stream().map(this::makerToPartitionAndFileID).collect(Collectors.toSet());
+
+ currentFileIDs.retainAll(tableFilesIDs);
+
+ if (!currentFileIDs.isEmpty() || hasCommitConflict(currentInstantAllMarkers, basePath)) {
+ LOG.warn("Conflict writing detected based on markers!\n"
+ + "Conflict markers: " + currentInstantAllMarkers + "\n"
+ + "Table markers: " + tableMarkers);
+ hasConflict.compareAndSet(false, true);
+ }
+ LOG.info("Finish batch marker checker in " + timer.endTimer() + " ms");
+
+ } catch (IOException e) {
+ throw new HoodieIOException("IOException occurs during checking marker conflict");
+ }
+ }
+
+ /**
+ * Get Candidate Instant to do conflict checking:
+ * 1. Skip current writer related instant(currentInstantTime)
+ * 2. Skip all instants after currentInstantTime
+ * 3. Skip dead writers related instants based on heart-beat
+ * @param instants
+ * @return
+ */
+ private List<String> getCandidateInstants(List<Path> instants, String currentInstantTime) {
Review Comment:
Can we adapt the common logic from `ConflictResolutionStrategy` instead of reinventing similar logic?
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/TimelineServerBasedWriteMarkers.java:
##########
@@ -132,14 +143,54 @@ protected Option<Path> create(String partitionPath, String dataFileName, IOType
HoodieTimer timer = HoodieTimer.start();
String markerFileName = getMarkerFileName(dataFileName, type);
- Map<String, String> paramsMap = new HashMap<>();
+ Map<String, String> paramsMap = initConfigMap(partitionPath, markerFileName);
+ boolean success = executeCreateMarkerRequest(paramsMap, partitionPath, markerFileName);
+ LOG.info("[timeline-server-based] Created marker file " + partitionPath + "/" + markerFileName
+ + " in " + timer.endTimer() + " ms");
+ if (success) {
+ return Option.of(new Path(FSUtils.getPartitionPath(markerDirPath, partitionPath), markerFileName));
+ } else {
+ return Option.empty();
+ }
+ }
+
+ @Override
+ public Option<Path> createWithEarlyConflictDetection(String partitionPath, String dataFileName, IOType type, boolean checkIfExists,
+ HoodieEarlyConflictDetectionStrategy resolutionStrategy,
+ Set<HoodieInstant> completedCommitInstants, HoodieWriteConfig config, String fileId) {
+ HoodieTimer timer = new HoodieTimer().startTimer();
+ String markerFileName = getMarkerFileName(dataFileName, type);
+ Map<String, String> paramsMap = initConfigMap(partitionPath, markerFileName);
+
+ paramsMap.put(MARKER_CONFLICT_CHECKER_BATCH_INTERVAL, config.getMarkerConflictCheckerBatchInterval());
+ paramsMap.put(MARKER_CONFLICT_CHECKER_PERIOD, config.getMarkerConflictCheckerPeriod());
paramsMap.put(MARKER_DIR_PATH_PARAM, markerDirPath.toString());
Review Comment:
this should be moved to `initConfigMap()` as well.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/SimpleTransactionDirectMarkerBasedEarlyConflictDetectionStrategy.java:
##########
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.table.marker;
+
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hudi.common.conflict.detection.HoodieTransactionDirectMarkerBasedEarlyConflictDetectionStrategy;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.exception.HoodieEarlyConflictDetectionException;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.ConcurrentModificationException;
+import java.util.Set;
+
+/**
+ * This strategy is used for direct marker writers, trying to do early conflict detection.
+ * It will use fileSystem api like list and exist directly to check if there is any marker file conflict.
+ */
+public class SimpleTransactionDirectMarkerBasedEarlyConflictDetectionStrategy extends HoodieTransactionDirectMarkerBasedEarlyConflictDetectionStrategy {
+ private static final Logger LOG = LogManager.getLogger(SimpleTransactionDirectMarkerBasedEarlyConflictDetectionStrategy.class);
+
+ @Override
+ public boolean hasMarkerConflict(String basePath, FileSystem fs, String partitionPath, String fileId, String instantTime,
Review Comment:
The logic here looks like the same as `SimpleDirectMarkerBasedEarlyConflictDetectionStrategy`. Does it make sense to move the transaction management inside the strategy to make it easier to understand?
##########
hudi-common/src/main/java/org/apache/hudi/common/conflict/detection/HoodieTransactionDirectMarkerBasedEarlyConflictDetectionStrategy.java:
##########
@@ -0,0 +1,23 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.conflict.detection;
+
+public abstract class HoodieTransactionDirectMarkerBasedEarlyConflictDetectionStrategy
Review Comment:
Do we still need this?
##########
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/client/TestHoodieClientMultiWriter.java:
##########
@@ -658,4 +740,56 @@ private JavaRDD<WriteStatus> startCommitForUpdate(HoodieWriteConfig writeConfig,
assertNoWriteErrors(statuses);
return result;
}
+
+ public static Stream<Arguments> configParams() {
+ Object[][] data =
+ new Object[][] {{"COPY_ON_WRITE", MarkerType.TIMELINE_SERVER_BASED.name()}, {"MERGE_ON_READ", MarkerType.TIMELINE_SERVER_BASED.name()},
+ {"MERGE_ON_READ", MarkerType.DIRECT.name()}, {"COPY_ON_WRITE", MarkerType.DIRECT.name()}};
+ return Stream.of(data).map(Arguments::of);
+ }
+
+ private HoodieWriteConfig buildWriteConfigForEarlyConflictDetect(String markerType, Properties properties) {
+ if (markerType.equalsIgnoreCase(MarkerType.DIRECT.name())) {
+ return getConfigBuilder()
+ .withHeartbeatIntervalInMs(3600 * 1000)
+ .withFileSystemViewConfig(FileSystemViewStorageConfig.newBuilder()
+ .withStorageType(FileSystemViewStorageType.MEMORY)
+ .withSecondaryStorageType(FileSystemViewStorageType.MEMORY).build())
+ .withCleanConfig(HoodieCleanConfig.newBuilder()
+ .withFailedWritesCleaningPolicy(HoodieFailedWritesCleaningPolicy.LAZY)
+ .withAutoClean(false).build())
+ .withArchivalConfig(HoodieArchivalConfig.newBuilder()
+ .withAutoArchive(false).build())
+ .withWriteConcurrencyMode(WriteConcurrencyMode.OPTIMISTIC_CONCURRENCY_CONTROL)
+ .withMarkersType(MarkerType.DIRECT.name())
+ .withLockConfig(HoodieLockConfig.newBuilder().withLockProvider(InProcessLockProvider.class)
+ .withEarlyConflictDetectionEnable(true)
+ .withEarlyConflictDetectionStrategy(SimpleDirectMarkerBasedEarlyConflictDetectionStrategy.class.getName())
Review Comment:
For direct markers, should we also test `SimpleTransactionDirectMarkerBasedEarlyConflictDetectionStrategy`?
##########
hudi-common/src/main/java/org/apache/hudi/common/conflict/detection/HoodieEarlyConflictDetectionStrategy.java:
##########
@@ -0,0 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.conflict.detection;
+
+public interface HoodieEarlyConflictDetectionStrategy {
Review Comment:
Move common methods here?
##########
hudi-common/src/main/java/org/apache/hudi/common/table/marker/MarkerOperation.java:
##########
@@ -29,6 +29,12 @@ public class MarkerOperation implements Serializable {
public static final String MARKER_DIR_PATH_PARAM = "markerdirpath";
public static final String MARKER_NAME_PARAM = "markername";
+ public static final String MARKER_CONFLICT_CHECKER_BATCH_INTERVAL = "batchinterval";
+ public static final String MARKER_CONFLICT_CHECKER_PERIOD = "period";
+ public static final String MARKER_CONFLICT_CHECKER_HEART_BEAT_INTERVAL = "heartbeatinterval";
+ public static final String MARKER_BASEPATH_PARAM = "basepath";
+ public static final String MARKER_CONFLICT_CHECKER_ENABLE = "HoodieEarlyConflictDetectionStrategy";
+ public static final String MARKER_CONFLICT_CHECKER_STRATEGY = "earlyconflictdetectionstrategy";
Review Comment:
Let's add `marker` as the prefix for the parameter naming and avoid capital letters.
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/MarkerHandler.java:
##########
@@ -150,7 +168,50 @@ public boolean doesMarkerDirExist(String markerDir) {
* @param markerName marker name
* @return the {@code CompletableFuture} instance for the request
*/
- public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName) {
+ public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName,
+ String batchInterval, String period, String maxAllowableHeartbeatIntervalInMs,
+ String basePath, String earlyConflictDetectionEnable,
+ String earlyConflictDetectionClassName) {
+ // Step1 do early conflict detection if enable
Review Comment:
Should the async conflict detection thread be started in the constructor of the `MarkerHandler`? It does not make sense to start the checker for every request. Anyway, the checker is doing the detection in a batching and async way.
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/MarkerHandler.java:
##########
@@ -150,7 +168,50 @@ public boolean doesMarkerDirExist(String markerDir) {
* @param markerName marker name
* @return the {@code CompletableFuture} instance for the request
*/
- public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName) {
+ public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName,
+ String batchInterval, String period, String maxAllowableHeartbeatIntervalInMs,
+ String basePath, String earlyConflictDetectionEnable,
+ String earlyConflictDetectionClassName) {
+ // Step1 do early conflict detection if enable
+ if (Boolean.parseBoolean(earlyConflictDetectionEnable)) {
+ try {
+ synchronized (earlyConflictDetectionLock) {
+ if (earlyConflictDetectionStrategy == null) {
+ earlyConflictDetectionStrategy = ReflectionUtils.loadClass(earlyConflictDetectionClassName);
+ }
Review Comment:
init this in constructor?
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/MarkerHandler.java:
##########
@@ -150,7 +168,50 @@ public boolean doesMarkerDirExist(String markerDir) {
* @param markerName marker name
* @return the {@code CompletableFuture} instance for the request
*/
- public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName) {
+ public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName,
+ String batchInterval, String period, String maxAllowableHeartbeatIntervalInMs,
+ String basePath, String earlyConflictDetectionEnable,
+ String earlyConflictDetectionClassName) {
+ // Step1 do early conflict detection if enable
+ if (Boolean.parseBoolean(earlyConflictDetectionEnable)) {
+ try {
+ synchronized (earlyConflictDetectionLock) {
+ if (earlyConflictDetectionStrategy == null) {
+ earlyConflictDetectionStrategy = ReflectionUtils.loadClass(earlyConflictDetectionClassName);
+ }
+
+ if (!markerDir.equalsIgnoreCase(currentMarkerDir)) {
+ this.currentMarkerDir = markerDir;
+ Set<String> actions = CollectionUtils.createSet(COMMIT_ACTION, DELTA_COMMIT_ACTION, REPLACE_COMMIT_ACTION);
+ Set<HoodieInstant> oldInstants = viewManager.getFileSystemView(basePath)
+ .getTimeline()
+ .filterCompletedInstants()
+ .filter(instant -> actions.contains(instant.getAction()))
+ .getInstants()
+ .collect(Collectors.toSet());
+
+ earlyConflictDetectionStrategy.fresh(batchInterval, period, markerDir, basePath, maxAllowableHeartbeatIntervalInMs, fileSystem,
+ this, oldInstants);
+ }
+ }
+
+ if (earlyConflictDetectionStrategy.hasMarkerConflict()) {
+ earlyConflictDetectionStrategy.resolveMarkerConflict(basePath, markerDir, markerName);
+ }
Review Comment:
Only this should be kept when doing batch marker creation in `MarkerDirState::processMarkerCreationRequests`.
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/marker/MarkerCheckerRunnable.java:
##########
@@ -0,0 +1,193 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.timeline.service.handlers.marker;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.common.model.HoodieCommitMetadata;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.HoodieTimer;
+import org.apache.hudi.common.util.MarkerUtils;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.hudi.timeline.service.handlers.MarkerHandler;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+public class MarkerCheckerRunnable implements Runnable {
+ private static final Logger LOG = LogManager.getLogger(MarkerCheckerRunnable.class);
+
+ private MarkerHandler markerHandler;
+ private String markerDir;
+ private String basePath;
+ private FileSystem fs;
+ private AtomicBoolean hasConflict;
+ private long maxAllowableHeartbeatIntervalInMs;
+ private Set<HoodieInstant> oldInstants;
+
+ public MarkerCheckerRunnable(AtomicBoolean hasConflict, MarkerHandler markerHandler, String markerDir,
Review Comment:
Is it easy to add a unit test for this checker?
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/marker/MarkerCheckerRunnable.java:
##########
@@ -0,0 +1,193 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.timeline.service.handlers.marker;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.common.model.HoodieCommitMetadata;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.HoodieTimer;
+import org.apache.hudi.common.util.MarkerUtils;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.hudi.timeline.service.handlers.MarkerHandler;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+public class MarkerCheckerRunnable implements Runnable {
+ private static final Logger LOG = LogManager.getLogger(MarkerCheckerRunnable.class);
+
+ private MarkerHandler markerHandler;
+ private String markerDir;
+ private String basePath;
+ private FileSystem fs;
+ private AtomicBoolean hasConflict;
+ private long maxAllowableHeartbeatIntervalInMs;
+ private Set<HoodieInstant> oldInstants;
+
+ public MarkerCheckerRunnable(AtomicBoolean hasConflict, MarkerHandler markerHandler, String markerDir,
+ String basePath, FileSystem fileSystem, long maxAllowableHeartbeatIntervalInMs,
+ Set<HoodieInstant> oldInstants) {
+ this.markerHandler = markerHandler;
+ this.markerDir = markerDir;
+ this.basePath = basePath;
+ this.fs = fileSystem;
+ this.hasConflict = hasConflict;
+ this.maxAllowableHeartbeatIntervalInMs = maxAllowableHeartbeatIntervalInMs;
+ this.oldInstants = oldInstants;
+ }
+
+ @Override
+ public void run() {
+ try {
+ if (!fs.exists(new Path(markerDir))) {
+ return;
+ }
+
+ HoodieTimer timer = new HoodieTimer().startTimer();
Review Comment:
nit: use `HoodieTimer.start()`;
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/marker/MarkerCheckerRunnable.java:
##########
@@ -0,0 +1,193 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.timeline.service.handlers.marker;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.common.model.HoodieCommitMetadata;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.HoodieTimer;
+import org.apache.hudi.common.util.MarkerUtils;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.hudi.timeline.service.handlers.MarkerHandler;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+public class MarkerCheckerRunnable implements Runnable {
Review Comment:
nit: rename to `MarkerBasedEarlyConflictDetectionRunnable`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028239400
##########
hudi-common/src/main/java/org/apache/hudi/common/conflict/detection/HoodieTransactionDirectMarkerBasedEarlyConflictDetectionStrategy.java:
##########
@@ -0,0 +1,23 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.conflict.detection;
+
+public abstract class HoodieTransactionDirectMarkerBasedEarlyConflictDetectionStrategy
Review Comment:
Removed!
##########
hudi-common/src/main/java/org/apache/hudi/common/table/marker/MarkerOperation.java:
##########
@@ -29,6 +29,12 @@ public class MarkerOperation implements Serializable {
public static final String MARKER_DIR_PATH_PARAM = "markerdirpath";
public static final String MARKER_NAME_PARAM = "markername";
+ public static final String MARKER_CONFLICT_CHECKER_BATCH_INTERVAL = "batchinterval";
+ public static final String MARKER_CONFLICT_CHECKER_PERIOD = "period";
+ public static final String MARKER_CONFLICT_CHECKER_HEART_BEAT_INTERVAL = "heartbeatinterval";
+ public static final String MARKER_BASEPATH_PARAM = "basepath";
+ public static final String MARKER_CONFLICT_CHECKER_ENABLE = "HoodieEarlyConflictDetectionStrategy";
+ public static final String MARKER_CONFLICT_CHECKER_STRATEGY = "earlyconflictdetectionstrategy";
Review Comment:
Changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028233986
##########
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/client/TestHoodieClientMultiWriter.java:
##########
@@ -658,4 +740,56 @@ private JavaRDD<WriteStatus> startCommitForUpdate(HoodieWriteConfig writeConfig,
assertNoWriteErrors(statuses);
return result;
}
+
+ public static Stream<Arguments> configParams() {
+ Object[][] data =
+ new Object[][] {{"COPY_ON_WRITE", MarkerType.TIMELINE_SERVER_BASED.name()}, {"MERGE_ON_READ", MarkerType.TIMELINE_SERVER_BASED.name()},
+ {"MERGE_ON_READ", MarkerType.DIRECT.name()}, {"COPY_ON_WRITE", MarkerType.DIRECT.name()}};
+ return Stream.of(data).map(Arguments::of);
+ }
+
+ private HoodieWriteConfig buildWriteConfigForEarlyConflictDetect(String markerType, Properties properties) {
+ if (markerType.equalsIgnoreCase(MarkerType.DIRECT.name())) {
+ return getConfigBuilder()
+ .withHeartbeatIntervalInMs(3600 * 1000)
+ .withFileSystemViewConfig(FileSystemViewStorageConfig.newBuilder()
+ .withStorageType(FileSystemViewStorageType.MEMORY)
+ .withSecondaryStorageType(FileSystemViewStorageType.MEMORY).build())
+ .withCleanConfig(HoodieCleanConfig.newBuilder()
+ .withFailedWritesCleaningPolicy(HoodieFailedWritesCleaningPolicy.LAZY)
+ .withAutoClean(false).build())
+ .withArchivalConfig(HoodieArchivalConfig.newBuilder()
+ .withAutoArchive(false).build())
+ .withWriteConcurrencyMode(WriteConcurrencyMode.OPTIMISTIC_CONCURRENCY_CONTROL)
+ .withMarkersType(MarkerType.DIRECT.name())
+ .withLockConfig(HoodieLockConfig.newBuilder().withLockProvider(InProcessLockProvider.class)
+ .withEarlyConflictDetectionEnable(true)
+ .withEarlyConflictDetectionStrategy(SimpleDirectMarkerBasedEarlyConflictDetectionStrategy.class.getName())
Review Comment:
Add!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028259347
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/MarkerHandler.java:
##########
@@ -150,7 +168,50 @@ public boolean doesMarkerDirExist(String markerDir) {
* @param markerName marker name
* @return the {@code CompletableFuture} instance for the request
*/
- public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName) {
+ public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName,
+ String batchInterval, String period, String maxAllowableHeartbeatIntervalInMs,
+ String basePath, String earlyConflictDetectionEnable,
+ String earlyConflictDetectionClassName) {
+ // Step1 do early conflict detection if enable
+ if (Boolean.parseBoolean(earlyConflictDetectionEnable)) {
+ try {
+ synchronized (earlyConflictDetectionLock) {
+ if (earlyConflictDetectionStrategy == null) {
+ earlyConflictDetectionStrategy = ReflectionUtils.loadClass(earlyConflictDetectionClassName);
+ }
+
+ if (!markerDir.equalsIgnoreCase(currentMarkerDir)) {
+ this.currentMarkerDir = markerDir;
+ Set<String> actions = CollectionUtils.createSet(COMMIT_ACTION, DELTA_COMMIT_ACTION, REPLACE_COMMIT_ACTION);
+ Set<HoodieInstant> oldInstants = viewManager.getFileSystemView(basePath)
+ .getTimeline()
+ .filterCompletedInstants()
+ .filter(instant -> actions.contains(instant.getAction()))
+ .getInstants()
+ .collect(Collectors.toSet());
+
+ earlyConflictDetectionStrategy.fresh(batchInterval, period, markerDir, basePath, maxAllowableHeartbeatIntervalInMs, fileSystem,
+ this, oldInstants);
+ }
+ }
+
+ if (earlyConflictDetectionStrategy.hasMarkerConflict()) {
+ earlyConflictDetectionStrategy.resolveMarkerConflict(basePath, markerDir, markerName);
Review Comment:
Same reason, this check is batch and async. For specific request get false result. It means maker checker find a conflict but maybe it is not current request related marker conflict.
So is it possible to let current executor to handle others' conflict based on timeline sever in async and batch mode. :)
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/marker/MarkerCheckerRunnable.java:
##########
@@ -0,0 +1,193 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.timeline.service.handlers.marker;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.common.model.HoodieCommitMetadata;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.HoodieTimer;
+import org.apache.hudi.common.util.MarkerUtils;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.hudi.timeline.service.handlers.MarkerHandler;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+public class MarkerCheckerRunnable implements Runnable {
+ private static final Logger LOG = LogManager.getLogger(MarkerCheckerRunnable.class);
+
+ private MarkerHandler markerHandler;
+ private String markerDir;
+ private String basePath;
+ private FileSystem fs;
+ private AtomicBoolean hasConflict;
+ private long maxAllowableHeartbeatIntervalInMs;
+ private Set<HoodieInstant> oldInstants;
+
+ public MarkerCheckerRunnable(AtomicBoolean hasConflict, MarkerHandler markerHandler, String markerDir,
+ String basePath, FileSystem fileSystem, long maxAllowableHeartbeatIntervalInMs,
+ Set<HoodieInstant> oldInstants) {
+ this.markerHandler = markerHandler;
+ this.markerDir = markerDir;
+ this.basePath = basePath;
+ this.fs = fileSystem;
+ this.hasConflict = hasConflict;
+ this.maxAllowableHeartbeatIntervalInMs = maxAllowableHeartbeatIntervalInMs;
+ this.oldInstants = oldInstants;
+ }
+
+ @Override
+ public void run() {
+ try {
+ if (!fs.exists(new Path(markerDir))) {
+ return;
+ }
+
+ HoodieTimer timer = new HoodieTimer().startTimer();
Review Comment:
changed!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1324163703
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 869baf7be4f626d10ca20606f7816b941203ec3a Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1374998160
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* c973c81cc138a320e562f293f1dc72828ed8fac6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138)
* c412635478d07b2bb37ad6738c06882f84b54001 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1375039580
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* c973c81cc138a320e562f293f1dc72828ed8fac6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138)
* c412635478d07b2bb37ad6738c06882f84b54001 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176)
* 6bb19747ad7841c458fd0f1adbbfb99d6be13a14 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1373633366
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* c973c81cc138a320e562f293f1dc72828ed8fac6 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1373196445
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 0447a7135e0cf136cd894291a1efd4374c3efd66 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206)
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1285078192
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 645766d09d376bc46a1e0b6af770d9e84f778b24 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1284787088
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1283550619
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 64819e4ab1ed12788e126072fa50e964fc62c257 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203)
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 645766d09d376bc46a1e0b6af770d9e84f778b24 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1291413721
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1312192736
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* e23ab613d4aad3844374b21d987f95ca6f5183e5 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584) Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600)
* 465536ff2afcf217c19ca00b4a90a5e04d372bc3 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1399842193
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1399873220
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214",
"triggerID" : "1376756472",
"triggerType" : "MANUAL"
}, {
"hash" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422",
"triggerID" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"triggerType" : "PUSH"
}, {
"hash" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14480",
"triggerID" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"triggerType" : "PUSH"
}, {
"hash" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14484",
"triggerID" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"triggerType" : "PUSH"
}, {
"hash" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14518",
"triggerID" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"triggerType" : "PUSH"
}, {
"hash" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14549",
"triggerID" : "1399842193",
"triggerType" : "MANUAL"
}, {
"hash" : "0a77616b5bd7b7d20259b3eb12d31e571f2af60e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14550",
"triggerID" : "0a77616b5bd7b7d20259b3eb12d31e571f2af60e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 501e47fb5c554a32acd7d2b410b5cdcf880f6057 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14518) Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14549)
* 0a77616b5bd7b7d20259b3eb12d31e571f2af60e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14550)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1376790832
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214",
"triggerID" : "1376756472",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 1b837ec65943331f6c805f723b088240ae66dfaf Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yanghua commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
yanghua commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1190026706
@zhangyue19921010 Would you please update the PR to fix the conflicts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1284822757
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 645766d09d376bc46a1e0b6af770d9e84f778b24 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1312418577
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 465536ff2afcf217c19ca00b4a90a5e04d372bc3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962) Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028230918
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/LockManager.java:
##########
@@ -62,6 +63,33 @@ public LockManager(HoodieWriteConfig writeConfig, FileSystem fs) {
metrics = new HoodieLockMetrics(writeConfig);
}
+ /**
+ * Try to have a lock at partitionPath + fileID level for different write handler.
+ * @param writeConfig
+ * @param fs
+ * @param partitionPath
+ * @param fileId
+ */
+ public LockManager(HoodieWriteConfig writeConfig, FileSystem fs, String partitionPath, String fileId) {
+ this.writeConfig = writeConfig;
+ this.hadoopConf = new SerializableConfiguration(fs.getConf());
+ TypedProperties props = refreshLockConfig(writeConfig, partitionPath + "/" + fileId);
+ this.lockConfiguration = new LockConfiguration(props);
+ maxRetries = lockConfiguration.getConfig().getInteger(LOCK_ACQUIRE_CLIENT_NUM_RETRIES_PROP_KEY,
+ Integer.parseInt(HoodieLockConfig.LOCK_ACQUIRE_CLIENT_NUM_RETRIES.defaultValue()));
+ maxWaitTimeInMs = lockConfiguration.getConfig().getLong(LOCK_ACQUIRE_CLIENT_RETRY_WAIT_TIME_IN_MILLIS_PROP_KEY,
+ Long.parseLong(HoodieLockConfig.LOCK_ACQUIRE_CLIENT_RETRY_WAIT_TIME_IN_MILLIS.defaultValue()));
Review Comment:
Changed.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/LockManager.java:
##########
@@ -62,6 +63,33 @@ public LockManager(HoodieWriteConfig writeConfig, FileSystem fs) {
metrics = new HoodieLockMetrics(writeConfig);
}
+ /**
+ * Try to have a lock at partitionPath + fileID level for different write handler.
+ * @param writeConfig
+ * @param fs
+ * @param partitionPath
+ * @param fileId
+ */
+ public LockManager(HoodieWriteConfig writeConfig, FileSystem fs, String partitionPath, String fileId) {
+ this.writeConfig = writeConfig;
+ this.hadoopConf = new SerializableConfiguration(fs.getConf());
+ TypedProperties props = refreshLockConfig(writeConfig, partitionPath + "/" + fileId);
+ this.lockConfiguration = new LockConfiguration(props);
+ maxRetries = lockConfiguration.getConfig().getInteger(LOCK_ACQUIRE_CLIENT_NUM_RETRIES_PROP_KEY,
+ Integer.parseInt(HoodieLockConfig.LOCK_ACQUIRE_CLIENT_NUM_RETRIES.defaultValue()));
+ maxWaitTimeInMs = lockConfiguration.getConfig().getLong(LOCK_ACQUIRE_CLIENT_RETRY_WAIT_TIME_IN_MILLIS_PROP_KEY,
+ Long.parseLong(HoodieLockConfig.LOCK_ACQUIRE_CLIENT_RETRY_WAIT_TIME_IN_MILLIS.defaultValue()));
+ }
+
+ /**
+ * rebuild lock related configs, only support ZK related lock for now.
+ */
+ private TypedProperties refreshLockConfig(HoodieWriteConfig writeConfig, String key) {
+ TypedProperties props = new TypedProperties(writeConfig.getProps());
+ props.setProperty(LockConfiguration.ZK_LOCK_KEY_PROP_KEY, key);
Review Comment:
Sure thing Changed. Also we could mark a TODO here to support more lock provider as next step
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] XuQianJin-Stars commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
XuQianJin-Stars commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028678716
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/SimpleDirectMarkerBasedEarlyConflictDetectionStrategy.java:
##########
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.table.marker;
+
+import org.apache.hudi.common.conflict.detection.HoodieDirectMarkerBasedEarlyConflictDetectionStrategy;
+import org.apache.hudi.common.fs.HoodieWrapperFileSystem;
+import org.apache.hudi.common.table.timeline.HoodieActiveTimeline;
+import org.apache.hudi.common.util.MarkerUtils;
+import org.apache.hudi.config.HoodieWriteConfig;
+import org.apache.hudi.exception.HoodieEarlyConflictDetectionException;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.ConcurrentModificationException;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+/**
+ * This strategy is used for direct marker writers, trying to do early conflict detection.
+ * It will use fileSystem api like list and exist directly to check if there is any marker file conflict.
+ */
+public class SimpleDirectMarkerBasedEarlyConflictDetectionStrategy extends HoodieDirectMarkerBasedEarlyConflictDetectionStrategy {
+
+ private static final Logger LOG = LogManager.getLogger(SimpleDirectMarkerBasedEarlyConflictDetectionStrategy.class);
+
+ public SimpleDirectMarkerBasedEarlyConflictDetectionStrategy(String basePath, HoodieWrapperFileSystem fs, String partitionPath, String fileId, String instantTime,
+ HoodieActiveTimeline activeTimeline, HoodieWriteConfig config, Boolean checkCommitConflict) {
+ super(basePath, fs, partitionPath, fileId, instantTime, activeTimeline, config, checkCommitConflict);
+ }
+
+ @Override
+ public boolean hasMarkerConflict() {
+ try {
+ return checkMarkerConflict(basePath, partitionPath, fileId, fs, instantTime)
+ || (checkCommitConflict && MarkerUtils.hasCommitConflict(Stream.of(fileId).collect(Collectors.toSet()), basePath, completedCommitInstants));
+ } catch (IOException e) {
+ LOG.warn("Exception occurs during create marker file in eager conflict detection mode.");
+ throw new HoodieIOException("Exception occurs during create marker file in eager conflict detection mode.", e);
+ }
+ }
+
+ @Override
+ public void resolveMarkerConflict(String basePath, String partitionPath, String dataFileName) {
+ throw new HoodieEarlyConflictDetectionException(new ConcurrentModificationException("Early conflict detected but cannot resolve conflicts for overlapping writes"));
Review Comment:
```
public void resolveMarkerConflict(String basePath, String partitionPath, String dataFileName, String instantTime, HoodieTableMetaClient metaClient) {
LOG.warn("Detected resolve conflict marker files: " + partitionPath + "/" + dataFileName + " for " + instantTime);
if (!metaClient.reloadActiveTimeline().filterPendingCompactionTimeline().containsInstant(instantTime)) {
throw new HoodieEarlyConflictDetectionException(new ConcurrentModificationException("Early conflict detected but cannot resolve conflicts for overlapping writes"));
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1028264254
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/marker/MarkerCheckerRunnable.java:
##########
@@ -0,0 +1,193 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.timeline.service.handlers.marker;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hudi.common.model.HoodieCommitMetadata;
+import org.apache.hudi.common.table.HoodieTableMetaClient;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.HoodieTimer;
+import org.apache.hudi.common.util.MarkerUtils;
+import org.apache.hudi.exception.HoodieIOException;
+import org.apache.hudi.timeline.service.handlers.MarkerHandler;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+public class MarkerCheckerRunnable implements Runnable {
+ private static final Logger LOG = LogManager.getLogger(MarkerCheckerRunnable.class);
+
+ private MarkerHandler markerHandler;
+ private String markerDir;
+ private String basePath;
+ private FileSystem fs;
+ private AtomicBoolean hasConflict;
+ private long maxAllowableHeartbeatIntervalInMs;
+ private Set<HoodieInstant> oldInstants;
+
+ public MarkerCheckerRunnable(AtomicBoolean hasConflict, MarkerHandler markerHandler, String markerDir,
+ String basePath, FileSystem fileSystem, long maxAllowableHeartbeatIntervalInMs,
+ Set<HoodieInstant> oldInstants) {
+ this.markerHandler = markerHandler;
+ this.markerDir = markerDir;
+ this.basePath = basePath;
+ this.fs = fileSystem;
+ this.hasConflict = hasConflict;
+ this.maxAllowableHeartbeatIntervalInMs = maxAllowableHeartbeatIntervalInMs;
+ this.oldInstants = oldInstants;
+ }
+
+ @Override
+ public void run() {
+ try {
+ if (!fs.exists(new Path(markerDir))) {
+ return;
+ }
+
+ HoodieTimer timer = new HoodieTimer().startTimer();
+ Set<String> currentInstantAllMarkers = markerHandler.getAllMarkers(markerDir);
+ Path tempPath = new Path(basePath + Path.SEPARATOR + HoodieTableMetaClient.TEMPFOLDER_NAME);
+
+ List<Path> instants = MarkerUtils.getAllMarkerDir(tempPath, fs);
+ List<String> candidate = getCandidateInstants(instants, markerDirToInstantTime(markerDir));
+ Set<String> tableMarkers = candidate.stream().flatMap(instant -> {
+ return MarkerUtils.readTimelineServerBasedMarkersFromFileSystemLocally(instant, fs).stream();
+ }).collect(Collectors.toSet());
+
+ Set<String> currentFileIDs = currentInstantAllMarkers.stream().map(this::makerToPartitionAndFileID).collect(Collectors.toSet());
+ Set<String> tableFilesIDs = tableMarkers.stream().map(this::makerToPartitionAndFileID).collect(Collectors.toSet());
+
+ currentFileIDs.retainAll(tableFilesIDs);
+
+ if (!currentFileIDs.isEmpty() || hasCommitConflict(currentInstantAllMarkers, basePath)) {
+ LOG.warn("Conflict writing detected based on markers!\n"
+ + "Conflict markers: " + currentInstantAllMarkers + "\n"
+ + "Table markers: " + tableMarkers);
+ hasConflict.compareAndSet(false, true);
+ }
+ LOG.info("Finish batch marker checker in " + timer.endTimer() + " ms");
+
+ } catch (IOException e) {
+ throw new HoodieIOException("IOException occurs during checking marker conflict");
+ }
+ }
+
+ /**
+ * Get Candidate Instant to do conflict checking:
+ * 1. Skip current writer related instant(currentInstantTime)
+ * 2. Skip all instants after currentInstantTime
+ * 3. Skip dead writers related instants based on heart-beat
+ * @param instants
+ * @return
+ */
+ private List<String> getCandidateInstants(List<Path> instants, String currentInstantTime) {
Review Comment:
Yeap, actually there are some diff here:
for occ getCandidateInstants which depends on a state:
```
// To find which instants are conflicting, we apply the following logic
// 1. Get completed instants timeline only for commits that have happened since the last successful write.
// 2. Get any scheduled or completed compaction or clustering operations that have started and/or finished
// after the current instant. We need to check for write conflicts since they may have mutated the same files
// that are being newly created by the current write.
```
For current early conflict detection getCandidateInstants:
```
/**
* Get Candidate Instant to do conflict checking:
* 1. Skip current writer related instant(currentInstantTime)
* 2. Skip all instants after currentInstantTime
* 3. Skip dead writers related instants based on heart-beat
* @param instants
* @return
*/
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1322015306
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 844b10a447845b27056b2520b67a624a0d1bd71a Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148)
* 71e0d1e91d2f540b441838263362c31251fa4ab5 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1029444656
##########
hudi-common/src/main/java/org/apache/hudi/common/util/MarkerUtils.java:
##########
@@ -208,10 +219,116 @@ public static Set<String> readMarkersFromFile(Path markersFilePath, Serializable
fsDataInputStream = fs.open(markersFilePath);
markers = new HashSet<>(FileIOUtils.readAsUTFStringLines(fsDataInputStream));
} catch (IOException e) {
- throw new HoodieIOException("Failed to read MARKERS file " + markersFilePath, e);
+ if (ignoreException) {
+ LOG.warn("IOException occurs during read MARKERS file, ", e);
+ } else {
+ throw new HoodieIOException("Failed to read MARKERS file " + markersFilePath, e);
+ }
} finally {
closeQuietly(fsDataInputStream);
}
return markers;
}
+
+ public static List<Path> getAllMarkerDir(Path tempPath, FileSystem fs) throws IOException {
+ return Arrays.stream(fs.listStatus(tempPath)).map(FileStatus::getPath).collect(Collectors.toList());
+ }
+
+ public static boolean hasCommitConflict(HoodieActiveTimeline activeTimeline, Set<String> currentFileIDs, Set<HoodieInstant> completedCommitInstants) {
+
+ Set<HoodieInstant> currentInstants = activeTimeline.reload().getCommitsTimeline().filterCompletedInstants().getInstants().collect(Collectors.toSet());
+
+ currentInstants.removeAll(completedCommitInstants);
+ Set<String> missingFileIDs = currentInstants.stream().flatMap(instant -> {
+ try {
+ return HoodieCommitMetadata.fromBytes(activeTimeline.getInstantDetails(instant).get(), HoodieCommitMetadata.class)
+ .getFileIdAndRelativePaths().keySet().stream();
+ } catch (Exception e) {
+ return Stream.empty();
+ }
+ }).collect(Collectors.toSet());
+ currentFileIDs.retainAll(missingFileIDs);
+ return !currentFileIDs.isEmpty();
+ }
+
+ /**
+ * Get Candidate Instant to do conflict checking:
+ * 1. Skip current writer related instant(currentInstantTime)
+ * 2. Skip all instants after currentInstantTime
+ * 3. Skip dead writers related instants based on heart-beat
+ * 4. Skip pending compaction instant (For now we don' do early conflict check with compact action)
+ * Because we don't want to let pending compaction block common writer.
+ * @param instants
+ * @return
+ */
+ public static List<String> getCandidateInstants(HoodieActiveTimeline activeTimeline, List<Path> instants, String currentInstantTime,
+ long maxAllowableHeartbeatIntervalInMs, FileSystem fs, String basePath) {
+
+ HoodieActiveTimeline reloadActive = activeTimeline.reload();
+
+ return instants.stream().map(Path::toString).filter(instantPath -> {
+ String instantTime = markerDirToInstantTime(instantPath);
+ return instantTime.compareToIgnoreCase(currentInstantTime) < 0
+ && !reloadActive.filterPendingCompactionTimeline().containsInstant(instantTime)
Review Comment:
Skip clustering instant and compaction instant when doing early conflict detection.
Because we don't want to let any pending compaction block common writer.
For example there is a failed compaction in active-timeline while common writer continues to write data to this failed compaction related file groups.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1321897189
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 465536ff2afcf217c19ca00b4a90a5e04d372bc3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962) Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975) Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976)
* 844b10a447845b27056b2520b67a624a0d1bd71a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1323651968
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 8a402c4c8574cca66047e614038a002cb24e8e94 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182)
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* b97bb164efb5264476ebdf3673396583ec90d1a6 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1373299095
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 0447a7135e0cf136cd894291a1efd4374c3efd66 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206)
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* c973c81cc138a320e562f293f1dc72828ed8fac6 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] zhangyue19921010 commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
zhangyue19921010 commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1376756472
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1376117122
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 6bb19747ad7841c458fd0f1adbbfb99d6be13a14 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179)
* be0d5b4e5a211b1ec9f3bdad65177e4667f9a222 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1376465146
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* be0d5b4e5a211b1ec9f3bdad65177e4667f9a222 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on a diff in pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
yihua commented on code in PR #6133:
URL: https://github.com/apache/hudi/pull/6133#discussion_r1080669381
##########
hudi-timeline-service/src/main/java/org/apache/hudi/timeline/service/handlers/MarkerHandler.java:
##########
@@ -150,7 +167,48 @@ public boolean doesMarkerDirExist(String markerDir) {
* @param markerName marker name
* @return the {@code CompletableFuture} instance for the request
*/
- public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName) {
+ public CompletableFuture<String> createMarker(Context context, String markerDir, String markerName, String basePath) {
+ // Step1 do early conflict detection if enable
+ if (timelineServiceConfig.earlyConflictDetectionEnable) {
+ try {
+ synchronized (earlyConflictDetectionLock) {
+ if (earlyConflictDetectionStrategy == null) {
+ earlyConflictDetectionStrategy = (HoodieTimelineServerBasedEarlyConflictDetectionStrategy) ReflectionUtils.loadClass(timelineServiceConfig.earlyConflictDetectionStrategy,
+ basePath, markerDir, markerName, timelineServiceConfig.checkCommitConflict);
+ }
+
+ // markerDir => $base_path/.hoodie/.temp/$instant_time
+ // If markerDir is changed like move to the next instant action, we need to fresh this earlyConflictDetectionStrategy.
+ // For specific instant related create marker action, we only call this check/fresh once
+ // instead of starting the conflict detector for every request
+ if (!markerDir.equalsIgnoreCase(currentMarkerDir)) {
+ this.currentMarkerDir = markerDir;
+ Set<String> actions = CollectionUtils.createSet(COMMIT_ACTION, DELTA_COMMIT_ACTION, REPLACE_COMMIT_ACTION);
+ Set<HoodieInstant> oldInstants = new HashSet<>(
+ viewManager.getFileSystemView(basePath)
+ .getTimeline()
+ .filterCompletedInstants()
+ .filter(instant -> actions.contains(instant.getAction()))
+ .getInstants());
+
+ earlyConflictDetectionStrategy.fresh(timelineServiceConfig.asyncConflictDetectorBatchIntervalMs,
+ timelineServiceConfig.asyncConflictDetectorBatchPeriodMs, markerDir, basePath, timelineServiceConfig.maxAllowableHeartbeatIntervalInMs, fileSystem,
+ this, oldInstants);
+ }
+ }
+
+ earlyConflictDetectionStrategy.detectAndResolveConflictIfNecessary();
+
+ } catch (HoodieEarlyConflictDetectionException e) {
+ LOG.warn("Detect write conflict, failed to create marker with early conflict detection enable", e);
+ return finishCreateMarkerFuture(context, markerDir, markerName);
+ } catch (Exception ex) {
+ LOG.warn("Catch exception during detect and resolve write conflict.");
Review Comment:
nit: Let's make the error message readable.
Also, in this case when early conflict detection fails to execute, we still allow the marker creation to continue.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/marker/WriteMarkers.java:
##########
@@ -60,8 +67,26 @@ public WriteMarkers(String basePath, String markerFolderPath, String instantTime
* @param type write IO type
* @return the marker path
*/
- public Option<Path> create(String partitionPath, String dataFileName, IOType type) {
- return create(partitionPath, dataFileName, type, false);
+ public Option<Path> create(String partitionPath, String dataFileName, IOType type, Option<HoodieWriteConfig> writeConfig,
+ Option<String> fileId, Option<HoodieTableMetaClient> metaClient) {
Review Comment:
It's not necessary to use `Option` for the arguments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1397910520
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214",
"triggerID" : "1376756472",
"triggerType" : "MANUAL"
}, {
"hash" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422",
"triggerID" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"triggerType" : "PUSH"
}, {
"hash" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14480",
"triggerID" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 67b3892f4bafe64ecf203a53bb86db22e61ae587 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422)
* a2980b73d4ad32976360804059fcd6df969b9f89 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14480)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1400007877
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10031",
"triggerID" : "fcaaf9d6dc063aed48319e335c181e913ac9106f",
"triggerType" : "PUSH"
}, {
"hash" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10033",
"triggerID" : "553fb00b590e9586ade21e86569ccb30d8dec7a9",
"triggerType" : "PUSH"
}, {
"hash" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dbe3db845908d261baa5a1aa71d19e0db55816de",
"triggerType" : "PUSH"
}, {
"hash" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10034",
"triggerID" : "66b7d1b75c7a201b5d108cd48a70363dd00a1795",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"triggerType" : "PUSH"
}, {
"hash" : "64819e4ab1ed12788e126072fa50e964fc62c257",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10203",
"triggerID" : "1235062563",
"triggerType" : "MANUAL"
}, {
"hash" : "678cce4a9748cb54a90a559384a0cb0443082535",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "678cce4a9748cb54a90a559384a0cb0443082535",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12326",
"triggerID" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"triggerType" : "PUSH"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12349",
"triggerID" : "1284215076",
"triggerType" : "MANUAL"
}, {
"hash" : "645766d09d376bc46a1e0b6af770d9e84f778b24",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12360",
"triggerID" : "1284787088",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12557",
"triggerID" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"triggerType" : "PUSH"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12584",
"triggerID" : "1291413721",
"triggerType" : "MANUAL"
}, {
"hash" : "e23ab613d4aad3844374b21d987f95ca6f5183e5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12600",
"triggerID" : "1291894521",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12962",
"triggerID" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"triggerType" : "PUSH"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12975",
"triggerID" : "1312417316",
"triggerType" : "MANUAL"
}, {
"hash" : "465536ff2afcf217c19ca00b4a90a5e04d372bc3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12976",
"triggerID" : "1312472305",
"triggerType" : "MANUAL"
}, {
"hash" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13148",
"triggerID" : "844b10a447845b27056b2520b67a624a0d1bd71a",
"triggerType" : "PUSH"
}, {
"hash" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13149",
"triggerID" : "71e0d1e91d2f540b441838263362c31251fa4ab5",
"triggerType" : "PUSH"
}, {
"hash" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fc5bf1ce7921bf25acc3659565457264d8b9dc2",
"triggerType" : "PUSH"
}, {
"hash" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13155",
"triggerID" : "316e5aefa89ec2dc35928af7c7da4b11a23aab20",
"triggerType" : "PUSH"
}, {
"hash" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0b74647767677a4cc1193295b493dc0537dd4c96",
"triggerType" : "PUSH"
}, {
"hash" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13157",
"triggerID" : "ffd8315ca6dc2c771e90b5fa983c0b41010605eb",
"triggerType" : "PUSH"
}, {
"hash" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13182",
"triggerID" : "8a402c4c8574cca66047e614038a002cb24e8e94",
"triggerType" : "PUSH"
}, {
"hash" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3369e5e8770cf9eb4c4d272f7c3af54933c992aa",
"triggerType" : "PUSH"
}, {
"hash" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13185",
"triggerID" : "b97bb164efb5264476ebdf3673396583ec90d1a6",
"triggerType" : "PUSH"
}, {
"hash" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ccecb4fa727cc254cf4780012c28bab24e6afde",
"triggerType" : "PUSH"
}, {
"hash" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13189",
"triggerID" : "869baf7be4f626d10ca20606f7816b941203ec3a",
"triggerType" : "PUSH"
}, {
"hash" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13206",
"triggerID" : "0447a7135e0cf136cd894291a1efd4374c3efd66",
"triggerType" : "PUSH"
}, {
"hash" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6fdf901df1086d6ecc07c7987b6a3212b08eaefb",
"triggerType" : "PUSH"
}, {
"hash" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14138",
"triggerID" : "c973c81cc138a320e562f293f1dc72828ed8fac6",
"triggerType" : "PUSH"
}, {
"hash" : "c412635478d07b2bb37ad6738c06882f84b54001",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14176",
"triggerID" : "c412635478d07b2bb37ad6738c06882f84b54001",
"triggerType" : "PUSH"
}, {
"hash" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14179",
"triggerID" : "6bb19747ad7841c458fd0f1adbbfb99d6be13a14",
"triggerType" : "PUSH"
}, {
"hash" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14200",
"triggerID" : "be0d5b4e5a211b1ec9f3bdad65177e4667f9a222",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14207",
"triggerID" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"triggerType" : "PUSH"
}, {
"hash" : "1b837ec65943331f6c805f723b088240ae66dfaf",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14214",
"triggerID" : "1376756472",
"triggerType" : "MANUAL"
}, {
"hash" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14422",
"triggerID" : "67b3892f4bafe64ecf203a53bb86db22e61ae587",
"triggerType" : "PUSH"
}, {
"hash" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14480",
"triggerID" : "a2980b73d4ad32976360804059fcd6df969b9f89",
"triggerType" : "PUSH"
}, {
"hash" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14484",
"triggerID" : "7344fabacee437adb4b55ca922df0f5fb14ae372",
"triggerType" : "PUSH"
}, {
"hash" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14518",
"triggerID" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"triggerType" : "PUSH"
}, {
"hash" : "501e47fb5c554a32acd7d2b410b5cdcf880f6057",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14549",
"triggerID" : "1399842193",
"triggerType" : "MANUAL"
}, {
"hash" : "0a77616b5bd7b7d20259b3eb12d31e571f2af60e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14550",
"triggerID" : "0a77616b5bd7b7d20259b3eb12d31e571f2af60e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dbe3db845908d261baa5a1aa71d19e0db55816de UNKNOWN
* 678cce4a9748cb54a90a559384a0cb0443082535 UNKNOWN
* 6fc5bf1ce7921bf25acc3659565457264d8b9dc2 UNKNOWN
* 0b74647767677a4cc1193295b493dc0537dd4c96 UNKNOWN
* 3369e5e8770cf9eb4c4d272f7c3af54933c992aa UNKNOWN
* 1ccecb4fa727cc254cf4780012c28bab24e6afde UNKNOWN
* 6fdf901df1086d6ecc07c7987b6a3212b08eaefb UNKNOWN
* 0a77616b5bd7b7d20259b3eb12d31e571f2af60e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14550)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on pull request #6133: [HUDI-1575] Early Conflict Detection For Multi-writer
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on PR #6133:
URL: https://github.com/apache/hudi/pull/6133#issuecomment-1398762897
I'm doing more thorough tests in CI. Please do not merge this PR now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org