You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "danny0405 (via GitHub)" <gi...@apache.org> on 2023/04/22 04:56:38 UTC
[GitHub] [hudi] danny0405 opened a new pull request, #8542: [HUDI-5887][HOTFIX] Always execute the lock action for LockManager
danny0405 opened a new pull request, #8542:
URL: https://github.com/apache/hudi/pull/8542
### Change Logs
The LockManager#lock is invoked from the TransactionManager which already takes the check for the requirement of a explicit lock.
### Impact
none
### Risk level (write none, low medium or high below)
none
### Documentation Update
_Describe any necessary documentation update if there is any new feature, config, or user-facing change_
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #8542: [HUDI-6123] Auto adjust lock configs only for single writer
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #8542:
URL: https://github.com/apache/hudi/pull/8542#discussion_r1290819723
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -2483,8 +2483,15 @@ public boolean areReleaseResourceEnabled() {
/**
* Returns whether the explicit guard of lock is required.
*/
- public boolean needsLockGuard() {
- return isMetadataTableEnabled() || getWriteConcurrencyMode().supportsOptimisticConcurrencyControl();
+ public boolean isLockRequired() {
+ return !isDefaultLockProvider() || getWriteConcurrencyMode().supportsOptimisticConcurrencyControl();
Review Comment:
Yeah, I was expecting the use to set up the optimistic concurrency control explicitly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on a diff in pull request #8542: [HUDI-6123] Auto adjust lock configs only for single writer
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #8542:
URL: https://github.com/apache/hudi/pull/8542#discussion_r1290687841
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -2483,8 +2483,15 @@ public boolean areReleaseResourceEnabled() {
/**
* Returns whether the explicit guard of lock is required.
*/
- public boolean needsLockGuard() {
- return isMetadataTableEnabled() || getWriteConcurrencyMode().supportsOptimisticConcurrencyControl();
+ public boolean isLockRequired() {
+ return !isDefaultLockProvider() || getWriteConcurrencyMode().supportsOptimisticConcurrencyControl();
Review Comment:
The user can still set ZK lock for a single writer with sync table services and this method returns false in this case, which is unexpected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8542: [HUDI-6123] Add lock guard by default for single table with MDT and sync table services
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8542:
URL: https://github.com/apache/hudi/pull/8542#issuecomment-1518586693
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550",
"triggerID" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"triggerType" : "PUSH"
}, {
"hash" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16554",
"triggerID" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"triggerType" : "PUSH"
}, {
"hash" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ccd692493bc1e1669f521bc1368324fa3fb7032f Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550)
* ec9f97ddc1509a940fc21431f34f6ace9925f79e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16554)
* 63210fe88b7ea31d5ac81c1c7753a8c375a74036 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8542: [HUDI-5887][HOTFIX] Always execute the lock action for LockManager
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8542:
URL: https://github.com/apache/hudi/pull/8542#issuecomment-1518514660
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550",
"triggerID" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ccd692493bc1e1669f521bc1368324fa3fb7032f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 merged pull request #8542: [HUDI-6123] Auto adjust lock configs only for single writer
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 merged PR #8542:
URL: https://github.com/apache/hudi/pull/8542
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8542: [HUDI-5887][HOTFIX] Always execute the lock action for LockManager
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8542:
URL: https://github.com/apache/hudi/pull/8542#issuecomment-1518539101
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550",
"triggerID" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"triggerType" : "PUSH"
}, {
"hash" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16554",
"triggerID" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ccd692493bc1e1669f521bc1368324fa3fb7032f Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550)
* ec9f97ddc1509a940fc21431f34f6ace9925f79e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16554)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8542: [HUDI-6123] Auto adjust lock configs only for single writer
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8542:
URL: https://github.com/apache/hudi/pull/8542#issuecomment-1518710744
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550",
"triggerID" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"triggerType" : "PUSH"
}, {
"hash" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16554",
"triggerID" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"triggerType" : "PUSH"
}, {
"hash" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16557",
"triggerID" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"triggerType" : "PUSH"
}, {
"hash" : "9f8ef0287c23c0c86156fa804fc8c7fafd79cd8d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16566",
"triggerID" : "9f8ef0287c23c0c86156fa804fc8c7fafd79cd8d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 63210fe88b7ea31d5ac81c1c7753a8c375a74036 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16557)
* 9f8ef0287c23c0c86156fa804fc8c7fafd79cd8d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16566)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8542: [HUDI-6123] Add lock guard by default for single table with MDT and sync table services
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8542:
URL: https://github.com/apache/hudi/pull/8542#issuecomment-1518588285
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550",
"triggerID" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"triggerType" : "PUSH"
}, {
"hash" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16554",
"triggerID" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"triggerType" : "PUSH"
}, {
"hash" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16557",
"triggerID" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ec9f97ddc1509a940fc21431f34f6ace9925f79e Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16554)
* 63210fe88b7ea31d5ac81c1c7753a8c375a74036 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16557)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8542: [HUDI-6123] Auto adjust lock configs only for single writer
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8542:
URL: https://github.com/apache/hudi/pull/8542#issuecomment-1518921765
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550",
"triggerID" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"triggerType" : "PUSH"
}, {
"hash" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16554",
"triggerID" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"triggerType" : "PUSH"
}, {
"hash" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16557",
"triggerID" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"triggerType" : "PUSH"
}, {
"hash" : "9f8ef0287c23c0c86156fa804fc8c7fafd79cd8d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16566",
"triggerID" : "9f8ef0287c23c0c86156fa804fc8c7fafd79cd8d",
"triggerType" : "PUSH"
}, {
"hash" : "3e1d71eda3332e13f6125ac2aa03b314839663ae",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16571",
"triggerID" : "3e1d71eda3332e13f6125ac2aa03b314839663ae",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9f8ef0287c23c0c86156fa804fc8c7fafd79cd8d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16566)
* 3e1d71eda3332e13f6125ac2aa03b314839663ae Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16571)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8542: [HUDI-6123] Auto adjust lock configs only for single writer
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8542:
URL: https://github.com/apache/hudi/pull/8542#issuecomment-1518944228
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550",
"triggerID" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"triggerType" : "PUSH"
}, {
"hash" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16554",
"triggerID" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"triggerType" : "PUSH"
}, {
"hash" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16557",
"triggerID" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"triggerType" : "PUSH"
}, {
"hash" : "9f8ef0287c23c0c86156fa804fc8c7fafd79cd8d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16566",
"triggerID" : "9f8ef0287c23c0c86156fa804fc8c7fafd79cd8d",
"triggerType" : "PUSH"
}, {
"hash" : "3e1d71eda3332e13f6125ac2aa03b314839663ae",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16571",
"triggerID" : "3e1d71eda3332e13f6125ac2aa03b314839663ae",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3e1d71eda3332e13f6125ac2aa03b314839663ae Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16571)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #8542: [HUDI-6123] Auto adjust lock configs only for single writer
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #8542:
URL: https://github.com/apache/hudi/pull/8542#discussion_r1174532773
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -2483,8 +2483,15 @@ public boolean areReleaseResourceEnabled() {
/**
* Returns whether the explicit guard of lock is required.
*/
- public boolean needsLockGuard() {
- return isMetadataTableEnabled() || getWriteConcurrencyMode().supportsOptimisticConcurrencyControl();
+ public boolean isLockRequired() {
+ return !isDefaultLockProvider() || getWriteConcurrencyMode().supportsOptimisticConcurrencyControl();
Review Comment:
Yes, to be compatible with the exising behavior, if the lock provider is reset by auto adjust, then it means it is definite that the lock is required.
MDT enabled does not mean we need a lock provider(check the premise of auto adjust lock actions), the change in #8111 is a little aggresive, so fix it altogether.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8542: [HUDI-6123] Auto adjust lock configs only for single writer
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8542:
URL: https://github.com/apache/hudi/pull/8542#issuecomment-1518709349
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550",
"triggerID" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"triggerType" : "PUSH"
}, {
"hash" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16554",
"triggerID" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"triggerType" : "PUSH"
}, {
"hash" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16557",
"triggerID" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"triggerType" : "PUSH"
}, {
"hash" : "9f8ef0287c23c0c86156fa804fc8c7fafd79cd8d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9f8ef0287c23c0c86156fa804fc8c7fafd79cd8d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 63210fe88b7ea31d5ac81c1c7753a8c375a74036 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16557)
* 9f8ef0287c23c0c86156fa804fc8c7fafd79cd8d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8542: [HUDI-5887][HOTFIX] Always execute the lock action for LockManager
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8542:
URL: https://github.com/apache/hudi/pull/8542#issuecomment-1518527440
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550",
"triggerID" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"triggerType" : "PUSH"
}, {
"hash" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ccd692493bc1e1669f521bc1368324fa3fb7032f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550)
* ec9f97ddc1509a940fc21431f34f6ace9925f79e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xiarixiaoyao commented on a diff in pull request #8542: [HUDI-6123] Auto adjust lock configs only for single writer
Posted by "xiarixiaoyao (via GitHub)" <gi...@apache.org>.
xiarixiaoyao commented on code in PR #8542:
URL: https://github.com/apache/hudi/pull/8542#discussion_r1174531825
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java:
##########
@@ -2483,8 +2483,15 @@ public boolean areReleaseResourceEnabled() {
/**
* Returns whether the explicit guard of lock is required.
*/
- public boolean needsLockGuard() {
- return isMetadataTableEnabled() || getWriteConcurrencyMode().supportsOptimisticConcurrencyControl();
+ public boolean isLockRequired() {
+ return !isDefaultLockProvider() || getWriteConcurrencyMode().supportsOptimisticConcurrencyControl();
Review Comment:
Default lock provider is zk lock.
i guess that when MDT enabled, **autoAdjustConfigsForConcurrencyMode**() will set InProcessLockProvider to LockProvider, so we use isDefaultLockProvider() instead of isMetadataTableEnabled()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on pull request #8542: [HUDI-6123] Auto adjust lock configs only for single writer
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on PR #8542:
URL: https://github.com/apache/hudi/pull/8542#issuecomment-1673934722
> For multiple streaming writers with no explicit lock provider set up, InProcessLockProvider should not be used.
In this case, user should explicitly set the lock provider as mentioned in the [docs](https://hudi.apache.org/docs/metadata#deployment-model-c-multi-writer). Auto config adjustment does not intend to solve this problem.
Also, we need to update the docs. This PR brings breaking changes to how configs work for the metadata table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8542: [HUDI-6123] Auto adjust lock configs only for single writer
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8542:
URL: https://github.com/apache/hudi/pull/8542#issuecomment-1518920481
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550",
"triggerID" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"triggerType" : "PUSH"
}, {
"hash" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16554",
"triggerID" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"triggerType" : "PUSH"
}, {
"hash" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16557",
"triggerID" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"triggerType" : "PUSH"
}, {
"hash" : "9f8ef0287c23c0c86156fa804fc8c7fafd79cd8d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16566",
"triggerID" : "9f8ef0287c23c0c86156fa804fc8c7fafd79cd8d",
"triggerType" : "PUSH"
}, {
"hash" : "3e1d71eda3332e13f6125ac2aa03b314839663ae",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3e1d71eda3332e13f6125ac2aa03b314839663ae",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9f8ef0287c23c0c86156fa804fc8c7fafd79cd8d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16566)
* 3e1d71eda3332e13f6125ac2aa03b314839663ae UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8542: [HUDI-6123] Auto adjust lock configs only for single writer
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8542:
URL: https://github.com/apache/hudi/pull/8542#issuecomment-1518893071
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550",
"triggerID" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"triggerType" : "PUSH"
}, {
"hash" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16554",
"triggerID" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"triggerType" : "PUSH"
}, {
"hash" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16557",
"triggerID" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"triggerType" : "PUSH"
}, {
"hash" : "9f8ef0287c23c0c86156fa804fc8c7fafd79cd8d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16566",
"triggerID" : "9f8ef0287c23c0c86156fa804fc8c7fafd79cd8d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9f8ef0287c23c0c86156fa804fc8c7fafd79cd8d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16566)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on pull request #8542: [HUDI-6123] Auto adjust lock configs only for single writer
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on PR #8542:
URL: https://github.com/apache/hudi/pull/8542#issuecomment-1673941293
I also feel that it is an overkill to leverage concurrency control mode to detect single writer as users may not set it correctly, making all the logic useless.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8542: [HUDI-5887][HOTFIX] Always execute the lock action for LockManager
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8542:
URL: https://github.com/apache/hudi/pull/8542#issuecomment-1518513191
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ccd692493bc1e1669f521bc1368324fa3fb7032f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8542: [HUDI-5887][HOTFIX] Always execute the lock action for LockManager
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8542:
URL: https://github.com/apache/hudi/pull/8542#issuecomment-1518529221
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550",
"triggerID" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"triggerType" : "PUSH"
}, {
"hash" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ccd692493bc1e1669f521bc1368324fa3fb7032f Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550)
* ec9f97ddc1509a940fc21431f34f6ace9925f79e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8542: [HUDI-6123] Add lock guard by default for single table with MDT and sync table services
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8542:
URL: https://github.com/apache/hudi/pull/8542#issuecomment-1518695903
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16550",
"triggerID" : "ccd692493bc1e1669f521bc1368324fa3fb7032f",
"triggerType" : "PUSH"
}, {
"hash" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16554",
"triggerID" : "ec9f97ddc1509a940fc21431f34f6ace9925f79e",
"triggerType" : "PUSH"
}, {
"hash" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16557",
"triggerID" : "63210fe88b7ea31d5ac81c1c7753a8c375a74036",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 63210fe88b7ea31d5ac81c1c7753a8c375a74036 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16557)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org