You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/11/27 00:41:37 UTC
[GitHub] [hudi] manojpec opened a new pull request #4134: [HUDI-2862] Table upgrade and metadata table bootstrap under the table lock
manojpec opened a new pull request #4134:
URL: https://github.com/apache/hudi/pull/4134
## What is the purpose of the pull request
- Today table upgrade need is detected by the first write client after upgrade
and if there is write concurrency mode configured it grabs the transaction
lock to protect the table upgrade from other concurrent writers. However, the
follow on metadata table creation and the initial bootstrapping also needs
similar global protection to avoid race in inflight commits and the metadata
table bootstrapping proces. Made the table upgrade and the follow-on metadata
table creation and there by the initial bootstrapping process under the
table level lock to avoid potential race with concurrent writers and other
async table services.
## Brief change log
*(for example:)*
- *Modify AnnotationLocation checkstyle rule in checkstyle.xml*
## Verify this pull request
*(Please pick either of the following options)*
This pull request is a trivial rework / code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end.*
- *Added HoodieClientWriteTest to verify the change.*
- *Manually verified the change by running a job locally.*
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #4134: [HUDI-2862] Table upgrade and metadata table bootstrap under the table lock (replaces #4124)
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on pull request #4134:
URL: https://github.com/apache/hudi/pull/4134#issuecomment-980482925
@hudi-bot azure run
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4134: [HUDI-2862] Table upgrade and metadata table bootstrap under the table lock
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4134:
URL: https://github.com/apache/hudi/pull/4134#issuecomment-980478263
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3837",
"triggerID" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"triggerType" : "PUSH"
}, {
"hash" : "2a59c48e20a0e20be83769a090bb3af6e517838d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2a59c48e20a0e20be83769a090bb3af6e517838d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 84a405a4c7da21d8c202d7507b91dc8ac2fc3489 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3837)
* 2a59c48e20a0e20be83769a090bb3af6e517838d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4134: [HUDI-2862] Table upgrade and metadata table bootstrap under the table lock
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4134:
URL: https://github.com/apache/hudi/pull/4134#issuecomment-980478263
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3837",
"triggerID" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"triggerType" : "PUSH"
}, {
"hash" : "2a59c48e20a0e20be83769a090bb3af6e517838d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2a59c48e20a0e20be83769a090bb3af6e517838d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 84a405a4c7da21d8c202d7507b91dc8ac2fc3489 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3837)
* 2a59c48e20a0e20be83769a090bb3af6e517838d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4134: [HUDI-2862] Table upgrade and metadata table bootstrap under the table lock
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4134:
URL: https://github.com/apache/hudi/pull/4134#issuecomment-980477137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 84a405a4c7da21d8c202d7507b91dc8ac2fc3489 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4134: [HUDI-2862] Table upgrade and metadata table bootstrap under the table lock (replaces #4124)
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4134:
URL: https://github.com/apache/hudi/pull/4134#issuecomment-980484284
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3837",
"triggerID" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"triggerType" : "PUSH"
}, {
"hash" : "2a59c48e20a0e20be83769a090bb3af6e517838d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3841",
"triggerID" : "2a59c48e20a0e20be83769a090bb3af6e517838d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 84a405a4c7da21d8c202d7507b91dc8ac2fc3489 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3837)
* 2a59c48e20a0e20be83769a090bb3af6e517838d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3841)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4134: [HUDI-2862] Table upgrade and metadata table bootstrap under the table lock (replaces #4124)
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4134:
URL: https://github.com/apache/hudi/pull/4134#issuecomment-980478575
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3837",
"triggerID" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"triggerType" : "PUSH"
}, {
"hash" : "2a59c48e20a0e20be83769a090bb3af6e517838d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2a59c48e20a0e20be83769a090bb3af6e517838d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 84a405a4c7da21d8c202d7507b91dc8ac2fc3489 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3837)
* 2a59c48e20a0e20be83769a090bb3af6e517838d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4134: [HUDI-2862] Table upgrade and metadata table bootstrap under the table lock
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4134:
URL: https://github.com/apache/hudi/pull/4134#issuecomment-980477137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 84a405a4c7da21d8c202d7507b91dc8ac2fc3489 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] manojpec commented on pull request #4134: [HUDI-2862] Table upgrade and metadata table bootstrap under the table lock (replaces #4124)
Posted by GitBox <gi...@apache.org>.
manojpec commented on pull request #4134:
URL: https://github.com/apache/hudi/pull/4134#issuecomment-980496446
This change is now pulled into https://github.com/apache/hudi/pull/4114. Closing this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4134: [HUDI-2862] Table upgrade and metadata table bootstrap under the table lock
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4134:
URL: https://github.com/apache/hudi/pull/4134#issuecomment-980478575
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3837",
"triggerID" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"triggerType" : "PUSH"
}, {
"hash" : "2a59c48e20a0e20be83769a090bb3af6e517838d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2a59c48e20a0e20be83769a090bb3af6e517838d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 84a405a4c7da21d8c202d7507b91dc8ac2fc3489 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3837)
* 2a59c48e20a0e20be83769a090bb3af6e517838d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #4134: [HUDI-2862] Table upgrade and metadata table bootstrap under the table lock (replaces #4124)
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on pull request #4134:
URL: https://github.com/apache/hudi/pull/4134#issuecomment-980480123
@hudi-bot azure run
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4134: [HUDI-2862] Table upgrade and metadata table bootstrap under the table lock (replaces #4124)
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4134:
URL: https://github.com/apache/hudi/pull/4134#issuecomment-980484284
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3837",
"triggerID" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"triggerType" : "PUSH"
}, {
"hash" : "2a59c48e20a0e20be83769a090bb3af6e517838d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3841",
"triggerID" : "2a59c48e20a0e20be83769a090bb3af6e517838d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 84a405a4c7da21d8c202d7507b91dc8ac2fc3489 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3837)
* 2a59c48e20a0e20be83769a090bb3af6e517838d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3841)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] manojpec closed pull request #4134: [HUDI-2862] Table upgrade and metadata table bootstrap under the table lock (replaces #4124)
Posted by GitBox <gi...@apache.org>.
manojpec closed pull request #4134:
URL: https://github.com/apache/hudi/pull/4134
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4134: [HUDI-2862] Table upgrade and metadata table bootstrap under the table lock (replaces #4124)
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4134:
URL: https://github.com/apache/hudi/pull/4134#issuecomment-980490105
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3837",
"triggerID" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"triggerType" : "PUSH"
}, {
"hash" : "2a59c48e20a0e20be83769a090bb3af6e517838d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3841",
"triggerID" : "2a59c48e20a0e20be83769a090bb3af6e517838d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2a59c48e20a0e20be83769a090bb3af6e517838d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3841)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4134: [HUDI-2862] Table upgrade and metadata table bootstrap under the table lock
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4134:
URL: https://github.com/apache/hudi/pull/4134#issuecomment-980477403
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3837",
"triggerID" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 84a405a4c7da21d8c202d7507b91dc8ac2fc3489 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3837)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4134: [HUDI-2862] Table upgrade and metadata table bootstrap under the table lock
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4134:
URL: https://github.com/apache/hudi/pull/4134#issuecomment-980477403
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3837",
"triggerID" : "84a405a4c7da21d8c202d7507b91dc8ac2fc3489",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 84a405a4c7da21d8c202d7507b91dc8ac2fc3489 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=3837)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org