You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "codope (via GitHub)" <gi...@apache.org> on 2023/01/28 08:07:43 UTC
[GitHub] [hudi] codope opened a new pull request, #7776: Enable schema reconciliation by default
codope opened a new pull request, #7776:
URL: https://github.com/apache/hudi/pull/7776
### Change Logs
Enable schema reconciliation by default as commit should go through with the wider schema in case of conflict.
### Impact
Schema will be reconciled by default.
### Risk level (write none, low medium or high below)
low
### Documentation Update
_Describe any necessary documentation update if there is any new feature, config, or user-facing change_
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on pull request #7776: [HUDI-5642] Enable schema reconciliation by default
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on PR #7776:
URL: https://github.com/apache/hudi/pull/7776#issuecomment-1407568524
The test failures are due to SchemaCompatibiltyException. One thing to note is that for an incoming schema with compatible projection of the latest table schema, the table schema is chosen but it may lose any data type promotion. See https://github.com/apache/hudi/blob/3979848a499131db594bbb49eb9ab160531a729d/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala#L566-L574
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7776: [HUDI-5642] Enable schema reconciliation by default
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7776:
URL: https://github.com/apache/hudi/pull/7776#issuecomment-1623230958
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a2dec7d1a6b80501e33250940984e2b2add6e812",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14708",
"triggerID" : "a2dec7d1a6b80501e33250940984e2b2add6e812",
"triggerType" : "PUSH"
}, {
"hash" : "67221fd5f81caad9a68f4ba16ca531f1f2804dda",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=18371",
"triggerID" : "67221fd5f81caad9a68f4ba16ca531f1f2804dda",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 67221fd5f81caad9a68f4ba16ca531f1f2804dda Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=18371)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7776: [HUDI-4071] Enable schema reconciliation by default
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7776:
URL: https://github.com/apache/hudi/pull/7776#issuecomment-1407390921
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a2dec7d1a6b80501e33250940984e2b2add6e812",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14708",
"triggerID" : "a2dec7d1a6b80501e33250940984e2b2add6e812",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a2dec7d1a6b80501e33250940984e2b2add6e812 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14708)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7776: [HUDI-5642] Enable schema reconciliation by default
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7776:
URL: https://github.com/apache/hudi/pull/7776#issuecomment-1623035452
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a2dec7d1a6b80501e33250940984e2b2add6e812",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14708",
"triggerID" : "a2dec7d1a6b80501e33250940984e2b2add6e812",
"triggerType" : "PUSH"
}, {
"hash" : "67221fd5f81caad9a68f4ba16ca531f1f2804dda",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=18371",
"triggerID" : "67221fd5f81caad9a68f4ba16ca531f1f2804dda",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a2dec7d1a6b80501e33250940984e2b2add6e812 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14708)
* 67221fd5f81caad9a68f4ba16ca531f1f2804dda Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=18371)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] vinothchandar commented on pull request #7776: [HUDI-5642] Enable schema reconciliation by default
Posted by "vinothchandar (via GitHub)" <gi...@apache.org>.
vinothchandar commented on PR #7776:
URL: https://github.com/apache/hudi/pull/7776#issuecomment-1522397041
@codope please help me understand why this is critical priority?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on pull request #7776: [HUDI-5642] Enable schema reconciliation by default
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on PR #7776:
URL: https://github.com/apache/hudi/pull/7776#issuecomment-1522676118
> please help me understand why this is critical priority?
this was planned to be done in one of the prev releases but there are some issues. I've lowered the priority for now, but we should take it up for 0.14.0. Updated the fix version on JIRA appropriately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7776: Enable schema reconciliation by default
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7776:
URL: https://github.com/apache/hudi/pull/7776#issuecomment-1407331058
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a2dec7d1a6b80501e33250940984e2b2add6e812",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a2dec7d1a6b80501e33250940984e2b2add6e812",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a2dec7d1a6b80501e33250940984e2b2add6e812 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #7776: Enable schema reconciliation by default
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on PR #7776:
URL: https://github.com/apache/hudi/pull/7776#issuecomment-1407342506
@codope I think we should create a JIRA issue for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7776: Enable schema reconciliation by default
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7776:
URL: https://github.com/apache/hudi/pull/7776#issuecomment-1407336127
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a2dec7d1a6b80501e33250940984e2b2add6e812",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14708",
"triggerID" : "a2dec7d1a6b80501e33250940984e2b2add6e812",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a2dec7d1a6b80501e33250940984e2b2add6e812 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14708)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on pull request #7776: [HUDI-5642] Enable schema reconciliation by default
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on PR #7776:
URL: https://github.com/apache/hudi/pull/7776#issuecomment-1623398489
> this was planned to be done in one of the prev releases but there are some issues. I've lowered the priority for now, but we should take it up for 0.14.0. Updated the fix version on JIRA appropriately.
That's cool, but we are not expecting to introduce any schema data type related regressions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7776: [HUDI-5642] Enable schema reconciliation by default
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7776:
URL: https://github.com/apache/hudi/pull/7776#issuecomment-1623028986
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a2dec7d1a6b80501e33250940984e2b2add6e812",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14708",
"triggerID" : "a2dec7d1a6b80501e33250940984e2b2add6e812",
"triggerType" : "PUSH"
}, {
"hash" : "67221fd5f81caad9a68f4ba16ca531f1f2804dda",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "67221fd5f81caad9a68f4ba16ca531f1f2804dda",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a2dec7d1a6b80501e33250940984e2b2add6e812 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=14708)
* 67221fd5f81caad9a68f4ba16ca531f1f2804dda UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org