You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/01/13 11:08:49 UTC
[GitHub] [hudi] YannByron opened a new pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
YannByron opened a new pull request #4587:
URL: https://github.com/apache/hudi/pull/4587
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*
## What is the purpose of the pull request
*(For example: This pull request adds quick-start document.)*
## Brief change log
*(for example:)*
- *Modify AnnotationLocation checkstyle rule in checkstyle.xml*
## Verify this pull request
*(Please pick either of the following options)*
This pull request is a trivial rework / code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end.*
- *Added HoodieClientWriteTest to verify the change.*
- *Manually verified the change by running a job locally.*
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012886240
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
}, {
"hash" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202",
"triggerID" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"triggerType" : "PUSH"
}, {
"hash" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5231",
"triggerID" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 790f50ec670c9247908c3c0fcdcaac34d0e02a21 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202)
* dec3b88d3cd1ea9475ce35b8789a721d940ae3f2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5231)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012282006
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
}, {
"hash" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202",
"triggerID" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 790f50ec670c9247908c3c0fcdcaac34d0e02a21 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1016063475
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
}, {
"hash" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202",
"triggerID" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"triggerType" : "PUSH"
}, {
"hash" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5231",
"triggerID" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"triggerType" : "PUSH"
}, {
"hash" : "4ca67729c349de96fe204323b20ce98137c434cb",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5340",
"triggerID" : "4ca67729c349de96fe204323b20ce98137c434cb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dec3b88d3cd1ea9475ce35b8789a721d940ae3f2 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5231)
* 4ca67729c349de96fe204323b20ce98137c434cb Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5340)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan merged pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
xushiyan merged pull request #4587:
URL: https://github.com/apache/hudi/pull/4587
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012879648
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
}, {
"hash" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202",
"triggerID" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"triggerType" : "PUSH"
}, {
"hash" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 790f50ec670c9247908c3c0fcdcaac34d0e02a21 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202)
* dec3b88d3cd1ea9475ce35b8789a721d940ae3f2 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012221921
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
}, {
"hash" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202",
"triggerID" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 91868d624fac0af68afbc399a0ba554c1e5bce93 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191)
* 790f50ec670c9247908c3c0fcdcaac34d0e02a21 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012221921
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
}, {
"hash" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202",
"triggerID" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 91868d624fac0af68afbc399a0ba554c1e5bce93 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191)
* 790f50ec670c9247908c3c0fcdcaac34d0e02a21 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012036868
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 91868d624fac0af68afbc399a0ba554c1e5bce93 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012879648
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
}, {
"hash" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202",
"triggerID" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"triggerType" : "PUSH"
}, {
"hash" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 790f50ec670c9247908c3c0fcdcaac34d0e02a21 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202)
* dec3b88d3cd1ea9475ce35b8789a721d940ae3f2 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1016061819
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
}, {
"hash" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202",
"triggerID" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"triggerType" : "PUSH"
}, {
"hash" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5231",
"triggerID" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"triggerType" : "PUSH"
}, {
"hash" : "4ca67729c349de96fe204323b20ce98137c434cb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4ca67729c349de96fe204323b20ce98137c434cb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dec3b88d3cd1ea9475ce35b8789a721d940ae3f2 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5231)
* 4ca67729c349de96fe204323b20ce98137c434cb UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1016061819
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
}, {
"hash" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202",
"triggerID" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"triggerType" : "PUSH"
}, {
"hash" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5231",
"triggerID" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"triggerType" : "PUSH"
}, {
"hash" : "4ca67729c349de96fe204323b20ce98137c434cb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4ca67729c349de96fe204323b20ce98137c434cb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dec3b88d3cd1ea9475ce35b8789a721d940ae3f2 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5231)
* 4ca67729c349de96fe204323b20ce98137c434cb UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012935446
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
}, {
"hash" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202",
"triggerID" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"triggerType" : "PUSH"
}, {
"hash" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5231",
"triggerID" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dec3b88d3cd1ea9475ce35b8789a721d940ae3f2 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5231)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
xiarixiaoyao commented on a change in pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#discussion_r783928372
##########
File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/AlterHoodieTableChangeColumnCommand.scala
##########
@@ -42,24 +42,27 @@ case class AlterHoodieTableChangeColumnCommand(
extends HoodieLeafRunnableCommand {
override def run(sparkSession: SparkSession): Seq[Row] = {
+ val resolver = sparkSession.sessionState.conf.resolver
val hoodieCatalogTable = HoodieCatalogTable(sparkSession, tableIdentifier)
- val resolver = sparkSession.sessionState.conf.resolver
- if (!resolver(columnName, newColumn.name)) {
- throw new AnalysisException(s"Can not support change column name for hudi table currently.")
- }
+ // Find the origin column from dataSchema by column name.
+ val originColumn = findColumnByName(hoodieCatalogTable.dataSchema, columnName, resolver).getOrElse(
+ throw new AnalysisException(s"Can't find column `$columnName` given table data columns " +
+ s"${hoodieCatalogTable.dataSchema.fieldNames.mkString("[`", "`, `", "`]")}")
+ )
+
Review comment:
I don't think we support rename operation now， Why remove relevant judgments？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012218620
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
}, {
"hash" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 91868d624fac0af68afbc399a0ba554c1e5bce93 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191)
* 790f50ec670c9247908c3c0fcdcaac34d0e02a21 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012086069
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 91868d624fac0af68afbc399a0ba554c1e5bce93 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012086069
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 91868d624fac0af68afbc399a0ba554c1e5bce93 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] YannByron commented on a change in pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
YannByron commented on a change in pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#discussion_r783983637
##########
File path: hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestAlterTable.scala
##########
@@ -53,24 +71,26 @@ class TestAlterTable extends TestHoodieSqlBase {
assertResult(true) (
spark.sessionState.catalog.tableExists(new TableIdentifier(newTableName))
)
+
val hadoopConf = spark.sessionState.newHadoopConf()
val metaClient = HoodieTableMetaClient.builder().setBasePath(tablePath)
.setConf(hadoopConf).build()
- assertResult(newTableName) (
- metaClient.getTableConfig.getTableName
- )
+ assertResult(newTableName) (metaClient.getTableConfig.getTableName)
+
+ // insert some data
spark.sql(s"insert into $newTableName values(1, 'a1', 10, 1000)")
- // Add table column
+ // add column
spark.sql(s"alter table $newTableName add columns(ext0 string)")
- val table = spark.sessionState.catalog.getTableMetadata(new TableIdentifier(newTableName))
+ catalogTable = spark.sessionState.catalog.getTableMetadata(new TableIdentifier(newTableName))
assertResult(Seq("id", "name", "price", "ts", "ext0")) {
- HoodieSqlUtils.removeMetaFields(table.schema).fields.map(_.name)
+ HoodieSqlUtils.removeMetaFields(catalogTable.schema).fields.map(_.name)
}
checkAnswer(s"select id, name, price, ts, ext0 from $newTableName")(
Seq(1, "a1", 10.0, 1000, null)
)
- // Alter table column type
+
+ // change column's data type
spark.sql(s"alter table $newTableName change column id id bigint")
Review comment:
I know that. See the details in https://issues.apache.org/jira/browse/HUDI-3237.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012039263
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 91868d624fac0af68afbc399a0ba554c1e5bce93 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] YannByron commented on a change in pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
YannByron commented on a change in pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#discussion_r784013199
##########
File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/AlterHoodieTableChangeColumnCommand.scala
##########
@@ -42,24 +42,27 @@ case class AlterHoodieTableChangeColumnCommand(
extends HoodieLeafRunnableCommand {
override def run(sparkSession: SparkSession): Seq[Row] = {
+ val resolver = sparkSession.sessionState.conf.resolver
val hoodieCatalogTable = HoodieCatalogTable(sparkSession, tableIdentifier)
- val resolver = sparkSession.sessionState.conf.resolver
- if (!resolver(columnName, newColumn.name)) {
- throw new AnalysisException(s"Can not support change column name for hudi table currently.")
- }
+ // Find the origin column from dataSchema by column name.
+ val originColumn = findColumnByName(hoodieCatalogTable.dataSchema, columnName, resolver).getOrElse(
+ throw new AnalysisException(s"Can't find column `$columnName` given table data columns " +
+ s"${hoodieCatalogTable.dataSchema.fieldNames.mkString("[`", "`, `", "`]")}")
+ )
+
Review comment:
I think It is better to have the same behavior as Spark.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012218620
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
}, {
"hash" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 91868d624fac0af68afbc399a0ba554c1e5bce93 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191)
* 790f50ec670c9247908c3c0fcdcaac34d0e02a21 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1016063475
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
}, {
"hash" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202",
"triggerID" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"triggerType" : "PUSH"
}, {
"hash" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5231",
"triggerID" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"triggerType" : "PUSH"
}, {
"hash" : "4ca67729c349de96fe204323b20ce98137c434cb",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5340",
"triggerID" : "4ca67729c349de96fe204323b20ce98137c434cb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dec3b88d3cd1ea9475ce35b8789a721d940ae3f2 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5231)
* 4ca67729c349de96fe204323b20ce98137c434cb Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5340)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012886240
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
}, {
"hash" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202",
"triggerID" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"triggerType" : "PUSH"
}, {
"hash" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5231",
"triggerID" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 790f50ec670c9247908c3c0fcdcaac34d0e02a21 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202)
* dec3b88d3cd1ea9475ce35b8789a721d940ae3f2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5231)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] YannByron commented on a change in pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
YannByron commented on a change in pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#discussion_r783986627
##########
File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/AlterHoodieTableChangeColumnCommand.scala
##########
@@ -42,24 +42,27 @@ case class AlterHoodieTableChangeColumnCommand(
extends HoodieLeafRunnableCommand {
override def run(sparkSession: SparkSession): Seq[Row] = {
+ val resolver = sparkSession.sessionState.conf.resolver
val hoodieCatalogTable = HoodieCatalogTable(sparkSession, tableIdentifier)
- val resolver = sparkSession.sessionState.conf.resolver
- if (!resolver(columnName, newColumn.name)) {
- throw new AnalysisException(s"Can not support change column name for hudi table currently.")
- }
+ // Find the origin column from dataSchema by column name.
+ val originColumn = findColumnByName(hoodieCatalogTable.dataSchema, columnName, resolver).getOrElse(
+ throw new AnalysisException(s"Can't find column `$columnName` given table data columns " +
+ s"${hoodieCatalogTable.dataSchema.fieldNames.mkString("[`", "`, `", "`]")}")
+ )
+
Review comment:
This changes also do not support rename operation. See the `findColumnByName`'s implement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
xiarixiaoyao commented on a change in pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#discussion_r783998567
##########
File path: hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/AlterHoodieTableChangeColumnCommand.scala
##########
@@ -42,24 +42,27 @@ case class AlterHoodieTableChangeColumnCommand(
extends HoodieLeafRunnableCommand {
override def run(sparkSession: SparkSession): Seq[Row] = {
+ val resolver = sparkSession.sessionState.conf.resolver
val hoodieCatalogTable = HoodieCatalogTable(sparkSession, tableIdentifier)
- val resolver = sparkSession.sessionState.conf.resolver
- if (!resolver(columnName, newColumn.name)) {
- throw new AnalysisException(s"Can not support change column name for hudi table currently.")
- }
+ // Find the origin column from dataSchema by column name.
+ val originColumn = findColumnByName(hoodieCatalogTable.dataSchema, columnName, resolver).getOrElse(
+ throw new AnalysisException(s"Can't find column `$columnName` given table data columns " +
+ s"${hoodieCatalogTable.dataSchema.fieldNames.mkString("[`", "`, `", "`]")}")
+ )
+
Review comment:
thanks . I mean, maybe we can throw a corresponding exception to tell the user that we don't support rename at present,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012036868
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 91868d624fac0af68afbc399a0ba554c1e5bce93 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012039263
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 91868d624fac0af68afbc399a0ba554c1e5bce93 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xiarixiaoyao commented on a change in pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
xiarixiaoyao commented on a change in pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#discussion_r783934791
##########
File path: hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestAlterTable.scala
##########
@@ -53,24 +71,26 @@ class TestAlterTable extends TestHoodieSqlBase {
assertResult(true) (
spark.sessionState.catalog.tableExists(new TableIdentifier(newTableName))
)
+
val hadoopConf = spark.sessionState.newHadoopConf()
val metaClient = HoodieTableMetaClient.builder().setBasePath(tablePath)
.setConf(hadoopConf).build()
- assertResult(newTableName) (
- metaClient.getTableConfig.getTableName
- )
+ assertResult(newTableName) (metaClient.getTableConfig.getTableName)
+
+ // insert some data
spark.sql(s"insert into $newTableName values(1, 'a1', 10, 1000)")
- // Add table column
+ // add column
spark.sql(s"alter table $newTableName add columns(ext0 string)")
- val table = spark.sessionState.catalog.getTableMetadata(new TableIdentifier(newTableName))
+ catalogTable = spark.sessionState.catalog.getTableMetadata(new TableIdentifier(newTableName))
assertResult(Seq("id", "name", "price", "ts", "ext0")) {
- HoodieSqlUtils.removeMetaFields(table.schema).fields.map(_.name)
+ HoodieSqlUtils.removeMetaFields(catalogTable.schema).fields.map(_.name)
}
checkAnswer(s"select id, name, price, ts, ext0 from $newTableName")(
Seq(1, "a1", 10.0, 1000, null)
)
- // Alter table column type
+
+ // change column's data type
spark.sql(s"alter table $newTableName change column id id bigint")
Review comment:
now， hudi on spark cannot support dataType change。 hudi use spark parquetFileFormat to read parquet file，but that reader is hardly support type change。 see the origin code of spark project **ParquetVectorUpdaterFactory.getUpdater**
This test is actually wrong， if you add **spark.sql(s"select id from $newTableName").show(false)** in line 95， this test will failed。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot removed a comment on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot removed a comment on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012282006
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
}, {
"hash" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202",
"triggerID" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 790f50ec670c9247908c3c0fcdcaac34d0e02a21 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1012935446
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
}, {
"hash" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202",
"triggerID" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"triggerType" : "PUSH"
}, {
"hash" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5231",
"triggerID" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dec3b88d3cd1ea9475ce35b8789a721d940ae3f2 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5231)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #4587: [HUDI-3236] use fields'comments persisted in catalog to fill in schema
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on pull request #4587:
URL: https://github.com/apache/hudi/pull/4587#issuecomment-1016095785
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5191",
"triggerID" : "91868d624fac0af68afbc399a0ba554c1e5bce93",
"triggerType" : "PUSH"
}, {
"hash" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5202",
"triggerID" : "790f50ec670c9247908c3c0fcdcaac34d0e02a21",
"triggerType" : "PUSH"
}, {
"hash" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5231",
"triggerID" : "dec3b88d3cd1ea9475ce35b8789a721d940ae3f2",
"triggerType" : "PUSH"
}, {
"hash" : "4ca67729c349de96fe204323b20ce98137c434cb",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5340",
"triggerID" : "4ca67729c349de96fe204323b20ce98137c434cb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4ca67729c349de96fe204323b20ce98137c434cb Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=5340)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org