You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/10/24 07:27:41 UTC
[GitHub] [hudi] shenshengli opened a new pull request, #7045: HUDI-5083 Fixed a bug when schema evolution
shenshengli opened a new pull request, #7045:
URL: https://github.com/apache/hudi/pull/7045
A bug occurs when the schema changes multiple times to a once existed column.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7045: [HUDI-5083]Fixed a bug when schema evolution
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7045:
URL: https://github.com/apache/hudi/pull/7045#issuecomment-1289948471
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c326073206cf20ca528c674678608d8713db7f1e",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12518",
"triggerID" : "c326073206cf20ca528c674678608d8713db7f1e",
"triggerType" : "PUSH"
}, {
"hash" : "14ce863d9d3d9931438747df715de640727ee971",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12556",
"triggerID" : "14ce863d9d3d9931438747df715de640727ee971",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c326073206cf20ca528c674678608d8713db7f1e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12518)
* 14ce863d9d3d9931438747df715de640727ee971 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12556)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] shenshengli commented on pull request #7045: [HUDI-5083]Fixed a bug when schema evolution
Posted by GitBox <gi...@apache.org>.
shenshengli commented on PR #7045:
URL: https://github.com/apache/hudi/pull/7045#issuecomment-1289939687
> @shenshengli 感谢您的贡献。请添加一些 UT 也许你可以添加流代码，在 TestSpark3DDL 的第 293 行之后
>
> ```
> // drop + rename + insert
> spark.sql(s"alter table $tableName drop column col6")
> spark.sql(s"alter table $tableName rename column col5 to col6")
> spark.sql(
> s"""
> | insert into $tableName values
> | (1,3,1,11,100001,'101.01',1001.0001,100009.0001,'aaaaaa','2021-12-25 12:01:01',true,'a01','2021-12-25'),
> | (11,3,1,11,100001,'101.01',1001.0001,100011.0001,'aaaaaa','2021-12-25 12:01:01',true,'a01','2021-12-25'),
> | (6,6,5,15,100005,'105.05',1005.0005,100007.0005,'aaaaaa','2021-12-26 12:05:05',false,'a05','2021-12-26')
> |""".stripMargin)
> checkAnswer(spark.sql(s"select col6 from $tableName where id = 1 or id = 6 or id = 11 order by id").collect())(
> Seq("aaaaaa"),
> Seq("aaaaaa"),
> Seq("aaaaaa")
> )
> ```
ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7045: HUDI-5083 Fixed a bug when schema evolution
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7045:
URL: https://github.com/apache/hudi/pull/7045#issuecomment-1288647382
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c326073206cf20ca528c674678608d8713db7f1e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12518",
"triggerID" : "c326073206cf20ca528c674678608d8713db7f1e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c326073206cf20ca528c674678608d8713db7f1e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12518)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xiarixiaoyao commented on pull request #7045: [HUDI-5083]Fixed a bug when schema evolution
Posted by GitBox <gi...@apache.org>.
xiarixiaoyao commented on PR #7045:
URL: https://github.com/apache/hudi/pull/7045#issuecomment-1288722652
@shenshengli
Thank you for your contribution. pls add some UT
maybe you can add fllow codes ， after line 293 in TestSpark3DDL
```
// drop + rename + insert
spark.sql(s"alter table $tableName drop column col6")
spark.sql(s"alter table $tableName rename column col5 to col6")
spark.sql(
s"""
| insert into $tableName values
| (1,3,1,11,100001,'101.01',1001.0001,100009.0001,'aaaaaa','2021-12-25 12:01:01',true,'a01','2021-12-25'),
| (11,3,1,11,100001,'101.01',1001.0001,100011.0001,'aaaaaa','2021-12-25 12:01:01',true,'a01','2021-12-25'),
| (6,6,5,15,100005,'105.05',1005.0005,100007.0005,'aaaaaa','2021-12-26 12:05:05',false,'a05','2021-12-26')
|""".stripMargin)
checkAnswer(spark.sql(s"select col6 from $tableName where id = 1 or id = 6 or id = 11 order by id").collect())(
Seq("aaaaaa"),
Seq("aaaaaa"),
Seq("aaaaaa")
)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xiarixiaoyao merged pull request #7045: [HUDI-5083]Fixed a bug when schema evolution
Posted by GitBox <gi...@apache.org>.
xiarixiaoyao merged PR #7045:
URL: https://github.com/apache/hudi/pull/7045
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7045: [HUDI-5083]Fixed a bug when schema evolution
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7045:
URL: https://github.com/apache/hudi/pull/7045#issuecomment-1289945138
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c326073206cf20ca528c674678608d8713db7f1e",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12518",
"triggerID" : "c326073206cf20ca528c674678608d8713db7f1e",
"triggerType" : "PUSH"
}, {
"hash" : "14ce863d9d3d9931438747df715de640727ee971",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "14ce863d9d3d9931438747df715de640727ee971",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c326073206cf20ca528c674678608d8713db7f1e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12518)
* 14ce863d9d3d9931438747df715de640727ee971 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7045: [HUDI-5083]Fixed a bug when schema evolution
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7045:
URL: https://github.com/apache/hudi/pull/7045#issuecomment-1290862720
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c326073206cf20ca528c674678608d8713db7f1e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12518",
"triggerID" : "c326073206cf20ca528c674678608d8713db7f1e",
"triggerType" : "PUSH"
}, {
"hash" : "14ce863d9d3d9931438747df715de640727ee971",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12556",
"triggerID" : "14ce863d9d3d9931438747df715de640727ee971",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 14ce863d9d3d9931438747df715de640727ee971 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12556)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xiarixiaoyao commented on pull request #7045: [HUDI-5083]Fixed a bug when schema evolution
Posted by GitBox <gi...@apache.org>.
xiarixiaoyao commented on PR #7045:
URL: https://github.com/apache/hudi/pull/7045#issuecomment-1295676508
@shenshengli
nice work, Thank you for your contribution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7045: [HUDI-5083]Fixed a bug when schema evolution
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7045:
URL: https://github.com/apache/hudi/pull/7045#issuecomment-1289273233
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c326073206cf20ca528c674678608d8713db7f1e",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12518",
"triggerID" : "c326073206cf20ca528c674678608d8713db7f1e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c326073206cf20ca528c674678608d8713db7f1e Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12518)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7045: HUDI-5083 Fixed a bug when schema evolution
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7045:
URL: https://github.com/apache/hudi/pull/7045#issuecomment-1288635283
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "c326073206cf20ca528c674678608d8713db7f1e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c326073206cf20ca528c674678608d8713db7f1e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c326073206cf20ca528c674678608d8713db7f1e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org