You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "yihua (via GitHub)" <gi...@apache.org> on 2023/08/09 21:57:11 UTC
[GitHub] [hudi] yihua opened a new pull request, #9411: [HUDI-6674] Add rollback info from metadata table in timeline commands
yihua opened a new pull request, #9411:
URL: https://github.com/apache/hudi/pull/9411
### Change Logs
This PR adds the rollback information from the metadata table to the output of the timeline commands in Hudi CLI, given that metadata data table also encounters more rollbacks now. To make the table concise, the rollback information is added to the "Action" (for data table) or "MT Action" (for metadata table) column, instead of having an independent column showing the information.
Here's the new output:
```
hudi:hoodie_table->timeline show active --limit 200 --show-time-seconds --show-rollback-info
╔═════╤═══════════════════╤═══════════════════╤═══════════╤════════════════╤════════════════╤════════════════╗
║ No. │ Instant │ Action │ State │ Requested │ Inflight │ Completed ║
║ │ │ │ │ Time │ Time │ Time ║
╠═════╪═══════════════════╪═══════════════════╪═══════════╪════════════════╪════════════════╪════════════════╣
...
╟─────┼───────────────────┼───────────────────┼───────────┼────────────────┼────────────────┼────────────────╢
║ 11 │ 20230807154601569 │ rollback │ COMPLETED │ 08-07 08:46:03 │ 08-07 08:46:03 │ 08-07 08:47:58 ║
║ │ │ Rolls back │ │ │ │ ║
║ │ │ 20230807154346625 │ │ │ │ ║
╟─────┼───────────────────┼───────────────────┼───────────┼────────────────┼────────────────┼────────────────╢
║ 12 │ 20230807154947753 │ rollback │ COMPLETED │ 08-07 08:49:49 │ 08-07 08:49:49 │ 08-07 08:51:46 ║
║ │ │ Rolls back │ │ │ │ ║
║ │ │ 20230807154720087 │ │ │ │ ║
╟─────┼───────────────────┼───────────────────┼───────────┼────────────────┼────────────────┼────────────────╢
║ 13 │ 20230807155105131 │ commit │ COMPLETED │ 08-07 08:51:47 │ 08-07 08:54:29 │ 08-07 08:55:42 ║
╟─────┼───────────────────┼───────────────────┼───────────┼────────────────┼────────────────┼────────────────╢
...
hudi:hoodie_table->timeline show active --with-metadata-table --limit 200 --show-time-seconds
╔═════╤══════════════════════╤═══════════════════╤═══════════╤════════════════╤════════════════╤════════════════╤══════════════════════╤═══════════╤════════════════╤════════════════╤════════════════╗
║ No. │ Instant │ Action │ State │ Requested │ Inflight │ Completed │ MT │ MT │ MT │ MT │ MT ║
║ │ │ │ │ Time │ Time │ Time │ Action │ State │ Requested │ Inflight │ Completed ║
║ │ │ │ │ │ │ │ │ │ Time │ Time │ Time ║
╠═════╪══════════════════════╪═══════════════════╪═══════════╪════════════════╪════════════════╪════════════════╪══════════════════════╪═══════════╪════════════════╪════════════════╪════════════════╣
...
╟─────┼──────────────────────┼───────────────────┼───────────┼────────────────┼────────────────┼────────────────┼──────────────────────┼───────────┼────────────────┼────────────────┼────────────────╢
║ 66 │ 20230807155157772 │ - │ - │ - │ - │ - │ rollback │ COMPLETED │ 08-07 08:51:59 │ 08-07 08:52:00 │ 08-07 08:52:01 ║
║ │ │ │ │ │ │ │ Rolls back │ │ │ │ ║
║ │ │ │ │ │ │ │ 20230807154406919 │ │ │ │ ║
╟─────┼──────────────────────┼───────────────────┼───────────┼────────────────┼────────────────┼────────────────┼──────────────────────┼───────────┼────────────────┼────────────────┼────────────────╢
║ 67 │ 20230807155547486 │ commit │ INFLIGHT │ 08-07 08:56:06 │ 08-07 08:58:27 │ - │ - │ - │ - │ - │ - ║
║ │ │ Rolled back by │ │ │ │ │ │ │ │ │ ║
║ │ │ 20230807160141230 │ │ │ │ │ │ │ │ │ ║
╟─────┼──────────────────────┼───────────────────┼───────────┼────────────────┼────────────────┼────────────────┼──────────────────────┼───────────┼────────────────┼────────────────┼────────────────╢
║ 68 │ 20230807155616690 │ clean │ COMPLETED │ 08-07 08:56:19 │ 08-07 08:56:19 │ 08-07 08:56:25 │ deltacommit │ COMPLETED │ 08-07 08:56:21 │ 08-07 08:56:22 │ 08-07 08:56:24 ║
╟─────┼──────────────────────┼───────────────────┼───────────┼────────────────┼────────────────┼────────────────┼──────────────────────┼───────────┼────────────────┼────────────────┼────────────────╢
║ 69 │ 20230807160141230 │ rollback │ INFLIGHT │ 08-07 09:01:44 │ 08-07 09:01:44 │ - │ deltacommit │ COMPLETED │ 08-07 09:01:46 │ 08-07 09:01:47 │ 08-07 09:01:50 ║
║ │ │ Rolls back │ │ │ │ │ │ │ │ │ ║
║ │ │ 20230807155547486 │ │ │ │ │ │ │ │ │ ║
╟─────┼──────────────────────┼───────────────────┼───────────┼────────────────┼────────────────┼────────────────┼──────────────────────┼───────────┼────────────────┼────────────────┼────────────────╢
...
```
compared to previous output:
```
hudi:hoodie_table->timeline show active --limit 200 --show-time-seconds --show-rollback-info
╔═════╤═══════════════════╤══════════╤═══════════╤═══════════════════╤════════════════╤════════════════╤════════════════╗
║ No. │ Instant │ Action │ State │ Rollback Info │ Requested │ Inflight │ Completed ║
║ │ │ │ │ │ Time │ Time │ Time ║
╠═════╪═══════════════════╪══════════╪═══════════╪═══════════════════╪════════════════╪════════════════╪════════════════╣
...
╟─────┼───────────────────┼──────────┼───────────┼───────────────────┼────────────────┼────────────────┼────────────────╢
║ 11 │ 20230807154601569 │ rollback │ COMPLETED │ Rolls back │ 08-07 08:46:03 │ 08-07 08:46:03 │ 08-07 08:47:58 ║
║ │ │ │ │ 20230807154346625 │ │ │ ║
╟─────┼───────────────────┼──────────┼───────────┼───────────────────┼────────────────┼────────────────┼────────────────╢
║ 12 │ 20230807154947753 │ rollback │ COMPLETED │ Rolls back │ 08-07 08:49:49 │ 08-07 08:49:49 │ 08-07 08:51:46 ║
║ │ │ │ │ 20230807154720087 │ │ │ ║
╟─────┼───────────────────┼──────────┼───────────┼───────────────────┼────────────────┼────────────────┼────────────────╢
║ 13 │ 20230807155105131 │ commit │ COMPLETED │ - │ 08-07 08:51:47 │ 08-07 08:54:29 │ 08-07 08:55:42 ║
╟─────┼───────────────────┼──────────┼───────────┼───────────────────┼────────────────┼────────────────┼────────────────╢
...
hudi:hoodie_table->timeline show active --with-metadata-table --limit 200 --show-time-seconds
╔═════╤══════════════════════╤══════════╤═══════════╤═══════════════════╤════════════════╤════════════════╤════════════════╤═════════════╤═══════════╤════════════════╤════════════════╤════════════════╗
║ No. │ Instant │ Action │ State │ Rollback Info │ Requested │ Inflight │ Completed │ MT │ MT │ MT │ MT │ MT ║
║ │ │ │ │ │ Time │ Time │ Time │ Action │ State │ Requested │ Inflight │ Completed ║
║ │ │ │ │ │ │ │ │ │ │ Time │ Time │ Time ║
╠═════╪══════════════════════╪══════════╪═══════════╪═══════════════════╪════════════════╪════════════════╪════════════════╪═════════════╪═══════════╪════════════════╪════════════════╪════════════════╣
...
╟─────┼──────────────────────┼──────────┼───────────┼───────────────────┼────────────────┼────────────────┼────────────────┼─────────────┼───────────┼────────────────┼────────────────┼────────────────╢
║ 66 │ 20230807155157772 │ - │ - │ - │ - │ - │ - │ rollback │ COMPLETED │ 08-07 08:51:59 │ 08-07 08:52:00 │ 08-07 08:52:01 ║
╟─────┼──────────────────────┼──────────┼───────────┼───────────────────┼────────────────┼────────────────┼────────────────┼─────────────┼───────────┼────────────────┼────────────────┼────────────────╢
║ 67 │ 20230807155547486 │ commit │ INFLIGHT │ Rolled back by │ 08-07 08:56:06 │ 08-07 08:58:27 │ - │ - │ - │ - │ - │ - ║
║ │ │ │ │ 20230807160141230 │ │ │ │ │ │ │ │ ║
╟─────┼──────────────────────┼──────────┼───────────┼───────────────────┼────────────────┼────────────────┼────────────────┼─────────────┼───────────┼────────────────┼────────────────┼────────────────╢
║ 68 │ 20230807155616690 │ clean │ COMPLETED │ - │ 08-07 08:56:19 │ 08-07 08:56:19 │ 08-07 08:56:25 │ deltacommit │ COMPLETED │ 08-07 08:56:21 │ 08-07 08:56:22 │ 08-07 08:56:24 ║
╟─────┼──────────────────────┼──────────┼───────────┼───────────────────┼────────────────┼────────────────┼────────────────┼─────────────┼───────────┼────────────────┼────────────────┼────────────────╢
║ 69 │ 20230807160141230 │ rollback │ INFLIGHT │ Rolls back │ 08-07 09:01:44 │ 08-07 09:01:44 │ - │ deltacommit │ COMPLETED │ 08-07 09:01:46 │ 08-07 09:01:47 │ 08-07 09:01:50 ║
║ │ │ │ │ 20230807155547486 │ │ │ │ │ │ │ │ ║
╟─────┼──────────────────────┼──────────┼───────────┼───────────────────┼────────────────┼────────────────┼────────────────┼─────────────┼───────────┼────────────────┼────────────────┼────────────────╢
...
```
### Impact
Surfaces rollback information from the metadata table as well and make it easier to read the output.
### Risk level
none
### Documentation Update
N/A
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua merged pull request #9411: [HUDI-6674] Add rollback info from metadata table in timeline commands
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua merged PR #9411:
URL: https://github.com/apache/hudi/pull/9411
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9411: [HUDI-6674] Add rollback info from metadata table in timeline commands
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9411:
URL: https://github.com/apache/hudi/pull/9411#issuecomment-1672282598
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6a8aa88016ab8c2b2cab779f45ac2ecd409f3742",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19234",
"triggerID" : "6a8aa88016ab8c2b2cab779f45ac2ecd409f3742",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6a8aa88016ab8c2b2cab779f45ac2ecd409f3742 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19234)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9411: [HUDI-6674] Add rollback info from metadata table in timeline commands
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9411:
URL: https://github.com/apache/hudi/pull/9411#issuecomment-1672224229
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6a8aa88016ab8c2b2cab779f45ac2ecd409f3742",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6a8aa88016ab8c2b2cab779f45ac2ecd409f3742",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6a8aa88016ab8c2b2cab779f45ac2ecd409f3742 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9411: [HUDI-6674] Add rollback info from metadata table in timeline commands
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9411:
URL: https://github.com/apache/hudi/pull/9411#issuecomment-1672514880
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6a8aa88016ab8c2b2cab779f45ac2ecd409f3742",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19234",
"triggerID" : "6a8aa88016ab8c2b2cab779f45ac2ecd409f3742",
"triggerType" : "PUSH"
}, {
"hash" : "6a8aa88016ab8c2b2cab779f45ac2ecd409f3742",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19233",
"triggerID" : "6a8aa88016ab8c2b2cab779f45ac2ecd409f3742",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6a8aa88016ab8c2b2cab779f45ac2ecd409f3742 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19234) Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19233)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9411: [HUDI-6674] Add rollback info from metadata table in timeline commands
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9411:
URL: https://github.com/apache/hudi/pull/9411#issuecomment-1672423674
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6a8aa88016ab8c2b2cab779f45ac2ecd409f3742",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19234",
"triggerID" : "6a8aa88016ab8c2b2cab779f45ac2ecd409f3742",
"triggerType" : "PUSH"
}, {
"hash" : "6a8aa88016ab8c2b2cab779f45ac2ecd409f3742",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19233",
"triggerID" : "6a8aa88016ab8c2b2cab779f45ac2ecd409f3742",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6a8aa88016ab8c2b2cab779f45ac2ecd409f3742 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19234) Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19233)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org