You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Vinoth Chandar (Jira)" <ji...@apache.org> on 2024/10/26 01:49:00 UTC
[jira] [Comment Edited] (HUDI-8227) Make table services operate on specific table versions
[ https://issues.apache.org/jira/browse/HUDI-8227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17893015#comment-17893015 ]
Vinoth Chandar edited comment on HUDI-8227 at 10/26/24 1:48 AM:
----------------------------------------------------------------
1. preventing table services from upgrading the table.. only writers should do that. need to study existing entry points..
2. today table services may not throw errors if table version is mismatched, because we instruct them to be offline for version upgrade.. Do we need a 0.15.1 or 0.14.x patch release for this failure..
3.1.x code writing t.v 6, working with 0.14 code running async table services.. what will break. if any? (concern is less about storage format, but more about behavior changes in code between 0.14 and 1.x) - can we limit the flow to : stop async table services, rollout 1.x with t.v=6 to writers, upgrade async table services to 1.x with t.v=6, start async table services.
4. Would upgrade/downgrade finsih/cancel all pending table services? - else we need to handle plans written by t.v=6 or t.v=8.. even in this case: the 1.x code needs to be able to execute table services as t.v=6 in the upgrade path?
was (Author: vc):
1. preventing table services from upgrading the table.. only writers should do that. need to study existing entry points..
2. today table services may not throw errors if table version is mismatched, because we instruct them to be offline for version upgrade.. Do we need a 0.15.1 or 0.14.x patch release for this failure..
3.1.x code writing t.v 6, working with 0.14 code running async table services.. what will break. if any? (concern is less about storage format, but more about behavior changes in code between 0.14 and 1.x) - can we limit the flow to : stop async table services, rollout 1.x with t.v=6 to writers, upgrade async table services to 1.x with t.v=6, start async table services.
4. Would upgrade/downgrade finsih/cancel all pending table services? - else we need to handle plans written by t.v=6 or t.v=8..
> Make table services operate on specific table versions
> ------------------------------------------------------
>
> Key: HUDI-8227
> URL: https://issues.apache.org/jira/browse/HUDI-8227
> Project: Apache Hudi
> Issue Type: Improvement
> Components: table-service
> Reporter: Vinoth Chandar
> Priority: Major
>
> We need table services to operate on tables based on the current table version. This generally helps us manage tables compatible with writer versions, as well as call them during upgrade/downgrade path.
> e.g [https://github.com/apache/hudi/pull/11866#discussion_r1765447834] talks about how validation may need to turned off to run compaction during upgrade/downgrade.
> Currently rollback is performed as a part of upgrade-downgrade, which also needs to be table version specific..
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)