[I] Refactor checkpoint translation APIs [hudi]

["hudi-bot (via GitHub)" <gi...@apache.org>]

hudi-bot opened a new issue, #17318:
URL: https://github.com/apache/hudi/issues/17318

   # Do we really need completely different relations based on checkpoint version.. I feel just passing in the checkpoint version to use is sufficient.. I worry there are too many V1, V2 classes to maintain here. [https://github.com/apache/hudi/pull/12342#discussion_r1864665107]
    # Can we introduce a factory.. {{Checkpoint.create(..)}} that will create the current/latest by default.. then we avoid use of version specific {{CheckpointV2}} and CheckpointV1` classes in regular code.. There should be a few places where we will handle CheckpointV1 and thats it.
    # 
   use {{HoodieTableVersion instead of int writeTableVersion - [https://github.com/apache/hudi/pull/12342#discussion_r1864669100]}}
   
   {{4. Code duplication - https://github.com/apache/hudi/pull/12342#discussion_r1864668618}}
   
   {{{}{}}}{{{}{}}}
   
   ## JIRA info
   
   - Link: https://issues.apache.org/jira/browse/HUDI-8627
   - Type: Sub-task
   - Parent: https://issues.apache.org/jira/browse/HUDI-9105
   - Fix version(s):
     - 1.1.0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Refactor checkpoint translation APIs [hudi]

["rangareddy (via GitHub)" <gi...@apache.org>]

rangareddy commented on issue #17318:
URL: https://github.com/apache/hudi/issues/17318#issuecomment-5365822579

   This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-8627).
   
   **Findings: mostly not done.**
   
   Against the items in the description:
   
   **1. "Too many V1, V2 classes to maintain" - unaddressed.** `hudi-common/src/main/java/org/apache/hudi/common/table/checkpoint/` still contains `StreamerCheckpointV1.java` and `StreamerCheckpointV2.java` alongside `Checkpoint.java`, `CheckpointUtils.java` and `UnresolvedStreamerCheckpointBasedOnCfg.java`. Usage breadth on `master`:
   
   | symbol | files referencing it |
   | --- | --- |
   | `CheckpointV1` | 38 |
   | `CheckpointV2` | 31 |
   | `StreamerCheckpointV1` | 37 |
   | `StreamerCheckpointV2` | 29 |
   
   So version-specific checkpoint types are spread across most of the streamer surface, not confined to "a few places" as the description hoped for.
   
   **2. A factory exists, but not in the requested shape.** `CheckpointUtils.createCheckpoint(String checkpointToResume)` at `:83` and `createCheckpoint(Checkpoint checkpointToResume)` at `:93`, rather than a `Checkpoint.create(..)` that returns the current/latest version by default. That is close enough to be worth deciding whether the remaining delta is real or whether item 2 can be marked done.
   
   **3.** The `HoodieTableVersion` instead of `int writeTableVersion` item was not separately verified.
   
   Note #17317 (HUDI-8625) wants to adopt this hierarchy for `HoodieSourceOffset`, so settling the shape here first would avoid reworking it twice.
   
   Keeping this open.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org