You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 10:40:39 UTC
[I] Making MT reads/writes table version specific [hudi]
hudi-bot opened a new issue, #16619:
URL: https://github.com/apache/hudi/issues/16619
In theory, given MT is another table, the reading/writing should be covered at the table level. But - the main thing that complicates this is the use of completion time file slicing in 1.x code to write/read MT. The problematic combination here is 1.0/tv=6 writer and being able to continue reading the MT from 0.14/0.15 reader.. The 1.0/tv=6 reader may handle this better since we have control over the reader code..
Without this, we need to bring back older restrictions around MT compaction/archival that existed in 0.x code.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-8229
- Type: Improvement
- Epic: https://issues.apache.org/jira/browse/HUDI-7856
- Fix version(s):
- 1.1.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Making MT reads/writes table version specific [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on issue #16619:
URL: https://github.com/apache/hudi/issues/16619#issuecomment-4795624627
Looks like the core architectural concern raised here has been addressed in master by [HUDI-9013](https://issues.apache.org/jira/browse/HUDI-9013) / PR #12948 (commit `aff16c48`, merged 2025-03-20), which added backwards-compatible MDT writer/reader support for table version 6:
- `HoodieBackedTableMetadataWriterTableVersionSix` — a v6-specific writer that restores the 0.x restrictions on MDT compaction scheduling (`validateCompactionScheduling`), uses the legacy instant-time suffix scheme (001/005/010), excludes v8-only partitions (SECONDARY_INDEX, EXPRESSION_INDEX, PARTITION_STATS), and uses the legacy rollback path.
- `SparkMetadataWriterFactory` routes to the v6 writer when `tableConfig.getTableVersion().lesserThan(HoodieTableVersion.EIGHT)`.
- `AbstractTableFileSystemView` gates completion-time-based file slicing on `tableVersion8AndAbove()`, so tv=6 reads/writes fall back to request-time slicing — compatible with 0.14/0.15 readers.
Is there remaining scope under HUDI-8229 that's still open, or can this be closed? cc whoever owns the epic HUDI-7856.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Making MT reads/writes table version specific [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan closed issue #16619: Making MT reads/writes table version specific
URL: https://github.com/apache/hudi/issues/16619
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org