You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "Lokesh Khurana (Jira)" <ji...@apache.org> on 2026/06/12 07:38:00 UTC
[jira] [Updated] (PHOENIX-7904) Online Schema Change Gap-Fix Initiative
[ https://issues.apache.org/jira/browse/PHOENIX-7904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lokesh Khurana updated PHOENIX-7904:
------------------------------------
Description:
The Online Schema Change (OSC) framework introduced under PHOENIX-6373 supports
zero-downtime ALTER TABLE for property changes (encoding scheme, immutable
storage scheme, etc.) by maintaining dual-write attribution between an old
physical HBase table and a transformed new physical, then swapping the
logical→physical pointer in SYSTEM.CATALOG. The framework is functionally
correct on the happy path but exposes correctness, durability, and operator-
visibility gaps under realistic production conditions.
This initiative ships in three phases:
* *Phase 0* — production-readiness gap-fix work on the shared framework.
* *Phase 1* — {{PHYSICAL_RENAME_ONLY}} transform type (`ALTER TABLE T SET PHYSICAL_TABLE_NAME = 'new'`).
* *Phase 2* — salt-bucket change
Gaps addressed:
* Cutover idempotency relies on live PTable reads that mutate during cutover itself
* Dual-write termination has no deterministic shutoff signal — partial-pass runs concurrently with stale-cache dual-write traffic.
* Old physical HBase table is never dropped, leaving stale data accessible to raw HBase clients indefinitely .
* Pause and failure variants leave dual-write attribution installed.
* Tenant view creation is not blocked during transform .
* Resume from PAUSED/FAILED runs partial-pass, missing writes accumulated in the parked window — silent data loss .
* Legacy Indexer coprocessor tables are not detected; transform runs with no dual-write — silent data loss.
* Existing index handling (drop-and-recreate at cutover) costs N×M full table scans — not viable for Phase 2's production workload .
* TransformType byte-to-enum deserializer throws on unknown bytes, blocking rolling upgrades that introduce new types.
* Modification-lock primitive is missing; ALTER + concurrent transform is racy.
Design doc :
{{[https://docs.google.com/document/d/1hfXMDETDf9aT2Qa_BWiyzEVeOfTNW55Ykp6ir2OtXAs|https://docs.google.com/document/d/1Zg3rfJFKXsVF-_1aW8ELB42mJxsYtSJRfKyyZHp555M/edit?tab=t.0]}}
was:
The Online Schema Change (OSC) framework introduced under PHOENIX-6373 supports
zero-downtime ALTER TABLE for property changes (encoding scheme, immutable
storage scheme, etc.) by maintaining dual-write attribution between an old
physical HBase table and a transformed new physical, then swapping the
logical→physical pointer in SYSTEM.CATALOG. The framework is functionally
correct on the happy path but exposes correctness, durability, and operator-
visibility gaps under realistic production conditions.
This initiative ships in three phases:
* *Phase 0* — production-readiness gap-fix work on the shared framework.
* *Phase 1* — {{PHYSICAL_RENAME_ONLY}} transform type (`ALTER TABLE T SET PHYSICAL_TABLE_NAME = 'new'`).
* *Phase 2* — salt-bucket change
Gaps addressed:
* Cutover idempotency relies on live PTable reads that mutate during cutover itself
* Dual-write termination has no deterministic shutoff signal — partial-pass runs concurrently with stale-cache dual-write traffic.
* Old physical HBase table is never dropped, leaving stale data accessible to raw HBase clients indefinitely .
* Pause and failure variants leave dual-write attribution installed.
* Tenant view creation is not blocked during transform .
* Resume from PAUSED/FAILED runs partial-pass, missing writes accumulated in the parked window — silent data loss .
* Legacy Indexer coprocessor tables are not detected; transform runs with no dual-write — silent data loss.
* Existing index handling (drop-and-recreate at cutover) costs N×M full table scans — not viable for Phase 2's production workload .
* TransformType byte-to-enum deserializer throws on unknown bytes, blocking rolling upgrades that introduce new types.
* Modification-lock primitive is missing; ALTER + concurrent transform is racy.
Design doc :
{{https://docs.google.com/document/d/1hfXMDETDf9aT2Qa_BWiyzEVeOfTNW55Ykp6ir2OtXAs}}
> Online Schema Change Gap-Fix Initiative
> ---------------------------------------
>
> Key: PHOENIX-7904
> URL: https://issues.apache.org/jira/browse/PHOENIX-7904
> Project: Phoenix
> Issue Type: Improvement
> Reporter: Lokesh Khurana
> Assignee: Lokesh Khurana
> Priority: Major
>
> The Online Schema Change (OSC) framework introduced under PHOENIX-6373 supports
> zero-downtime ALTER TABLE for property changes (encoding scheme, immutable
> storage scheme, etc.) by maintaining dual-write attribution between an old
> physical HBase table and a transformed new physical, then swapping the
> logical→physical pointer in SYSTEM.CATALOG. The framework is functionally
> correct on the happy path but exposes correctness, durability, and operator-
> visibility gaps under realistic production conditions.
>
> This initiative ships in three phases:
> * *Phase 0* — production-readiness gap-fix work on the shared framework.
> * *Phase 1* — {{PHYSICAL_RENAME_ONLY}} transform type (`ALTER TABLE T SET PHYSICAL_TABLE_NAME = 'new'`).
> * *Phase 2* — salt-bucket change
>
> Gaps addressed:
> * Cutover idempotency relies on live PTable reads that mutate during cutover itself
> * Dual-write termination has no deterministic shutoff signal — partial-pass runs concurrently with stale-cache dual-write traffic.
> * Old physical HBase table is never dropped, leaving stale data accessible to raw HBase clients indefinitely .
> * Pause and failure variants leave dual-write attribution installed.
> * Tenant view creation is not blocked during transform .
> * Resume from PAUSED/FAILED runs partial-pass, missing writes accumulated in the parked window — silent data loss .
> * Legacy Indexer coprocessor tables are not detected; transform runs with no dual-write — silent data loss.
> * Existing index handling (drop-and-recreate at cutover) costs N×M full table scans — not viable for Phase 2's production workload .
> * TransformType byte-to-enum deserializer throws on unknown bytes, blocking rolling upgrades that introduce new types.
> * Modification-lock primitive is missing; ALTER + concurrent transform is racy.
>
> Design doc :
> {{[https://docs.google.com/document/d/1hfXMDETDf9aT2Qa_BWiyzEVeOfTNW55Ykp6ir2OtXAs|https://docs.google.com/document/d/1Zg3rfJFKXsVF-_1aW8ELB42mJxsYtSJRfKyyZHp555M/edit?tab=t.0]}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)