You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@fluss.apache.org by liu han <ha...@gmail.com> on 2026/03/25 14:54:56 UTC
[Discuss] FIP-33: JBOD Phase 1 - Local Multi-Directory Support
liu han <ha...@gmail.com>
5:08 PM (5 hours ago)
to dev
Hi devs,
I'd like to start a discussion about FIP-33: JBOD Phase 1 - Local
Multi-Directory Support [1].
Fluss currently assumes a single `data.dir` on each TabletServer. This
works for single-disk deployments, but it does not make good use of servers
with multiple local disks, where we would like to better utilize available
disk capacity and I/O bandwidth.
To support JBOD incrementally and keep the design low-risk, we plan to
introduce it in three phases:
1. Phase 1: Local Multi-Directory Support
2. Phase 2: Single-Disk Failure Isolation
3. Phase 3: Inter-Disk Data Rebalancing
This FIP focuses on Phase 1 only.
The goal of Phase 1 is to enable multiple local data directories on
TabletServer, support static bucket placement across directories, recover
bucket placement after restart, and keep `log + kv` co-located for primary
key tables, while remaining compatible with the existing single-directory
deployment model.
This phase does not introduce metadata, RPC, protobuf, or coordinator
changes. It also does not cover disk failure isolation or inter-disk
migration, which are planned for later phases.
Any feedback and suggestions are welcome!
[1]
https://cwiki.apache.org/confluence/display/FLUSS/FIP-33%3A+JBOD+Phase+1+-+Local+Multi-Directory+Support
Re: [Discuss] FIP-33: JBOD Phase 1 - Local Multi-Directory Support
Posted by liu han <ha...@gmail.com>.
Hi Jark,
Thanks a lot for reviewing FIP-33 and for the support.
Glad to hear the overall design looks good to you. I’ll keep collecting
feedback from the community, and if there are no major concerns, I’ll move
forward with the vote for Phase 1.
Best,
Liu Han
On Fri, Mar 27, 2026 at 2:53 PM Jark Wu <im...@gmail.com> wrote:
> Hi Liu Han,
>
> Thanks for driving this proposal. I've reviewed FIP-33 and the overall
> design looks good to me (LGTM).
>
> Looking forward to the vote and follow-up phases.
>
> Best,
> Jark
>
>
> On Wed, 25 Mar 2026 at 22:55, liu han <ha...@gmail.com> wrote:
> >
> > liu han <ha...@gmail.com>
> > 5:08 PM (5 hours ago)
> > to dev
> > Hi devs,
> >
> > I'd like to start a discussion about FIP-33: JBOD Phase 1 - Local
> > Multi-Directory Support [1].
> >
> > Fluss currently assumes a single `data.dir` on each TabletServer. This
> > works for single-disk deployments, but it does not make good use of
> servers
> > with multiple local disks, where we would like to better utilize
> available
> > disk capacity and I/O bandwidth.
> >
> > To support JBOD incrementally and keep the design low-risk, we plan to
> > introduce it in three phases:
> >
> > 1. Phase 1: Local Multi-Directory Support
> > 2. Phase 2: Single-Disk Failure Isolation
> > 3. Phase 3: Inter-Disk Data Rebalancing
> >
> > This FIP focuses on Phase 1 only.
> >
> > The goal of Phase 1 is to enable multiple local data directories on
> > TabletServer, support static bucket placement across directories, recover
> > bucket placement after restart, and keep `log + kv` co-located for
> primary
> > key tables, while remaining compatible with the existing single-directory
> > deployment model.
> >
> > This phase does not introduce metadata, RPC, protobuf, or coordinator
> > changes. It also does not cover disk failure isolation or inter-disk
> > migration, which are planned for later phases.
> >
> > Any feedback and suggestions are welcome!
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/display/FLUSS/FIP-33%3A+JBOD+Phase+1+-+Local+Multi-Directory+Support
>
Re: [Discuss] FIP-33: JBOD Phase 1 - Local Multi-Directory Support
Posted by Jark Wu <im...@gmail.com>.
Hi Liu Han,
Thanks for driving this proposal. I've reviewed FIP-33 and the overall
design looks good to me (LGTM).
Looking forward to the vote and follow-up phases.
Best,
Jark
On Wed, 25 Mar 2026 at 22:55, liu han <ha...@gmail.com> wrote:
>
> liu han <ha...@gmail.com>
> 5:08 PM (5 hours ago)
> to dev
> Hi devs,
>
> I'd like to start a discussion about FIP-33: JBOD Phase 1 - Local
> Multi-Directory Support [1].
>
> Fluss currently assumes a single `data.dir` on each TabletServer. This
> works for single-disk deployments, but it does not make good use of servers
> with multiple local disks, where we would like to better utilize available
> disk capacity and I/O bandwidth.
>
> To support JBOD incrementally and keep the design low-risk, we plan to
> introduce it in three phases:
>
> 1. Phase 1: Local Multi-Directory Support
> 2. Phase 2: Single-Disk Failure Isolation
> 3. Phase 3: Inter-Disk Data Rebalancing
>
> This FIP focuses on Phase 1 only.
>
> The goal of Phase 1 is to enable multiple local data directories on
> TabletServer, support static bucket placement across directories, recover
> bucket placement after restart, and keep `log + kv` co-located for primary
> key tables, while remaining compatible with the existing single-directory
> deployment model.
>
> This phase does not introduce metadata, RPC, protobuf, or coordinator
> changes. It also does not cover disk failure isolation or inter-disk
> migration, which are planned for later phases.
>
> Any feedback and suggestions are welcome!
>
> [1]
> https://cwiki.apache.org/confluence/display/FLUSS/FIP-33%3A+JBOD+Phase+1+-+Local+Multi-Directory+Support