You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@celeborn.apache.org by Aravind Patnam <ak...@gmail.com> on 2026/05/06 06:52:36 UTC

[VOTE] CIP 22: Encryption at Rest for Celeborn Shuffle Data

Hi all,

This is a call to vote to contribute CIP 22: Encryption at Rest for
Celeborn Shuffle Data
<https://docs.google.com/document/d/1xBrLtpb8bk8CdJENiM3aLJCbFThRKLor8uAjSsfXoxE/edit?usp=sharing>to
Apache Celeborn.

Please do vote accordingly:
[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and the reason)

Thanks once again!!

-- 
Aravind K. Patnam

Re: [VOTE] CIP 22: Encryption at Rest for Celeborn Shuffle Data

Posted by Aravind Patnam <ak...@gmail.com>.
Hey folks,

I think it has been more than 72h (3 days) since this thread started.
This CIP has passed with 3 +1 votes, and has no -1 votes.

Excited to start contributing this to Apache Celeborn!! Thanks everyone for
the feedback and discussion :)

Thanks,
Aravind

On Sun, May 10, 2026 at 5:41 AM Enrico Olivelli <eo...@gmail.com> wrote:

> +1 (non binding)
>
> Enrico
>
> Il giorno dom 10 mag 2026 alle ore 07:19 Karthik Prabhakar <
> knkarthik01@gmail.com> ha scritto:
>
> > +1 (non-binding)
> >
> > Thanks Aravind, glad to see this shaped into a CIP. The client-side
> > approach has real advantages - sendfile path stays intact, workers stay
> out
> > of key handling, clean Spark integration.
> >
> > Worth flagging for the record that CIP-22 is narrower than CELEBORN-2311
> > <https://issues.apache.org/jira/browse/CELEBORN-2311> it's Spark-only
> > (Flink/MR clients uncovered), and spark.io.encryption derives keys
> locally
> > with no KEK/DEK split or external custody - which is the part that
> matters
> > for the HIPAA/PCI/SOC 2/FedRAMP deployments I had in mind. The KeyService
> > SPI with pluggable KMS providers from CELEBORN-2311 is still needed for
> > those.
> >
> > Suggest landing this as the Spark path and keeping CELEBORN-2311 open as
> > the umbrella for the engine-agnostic worker-side track. Happy to drive
> that
> > once this lands.
> >
> > - Karthik
> >
> > On Wed, May 6, 2026 at 10:16 AM Mridul Muralidharan <mr...@gmail.com>
> > wrote:
> >
> > > +1
> > >
> > > This is based on what we are currently in the process of rolling out
> > > internally in production.
> > >
> > > Regards,
> > > Mridul
> > >
> > > On Wed, May 6, 2026 at 1:53 AM Aravind Patnam <ak...@gmail.com>
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > This is a call to vote to contribute CIP 22: Encryption at Rest for
> > > > Celeborn Shuffle Data
> > > > <
> > > >
> > >
> >
> https://docs.google.com/document/d/1xBrLtpb8bk8CdJENiM3aLJCbFThRKLor8uAjSsfXoxE/edit?usp=sharing
> > > > >to
> > > > Apache Celeborn.
> > > >
> > > > Please do vote accordingly:
> > > > [ ] +1 approve
> > > > [ ] +0 no opinion
> > > > [ ] -1 disapprove (and the reason)
> > > >
> > > > Thanks once again!!
> > > >
> > > > --
> > > > Aravind K. Patnam
> > > >
> > >
> >
> >
> > --
> > Thanks,
> > Karthik
> >
>


-- 
Aravind K. Patnam

Re: [VOTE] CIP 22: Encryption at Rest for Celeborn Shuffle Data

Posted by Enrico Olivelli <eo...@gmail.com>.
+1 (non binding)

Enrico

Il giorno dom 10 mag 2026 alle ore 07:19 Karthik Prabhakar <
knkarthik01@gmail.com> ha scritto:

> +1 (non-binding)
>
> Thanks Aravind, glad to see this shaped into a CIP. The client-side
> approach has real advantages - sendfile path stays intact, workers stay out
> of key handling, clean Spark integration.
>
> Worth flagging for the record that CIP-22 is narrower than CELEBORN-2311
> <https://issues.apache.org/jira/browse/CELEBORN-2311> it's Spark-only
> (Flink/MR clients uncovered), and spark.io.encryption derives keys locally
> with no KEK/DEK split or external custody - which is the part that matters
> for the HIPAA/PCI/SOC 2/FedRAMP deployments I had in mind. The KeyService
> SPI with pluggable KMS providers from CELEBORN-2311 is still needed for
> those.
>
> Suggest landing this as the Spark path and keeping CELEBORN-2311 open as
> the umbrella for the engine-agnostic worker-side track. Happy to drive that
> once this lands.
>
> - Karthik
>
> On Wed, May 6, 2026 at 10:16 AM Mridul Muralidharan <mr...@gmail.com>
> wrote:
>
> > +1
> >
> > This is based on what we are currently in the process of rolling out
> > internally in production.
> >
> > Regards,
> > Mridul
> >
> > On Wed, May 6, 2026 at 1:53 AM Aravind Patnam <ak...@gmail.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > This is a call to vote to contribute CIP 22: Encryption at Rest for
> > > Celeborn Shuffle Data
> > > <
> > >
> >
> https://docs.google.com/document/d/1xBrLtpb8bk8CdJENiM3aLJCbFThRKLor8uAjSsfXoxE/edit?usp=sharing
> > > >to
> > > Apache Celeborn.
> > >
> > > Please do vote accordingly:
> > > [ ] +1 approve
> > > [ ] +0 no opinion
> > > [ ] -1 disapprove (and the reason)
> > >
> > > Thanks once again!!
> > >
> > > --
> > > Aravind K. Patnam
> > >
> >
>
>
> --
> Thanks,
> Karthik
>

Re: [VOTE] CIP 22: Encryption at Rest for Celeborn Shuffle Data

Posted by Karthik Prabhakar <kn...@gmail.com>.
+1 (non-binding)

Thanks Aravind, glad to see this shaped into a CIP. The client-side
approach has real advantages - sendfile path stays intact, workers stay out
of key handling, clean Spark integration.

Worth flagging for the record that CIP-22 is narrower than CELEBORN-2311
<https://issues.apache.org/jira/browse/CELEBORN-2311> it's Spark-only
(Flink/MR clients uncovered), and spark.io.encryption derives keys locally
with no KEK/DEK split or external custody - which is the part that matters
for the HIPAA/PCI/SOC 2/FedRAMP deployments I had in mind. The KeyService
SPI with pluggable KMS providers from CELEBORN-2311 is still needed for
those.

Suggest landing this as the Spark path and keeping CELEBORN-2311 open as
the umbrella for the engine-agnostic worker-side track. Happy to drive that
once this lands.

- Karthik

On Wed, May 6, 2026 at 10:16 AM Mridul Muralidharan <mr...@gmail.com>
wrote:

> +1
>
> This is based on what we are currently in the process of rolling out
> internally in production.
>
> Regards,
> Mridul
>
> On Wed, May 6, 2026 at 1:53 AM Aravind Patnam <ak...@gmail.com>
> wrote:
>
> > Hi all,
> >
> > This is a call to vote to contribute CIP 22: Encryption at Rest for
> > Celeborn Shuffle Data
> > <
> >
> https://docs.google.com/document/d/1xBrLtpb8bk8CdJENiM3aLJCbFThRKLor8uAjSsfXoxE/edit?usp=sharing
> > >to
> > Apache Celeborn.
> >
> > Please do vote accordingly:
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove (and the reason)
> >
> > Thanks once again!!
> >
> > --
> > Aravind K. Patnam
> >
>


-- 
Thanks,
Karthik

Re: [VOTE] CIP 22: Encryption at Rest for Celeborn Shuffle Data

Posted by Mridul Muralidharan <mr...@gmail.com>.
+1

This is based on what we are currently in the process of rolling out
internally in production.

Regards,
Mridul

On Wed, May 6, 2026 at 1:53 AM Aravind Patnam <ak...@gmail.com> wrote:

> Hi all,
>
> This is a call to vote to contribute CIP 22: Encryption at Rest for
> Celeborn Shuffle Data
> <
> https://docs.google.com/document/d/1xBrLtpb8bk8CdJENiM3aLJCbFThRKLor8uAjSsfXoxE/edit?usp=sharing
> >to
> Apache Celeborn.
>
> Please do vote accordingly:
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and the reason)
>
> Thanks once again!!
>
> --
> Aravind K. Patnam
>