You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@fluss.apache.org by Lorenzo Affetti via dev <de...@fluss.apache.org> on 2026/06/12 14:44:05 UTC
[DISCUSS] FIP-29: (m)TLS support
Hello community!
I would like to start a discussion about FIP-29: (m)TLS support.
Here is the motivation:
All Fluss RPC traffic — between clients and servers, and between servers —
currently travels over the network *in plaintext*.
This has two consequences:
1. *Confidentiality.* Table data, metadata, and even the credentials
exchanged by the existing application-layer authentication are visible to
anyone able to observe the network. In shared, cloud, or multi-tenant
deployments this is unacceptable.
2. *Strong client authentication.* Fluss has a pluggable authentication
framework, but it has no certificate-based mechanism. Operators who
standardize on PKI / mutual TLS for service-to-service identity (a very
common requirement) cannot integrate Fluss.
This FIP adds:
- *TLS transport encryption* for the Netty RPC layer, enabled per server
listener and on the client.
- *Mutual TLS (mTLS) authentication*, where the server derives the
authenticated FlussPrincipal from the client's X.509 certificate,
integrated into the existing authentication/authorization pipeline.
Eager to hear about your thoughts!
--
Lorenzo Affetti
Team Leader of Stream Storage
lorenzo.affetti@ververica.com
www.ververica.com
------------------------------
<https://www.ververica.com/>
Ververica GmbH | Herzogspitalstrasse 24 | 80331 München | Germany
Follow us:
<https://www.linkedin.com/company/ververica/posts/?feedView=all>
<ht...@ververica>
<https://open.spotify.com/show/2XME9h8iBOyr6YupqM99ir?si=87b064644add42a1>Available
on: <https://aws.amazon.com/marketplace/pp/prodview-luvmqd6leha4i>
<https://marketplace.microsoft.com/en-us/product/saas/ververica.vvc_managed?tab=Overview>
Pflichtangaben/Mandatory Information
<https://www.ververica.com/mandatory-information>
Re: [DISCUSS] FIP-29: (m)TLS support
Posted by Anton Borisov <bo...@gmail.com>.
Hi Lorenzo,
Thanks for the FIP, I went through it and it feels solid.
Direction makes sense to me: keeping TLS orthogonal to auth fits the
per-listener plugin model and the SPI change is one
backward-compatible method.
+1 to Giannis on zero-copy, and worth pinning down concretely. The
fetch path uses sendfile today (FileRegionSend ->
FlussFileRegion.transferTo -> FileChannel.transferTo), and SslHandler
can't wrap a FileRegion, so TLS listeners fall back to
copy-through-the-engine and lose zero-copy. Same trade-off Kafka makes
with SSL. The OpenSSL provider (already future work) cuts the
encryption CPU but doesn't bring sendfile back. Only kTLS does, and
that's Linux-only + native transport + experimental in Netty, so I'd
park it explicitly as out of scope rather than leave it implied.
Either way, it is worth calling the throughput hit out in the FIP so
operators aren't surprised.
A few more things:
1. Config validation. security.ssl.enabled.listeners (transport) and
security.protocol.map (auth) are set independently, but mTLS only
works over TLS. A listener mapped to mTLS but left out of the SSL set
starts clean, then every connection dies at AuthenticateRequest on a
null sslSession(), which the FIP already documents. A config typo
surfacing as a per-request runtime failure. Shall we validate at
startup instead: mTLS listener must be SSL-enabled and have a
truststore, TLS listener needs a keystore, client mTLS needs the SSL
switch plus a keystore? Cheap, and it fails loud and early.
2. Observability vs the reload machinery. Metrics are deferred to
logs; I'd pull two into v1. A handshake-failure counter and that's
just matching Kafka, which exposes failed-authentication-total/-rate
with SSL handshake failures counted. And a cert days-to-expiry gauge,
which Kafka notably lacks, and that gap has taken clusters down
(Strimzi had a proposal to add expiry monitoring, don't remember the
status and if it's merged or not though), so worth shipping rather
than repeating. Handshake failure is below the normal RPC error path,
so logs-only is effectively invisible in prod. Meanwhile the reactive
self-healing reload (content hashing, rate limiting) is heavier than
anything Kafka ships - the auto-reload work there (KIP-687/KIP-1119)
is just file-watch + interval, and the long-standing path is
operator-triggered dynamic config. Shall we ship interval reload +
those two metrics in v1 and defer the self-healer? WDYT?
3. Two clarifications.
(a) The FIP secures RPC but says nothing about the server<->ZK
connection, remote storage. Let's just say smth in Non-Goals, so we're
not implying the cluster is end-to-end encrypted by default with this
FIP implementation, but ZK/remote storage encryption is handled by
their own mechanism.
(b) SslPrincipalMapper is "Kafka-compatible" but v1 ships only DEFAULT
+ basic rules. A rule copied from a working Kafka config could map to
the wrong principal silently - a silent authz failure, not just wasted
work. Shall we fail-fast on rules we don't fully support rather than
fall through?
-- Anton
пт, 19 июн. 2026 г. в 05:58, Giannis Polyzos <ip...@gmail.com>:
>
> Hi Lorenzo,
> thank you for the really nice and detailed FIP.
>
> May I suggest attaching the link to the proposal?
> https://cwiki.apache.org/confluence/display/FLUSS/FIP-29%3A+%28m%29TLS+Support
>
> Overall the proposal looks great and looks solid to me.
> Two questions I have just for clarity:
> 1. Netty’s SslHandler can’t encrypt a FileRegion.. Does this mean that TLS
> enables clients will lose zero-copy? Or is there a workaround for this? If
> not I think maybe it would be good to explicitly mention this in the
> proposal so users are aware of this trade-off
>
> 2. Does the design assume always short-lived credentials?
>
> Overall it looks good to me and it’s a +1 on my side.
>
> Best,
> Giannis
>
> On Fri, 12 Jun 2026 at 5:44 PM, Lorenzo Affetti via dev <
> dev@fluss.apache.org> wrote:
>
> > Hello community!
> >
> > I would like to start a discussion about FIP-29: (m)TLS support.
> >
> > Here is the motivation:
> >
> > All Fluss RPC traffic — between clients and servers, and between servers —
> > currently travels over the network *in plaintext*.
> >
> > This has two consequences:
> >
> > 1. *Confidentiality.* Table data, metadata, and even the credentials
> > exchanged by the existing application-layer authentication are visible
> > to
> > anyone able to observe the network. In shared, cloud, or multi-tenant
> > deployments this is unacceptable.
> > 2. *Strong client authentication.* Fluss has a pluggable authentication
> > framework, but it has no certificate-based mechanism. Operators who
> > standardize on PKI / mutual TLS for service-to-service identity (a very
> > common requirement) cannot integrate Fluss.
> >
> > This FIP adds:
> >
> > - *TLS transport encryption* for the Netty RPC layer, enabled per server
> > listener and on the client.
> > - *Mutual TLS (mTLS) authentication*, where the server derives the
> > authenticated FlussPrincipal from the client's X.509 certificate,
> > integrated into the existing authentication/authorization pipeline.
> >
> >
> > Eager to hear about your thoughts!
> >
> > --
> > Lorenzo Affetti
> > Team Leader of Stream Storage
> > lorenzo.affetti@ververica.com
> > www.ververica.com
> > ------------------------------
> >
> > <https://www.ververica.com/>
> > Ververica GmbH | Herzogspitalstrasse 24 | 80331 München | Germany
> > <https://www.google.com/maps/search/Herzogspitalstrasse+24+%7C+80331+M%C3%BCnchen+%7C+Germany?entry=gmail&source=g>
> >
> > Follow us:
> > <https://www.linkedin.com/company/ververica/posts/?feedView=all>
> > <ht...@ververica>
> > <https://open.spotify.com/show/2XME9h8iBOyr6YupqM99ir?si=87b064644add42a1
> > >Available
> > on: <https://aws.amazon.com/marketplace/pp/prodview-luvmqd6leha4i>
> > <
> > https://marketplace.microsoft.com/en-us/product/saas/ververica.vvc_managed?tab=Overview
> > >
> >
> > Pflichtangaben/Mandatory Information
> > <https://www.ververica.com/mandatory-information>
> >
Re: [DISCUSS] FIP-29: (m)TLS support
Posted by Giannis Polyzos <ip...@gmail.com>.
Hi Lorenzo and all,
I think this proposal is also solid and after addressing any open comments
we can start the Vote.
Best,
Giannis
On Sun, 28 Jun 2026 at 4:40 AM, Forward Xu <fo...@gmail.com> wrote:
> Hi Lorenzo,
>
> Thanks for driving this FIP — adding TLS/mTLS support is an important step
> for running Fluss in shared, cloud, and multi-tenant environments, so big
> +1 from me on the direction.
>
> I'd also like to echo the two points Giannis raised, as I think they're
> worth addressing explicitly in the proposal:
>
> 1.
>
> *Zero-copy trade-off.* Since Netty's SslHandler can't encrypt a
> FileRegion, enabling TLS effectively means losing the zero-copy path. It
> would be great to call this out explicitly in the FIP so users
> understand
> the performance trade-off when turning TLS on, and to clarify whether it
> applies only to TLS-enabled listeners (so plaintext internal paths can
> keep
> zero-copy).
> 2.
>
> *Credential lifetime.* Like Giannis, I'd find it helpful to know whether
> the design assumes short-lived credentials, or whether certificate
> rotation
> / revocation (CRL/OCSP) is left to the operator's PKI.
>
> Overall this looks solid to me. Thanks again for the work!
>
> Best, ForwardXu
>
> Anton Borisov <bo...@gmail.com> 于2026年6月26日周五 18:57写道：
>
> > Hi Lorenzo,
> >
> > Thank you for the clarifications, this makes sense to me.
> >
> > -- Anton
> >
> > ср, 24 июн. 2026 г. в 16:16, Lorenzo Affetti via dev <
> dev@fluss.apache.org
> > >:
> >
> > >
> > > Hi Giannis, Anton,
> > >
> > > Thanks for the detailed reviews. I've updated the FIP to address all
> > > points raised. You can find the summary of changes below and a diff
> > attached for your convenience.
> > >
> > > ---
> > >
> > > Zero-copy (Giannis Q1, Anton)
> > >
> > > This was a big oversight on my part; I apologize.
> > > Thanks for unveiling this.
> > >
> > > You are both correct. TLS and sendfile are mutually exclusive:
> SslHandler
> > > requires bytes in JVM-managed memory before encryption, which is
> > > incompatible with FileChannel.transferTo(). On TLS-enabled listeners,
> log
> > > fetch responses are copied into a buffer in the JVM before encryption.
> > > Plaintext listeners are unaffected. This is the same trade-off Apache
> > Kafka
> > > makes with SSL.
> > >
> > > I've added a "Performance trade-off: zero-copy" subsection to the FIP
> > that
> > > calls this out explicitly, so operators can account for it in capacity
> > > planning. kTLS is now explicitly listed as Out of Scope rather than
> left
> > > implied.
> > >
> > > ---
> > >
> > > Short-lived credentials (Giannis Q2)
> > >
> > > The design does not assume short-lived credentials. Hot-reload is
> > entirely
> > > opt-in: setting security.ssl.reload.interval=0 disables polling, and
> the
> > > system works correctly with static long-lived certificates. Rotation in
> > that
> > > model is handled with an explicit rolling restart, exactly as any other
> > > config change today. The 5-minute default targets cloud-native
> > deployments;
> > > if files never change, reload is a no-op.
> > >
> > > I've added an explicit note on static/long-lived certs to the
> hot-reload
> > > section.
> > >
> > > ---
> > >
> > > Non-goals (Anton)
> > >
> > > Added a dedicated Non-Goals section covering ZooKeeper connection
> > security,
> > > remote storage encryption, and data at rest. The FIP now states
> > explicitly
> > > that enabling RPC TLS does not imply end-to-end cluster encryption.
> > >
> > > ---
> > >
> > > Configuration validation (Anton)
> > >
> > > Expanded to a fail-fast startup validation table covering the cases you
> > > listed.
> > >
> > > ---
> > >
> > > Metrics (Anton)
> > >
> > > The two metrics you flagged are promoted to v1:
> > >
> > > - tls.handshake.failures (counter, per listener): matches
> > > Kafka's failed-authentication-total. Handshake failures are below
> > > the normal RPC error path; logs-only is effectively invisible in
> > > production.
> > >
> > > - tls.certificate.days.to.expiry (gauge, per listener): fills the
> > > gap Kafka has never closed, enabling expiry alerting before it
> > > becomes a handshake failure.
> > >
> > > On the reactive self-healer: I agree the interval-only model is
> > > simpler and closer to what Kafka ships. However, the reactive trigger
> > > provides a concrete safety guarantee that the interval-only model
> > > cannot: a certificate renewed just after a poll is adopted on the
> > > next connection attempt rather than after up to reload.interval. For
> > > short-lived certs (e.g., 1-hour SPIFFE SVIDs) with a 5-minute poll
> > > interval, that window matters. The reactive logic is also isolated and
> > > safe by construction: rate-limited, never swaps in a broken context,
> > > and does not retry the triggering connection. I'd like to keep it in
> > > v1. Happy to discuss further on the list if you feel strongly about
> > > deferring it.
> > >
> > > ---
> > >
> > > SslPrincipalMapper (Anton)
> > >
> > > Added explicit fail-fast behaviour: unsupported rule grammar causes the
> > > server to refuse to start, not fail at authentication time. This is the
> > > right place — the rules are static config, validation is cheap (regex
> > > compile), and startup failure is far more diagnosable than a per-
> > > connection authorization failure that looks like an ACL
> misconfiguration.
> > > The validation table in the FIP now has an eighth row for this.
> > >
> > > ---
> > >
> > > Let me know if
> > > anything needs further discussion.
> > >
> > > Best,
> > > Lorenzo
> > >
> > > On Fri, Jun 19, 2026 at 6:58 AM Giannis Polyzos <ipolyzos.se@gmail.com
> >
> > wrote:
> > >>
> > >> Hi Lorenzo,
> > >> thank you for the really nice and detailed FIP.
> > >>
> > >> May I suggest attaching the link to the proposal?
> > >>
> >
> https://cwiki.apache.org/confluence/display/FLUSS/FIP-29%3A+%28m%29TLS+Support
> > >>
> > >> Overall the proposal looks great and looks solid to me.
> > >> Two questions I have just for clarity:
> > >> 1. Netty’s SslHandler can’t encrypt a FileRegion.. Does this mean
> that
> > TLS enables clients will lose zero-copy? Or is there a workaround for
> this?
> > If not I think maybe it would be good to explicitly mention this in the
> > proposal so users are aware of this trade-off
> > >>
> > >> 2. Does the design assume always short-lived credentials?
> > >>
> > >> Overall it looks good to me and it’s a +1 on my side.
> > >>
> > >> Best,
> > >> Giannis
> > >>
> > >> On Fri, 12 Jun 2026 at 5:44 PM, Lorenzo Affetti via dev <
> > dev@fluss.apache.org> wrote:
> > >>>
> > >>> Hello community!
> > >>>
> > >>> I would like to start a discussion about FIP-29: (m)TLS support.
> > >>>
> > >>> Here is the motivation:
> > >>>
> > >>> All Fluss RPC traffic — between clients and servers, and between
> > servers —
> > >>> currently travels over the network *in plaintext*.
> > >>>
> > >>> This has two consequences:
> > >>>
> > >>> 1. *Confidentiality.* Table data, metadata, and even the
> credentials
> > >>> exchanged by the existing application-layer authentication are
> > visible to
> > >>> anyone able to observe the network. In shared, cloud, or
> > multi-tenant
> > >>> deployments this is unacceptable.
> > >>> 2. *Strong client authentication.* Fluss has a pluggable
> > authentication
> > >>> framework, but it has no certificate-based mechanism. Operators
> who
> > >>> standardize on PKI / mutual TLS for service-to-service identity (a
> > very
> > >>> common requirement) cannot integrate Fluss.
> > >>>
> > >>> This FIP adds:
> > >>>
> > >>> - *TLS transport encryption* for the Netty RPC layer, enabled per
> > server
> > >>> listener and on the client.
> > >>> - *Mutual TLS (mTLS) authentication*, where the server derives the
> > >>> authenticated FlussPrincipal from the client's X.509 certificate,
> > >>> integrated into the existing authentication/authorization
> pipeline.
> > >>>
> > >>>
> > >>> Eager to hear about your thoughts!
> > >>>
> > >>> --
> > >>> Lorenzo Affetti
> > >>> Team Leader of Stream Storage
> > >>> lorenzo.affetti@ververica.com
> > >>> www.ververica.com
> > >>> ------------------------------
> > >>>
> > >>> <https://www.ververica.com/>
> > >>> Ververica GmbH | Herzogspitalstrasse 24 | 80331 München | Germany
> > >>>
> > >>> Follow us:
> > >>> <https://www.linkedin.com/company/ververica/posts/?feedView=all>
> > >>> <ht...@ververica>
> > >>> <
> > https://open.spotify.com/show/2XME9h8iBOyr6YupqM99ir?si=87b064644add42a1
> > >Available
> > >>> on: <https://aws.amazon.com/marketplace/pp/prodview-luvmqd6leha4i>
> > >>> <
> >
> https://marketplace.microsoft.com/en-us/product/saas/ververica.vvc_managed?tab=Overview
> > >
> > >>>
> > >>> Pflichtangaben/Mandatory Information
> > >>> <https://www.ververica.com/mandatory-information>
> > >
> > >
> > >
> > > --
> > > Lorenzo Affetti
> > > Senior Software Engineer @ Flink Team
> > > Ververica
> >
>
Re: [DISCUSS] FIP-29: (m)TLS support
Posted by Forward Xu <fo...@gmail.com>.
Hi Lorenzo,
Thanks for driving this FIP — adding TLS/mTLS support is an important step
for running Fluss in shared, cloud, and multi-tenant environments, so big
+1 from me on the direction.
I'd also like to echo the two points Giannis raised, as I think they're
worth addressing explicitly in the proposal:
1.
*Zero-copy trade-off.* Since Netty's SslHandler can't encrypt a
FileRegion, enabling TLS effectively means losing the zero-copy path. It
would be great to call this out explicitly in the FIP so users understand
the performance trade-off when turning TLS on, and to clarify whether it
applies only to TLS-enabled listeners (so plaintext internal paths can keep
zero-copy).
2.
*Credential lifetime.* Like Giannis, I'd find it helpful to know whether
the design assumes short-lived credentials, or whether certificate rotation
/ revocation (CRL/OCSP) is left to the operator's PKI.
Overall this looks solid to me. Thanks again for the work!
Best, ForwardXu
Anton Borisov <bo...@gmail.com> 于2026年6月26日周五 18:57写道：
> Hi Lorenzo,
>
> Thank you for the clarifications, this makes sense to me.
>
> -- Anton
>
> ср, 24 июн. 2026 г. в 16:16, Lorenzo Affetti via dev <dev@fluss.apache.org
> >:
>
> >
> > Hi Giannis, Anton,
> >
> > Thanks for the detailed reviews. I've updated the FIP to address all
> > points raised. You can find the summary of changes below and a diff
> attached for your convenience.
> >
> > ---
> >
> > Zero-copy (Giannis Q1, Anton)
> >
> > This was a big oversight on my part; I apologize.
> > Thanks for unveiling this.
> >
> > You are both correct. TLS and sendfile are mutually exclusive: SslHandler
> > requires bytes in JVM-managed memory before encryption, which is
> > incompatible with FileChannel.transferTo(). On TLS-enabled listeners, log
> > fetch responses are copied into a buffer in the JVM before encryption.
> > Plaintext listeners are unaffected. This is the same trade-off Apache
> Kafka
> > makes with SSL.
> >
> > I've added a "Performance trade-off: zero-copy" subsection to the FIP
> that
> > calls this out explicitly, so operators can account for it in capacity
> > planning. kTLS is now explicitly listed as Out of Scope rather than left
> > implied.
> >
> > ---
> >
> > Short-lived credentials (Giannis Q2)
> >
> > The design does not assume short-lived credentials. Hot-reload is
> entirely
> > opt-in: setting security.ssl.reload.interval=0 disables polling, and the
> > system works correctly with static long-lived certificates. Rotation in
> that
> > model is handled with an explicit rolling restart, exactly as any other
> > config change today. The 5-minute default targets cloud-native
> deployments;
> > if files never change, reload is a no-op.
> >
> > I've added an explicit note on static/long-lived certs to the hot-reload
> > section.
> >
> > ---
> >
> > Non-goals (Anton)
> >
> > Added a dedicated Non-Goals section covering ZooKeeper connection
> security,
> > remote storage encryption, and data at rest. The FIP now states
> explicitly
> > that enabling RPC TLS does not imply end-to-end cluster encryption.
> >
> > ---
> >
> > Configuration validation (Anton)
> >
> > Expanded to a fail-fast startup validation table covering the cases you
> > listed.
> >
> > ---
> >
> > Metrics (Anton)
> >
> > The two metrics you flagged are promoted to v1:
> >
> > - tls.handshake.failures (counter, per listener): matches
> > Kafka's failed-authentication-total. Handshake failures are below
> > the normal RPC error path; logs-only is effectively invisible in
> > production.
> >
> > - tls.certificate.days.to.expiry (gauge, per listener): fills the
> > gap Kafka has never closed, enabling expiry alerting before it
> > becomes a handshake failure.
> >
> > On the reactive self-healer: I agree the interval-only model is
> > simpler and closer to what Kafka ships. However, the reactive trigger
> > provides a concrete safety guarantee that the interval-only model
> > cannot: a certificate renewed just after a poll is adopted on the
> > next connection attempt rather than after up to reload.interval. For
> > short-lived certs (e.g., 1-hour SPIFFE SVIDs) with a 5-minute poll
> > interval, that window matters. The reactive logic is also isolated and
> > safe by construction: rate-limited, never swaps in a broken context,
> > and does not retry the triggering connection. I'd like to keep it in
> > v1. Happy to discuss further on the list if you feel strongly about
> > deferring it.
> >
> > ---
> >
> > SslPrincipalMapper (Anton)
> >
> > Added explicit fail-fast behaviour: unsupported rule grammar causes the
> > server to refuse to start, not fail at authentication time. This is the
> > right place — the rules are static config, validation is cheap (regex
> > compile), and startup failure is far more diagnosable than a per-
> > connection authorization failure that looks like an ACL misconfiguration.
> > The validation table in the FIP now has an eighth row for this.
> >
> > ---
> >
> > Let me know if
> > anything needs further discussion.
> >
> > Best,
> > Lorenzo
> >
> > On Fri, Jun 19, 2026 at 6:58 AM Giannis Polyzos <ip...@gmail.com>
> wrote:
> >>
> >> Hi Lorenzo,
> >> thank you for the really nice and detailed FIP.
> >>
> >> May I suggest attaching the link to the proposal?
> >>
> https://cwiki.apache.org/confluence/display/FLUSS/FIP-29%3A+%28m%29TLS+Support
> >>
> >> Overall the proposal looks great and looks solid to me.
> >> Two questions I have just for clarity:
> >> 1. Netty’s SslHandler can’t encrypt a FileRegion.. Does this mean that
> TLS enables clients will lose zero-copy? Or is there a workaround for this?
> If not I think maybe it would be good to explicitly mention this in the
> proposal so users are aware of this trade-off
> >>
> >> 2. Does the design assume always short-lived credentials?
> >>
> >> Overall it looks good to me and it’s a +1 on my side.
> >>
> >> Best,
> >> Giannis
> >>
> >> On Fri, 12 Jun 2026 at 5:44 PM, Lorenzo Affetti via dev <
> dev@fluss.apache.org> wrote:
> >>>
> >>> Hello community!
> >>>
> >>> I would like to start a discussion about FIP-29: (m)TLS support.
> >>>
> >>> Here is the motivation:
> >>>
> >>> All Fluss RPC traffic — between clients and servers, and between
> servers —
> >>> currently travels over the network *in plaintext*.
> >>>
> >>> This has two consequences:
> >>>
> >>> 1. *Confidentiality.* Table data, metadata, and even the credentials
> >>> exchanged by the existing application-layer authentication are
> visible to
> >>> anyone able to observe the network. In shared, cloud, or
> multi-tenant
> >>> deployments this is unacceptable.
> >>> 2. *Strong client authentication.* Fluss has a pluggable
> authentication
> >>> framework, but it has no certificate-based mechanism. Operators who
> >>> standardize on PKI / mutual TLS for service-to-service identity (a
> very
> >>> common requirement) cannot integrate Fluss.
> >>>
> >>> This FIP adds:
> >>>
> >>> - *TLS transport encryption* for the Netty RPC layer, enabled per
> server
> >>> listener and on the client.
> >>> - *Mutual TLS (mTLS) authentication*, where the server derives the
> >>> authenticated FlussPrincipal from the client's X.509 certificate,
> >>> integrated into the existing authentication/authorization pipeline.
> >>>
> >>>
> >>> Eager to hear about your thoughts!
> >>>
> >>> --
> >>> Lorenzo Affetti
> >>> Team Leader of Stream Storage
> >>> lorenzo.affetti@ververica.com
> >>> www.ververica.com
> >>> ------------------------------
> >>>
> >>> <https://www.ververica.com/>
> >>> Ververica GmbH | Herzogspitalstrasse 24 | 80331 München | Germany
> >>>
> >>> Follow us:
> >>> <https://www.linkedin.com/company/ververica/posts/?feedView=all>
> >>> <ht...@ververica>
> >>> <
> https://open.spotify.com/show/2XME9h8iBOyr6YupqM99ir?si=87b064644add42a1
> >Available
> >>> on: <https://aws.amazon.com/marketplace/pp/prodview-luvmqd6leha4i>
> >>> <
> https://marketplace.microsoft.com/en-us/product/saas/ververica.vvc_managed?tab=Overview
> >
> >>>
> >>> Pflichtangaben/Mandatory Information
> >>> <https://www.ververica.com/mandatory-information>
> >
> >
> >
> > --
> > Lorenzo Affetti
> > Senior Software Engineer @ Flink Team
> > Ververica
>
Re: [DISCUSS] FIP-29: (m)TLS support
Posted by Anton Borisov <bo...@gmail.com>.
Hi Lorenzo,
Thank you for the clarifications, this makes sense to me.
-- Anton
ср, 24 июн. 2026 г. в 16:16, Lorenzo Affetti via dev <de...@fluss.apache.org>:
>
> Hi Giannis, Anton,
>
> Thanks for the detailed reviews. I've updated the FIP to address all
> points raised. You can find the summary of changes below and a diff attached for your convenience.
>
> ---
>
> Zero-copy (Giannis Q1, Anton)
>
> This was a big oversight on my part; I apologize.
> Thanks for unveiling this.
>
> You are both correct. TLS and sendfile are mutually exclusive: SslHandler
> requires bytes in JVM-managed memory before encryption, which is
> incompatible with FileChannel.transferTo(). On TLS-enabled listeners, log
> fetch responses are copied into a buffer in the JVM before encryption.
> Plaintext listeners are unaffected. This is the same trade-off Apache Kafka
> makes with SSL.
>
> I've added a "Performance trade-off: zero-copy" subsection to the FIP that
> calls this out explicitly, so operators can account for it in capacity
> planning. kTLS is now explicitly listed as Out of Scope rather than left
> implied.
>
> ---
>
> Short-lived credentials (Giannis Q2)
>
> The design does not assume short-lived credentials. Hot-reload is entirely
> opt-in: setting security.ssl.reload.interval=0 disables polling, and the
> system works correctly with static long-lived certificates. Rotation in that
> model is handled with an explicit rolling restart, exactly as any other
> config change today. The 5-minute default targets cloud-native deployments;
> if files never change, reload is a no-op.
>
> I've added an explicit note on static/long-lived certs to the hot-reload
> section.
>
> ---
>
> Non-goals (Anton)
>
> Added a dedicated Non-Goals section covering ZooKeeper connection security,
> remote storage encryption, and data at rest. The FIP now states explicitly
> that enabling RPC TLS does not imply end-to-end cluster encryption.
>
> ---
>
> Configuration validation (Anton)
>
> Expanded to a fail-fast startup validation table covering the cases you
> listed.
>
> ---
>
> Metrics (Anton)
>
> The two metrics you flagged are promoted to v1:
>
> - tls.handshake.failures (counter, per listener): matches
> Kafka's failed-authentication-total. Handshake failures are below
> the normal RPC error path; logs-only is effectively invisible in
> production.
>
> - tls.certificate.days.to.expiry (gauge, per listener): fills the
> gap Kafka has never closed, enabling expiry alerting before it
> becomes a handshake failure.
>
> On the reactive self-healer: I agree the interval-only model is
> simpler and closer to what Kafka ships. However, the reactive trigger
> provides a concrete safety guarantee that the interval-only model
> cannot: a certificate renewed just after a poll is adopted on the
> next connection attempt rather than after up to reload.interval. For
> short-lived certs (e.g., 1-hour SPIFFE SVIDs) with a 5-minute poll
> interval, that window matters. The reactive logic is also isolated and
> safe by construction: rate-limited, never swaps in a broken context,
> and does not retry the triggering connection. I'd like to keep it in
> v1. Happy to discuss further on the list if you feel strongly about
> deferring it.
>
> ---
>
> SslPrincipalMapper (Anton)
>
> Added explicit fail-fast behaviour: unsupported rule grammar causes the
> server to refuse to start, not fail at authentication time. This is the
> right place — the rules are static config, validation is cheap (regex
> compile), and startup failure is far more diagnosable than a per-
> connection authorization failure that looks like an ACL misconfiguration.
> The validation table in the FIP now has an eighth row for this.
>
> ---
>
> Let me know if
> anything needs further discussion.
>
> Best,
> Lorenzo
>
> On Fri, Jun 19, 2026 at 6:58 AM Giannis Polyzos <ip...@gmail.com> wrote:
>>
>> Hi Lorenzo,
>> thank you for the really nice and detailed FIP.
>>
>> May I suggest attaching the link to the proposal?
>> https://cwiki.apache.org/confluence/display/FLUSS/FIP-29%3A+%28m%29TLS+Support
>>
>> Overall the proposal looks great and looks solid to me.
>> Two questions I have just for clarity:
>> 1. Netty’s SslHandler can’t encrypt a FileRegion.. Does this mean that TLS enables clients will lose zero-copy? Or is there a workaround for this? If not I think maybe it would be good to explicitly mention this in the proposal so users are aware of this trade-off
>>
>> 2. Does the design assume always short-lived credentials?
>>
>> Overall it looks good to me and it’s a +1 on my side.
>>
>> Best,
>> Giannis
>>
>> On Fri, 12 Jun 2026 at 5:44 PM, Lorenzo Affetti via dev <de...@fluss.apache.org> wrote:
>>>
>>> Hello community!
>>>
>>> I would like to start a discussion about FIP-29: (m)TLS support.
>>>
>>> Here is the motivation:
>>>
>>> All Fluss RPC traffic — between clients and servers, and between servers —
>>> currently travels over the network *in plaintext*.
>>>
>>> This has two consequences:
>>>
>>> 1. *Confidentiality.* Table data, metadata, and even the credentials
>>> exchanged by the existing application-layer authentication are visible to
>>> anyone able to observe the network. In shared, cloud, or multi-tenant
>>> deployments this is unacceptable.
>>> 2. *Strong client authentication.* Fluss has a pluggable authentication
>>> framework, but it has no certificate-based mechanism. Operators who
>>> standardize on PKI / mutual TLS for service-to-service identity (a very
>>> common requirement) cannot integrate Fluss.
>>>
>>> This FIP adds:
>>>
>>> - *TLS transport encryption* for the Netty RPC layer, enabled per server
>>> listener and on the client.
>>> - *Mutual TLS (mTLS) authentication*, where the server derives the
>>> authenticated FlussPrincipal from the client's X.509 certificate,
>>> integrated into the existing authentication/authorization pipeline.
>>>
>>>
>>> Eager to hear about your thoughts!
>>>
>>> --
>>> Lorenzo Affetti
>>> Team Leader of Stream Storage
>>> lorenzo.affetti@ververica.com
>>> www.ververica.com
>>> ------------------------------
>>>
>>> <https://www.ververica.com/>
>>> Ververica GmbH | Herzogspitalstrasse 24 | 80331 München | Germany
>>>
>>> Follow us:
>>> <https://www.linkedin.com/company/ververica/posts/?feedView=all>
>>> <ht...@ververica>
>>> <https://open.spotify.com/show/2XME9h8iBOyr6YupqM99ir?si=87b064644add42a1>Available
>>> on: <https://aws.amazon.com/marketplace/pp/prodview-luvmqd6leha4i>
>>> <https://marketplace.microsoft.com/en-us/product/saas/ververica.vvc_managed?tab=Overview>
>>>
>>> Pflichtangaben/Mandatory Information
>>> <https://www.ververica.com/mandatory-information>
>
>
>
> --
> Lorenzo Affetti
> Senior Software Engineer @ Flink Team
> Ververica
Re: [DISCUSS] FIP-29: (m)TLS support
Posted by Lorenzo Affetti via dev <de...@fluss.apache.org>.
Hi Giannis, Anton,
Thanks for the detailed reviews. I've updated the FIP to address all
points raised. You can find the summary of changes below and a diff
attached for your convenience.
---
Zero-copy (Giannis Q1, Anton)
This was a big oversight on my part; I apologize.
Thanks for unveiling this.
You are both correct. TLS and sendfile are mutually exclusive: SslHandler
requires bytes in JVM-managed memory before encryption, which is
incompatible with FileChannel.transferTo(). On TLS-enabled listeners, log
fetch responses are copied into a buffer in the JVM before encryption.
Plaintext listeners are unaffected. This is the same trade-off Apache Kafka
makes with SSL.
I've added a "Performance trade-off: zero-copy" subsection to the FIP that
calls this out explicitly, so operators can account for it in capacity
planning. kTLS is now explicitly listed as Out of Scope rather than left
implied.
---
Short-lived credentials (Giannis Q2)
The design does not assume short-lived credentials. Hot-reload is entirely
opt-in: setting security.ssl.reload.interval=0 disables polling, and the
system works correctly with static long-lived certificates. Rotation in that
model is handled with an explicit rolling restart, exactly as any other
config change today. The 5-minute default targets cloud-native deployments;
if files never change, reload is a no-op.
I've added an explicit note on static/long-lived certs to the hot-reload
section.
---
Non-goals (Anton)
Added a dedicated Non-Goals section covering ZooKeeper connection security,
remote storage encryption, and data at rest. The FIP now states explicitly
that enabling RPC TLS does not imply end-to-end cluster encryption.
---
Configuration validation (Anton)
Expanded to a fail-fast startup validation table covering the cases you
listed.
---
Metrics (Anton)
The two metrics you flagged are promoted to v1:
- tls.handshake.failures (counter, per listener): matches
Kafka's failed-authentication-total. Handshake failures are below
the normal RPC error path; logs-only is effectively invisible in
production.
- tls.certificate.days.to.expiry (gauge, per listener): fills the
gap Kafka has never closed, enabling expiry alerting before it
becomes a handshake failure.
On the reactive self-healer: I agree the interval-only model is
simpler and closer to what Kafka ships. However, the reactive trigger
provides a concrete safety guarantee that the interval-only model
cannot: a certificate renewed just after a poll is adopted on the
next connection attempt rather than after up to reload.interval. For
short-lived certs (e.g., 1-hour SPIFFE SVIDs) with a 5-minute poll
interval, that window matters. The reactive logic is also isolated and
safe by construction: rate-limited, never swaps in a broken context,
and does not retry the triggering connection. I'd like to keep it in
v1. Happy to discuss further on the list if you feel strongly about
deferring it.
---
SslPrincipalMapper (Anton)
Added explicit fail-fast behaviour: unsupported rule grammar causes the
server to refuse to start, not fail at authentication time. This is the
right place — the rules are static config, validation is cheap (regex
compile), and startup failure is far more diagnosable than a per-
connection authorization failure that looks like an ACL misconfiguration.
The validation table in the FIP now has an eighth row for this.
---
Let me know if
anything needs further discussion.
Best,
Lorenzo
On Fri, Jun 19, 2026 at 6:58 AM Giannis Polyzos <ip...@gmail.com>
wrote:
> Hi Lorenzo,
> thank you for the really nice and detailed FIP.
>
> May I suggest attaching the link to the proposal?
>
> https://cwiki.apache.org/confluence/display/FLUSS/FIP-29%3A+%28m%29TLS+Support
>
> Overall the proposal looks great and looks solid to me.
> Two questions I have just for clarity:
> 1. Netty’s SslHandler can’t encrypt a FileRegion.. Does this mean that
> TLS enables clients will lose zero-copy? Or is there a workaround for this?
> If not I think maybe it would be good to explicitly mention this in the
> proposal so users are aware of this trade-off
>
> 2. Does the design assume always short-lived credentials?
>
> Overall it looks good to me and it’s a +1 on my side.
>
> Best,
> Giannis
>
> On Fri, 12 Jun 2026 at 5:44 PM, Lorenzo Affetti via dev <
> dev@fluss.apache.org> wrote:
>
>> Hello community!
>>
>> I would like to start a discussion about FIP-29: (m)TLS support.
>>
>> Here is the motivation:
>>
>> All Fluss RPC traffic — between clients and servers, and between servers —
>> currently travels over the network *in plaintext*.
>>
>> This has two consequences:
>>
>> 1. *Confidentiality.* Table data, metadata, and even the credentials
>> exchanged by the existing application-layer authentication are visible
>> to
>> anyone able to observe the network. In shared, cloud, or multi-tenant
>> deployments this is unacceptable.
>> 2. *Strong client authentication.* Fluss has a pluggable authentication
>> framework, but it has no certificate-based mechanism. Operators who
>> standardize on PKI / mutual TLS for service-to-service identity (a very
>> common requirement) cannot integrate Fluss.
>>
>> This FIP adds:
>>
>> - *TLS transport encryption* for the Netty RPC layer, enabled per
>> server
>> listener and on the client.
>> - *Mutual TLS (mTLS) authentication*, where the server derives the
>> authenticated FlussPrincipal from the client's X.509 certificate,
>> integrated into the existing authentication/authorization pipeline.
>>
>>
>> Eager to hear about your thoughts!
>>
>> --
>> Lorenzo Affetti
>> Team Leader of Stream Storage
>> lorenzo.affetti@ververica.com
>> www.ververica.com
>> ------------------------------
>>
>> <https://www.ververica.com/>
>> Ververica GmbH | Herzogspitalstrasse 24 | 80331 München | Germany
>> <https://www.google.com/maps/search/Herzogspitalstrasse+24+%7C+80331+M%C3%BCnchen+%7C+Germany?entry=gmail&source=g>
>>
>> Follow us:
>> <https://www.linkedin.com/company/ververica/posts/?feedView=all>
>> <ht...@ververica>
>> <https://open.spotify.com/show/2XME9h8iBOyr6YupqM99ir?si=87b064644add42a1
>> >Available
>> on: <https://aws.amazon.com/marketplace/pp/prodview-luvmqd6leha4i>
>> <
>> https://marketplace.microsoft.com/en-us/product/saas/ververica.vvc_managed?tab=Overview
>> >
>>
>> Pflichtangaben/Mandatory Information
>> <https://www.ververica.com/mandatory-information>
>>
>
--
Lorenzo Affetti
Senior Software Engineer @ Flink Team
Ververica <http://www.ververica.com>
Re: [DISCUSS] FIP-29: (m)TLS support
Posted by Giannis Polyzos <ip...@gmail.com>.
Hi Lorenzo,
thank you for the really nice and detailed FIP.
May I suggest attaching the link to the proposal?
https://cwiki.apache.org/confluence/display/FLUSS/FIP-29%3A+%28m%29TLS+Support
Overall the proposal looks great and looks solid to me.
Two questions I have just for clarity:
1. Netty’s SslHandler can’t encrypt a FileRegion.. Does this mean that TLS
enables clients will lose zero-copy? Or is there a workaround for this? If
not I think maybe it would be good to explicitly mention this in the
proposal so users are aware of this trade-off
2. Does the design assume always short-lived credentials?
Overall it looks good to me and it’s a +1 on my side.
Best,
Giannis
On Fri, 12 Jun 2026 at 5:44 PM, Lorenzo Affetti via dev <
dev@fluss.apache.org> wrote:
> Hello community!
>
> I would like to start a discussion about FIP-29: (m)TLS support.
>
> Here is the motivation:
>
> All Fluss RPC traffic — between clients and servers, and between servers —
> currently travels over the network *in plaintext*.
>
> This has two consequences:
>
> 1. *Confidentiality.* Table data, metadata, and even the credentials
> exchanged by the existing application-layer authentication are visible
> to
> anyone able to observe the network. In shared, cloud, or multi-tenant
> deployments this is unacceptable.
> 2. *Strong client authentication.* Fluss has a pluggable authentication
> framework, but it has no certificate-based mechanism. Operators who
> standardize on PKI / mutual TLS for service-to-service identity (a very
> common requirement) cannot integrate Fluss.
>
> This FIP adds:
>
> - *TLS transport encryption* for the Netty RPC layer, enabled per server
> listener and on the client.
> - *Mutual TLS (mTLS) authentication*, where the server derives the
> authenticated FlussPrincipal from the client's X.509 certificate,
> integrated into the existing authentication/authorization pipeline.
>
>
> Eager to hear about your thoughts!
>
> --
> Lorenzo Affetti
> Team Leader of Stream Storage
> lorenzo.affetti@ververica.com
> www.ververica.com
> ------------------------------
>
> <https://www.ververica.com/>
> Ververica GmbH | Herzogspitalstrasse 24 | 80331 München | Germany
> <https://www.google.com/maps/search/Herzogspitalstrasse+24+%7C+80331+M%C3%BCnchen+%7C+Germany?entry=gmail&source=g>
>
> Follow us:
> <https://www.linkedin.com/company/ververica/posts/?feedView=all>
> <ht...@ververica>
> <https://open.spotify.com/show/2XME9h8iBOyr6YupqM99ir?si=87b064644add42a1
> >Available
> on: <https://aws.amazon.com/marketplace/pp/prodview-luvmqd6leha4i>
> <
> https://marketplace.microsoft.com/en-us/product/saas/ververica.vvc_managed?tab=Overview
> >
>
> Pflichtangaben/Mandatory Information
> <https://www.ververica.com/mandatory-information>
>