You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@fluss.apache.org by Giannis Polyzos <ip...@gmail.com> on 2026/01/12 13:26:59 UTC
[DISCUSS] Fluss Messaging 2026
Hi everyone,
Over the last year, Fluss has grown a lot, and I think it's a good time to
update our core message on the website to reflect that.
I would like to propose the following.
Promote Fluss as: Streaming Storage For Real-Time Data & AI
another alternative Streaming Storage For Real-Time Data & Intelligent
Systems
and update our 6 core capabilities:
*1. Sub-Second Data Freshness: *Continuous ingestion and immediate
availability of data enable low-latency analytics and real-time
decision-making at scale.
*2.* *Streaming & Lakehouse Unification:* Streaming-native storage with
low-latency access on top of the lakehouse, using tables as a single
abstraction to unify real-time and historical data across engines.
*3. Columnar Streaming:* Based on *Apache Arrow *it allows database
primitives on data streams and techniques like column pruning and predicate
pushdown. This ensures engines read only the data they need, minimizing I/O
and network costs.
*4. Compute–Storage Separation:* Stream processors focus on pure
computation while Fluss manages state and storage, with features like
deduplication, partial updates, delta joins, and aggregation merge engines.
*5. ML & AI–Ready Storage:* A unified storage layer supporting row-based,
columnar, vector, and multi-modal data, enabling real-time feature stores
and a centralized data repository for ML and AI systems.
*6. Changelogs & Decision Tracking:* Built-in changelog generation provides
an append-only history of state and decision evolution, enabling auditing,
reproducibility, and deep system observability.
Any suggestions and thoughts to revisit or frame the above are highly
welcomed.
Best,
Giannis
Re: [DISCUSS] Fluss Messaging 2026
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
+1
It looks great!
Just two comments:
- I think this discussion should be moved to the dev@ mailing list, as I
don't see anything here that needs to remain private.
- On a personal note, I plan to be more active in Fluss and am happy to
work with you all on these topics.
Regards,
JB
On Mon, Jan 12, 2026 at 2:26 PM Giannis Polyzos <ip...@gmail.com>
wrote:
> Hi everyone,
>
> Over the last year, Fluss has grown a lot, and I think it's a good time to
> update our core message on the website to reflect that.
>
> I would like to propose the following.
>
> Promote Fluss as: Streaming Storage For Real-Time Data & AI
> another alternative Streaming Storage For Real-Time Data & Intelligent
> Systems
>
> and update our 6 core capabilities:
>
> *1. Sub-Second Data Freshness: *Continuous ingestion and immediate
> availability of data enable low-latency analytics and real-time
> decision-making at scale.
> *2.* *Streaming & Lakehouse Unification:* Streaming-native storage with
> low-latency access on top of the lakehouse, using tables as a single
> abstraction to unify real-time and historical data across engines.
> *3. Columnar Streaming:* Based on *Apache Arrow *it allows database
> primitives on data streams and techniques like column pruning and predicate
> pushdown. This ensures engines read only the data they need, minimizing I/O
> and network costs.
> *4. Compute–Storage Separation:* Stream processors focus on pure
> computation while Fluss manages state and storage, with features like
> deduplication, partial updates, delta joins, and aggregation merge engines.
> *5. ML & AI–Ready Storage:* A unified storage layer supporting row-based,
> columnar, vector, and multi-modal data, enabling real-time feature stores
> and a centralized data repository for ML and AI systems.
> *6. Changelogs & Decision Tracking:* Built-in changelog generation
> provides an append-only history of state and decision evolution, enabling
> auditing, reproducibility, and deep system observability.
>
> Any suggestions and thoughts to revisit or frame the above are highly
> welcomed.
>
> Best,
> Giannis
>
Re: [DISCUSS] Fluss Messaging 2026
Posted by Lorenzo Affetti via dev <de...@fluss.apache.org>.
Hello Otters!
I was reading through this one, and I find Giannis proposal excellent.
I really love this new pillar:
*4. Compute–Storage Separation:* Stream processors focus on pure
> computation while Fluss manages state and storage, with features like
> deduplication, partial updates, delta joins, and aggregation merge
engines.
I find this idea of zero-state computation in Flink and other engines quite
fascinating, especially for the use case of streaming applications, outside
of the analytics realm.
What about:
Stream & AI Storage for Real-Time Computation?
"Stream & AI Storage" ->
- the storage for your streams and your model data, vectors, etc.
- AI is more integrated in the title and less of a buzzword added at the
end
"For Real-Time Computation" -> A bit less sharp than the "Analytics" one,
while being a bit more specific than "Data", but not limiting Fluss to
Analytics only.
Only my 5 cents, I am quite new to the community, I may be missing some
crucial aspect.
Thank you guys.
On Sat, Jan 17, 2026 at 5:38 PM Jark Wu <im...@gmail.com> wrote:
> Thank you, Giannis!
>
> +1 for a blog.
>
> Maybe we can announce the new 0.9 version with the new title/message,
> and explain what AI means in the context of Fluss with all the new
> features shipped in 0.9 to highlight the AI-ready.
>
> Best,
> Jark
>
> On Sun, 18 Jan 2026 at 00:30, Giannis Polyzos <ip...@gmail.com>
> wrote:
> >
> > Absolutely this makes sense and the more explicit we are the better 👍
> >
> > Streaming Storage for Real-Time Analytics & AI
> >
> > Maybe it’s worth also putting together a blog post to explain what AI
> means
> > in the context of Fluss, so users can better understand the use cases.
> >
> > Best,
> > Giannis
> >
> >
> >
> > On Sat, 17 Jan 2026 at 6:26 PM, Jark Wu <im...@gmail.com> wrote:
> >
> > > Thanks, Giannis!
> > >
> > > I’m really excited about the new messaging. It clearly showcases
> > > Fluss’s new features and positions us firmly in the AI era.
> > >
> > > I’m just a bit torn on the title. “Streaming Storage for Real-Time
> > > Data” alone doesn’t clearly differentiate us from Kafka, which also
> > > fits that description. What if we keep the keyword “Analytics” to
> > > sharpen our positioning? For example: “Streaming Storage for Real-Time
> > > Analytics & AI”. This would maintain continuity with our existing
> > > messaging, ensuring a smooth, incremental evolution that won’t
> > > surprise users, while better highlighting Fluss’s unique value in
> > > powering analytical workloads.
> > >
> > > Best,
> > > Jark
> > >
> > > On Sat, 17 Jan 2026 at 02:44, Mehul Batra <me...@gmail.com>
> > > wrote:
> > > >
> > > > +1
> > > > Best Regards,
> > > > Mehul Batra
> > > >
> > > > On Mon, Jan 12, 2026 at 6:57 PM Giannis Polyzos <
> ipolyzos.se@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi everyone,
> > > > >
> > > > > Over the last year, Fluss has grown a lot, and I think it's a good
> > > time to
> > > > > update our core message on the website to reflect that.
> > > > >
> > > > > I would like to propose the following.
> > > > >
> > > > > Promote Fluss as: Streaming Storage For Real-Time Data & AI
> > > > > another alternative Streaming Storage For Real-Time Data &
> Intelligent
> > > > > Systems
> > > > >
> > > > > and update our 6 core capabilities:
> > > > >
> > > > > *1. Sub-Second Data Freshness: *Continuous ingestion and immediate
> > > > > availability of data enable low-latency analytics and real-time
> > > > > decision-making at scale.
> > > > > *2.* *Streaming & Lakehouse Unification:* Streaming-native storage
> with
> > > > > low-latency access on top of the lakehouse, using tables as a
> single
> > > > > abstraction to unify real-time and historical data across engines.
> > > > > *3. Columnar Streaming:* Based on *Apache Arrow *it allows database
> > > > > primitives on data streams and techniques like column pruning and
> > > predicate
> > > > > pushdown. This ensures engines read only the data they need,
> > > minimizing I/O
> > > > > and network costs.
> > > > > *4. Compute–Storage Separation:* Stream processors focus on pure
> > > > > computation while Fluss manages state and storage, with features
> like
> > > > > deduplication, partial updates, delta joins, and aggregation merge
> > > engines.
> > > > > *5. ML & AI–Ready Storage:* A unified storage layer supporting
> > > row-based,
> > > > > columnar, vector, and multi-modal data, enabling real-time feature
> > > stores
> > > > > and a centralized data repository for ML and AI systems.
> > > > > *6. Changelogs & Decision Tracking:* Built-in changelog generation
> > > > > provides an append-only history of state and decision evolution,
> > > enabling
> > > > > auditing, reproducibility, and deep system observability.
> > > > >
> > > > > Any suggestions and thoughts to revisit or frame the above are
> highly
> > > > > welcomed.
> > > > >
> > > > > Best,
> > > > > Giannis
> > > > >
> > >
>
--
Lorenzo Affetti
Senior Software Engineer @ Flink Team
Ververica <http://www.ververica.com>
Re: [DISCUSS] Fluss Messaging 2026
Posted by Jark Wu <im...@gmail.com>.
Thank you, Giannis!
+1 for a blog.
Maybe we can announce the new 0.9 version with the new title/message,
and explain what AI means in the context of Fluss with all the new
features shipped in 0.9 to highlight the AI-ready.
Best,
Jark
On Sun, 18 Jan 2026 at 00:30, Giannis Polyzos <ip...@gmail.com> wrote:
>
> Absolutely this makes sense and the more explicit we are the better 👍
>
> Streaming Storage for Real-Time Analytics & AI
>
> Maybe it’s worth also putting together a blog post to explain what AI means
> in the context of Fluss, so users can better understand the use cases.
>
> Best,
> Giannis
>
>
>
> On Sat, 17 Jan 2026 at 6:26 PM, Jark Wu <im...@gmail.com> wrote:
>
> > Thanks, Giannis!
> >
> > I’m really excited about the new messaging. It clearly showcases
> > Fluss’s new features and positions us firmly in the AI era.
> >
> > I’m just a bit torn on the title. “Streaming Storage for Real-Time
> > Data” alone doesn’t clearly differentiate us from Kafka, which also
> > fits that description. What if we keep the keyword “Analytics” to
> > sharpen our positioning? For example: “Streaming Storage for Real-Time
> > Analytics & AI”. This would maintain continuity with our existing
> > messaging, ensuring a smooth, incremental evolution that won’t
> > surprise users, while better highlighting Fluss’s unique value in
> > powering analytical workloads.
> >
> > Best,
> > Jark
> >
> > On Sat, 17 Jan 2026 at 02:44, Mehul Batra <me...@gmail.com>
> > wrote:
> > >
> > > +1
> > > Best Regards,
> > > Mehul Batra
> > >
> > > On Mon, Jan 12, 2026 at 6:57 PM Giannis Polyzos <ip...@gmail.com>
> > > wrote:
> > >
> > > > Hi everyone,
> > > >
> > > > Over the last year, Fluss has grown a lot, and I think it's a good
> > time to
> > > > update our core message on the website to reflect that.
> > > >
> > > > I would like to propose the following.
> > > >
> > > > Promote Fluss as: Streaming Storage For Real-Time Data & AI
> > > > another alternative Streaming Storage For Real-Time Data & Intelligent
> > > > Systems
> > > >
> > > > and update our 6 core capabilities:
> > > >
> > > > *1. Sub-Second Data Freshness: *Continuous ingestion and immediate
> > > > availability of data enable low-latency analytics and real-time
> > > > decision-making at scale.
> > > > *2.* *Streaming & Lakehouse Unification:* Streaming-native storage with
> > > > low-latency access on top of the lakehouse, using tables as a single
> > > > abstraction to unify real-time and historical data across engines.
> > > > *3. Columnar Streaming:* Based on *Apache Arrow *it allows database
> > > > primitives on data streams and techniques like column pruning and
> > predicate
> > > > pushdown. This ensures engines read only the data they need,
> > minimizing I/O
> > > > and network costs.
> > > > *4. Compute–Storage Separation:* Stream processors focus on pure
> > > > computation while Fluss manages state and storage, with features like
> > > > deduplication, partial updates, delta joins, and aggregation merge
> > engines.
> > > > *5. ML & AI–Ready Storage:* A unified storage layer supporting
> > row-based,
> > > > columnar, vector, and multi-modal data, enabling real-time feature
> > stores
> > > > and a centralized data repository for ML and AI systems.
> > > > *6. Changelogs & Decision Tracking:* Built-in changelog generation
> > > > provides an append-only history of state and decision evolution,
> > enabling
> > > > auditing, reproducibility, and deep system observability.
> > > >
> > > > Any suggestions and thoughts to revisit or frame the above are highly
> > > > welcomed.
> > > >
> > > > Best,
> > > > Giannis
> > > >
> >
Re: [DISCUSS] Fluss Messaging 2026
Posted by Giannis Polyzos <ip...@gmail.com>.
Absolutely this makes sense and the more explicit we are the better 👍
Streaming Storage for Real-Time Analytics & AI
Maybe it’s worth also putting together a blog post to explain what AI means
in the context of Fluss, so users can better understand the use cases.
Best,
Giannis
On Sat, 17 Jan 2026 at 6:26 PM, Jark Wu <im...@gmail.com> wrote:
> Thanks, Giannis!
>
> I’m really excited about the new messaging. It clearly showcases
> Fluss’s new features and positions us firmly in the AI era.
>
> I’m just a bit torn on the title. “Streaming Storage for Real-Time
> Data” alone doesn’t clearly differentiate us from Kafka, which also
> fits that description. What if we keep the keyword “Analytics” to
> sharpen our positioning? For example: “Streaming Storage for Real-Time
> Analytics & AI”. This would maintain continuity with our existing
> messaging, ensuring a smooth, incremental evolution that won’t
> surprise users, while better highlighting Fluss’s unique value in
> powering analytical workloads.
>
> Best,
> Jark
>
> On Sat, 17 Jan 2026 at 02:44, Mehul Batra <me...@gmail.com>
> wrote:
> >
> > +1
> > Best Regards,
> > Mehul Batra
> >
> > On Mon, Jan 12, 2026 at 6:57 PM Giannis Polyzos <ip...@gmail.com>
> > wrote:
> >
> > > Hi everyone,
> > >
> > > Over the last year, Fluss has grown a lot, and I think it's a good
> time to
> > > update our core message on the website to reflect that.
> > >
> > > I would like to propose the following.
> > >
> > > Promote Fluss as: Streaming Storage For Real-Time Data & AI
> > > another alternative Streaming Storage For Real-Time Data & Intelligent
> > > Systems
> > >
> > > and update our 6 core capabilities:
> > >
> > > *1. Sub-Second Data Freshness: *Continuous ingestion and immediate
> > > availability of data enable low-latency analytics and real-time
> > > decision-making at scale.
> > > *2.* *Streaming & Lakehouse Unification:* Streaming-native storage with
> > > low-latency access on top of the lakehouse, using tables as a single
> > > abstraction to unify real-time and historical data across engines.
> > > *3. Columnar Streaming:* Based on *Apache Arrow *it allows database
> > > primitives on data streams and techniques like column pruning and
> predicate
> > > pushdown. This ensures engines read only the data they need,
> minimizing I/O
> > > and network costs.
> > > *4. Compute–Storage Separation:* Stream processors focus on pure
> > > computation while Fluss manages state and storage, with features like
> > > deduplication, partial updates, delta joins, and aggregation merge
> engines.
> > > *5. ML & AI–Ready Storage:* A unified storage layer supporting
> row-based,
> > > columnar, vector, and multi-modal data, enabling real-time feature
> stores
> > > and a centralized data repository for ML and AI systems.
> > > *6. Changelogs & Decision Tracking:* Built-in changelog generation
> > > provides an append-only history of state and decision evolution,
> enabling
> > > auditing, reproducibility, and deep system observability.
> > >
> > > Any suggestions and thoughts to revisit or frame the above are highly
> > > welcomed.
> > >
> > > Best,
> > > Giannis
> > >
>
Re: [DISCUSS] Fluss Messaging 2026
Posted by Jark Wu <im...@gmail.com>.
Thanks, Giannis!
I’m really excited about the new messaging. It clearly showcases
Fluss’s new features and positions us firmly in the AI era.
I’m just a bit torn on the title. “Streaming Storage for Real-Time
Data” alone doesn’t clearly differentiate us from Kafka, which also
fits that description. What if we keep the keyword “Analytics” to
sharpen our positioning? For example: “Streaming Storage for Real-Time
Analytics & AI”. This would maintain continuity with our existing
messaging, ensuring a smooth, incremental evolution that won’t
surprise users, while better highlighting Fluss’s unique value in
powering analytical workloads.
Best,
Jark
On Sat, 17 Jan 2026 at 02:44, Mehul Batra <me...@gmail.com> wrote:
>
> +1
> Best Regards,
> Mehul Batra
>
> On Mon, Jan 12, 2026 at 6:57 PM Giannis Polyzos <ip...@gmail.com>
> wrote:
>
> > Hi everyone,
> >
> > Over the last year, Fluss has grown a lot, and I think it's a good time to
> > update our core message on the website to reflect that.
> >
> > I would like to propose the following.
> >
> > Promote Fluss as: Streaming Storage For Real-Time Data & AI
> > another alternative Streaming Storage For Real-Time Data & Intelligent
> > Systems
> >
> > and update our 6 core capabilities:
> >
> > *1. Sub-Second Data Freshness: *Continuous ingestion and immediate
> > availability of data enable low-latency analytics and real-time
> > decision-making at scale.
> > *2.* *Streaming & Lakehouse Unification:* Streaming-native storage with
> > low-latency access on top of the lakehouse, using tables as a single
> > abstraction to unify real-time and historical data across engines.
> > *3. Columnar Streaming:* Based on *Apache Arrow *it allows database
> > primitives on data streams and techniques like column pruning and predicate
> > pushdown. This ensures engines read only the data they need, minimizing I/O
> > and network costs.
> > *4. Compute–Storage Separation:* Stream processors focus on pure
> > computation while Fluss manages state and storage, with features like
> > deduplication, partial updates, delta joins, and aggregation merge engines.
> > *5. ML & AI–Ready Storage:* A unified storage layer supporting row-based,
> > columnar, vector, and multi-modal data, enabling real-time feature stores
> > and a centralized data repository for ML and AI systems.
> > *6. Changelogs & Decision Tracking:* Built-in changelog generation
> > provides an append-only history of state and decision evolution, enabling
> > auditing, reproducibility, and deep system observability.
> >
> > Any suggestions and thoughts to revisit or frame the above are highly
> > welcomed.
> >
> > Best,
> > Giannis
> >
Re: [DISCUSS] Fluss Messaging 2026
Posted by Mehul Batra <me...@gmail.com>.
+1
Best Regards,
Mehul Batra
On Mon, Jan 12, 2026 at 6:57 PM Giannis Polyzos <ip...@gmail.com>
wrote:
> Hi everyone,
>
> Over the last year, Fluss has grown a lot, and I think it's a good time to
> update our core message on the website to reflect that.
>
> I would like to propose the following.
>
> Promote Fluss as: Streaming Storage For Real-Time Data & AI
> another alternative Streaming Storage For Real-Time Data & Intelligent
> Systems
>
> and update our 6 core capabilities:
>
> *1. Sub-Second Data Freshness: *Continuous ingestion and immediate
> availability of data enable low-latency analytics and real-time
> decision-making at scale.
> *2.* *Streaming & Lakehouse Unification:* Streaming-native storage with
> low-latency access on top of the lakehouse, using tables as a single
> abstraction to unify real-time and historical data across engines.
> *3. Columnar Streaming:* Based on *Apache Arrow *it allows database
> primitives on data streams and techniques like column pruning and predicate
> pushdown. This ensures engines read only the data they need, minimizing I/O
> and network costs.
> *4. Compute–Storage Separation:* Stream processors focus on pure
> computation while Fluss manages state and storage, with features like
> deduplication, partial updates, delta joins, and aggregation merge engines.
> *5. ML & AI–Ready Storage:* A unified storage layer supporting row-based,
> columnar, vector, and multi-modal data, enabling real-time feature stores
> and a centralized data repository for ML and AI systems.
> *6. Changelogs & Decision Tracking:* Built-in changelog generation
> provides an append-only history of state and decision evolution, enabling
> auditing, reproducibility, and deep system observability.
>
> Any suggestions and thoughts to revisit or frame the above are highly
> welcomed.
>
> Best,
> Giannis
>