You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Martijn Visser <ma...@apache.org> on 2026/01/07 10:01:32 UTC
[DISCUSS] FLIP-561: Restructure Flink documentation
Hi everyone,
I'd like to start a discussion on FLIP-561 [1] to restructure the Apache
Flink documentation to improve discoverability and usability of them. The
Flink documentation has evolved organically over many years. While
comprehensive, users face several navigation challenges:
- The distinction between "Try Flink", "Learn Flink", and "Concepts" is
unclear (these blur the Diátaxis categories of tutorials, explanation, and
reference)
- SQL is bundled with Table API, forcing SQL-only users through
programmatic content
- Streaming concepts like Dynamic Tables are buried in Table API, though
they are general relational streaming concepts
- Python documentation duplicates the structure of both Table API and
DataStream API
- Connector documentation is fragmented by API rather than by system
This FLIP also refers to previously discussed and/or voted on FLIP-60 [2]
and FLIP-541 [3]
The FLIP contains the new proposed information architecture and a couple of
screenshots from my local changes. You can also use this live website [4]
that contains mostly the refactored menu structure, content that has been
moved to a new location and some new content (like First Steps).
I'd appreciate feedback on the proposed structure and rest of the FLIP.
Thanks,
Martijn
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
[2]
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
[3]
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
[4]
https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
Re: [DISCUSS] FLIP-561: Restructure Flink documentation
Posted by Martijn Visser <ma...@apache.org>.
Hi Zhe Wang,
Thanks for the feedback! Yes, this should be moved there too, and in the
follow-up FLIP on connectors integrated into one page. I've updated the
FLIP, thanks for the input!
Best regards,
Martijn
On Tue, Jan 13, 2026 at 5:05 AM Zhe Wang <ja...@gmail.com> wrote:
> Hi Martijn,
>
> Big +1, the new structure looks more clear.
> I have a small question,
> "User-defined Sources & Sinks" seems to define the source & sink for
> connectors, shall we move it in Connectors?
>
> Best regards,
> Zhe Wang
>
> Martijn Visser <ma...@apache.org> 于2026年1月12日周一 21:18写道：
>
> > I've included in the FLIP that the docs contribution guide should be
> > updated, so that it's clear what should go into Learn Flink vs Concepts.
> >
> > I'm planning to open a vote on this later this week, so that there's
> still
> > some time for other developers or users to chip-in if they want to.
> >
> > Best regards,
> >
> > Martijn
> >
> > On Fri, Jan 9, 2026 at 2:14 AM Xintong Song <to...@gmail.com>
> wrote:
> >
> > > > - Learn Flink: Learning-oriented tutorials where you follow along and
> > > build
> > > > something. The focus is on either snippet-based or guided, hands-on
> > > > learning paths.
> > > > - Concepts: Explanation-oriented content about foundational ideas and
> > > > mental models. The focus is on understanding how things work.
> > > >
> > > > The boundary isn't perfectly clean in the current proposal - for
> > example,
> > > > "Streaming Analytics" in Learn Flink has conceptual elements. The
> > guiding
> > > > principle is: if content is primarily "follow these steps to learn
> X",
> > it
> > > > belongs in Learn Flink. If it's primarily "here's how X works under
> the
> > > > hood", it belongs in Concepts.
> > > >
> > > > We could refine this further, but the core distinction is (either
> > snippet
> > > > or hands-on) learning vs. theoretical understanding.
> > >
> > >
> > > Thanks for the explanation. Sounds good to me.
> > >
> > >
> > > Best,
> > >
> > > Xintong
> > >
> > >
> > >
> > > On Fri, Jan 9, 2026 at 8:04 AM David Anderson <da...@apache.org>
> > > wrote:
> > >
> > > > Martijn, thanks for taking this on. This looks great.
> > > >
> > > > I think it's smart to limit the scope to something manageable and
> > > concrete
> > > > (as you've done).
> > > >
> > > > Maybe we can add a couple more bullet points to the Future Work
> > section.
> > > A
> > > > reorg of the Deployment and Operations sections could be listed
> there,
> > > > along with a reorg of some of the explanatory and learning content to
> > > > better follow the diataxis model. I have some ideas, and look forward
> > to
> > > > helping out there in the future.
> > > >
> > > > David
> > > >
> > > > On Thu, Jan 8, 2026 at 7:33 AM Ryan van Huuksloot via dev <
> > > > dev@flink.apache.org> wrote:
> > > >
> > > > > >Absolutely. I intentionally scoped Connectors restructuring as a
> > > > separate
> > > > > follow-up FLIP (noted in the Future Work section), because it
> > requires
> > > a
> > > > > bit more reorganization given the docs are in the external
> connector
> > > > repos
> > > > > too and I have some additional ideas too. The vision is to organize
> > by
> > > > > system (Kafka, JDBC, etc.) with tabs showing SQL/Table
> API/DataStream
> > > > usage
> > > > > where applicable, plus a dedicated Formats subsection with proper
> > > > context.
> > > > >
> > > > > Perfect, I missed that. Sounds like a good plan.
> > > > > Thanks for leading this effort!
> > > > >
> > > > > Ryan van Huuksloot
> > > > > Staff Engineer, Infrastructure | Streaming Platform
> > > > > [image: Shopify]
> > > > > <
> > >
> https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Jan 8, 2026 at 10:14 AM Martijn Visser <
> > > martijnvisser@apache.org
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > Thanks for the feedback! Let me go through them one-by-one:
> > > > > >
> > > > > > > I like the way that the data types are currently documented
> > showing
> > > > how
> > > > > > each are implemented in SQL and the APIs, this is useful for
> Flink
> > > > > > contributors.Is there a way to keep this view in the structure
> > > > somewhere
> > > > > > maybe in a contributor section?
> > > > > >
> > > > > > The Data Types content (including the SQL/Java/Scala type
> mappings)
> > > is
> > > > > > moving to SQL Reference, not being removed. The
> > contributor-friendly
> > > > view
> > > > > > will be preserved. We could add add a cross-link from Table API
> > > > > > documentation to ensure discoverability from both entry points.
> > > > > >
> > > > > > > You mention the Diátaxis documentation framework - I wondered
> > > about
> > > > > the
> > > > > > how-to guides (task-oriented), and the reference material. Do we
> > > think
> > > > > > there is no how to guides?
> > > > > >
> > > > > > That's correct indeed. The current Flink docs are primarily
> > tutorials
> > > > and
> > > > > > reference material. True task-oriented how-to guides are largely
> > > > missing
> > > > > or
> > > > > > scattered across blog posts. Since this is mostly a restructuring
> > of
> > > > > > existing content, it won't immediately solve that problem. I'd
> > rather
> > > > > first
> > > > > > focus on this effort and the others that are currently out of
> scope
> > > of
> > > > > the
> > > > > > FLIP, and then work on this one.
> > > > > >
> > > > > > > In the past I have used snippets for documentation that is
> > > > duplicated,
> > > > > do
> > > > > > we plan to use something like that for the duplicated content in
> > SQL
> > > > and
> > > > > > table APIs? So we only have information authored once?
> > > > > >
> > > > > > Can you give an example of where you used snippets? Hugo does
> > support
> > > > > > content reuse via shortcodes and partials, so when needed we can
> > > > leverage
> > > > > > it. For most cases, cross-linking may be simpler to maintain.
> > > > > >
> > > > > > > One additional thing that may be worth evaluating changing is
> the
> > > > > > Connectors section.
> > > > > >
> > > > > > Absolutely. I intentionally scoped Connectors restructuring as a
> > > > separate
> > > > > > follow-up FLIP (noted in the Future Work section), because it
> > > requires
> > > > a
> > > > > > bit more reorganization given the docs are in the external
> > connector
> > > > > repos
> > > > > > too and I have some additional ideas too. The vision is to
> organize
> > > by
> > > > > > system (Kafka, JDBC, etc.) with tabs showing SQL/Table
> > API/DataStream
> > > > > usage
> > > > > > where applicable, plus a dedicated Formats subsection with proper
> > > > > context.
> > > > > >
> > > > > > > We might need a "Built-in Functions" section under "Flink APIs
> -
> > > > Table
> > > > > > API - Functions" that redirects to "Flink SQL - Functions -
> > Built-in
> > > > > > Functions". This would help users navigate more easily.
> > > > > >
> > > > > > Good suggestion. I'll add a link or redirect under "Flink APIs →
> > > Table
> > > > > API
> > > > > > → Functions" pointing to "Flink SQL → Functions → Built-in
> > > Functions".
> > > > > This
> > > > > > follows the same pattern as Data Types, by maintaining content in
> > one
> > > > > > location, cross-link from related sections to ensure
> > discoverability
> > > > > > without duplication.
> > > > > >
> > > > > > > One quick question, what are the different purposes of Learn
> > Flink
> > > > and
> > > > > > Concepts. It seems Learn Flink also contains some api-agnostic
> > > > concepts,
> > > > > as
> > > > > > well as introductions to different api sets. What is the
> boundary?
> > > > > >
> > > > > > Good question. The intended distinction is like this:
> > > > > >
> > > > > > - Learn Flink: Learning-oriented tutorials where you follow along
> > and
> > > > > build
> > > > > > something. The focus is on either snippet-based or guided,
> hands-on
> > > > > > learning paths.
> > > > > > - Concepts: Explanation-oriented content about foundational ideas
> > and
> > > > > > mental models. The focus is on understanding how things work.
> > > > > >
> > > > > > The boundary isn't perfectly clean in the current proposal - for
> > > > example,
> > > > > > "Streaming Analytics" in Learn Flink has conceptual elements. The
> > > > guiding
> > > > > > principle is: if content is primarily "follow these steps to
> learn
> > > X",
> > > > it
> > > > > > belongs in Learn Flink. If it's primarily "here's how X works
> under
> > > the
> > > > > > hood", it belongs in Concepts.
> > > > > >
> > > > > > We could refine this further, but the core distinction is (either
> > > > snippet
> > > > > > or hands-on) learning vs. theoretical understanding.
> > > > > >
> > > > > > Best regards,
> > > > > >
> > > > > > Martijn
> > > > > >
> > > > > > On Thu, Jan 8, 2026 at 4:58 AM Xintong Song <
> tonysong820@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Big +1! Thanks for putting this together, Martijn. This sounds
> > > great.
> > > > > > >
> > > > > > > One quick question, what are the different purposes of Learn
> > Flink
> > > > and
> > > > > > > Concepts. It seems Learn Flink also contains some api-agnostic
> > > > > concepts,
> > > > > > as
> > > > > > > well as introductions to different api sets. What is the
> > boundary?
> > > > > > >
> > > > > > > Best,
> > > > > > >
> > > > > > > Xintong
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Jan 8, 2026 at 10:43 AM dylanhz <dy...@163.com>
> wrote:
> > > > > > >
> > > > > > > > Hi Martin,
> > > > > > > >
> > > > > > > >
> > > > > > > > This proposal looks great to me. I think the new structure is
> > > well
> > > > > > > > organized and easy to understand.
> > > > > > > >
> > > > > > > >
> > > > > > > > I have only one small suggestion:
> > > > > > > > We might need a "Built-in Functions" section under "Flink
> APIs
> > -
> > > > > Table
> > > > > > > API
> > > > > > > > - Functions" that redirects to "Flink SQL - Functions -
> > Built-in
> > > > > > > > Functions". This would help users navigate more easily.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > > Best regards,
> > > > > > > > dylanhz
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > At 2026-01-07 18:01:32, "Martijn Visser" <
> > > martijnvisser@apache.org
> > > > >
> > > > > > > wrote:
> > > > > > > > >Hi everyone,
> > > > > > > > >
> > > > > > > > >I'd like to start a discussion on FLIP-561 [1] to
> restructure
> > > the
> > > > > > Apache
> > > > > > > > >Flink documentation to improve discoverability and usability
> > of
> > > > > them.
> > > > > > > The
> > > > > > > > >Flink documentation has evolved organically over many years.
> > > While
> > > > > > > > >comprehensive, users face several navigation challenges:
> > > > > > > > >
> > > > > > > > >- The distinction between "Try Flink", "Learn Flink", and
> > > > "Concepts"
> > > > > > is
> > > > > > > > >unclear (these blur the Diátaxis categories of tutorials,
> > > > > explanation,
> > > > > > > and
> > > > > > > > >reference)
> > > > > > > > >- SQL is bundled with Table API, forcing SQL-only users
> > through
> > > > > > > > >programmatic content
> > > > > > > > >- Streaming concepts like Dynamic Tables are buried in Table
> > > API,
> > > > > > though
> > > > > > > > >they are general relational streaming concepts
> > > > > > > > >- Python documentation duplicates the structure of both
> Table
> > > API
> > > > > and
> > > > > > > > >DataStream API
> > > > > > > > >- Connector documentation is fragmented by API rather than
> by
> > > > system
> > > > > > > > >
> > > > > > > > >This FLIP also refers to previously discussed and/or voted
> on
> > > > > FLIP-60
> > > > > > > [2]
> > > > > > > > >and FLIP-541 [3]
> > > > > > > > >
> > > > > > > > >The FLIP contains the new proposed information architecture
> > and
> > > a
> > > > > > couple
> > > > > > > > of
> > > > > > > > >screenshots from my local changes. You can also use this
> live
> > > > > website
> > > > > > > [4]
> > > > > > > > >that contains mostly the refactored menu structure, content
> > that
> > > > has
> > > > > > > been
> > > > > > > > >moved to a new location and some new content (like First
> > Steps).
> > > > > > > > >
> > > > > > > > >I'd appreciate feedback on the proposed structure and rest
> of
> > > the
> > > > > > FLIP.
> > > > > > > > >
> > > > > > > > >Thanks,
> > > > > > > > >
> > > > > > > > >Martijn
> > > > > > > > >
> > > > > > > > >[1]
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
> > > > > > > > >[2]
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
> > > > > > > > >[3]
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
> > > > > > > > >[4]
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] FLIP-561: Restructure Flink documentation
Posted by Zhe Wang <ja...@gmail.com>.
Hi Martijn,
Big +1, the new structure looks more clear.
I have a small question,
"User-defined Sources & Sinks" seems to define the source & sink for
connectors, shall we move it in Connectors?
Best regards,
Zhe Wang
Martijn Visser <ma...@apache.org> 于2026年1月12日周一 21:18写道：
> I've included in the FLIP that the docs contribution guide should be
> updated, so that it's clear what should go into Learn Flink vs Concepts.
>
> I'm planning to open a vote on this later this week, so that there's still
> some time for other developers or users to chip-in if they want to.
>
> Best regards,
>
> Martijn
>
> On Fri, Jan 9, 2026 at 2:14 AM Xintong Song <to...@gmail.com> wrote:
>
> > > - Learn Flink: Learning-oriented tutorials where you follow along and
> > build
> > > something. The focus is on either snippet-based or guided, hands-on
> > > learning paths.
> > > - Concepts: Explanation-oriented content about foundational ideas and
> > > mental models. The focus is on understanding how things work.
> > >
> > > The boundary isn't perfectly clean in the current proposal - for
> example,
> > > "Streaming Analytics" in Learn Flink has conceptual elements. The
> guiding
> > > principle is: if content is primarily "follow these steps to learn X",
> it
> > > belongs in Learn Flink. If it's primarily "here's how X works under the
> > > hood", it belongs in Concepts.
> > >
> > > We could refine this further, but the core distinction is (either
> snippet
> > > or hands-on) learning vs. theoretical understanding.
> >
> >
> > Thanks for the explanation. Sounds good to me.
> >
> >
> > Best,
> >
> > Xintong
> >
> >
> >
> > On Fri, Jan 9, 2026 at 8:04 AM David Anderson <da...@apache.org>
> > wrote:
> >
> > > Martijn, thanks for taking this on. This looks great.
> > >
> > > I think it's smart to limit the scope to something manageable and
> > concrete
> > > (as you've done).
> > >
> > > Maybe we can add a couple more bullet points to the Future Work
> section.
> > A
> > > reorg of the Deployment and Operations sections could be listed there,
> > > along with a reorg of some of the explanatory and learning content to
> > > better follow the diataxis model. I have some ideas, and look forward
> to
> > > helping out there in the future.
> > >
> > > David
> > >
> > > On Thu, Jan 8, 2026 at 7:33 AM Ryan van Huuksloot via dev <
> > > dev@flink.apache.org> wrote:
> > >
> > > > >Absolutely. I intentionally scoped Connectors restructuring as a
> > > separate
> > > > follow-up FLIP (noted in the Future Work section), because it
> requires
> > a
> > > > bit more reorganization given the docs are in the external connector
> > > repos
> > > > too and I have some additional ideas too. The vision is to organize
> by
> > > > system (Kafka, JDBC, etc.) with tabs showing SQL/Table API/DataStream
> > > usage
> > > > where applicable, plus a dedicated Formats subsection with proper
> > > context.
> > > >
> > > > Perfect, I missed that. Sounds like a good plan.
> > > > Thanks for leading this effort!
> > > >
> > > > Ryan van Huuksloot
> > > > Staff Engineer, Infrastructure | Streaming Platform
> > > > [image: Shopify]
> > > > <
> > https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email
> > > >
> > > >
> > > >
> > > > On Thu, Jan 8, 2026 at 10:14 AM Martijn Visser <
> > martijnvisser@apache.org
> > > >
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > Thanks for the feedback! Let me go through them one-by-one:
> > > > >
> > > > > > I like the way that the data types are currently documented
> showing
> > > how
> > > > > each are implemented in SQL and the APIs, this is useful for Flink
> > > > > contributors.Is there a way to keep this view in the structure
> > > somewhere
> > > > > maybe in a contributor section?
> > > > >
> > > > > The Data Types content (including the SQL/Java/Scala type mappings)
> > is
> > > > > moving to SQL Reference, not being removed. The
> contributor-friendly
> > > view
> > > > > will be preserved. We could add add a cross-link from Table API
> > > > > documentation to ensure discoverability from both entry points.
> > > > >
> > > > > > You mention the Diátaxis documentation framework - I wondered
> > about
> > > > the
> > > > > how-to guides (task-oriented), and the reference material. Do we
> > think
> > > > > there is no how to guides?
> > > > >
> > > > > That's correct indeed. The current Flink docs are primarily
> tutorials
> > > and
> > > > > reference material. True task-oriented how-to guides are largely
> > > missing
> > > > or
> > > > > scattered across blog posts. Since this is mostly a restructuring
> of
> > > > > existing content, it won't immediately solve that problem. I'd
> rather
> > > > first
> > > > > focus on this effort and the others that are currently out of scope
> > of
> > > > the
> > > > > FLIP, and then work on this one.
> > > > >
> > > > > > In the past I have used snippets for documentation that is
> > > duplicated,
> > > > do
> > > > > we plan to use something like that for the duplicated content in
> SQL
> > > and
> > > > > table APIs? So we only have information authored once?
> > > > >
> > > > > Can you give an example of where you used snippets? Hugo does
> support
> > > > > content reuse via shortcodes and partials, so when needed we can
> > > leverage
> > > > > it. For most cases, cross-linking may be simpler to maintain.
> > > > >
> > > > > > One additional thing that may be worth evaluating changing is the
> > > > > Connectors section.
> > > > >
> > > > > Absolutely. I intentionally scoped Connectors restructuring as a
> > > separate
> > > > > follow-up FLIP (noted in the Future Work section), because it
> > requires
> > > a
> > > > > bit more reorganization given the docs are in the external
> connector
> > > > repos
> > > > > too and I have some additional ideas too. The vision is to organize
> > by
> > > > > system (Kafka, JDBC, etc.) with tabs showing SQL/Table
> API/DataStream
> > > > usage
> > > > > where applicable, plus a dedicated Formats subsection with proper
> > > > context.
> > > > >
> > > > > > We might need a "Built-in Functions" section under "Flink APIs -
> > > Table
> > > > > API - Functions" that redirects to "Flink SQL - Functions -
> Built-in
> > > > > Functions". This would help users navigate more easily.
> > > > >
> > > > > Good suggestion. I'll add a link or redirect under "Flink APIs →
> > Table
> > > > API
> > > > > → Functions" pointing to "Flink SQL → Functions → Built-in
> > Functions".
> > > > This
> > > > > follows the same pattern as Data Types, by maintaining content in
> one
> > > > > location, cross-link from related sections to ensure
> discoverability
> > > > > without duplication.
> > > > >
> > > > > > One quick question, what are the different purposes of Learn
> Flink
> > > and
> > > > > Concepts. It seems Learn Flink also contains some api-agnostic
> > > concepts,
> > > > as
> > > > > well as introductions to different api sets. What is the boundary?
> > > > >
> > > > > Good question. The intended distinction is like this:
> > > > >
> > > > > - Learn Flink: Learning-oriented tutorials where you follow along
> and
> > > > build
> > > > > something. The focus is on either snippet-based or guided, hands-on
> > > > > learning paths.
> > > > > - Concepts: Explanation-oriented content about foundational ideas
> and
> > > > > mental models. The focus is on understanding how things work.
> > > > >
> > > > > The boundary isn't perfectly clean in the current proposal - for
> > > example,
> > > > > "Streaming Analytics" in Learn Flink has conceptual elements. The
> > > guiding
> > > > > principle is: if content is primarily "follow these steps to learn
> > X",
> > > it
> > > > > belongs in Learn Flink. If it's primarily "here's how X works under
> > the
> > > > > hood", it belongs in Concepts.
> > > > >
> > > > > We could refine this further, but the core distinction is (either
> > > snippet
> > > > > or hands-on) learning vs. theoretical understanding.
> > > > >
> > > > > Best regards,
> > > > >
> > > > > Martijn
> > > > >
> > > > > On Thu, Jan 8, 2026 at 4:58 AM Xintong Song <tonysong820@gmail.com
> >
> > > > wrote:
> > > > >
> > > > > > Big +1! Thanks for putting this together, Martijn. This sounds
> > great.
> > > > > >
> > > > > > One quick question, what are the different purposes of Learn
> Flink
> > > and
> > > > > > Concepts. It seems Learn Flink also contains some api-agnostic
> > > > concepts,
> > > > > as
> > > > > > well as introductions to different api sets. What is the
> boundary?
> > > > > >
> > > > > > Best,
> > > > > >
> > > > > > Xintong
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Jan 8, 2026 at 10:43 AM dylanhz <dy...@163.com> wrote:
> > > > > >
> > > > > > > Hi Martin,
> > > > > > >
> > > > > > >
> > > > > > > This proposal looks great to me. I think the new structure is
> > well
> > > > > > > organized and easy to understand.
> > > > > > >
> > > > > > >
> > > > > > > I have only one small suggestion:
> > > > > > > We might need a "Built-in Functions" section under "Flink APIs
> -
> > > > Table
> > > > > > API
> > > > > > > - Functions" that redirects to "Flink SQL - Functions -
> Built-in
> > > > > > > Functions". This would help users navigate more easily.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > > Best regards,
> > > > > > > dylanhz
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > At 2026-01-07 18:01:32, "Martijn Visser" <
> > martijnvisser@apache.org
> > > >
> > > > > > wrote:
> > > > > > > >Hi everyone,
> > > > > > > >
> > > > > > > >I'd like to start a discussion on FLIP-561 [1] to restructure
> > the
> > > > > Apache
> > > > > > > >Flink documentation to improve discoverability and usability
> of
> > > > them.
> > > > > > The
> > > > > > > >Flink documentation has evolved organically over many years.
> > While
> > > > > > > >comprehensive, users face several navigation challenges:
> > > > > > > >
> > > > > > > >- The distinction between "Try Flink", "Learn Flink", and
> > > "Concepts"
> > > > > is
> > > > > > > >unclear (these blur the Diátaxis categories of tutorials,
> > > > explanation,
> > > > > > and
> > > > > > > >reference)
> > > > > > > >- SQL is bundled with Table API, forcing SQL-only users
> through
> > > > > > > >programmatic content
> > > > > > > >- Streaming concepts like Dynamic Tables are buried in Table
> > API,
> > > > > though
> > > > > > > >they are general relational streaming concepts
> > > > > > > >- Python documentation duplicates the structure of both Table
> > API
> > > > and
> > > > > > > >DataStream API
> > > > > > > >- Connector documentation is fragmented by API rather than by
> > > system
> > > > > > > >
> > > > > > > >This FLIP also refers to previously discussed and/or voted on
> > > > FLIP-60
> > > > > > [2]
> > > > > > > >and FLIP-541 [3]
> > > > > > > >
> > > > > > > >The FLIP contains the new proposed information architecture
> and
> > a
> > > > > couple
> > > > > > > of
> > > > > > > >screenshots from my local changes. You can also use this live
> > > > website
> > > > > > [4]
> > > > > > > >that contains mostly the refactored menu structure, content
> that
> > > has
> > > > > > been
> > > > > > > >moved to a new location and some new content (like First
> Steps).
> > > > > > > >
> > > > > > > >I'd appreciate feedback on the proposed structure and rest of
> > the
> > > > > FLIP.
> > > > > > > >
> > > > > > > >Thanks,
> > > > > > > >
> > > > > > > >Martijn
> > > > > > > >
> > > > > > > >[1]
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
> > > > > > > >[2]
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
> > > > > > > >[3]
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
> > > > > > > >[4]
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] FLIP-561: Restructure Flink documentation
Posted by Martijn Visser <ma...@apache.org>.
I've included in the FLIP that the docs contribution guide should be
updated, so that it's clear what should go into Learn Flink vs Concepts.
I'm planning to open a vote on this later this week, so that there's still
some time for other developers or users to chip-in if they want to.
Best regards,
Martijn
On Fri, Jan 9, 2026 at 2:14 AM Xintong Song <to...@gmail.com> wrote:
> > - Learn Flink: Learning-oriented tutorials where you follow along and
> build
> > something. The focus is on either snippet-based or guided, hands-on
> > learning paths.
> > - Concepts: Explanation-oriented content about foundational ideas and
> > mental models. The focus is on understanding how things work.
> >
> > The boundary isn't perfectly clean in the current proposal - for example,
> > "Streaming Analytics" in Learn Flink has conceptual elements. The guiding
> > principle is: if content is primarily "follow these steps to learn X", it
> > belongs in Learn Flink. If it's primarily "here's how X works under the
> > hood", it belongs in Concepts.
> >
> > We could refine this further, but the core distinction is (either snippet
> > or hands-on) learning vs. theoretical understanding.
>
>
> Thanks for the explanation. Sounds good to me.
>
>
> Best,
>
> Xintong
>
>
>
> On Fri, Jan 9, 2026 at 8:04 AM David Anderson <da...@apache.org>
> wrote:
>
> > Martijn, thanks for taking this on. This looks great.
> >
> > I think it's smart to limit the scope to something manageable and
> concrete
> > (as you've done).
> >
> > Maybe we can add a couple more bullet points to the Future Work section.
> A
> > reorg of the Deployment and Operations sections could be listed there,
> > along with a reorg of some of the explanatory and learning content to
> > better follow the diataxis model. I have some ideas, and look forward to
> > helping out there in the future.
> >
> > David
> >
> > On Thu, Jan 8, 2026 at 7:33 AM Ryan van Huuksloot via dev <
> > dev@flink.apache.org> wrote:
> >
> > > >Absolutely. I intentionally scoped Connectors restructuring as a
> > separate
> > > follow-up FLIP (noted in the Future Work section), because it requires
> a
> > > bit more reorganization given the docs are in the external connector
> > repos
> > > too and I have some additional ideas too. The vision is to organize by
> > > system (Kafka, JDBC, etc.) with tabs showing SQL/Table API/DataStream
> > usage
> > > where applicable, plus a dedicated Formats subsection with proper
> > context.
> > >
> > > Perfect, I missed that. Sounds like a good plan.
> > > Thanks for leading this effort!
> > >
> > > Ryan van Huuksloot
> > > Staff Engineer, Infrastructure | Streaming Platform
> > > [image: Shopify]
> > > <
> https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email
> > >
> > >
> > >
> > > On Thu, Jan 8, 2026 at 10:14 AM Martijn Visser <
> martijnvisser@apache.org
> > >
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > Thanks for the feedback! Let me go through them one-by-one:
> > > >
> > > > > I like the way that the data types are currently documented showing
> > how
> > > > each are implemented in SQL and the APIs, this is useful for Flink
> > > > contributors.Is there a way to keep this view in the structure
> > somewhere
> > > > maybe in a contributor section?
> > > >
> > > > The Data Types content (including the SQL/Java/Scala type mappings)
> is
> > > > moving to SQL Reference, not being removed. The contributor-friendly
> > view
> > > > will be preserved. We could add add a cross-link from Table API
> > > > documentation to ensure discoverability from both entry points.
> > > >
> > > > > You mention the Diátaxis documentation framework - I wondered
> about
> > > the
> > > > how-to guides (task-oriented), and the reference material. Do we
> think
> > > > there is no how to guides?
> > > >
> > > > That's correct indeed. The current Flink docs are primarily tutorials
> > and
> > > > reference material. True task-oriented how-to guides are largely
> > missing
> > > or
> > > > scattered across blog posts. Since this is mostly a restructuring of
> > > > existing content, it won't immediately solve that problem. I'd rather
> > > first
> > > > focus on this effort and the others that are currently out of scope
> of
> > > the
> > > > FLIP, and then work on this one.
> > > >
> > > > > In the past I have used snippets for documentation that is
> > duplicated,
> > > do
> > > > we plan to use something like that for the duplicated content in SQL
> > and
> > > > table APIs? So we only have information authored once?
> > > >
> > > > Can you give an example of where you used snippets? Hugo does support
> > > > content reuse via shortcodes and partials, so when needed we can
> > leverage
> > > > it. For most cases, cross-linking may be simpler to maintain.
> > > >
> > > > > One additional thing that may be worth evaluating changing is the
> > > > Connectors section.
> > > >
> > > > Absolutely. I intentionally scoped Connectors restructuring as a
> > separate
> > > > follow-up FLIP (noted in the Future Work section), because it
> requires
> > a
> > > > bit more reorganization given the docs are in the external connector
> > > repos
> > > > too and I have some additional ideas too. The vision is to organize
> by
> > > > system (Kafka, JDBC, etc.) with tabs showing SQL/Table API/DataStream
> > > usage
> > > > where applicable, plus a dedicated Formats subsection with proper
> > > context.
> > > >
> > > > > We might need a "Built-in Functions" section under "Flink APIs -
> > Table
> > > > API - Functions" that redirects to "Flink SQL - Functions - Built-in
> > > > Functions". This would help users navigate more easily.
> > > >
> > > > Good suggestion. I'll add a link or redirect under "Flink APIs →
> Table
> > > API
> > > > → Functions" pointing to "Flink SQL → Functions → Built-in
> Functions".
> > > This
> > > > follows the same pattern as Data Types, by maintaining content in one
> > > > location, cross-link from related sections to ensure discoverability
> > > > without duplication.
> > > >
> > > > > One quick question, what are the different purposes of Learn Flink
> > and
> > > > Concepts. It seems Learn Flink also contains some api-agnostic
> > concepts,
> > > as
> > > > well as introductions to different api sets. What is the boundary?
> > > >
> > > > Good question. The intended distinction is like this:
> > > >
> > > > - Learn Flink: Learning-oriented tutorials where you follow along and
> > > build
> > > > something. The focus is on either snippet-based or guided, hands-on
> > > > learning paths.
> > > > - Concepts: Explanation-oriented content about foundational ideas and
> > > > mental models. The focus is on understanding how things work.
> > > >
> > > > The boundary isn't perfectly clean in the current proposal - for
> > example,
> > > > "Streaming Analytics" in Learn Flink has conceptual elements. The
> > guiding
> > > > principle is: if content is primarily "follow these steps to learn
> X",
> > it
> > > > belongs in Learn Flink. If it's primarily "here's how X works under
> the
> > > > hood", it belongs in Concepts.
> > > >
> > > > We could refine this further, but the core distinction is (either
> > snippet
> > > > or hands-on) learning vs. theoretical understanding.
> > > >
> > > > Best regards,
> > > >
> > > > Martijn
> > > >
> > > > On Thu, Jan 8, 2026 at 4:58 AM Xintong Song <to...@gmail.com>
> > > wrote:
> > > >
> > > > > Big +1! Thanks for putting this together, Martijn. This sounds
> great.
> > > > >
> > > > > One quick question, what are the different purposes of Learn Flink
> > and
> > > > > Concepts. It seems Learn Flink also contains some api-agnostic
> > > concepts,
> > > > as
> > > > > well as introductions to different api sets. What is the boundary?
> > > > >
> > > > > Best,
> > > > >
> > > > > Xintong
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Jan 8, 2026 at 10:43 AM dylanhz <dy...@163.com> wrote:
> > > > >
> > > > > > Hi Martin,
> > > > > >
> > > > > >
> > > > > > This proposal looks great to me. I think the new structure is
> well
> > > > > > organized and easy to understand.
> > > > > >
> > > > > >
> > > > > > I have only one small suggestion:
> > > > > > We might need a "Built-in Functions" section under "Flink APIs -
> > > Table
> > > > > API
> > > > > > - Functions" that redirects to "Flink SQL - Functions - Built-in
> > > > > > Functions". This would help users navigate more easily.
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Best regards,
> > > > > > dylanhz
> > > > > >
> > > > > >
> > > > > >
> > > > > > At 2026-01-07 18:01:32, "Martijn Visser" <
> martijnvisser@apache.org
> > >
> > > > > wrote:
> > > > > > >Hi everyone,
> > > > > > >
> > > > > > >I'd like to start a discussion on FLIP-561 [1] to restructure
> the
> > > > Apache
> > > > > > >Flink documentation to improve discoverability and usability of
> > > them.
> > > > > The
> > > > > > >Flink documentation has evolved organically over many years.
> While
> > > > > > >comprehensive, users face several navigation challenges:
> > > > > > >
> > > > > > >- The distinction between "Try Flink", "Learn Flink", and
> > "Concepts"
> > > > is
> > > > > > >unclear (these blur the Diátaxis categories of tutorials,
> > > explanation,
> > > > > and
> > > > > > >reference)
> > > > > > >- SQL is bundled with Table API, forcing SQL-only users through
> > > > > > >programmatic content
> > > > > > >- Streaming concepts like Dynamic Tables are buried in Table
> API,
> > > > though
> > > > > > >they are general relational streaming concepts
> > > > > > >- Python documentation duplicates the structure of both Table
> API
> > > and
> > > > > > >DataStream API
> > > > > > >- Connector documentation is fragmented by API rather than by
> > system
> > > > > > >
> > > > > > >This FLIP also refers to previously discussed and/or voted on
> > > FLIP-60
> > > > > [2]
> > > > > > >and FLIP-541 [3]
> > > > > > >
> > > > > > >The FLIP contains the new proposed information architecture and
> a
> > > > couple
> > > > > > of
> > > > > > >screenshots from my local changes. You can also use this live
> > > website
> > > > > [4]
> > > > > > >that contains mostly the refactored menu structure, content that
> > has
> > > > > been
> > > > > > >moved to a new location and some new content (like First Steps).
> > > > > > >
> > > > > > >I'd appreciate feedback on the proposed structure and rest of
> the
> > > > FLIP.
> > > > > > >
> > > > > > >Thanks,
> > > > > > >
> > > > > > >Martijn
> > > > > > >
> > > > > > >[1]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
> > > > > > >[2]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
> > > > > > >[3]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
> > > > > > >[4]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] FLIP-561: Restructure Flink documentation
Posted by Martijn Visser <ma...@apache.org>.
I've included in the FLIP that the docs contribution guide should be
updated, so that it's clear what should go into Learn Flink vs Concepts.
I'm planning to open a vote on this later this week, so that there's still
some time for other developers or users to chip-in if they want to.
Best regards,
Martijn
On Fri, Jan 9, 2026 at 2:14 AM Xintong Song <to...@gmail.com> wrote:
> > - Learn Flink: Learning-oriented tutorials where you follow along and
> build
> > something. The focus is on either snippet-based or guided, hands-on
> > learning paths.
> > - Concepts: Explanation-oriented content about foundational ideas and
> > mental models. The focus is on understanding how things work.
> >
> > The boundary isn't perfectly clean in the current proposal - for example,
> > "Streaming Analytics" in Learn Flink has conceptual elements. The guiding
> > principle is: if content is primarily "follow these steps to learn X", it
> > belongs in Learn Flink. If it's primarily "here's how X works under the
> > hood", it belongs in Concepts.
> >
> > We could refine this further, but the core distinction is (either snippet
> > or hands-on) learning vs. theoretical understanding.
>
>
> Thanks for the explanation. Sounds good to me.
>
>
> Best,
>
> Xintong
>
>
>
> On Fri, Jan 9, 2026 at 8:04 AM David Anderson <da...@apache.org>
> wrote:
>
> > Martijn, thanks for taking this on. This looks great.
> >
> > I think it's smart to limit the scope to something manageable and
> concrete
> > (as you've done).
> >
> > Maybe we can add a couple more bullet points to the Future Work section.
> A
> > reorg of the Deployment and Operations sections could be listed there,
> > along with a reorg of some of the explanatory and learning content to
> > better follow the diataxis model. I have some ideas, and look forward to
> > helping out there in the future.
> >
> > David
> >
> > On Thu, Jan 8, 2026 at 7:33 AM Ryan van Huuksloot via dev <
> > dev@flink.apache.org> wrote:
> >
> > > >Absolutely. I intentionally scoped Connectors restructuring as a
> > separate
> > > follow-up FLIP (noted in the Future Work section), because it requires
> a
> > > bit more reorganization given the docs are in the external connector
> > repos
> > > too and I have some additional ideas too. The vision is to organize by
> > > system (Kafka, JDBC, etc.) with tabs showing SQL/Table API/DataStream
> > usage
> > > where applicable, plus a dedicated Formats subsection with proper
> > context.
> > >
> > > Perfect, I missed that. Sounds like a good plan.
> > > Thanks for leading this effort!
> > >
> > > Ryan van Huuksloot
> > > Staff Engineer, Infrastructure | Streaming Platform
> > > [image: Shopify]
> > > <
> https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email
> > >
> > >
> > >
> > > On Thu, Jan 8, 2026 at 10:14 AM Martijn Visser <
> martijnvisser@apache.org
> > >
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > Thanks for the feedback! Let me go through them one-by-one:
> > > >
> > > > > I like the way that the data types are currently documented showing
> > how
> > > > each are implemented in SQL and the APIs, this is useful for Flink
> > > > contributors.Is there a way to keep this view in the structure
> > somewhere
> > > > maybe in a contributor section?
> > > >
> > > > The Data Types content (including the SQL/Java/Scala type mappings)
> is
> > > > moving to SQL Reference, not being removed. The contributor-friendly
> > view
> > > > will be preserved. We could add add a cross-link from Table API
> > > > documentation to ensure discoverability from both entry points.
> > > >
> > > > > You mention the Diátaxis documentation framework - I wondered
> about
> > > the
> > > > how-to guides (task-oriented), and the reference material. Do we
> think
> > > > there is no how to guides?
> > > >
> > > > That's correct indeed. The current Flink docs are primarily tutorials
> > and
> > > > reference material. True task-oriented how-to guides are largely
> > missing
> > > or
> > > > scattered across blog posts. Since this is mostly a restructuring of
> > > > existing content, it won't immediately solve that problem. I'd rather
> > > first
> > > > focus on this effort and the others that are currently out of scope
> of
> > > the
> > > > FLIP, and then work on this one.
> > > >
> > > > > In the past I have used snippets for documentation that is
> > duplicated,
> > > do
> > > > we plan to use something like that for the duplicated content in SQL
> > and
> > > > table APIs? So we only have information authored once?
> > > >
> > > > Can you give an example of where you used snippets? Hugo does support
> > > > content reuse via shortcodes and partials, so when needed we can
> > leverage
> > > > it. For most cases, cross-linking may be simpler to maintain.
> > > >
> > > > > One additional thing that may be worth evaluating changing is the
> > > > Connectors section.
> > > >
> > > > Absolutely. I intentionally scoped Connectors restructuring as a
> > separate
> > > > follow-up FLIP (noted in the Future Work section), because it
> requires
> > a
> > > > bit more reorganization given the docs are in the external connector
> > > repos
> > > > too and I have some additional ideas too. The vision is to organize
> by
> > > > system (Kafka, JDBC, etc.) with tabs showing SQL/Table API/DataStream
> > > usage
> > > > where applicable, plus a dedicated Formats subsection with proper
> > > context.
> > > >
> > > > > We might need a "Built-in Functions" section under "Flink APIs -
> > Table
> > > > API - Functions" that redirects to "Flink SQL - Functions - Built-in
> > > > Functions". This would help users navigate more easily.
> > > >
> > > > Good suggestion. I'll add a link or redirect under "Flink APIs →
> Table
> > > API
> > > > → Functions" pointing to "Flink SQL → Functions → Built-in
> Functions".
> > > This
> > > > follows the same pattern as Data Types, by maintaining content in one
> > > > location, cross-link from related sections to ensure discoverability
> > > > without duplication.
> > > >
> > > > > One quick question, what are the different purposes of Learn Flink
> > and
> > > > Concepts. It seems Learn Flink also contains some api-agnostic
> > concepts,
> > > as
> > > > well as introductions to different api sets. What is the boundary?
> > > >
> > > > Good question. The intended distinction is like this:
> > > >
> > > > - Learn Flink: Learning-oriented tutorials where you follow along and
> > > build
> > > > something. The focus is on either snippet-based or guided, hands-on
> > > > learning paths.
> > > > - Concepts: Explanation-oriented content about foundational ideas and
> > > > mental models. The focus is on understanding how things work.
> > > >
> > > > The boundary isn't perfectly clean in the current proposal - for
> > example,
> > > > "Streaming Analytics" in Learn Flink has conceptual elements. The
> > guiding
> > > > principle is: if content is primarily "follow these steps to learn
> X",
> > it
> > > > belongs in Learn Flink. If it's primarily "here's how X works under
> the
> > > > hood", it belongs in Concepts.
> > > >
> > > > We could refine this further, but the core distinction is (either
> > snippet
> > > > or hands-on) learning vs. theoretical understanding.
> > > >
> > > > Best regards,
> > > >
> > > > Martijn
> > > >
> > > > On Thu, Jan 8, 2026 at 4:58 AM Xintong Song <to...@gmail.com>
> > > wrote:
> > > >
> > > > > Big +1! Thanks for putting this together, Martijn. This sounds
> great.
> > > > >
> > > > > One quick question, what are the different purposes of Learn Flink
> > and
> > > > > Concepts. It seems Learn Flink also contains some api-agnostic
> > > concepts,
> > > > as
> > > > > well as introductions to different api sets. What is the boundary?
> > > > >
> > > > > Best,
> > > > >
> > > > > Xintong
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Jan 8, 2026 at 10:43 AM dylanhz <dy...@163.com> wrote:
> > > > >
> > > > > > Hi Martin,
> > > > > >
> > > > > >
> > > > > > This proposal looks great to me. I think the new structure is
> well
> > > > > > organized and easy to understand.
> > > > > >
> > > > > >
> > > > > > I have only one small suggestion:
> > > > > > We might need a "Built-in Functions" section under "Flink APIs -
> > > Table
> > > > > API
> > > > > > - Functions" that redirects to "Flink SQL - Functions - Built-in
> > > > > > Functions". This would help users navigate more easily.
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Best regards,
> > > > > > dylanhz
> > > > > >
> > > > > >
> > > > > >
> > > > > > At 2026-01-07 18:01:32, "Martijn Visser" <
> martijnvisser@apache.org
> > >
> > > > > wrote:
> > > > > > >Hi everyone,
> > > > > > >
> > > > > > >I'd like to start a discussion on FLIP-561 [1] to restructure
> the
> > > > Apache
> > > > > > >Flink documentation to improve discoverability and usability of
> > > them.
> > > > > The
> > > > > > >Flink documentation has evolved organically over many years.
> While
> > > > > > >comprehensive, users face several navigation challenges:
> > > > > > >
> > > > > > >- The distinction between "Try Flink", "Learn Flink", and
> > "Concepts"
> > > > is
> > > > > > >unclear (these blur the Diátaxis categories of tutorials,
> > > explanation,
> > > > > and
> > > > > > >reference)
> > > > > > >- SQL is bundled with Table API, forcing SQL-only users through
> > > > > > >programmatic content
> > > > > > >- Streaming concepts like Dynamic Tables are buried in Table
> API,
> > > > though
> > > > > > >they are general relational streaming concepts
> > > > > > >- Python documentation duplicates the structure of both Table
> API
> > > and
> > > > > > >DataStream API
> > > > > > >- Connector documentation is fragmented by API rather than by
> > system
> > > > > > >
> > > > > > >This FLIP also refers to previously discussed and/or voted on
> > > FLIP-60
> > > > > [2]
> > > > > > >and FLIP-541 [3]
> > > > > > >
> > > > > > >The FLIP contains the new proposed information architecture and
> a
> > > > couple
> > > > > > of
> > > > > > >screenshots from my local changes. You can also use this live
> > > website
> > > > > [4]
> > > > > > >that contains mostly the refactored menu structure, content that
> > has
> > > > > been
> > > > > > >moved to a new location and some new content (like First Steps).
> > > > > > >
> > > > > > >I'd appreciate feedback on the proposed structure and rest of
> the
> > > > FLIP.
> > > > > > >
> > > > > > >Thanks,
> > > > > > >
> > > > > > >Martijn
> > > > > > >
> > > > > > >[1]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
> > > > > > >[2]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
> > > > > > >[3]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
> > > > > > >[4]
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] FLIP-561: Restructure Flink documentation
Posted by Xintong Song <to...@gmail.com>.
> - Learn Flink: Learning-oriented tutorials where you follow along and
build
> something. The focus is on either snippet-based or guided, hands-on
> learning paths.
> - Concepts: Explanation-oriented content about foundational ideas and
> mental models. The focus is on understanding how things work.
>
> The boundary isn't perfectly clean in the current proposal - for example,
> "Streaming Analytics" in Learn Flink has conceptual elements. The guiding
> principle is: if content is primarily "follow these steps to learn X", it
> belongs in Learn Flink. If it's primarily "here's how X works under the
> hood", it belongs in Concepts.
>
> We could refine this further, but the core distinction is (either snippet
> or hands-on) learning vs. theoretical understanding.
Thanks for the explanation. Sounds good to me.
Best,
Xintong
On Fri, Jan 9, 2026 at 8:04 AM David Anderson <da...@apache.org> wrote:
> Martijn, thanks for taking this on. This looks great.
>
> I think it's smart to limit the scope to something manageable and concrete
> (as you've done).
>
> Maybe we can add a couple more bullet points to the Future Work section. A
> reorg of the Deployment and Operations sections could be listed there,
> along with a reorg of some of the explanatory and learning content to
> better follow the diataxis model. I have some ideas, and look forward to
> helping out there in the future.
>
> David
>
> On Thu, Jan 8, 2026 at 7:33 AM Ryan van Huuksloot via dev <
> dev@flink.apache.org> wrote:
>
> > >Absolutely. I intentionally scoped Connectors restructuring as a
> separate
> > follow-up FLIP (noted in the Future Work section), because it requires a
> > bit more reorganization given the docs are in the external connector
> repos
> > too and I have some additional ideas too. The vision is to organize by
> > system (Kafka, JDBC, etc.) with tabs showing SQL/Table API/DataStream
> usage
> > where applicable, plus a dedicated Formats subsection with proper
> context.
> >
> > Perfect, I missed that. Sounds like a good plan.
> > Thanks for leading this effort!
> >
> > Ryan van Huuksloot
> > Staff Engineer, Infrastructure | Streaming Platform
> > [image: Shopify]
> > <https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email
> >
> >
> >
> > On Thu, Jan 8, 2026 at 10:14 AM Martijn Visser <martijnvisser@apache.org
> >
> > wrote:
> >
> > > Hi all,
> > >
> > > Thanks for the feedback! Let me go through them one-by-one:
> > >
> > > > I like the way that the data types are currently documented showing
> how
> > > each are implemented in SQL and the APIs, this is useful for Flink
> > > contributors.Is there a way to keep this view in the structure
> somewhere
> > > maybe in a contributor section?
> > >
> > > The Data Types content (including the SQL/Java/Scala type mappings) is
> > > moving to SQL Reference, not being removed. The contributor-friendly
> view
> > > will be preserved. We could add add a cross-link from Table API
> > > documentation to ensure discoverability from both entry points.
> > >
> > > > You mention the Diátaxis documentation framework - I wondered about
> > the
> > > how-to guides (task-oriented), and the reference material. Do we think
> > > there is no how to guides?
> > >
> > > That's correct indeed. The current Flink docs are primarily tutorials
> and
> > > reference material. True task-oriented how-to guides are largely
> missing
> > or
> > > scattered across blog posts. Since this is mostly a restructuring of
> > > existing content, it won't immediately solve that problem. I'd rather
> > first
> > > focus on this effort and the others that are currently out of scope of
> > the
> > > FLIP, and then work on this one.
> > >
> > > > In the past I have used snippets for documentation that is
> duplicated,
> > do
> > > we plan to use something like that for the duplicated content in SQL
> and
> > > table APIs? So we only have information authored once?
> > >
> > > Can you give an example of where you used snippets? Hugo does support
> > > content reuse via shortcodes and partials, so when needed we can
> leverage
> > > it. For most cases, cross-linking may be simpler to maintain.
> > >
> > > > One additional thing that may be worth evaluating changing is the
> > > Connectors section.
> > >
> > > Absolutely. I intentionally scoped Connectors restructuring as a
> separate
> > > follow-up FLIP (noted in the Future Work section), because it requires
> a
> > > bit more reorganization given the docs are in the external connector
> > repos
> > > too and I have some additional ideas too. The vision is to organize by
> > > system (Kafka, JDBC, etc.) with tabs showing SQL/Table API/DataStream
> > usage
> > > where applicable, plus a dedicated Formats subsection with proper
> > context.
> > >
> > > > We might need a "Built-in Functions" section under "Flink APIs -
> Table
> > > API - Functions" that redirects to "Flink SQL - Functions - Built-in
> > > Functions". This would help users navigate more easily.
> > >
> > > Good suggestion. I'll add a link or redirect under "Flink APIs → Table
> > API
> > > → Functions" pointing to "Flink SQL → Functions → Built-in Functions".
> > This
> > > follows the same pattern as Data Types, by maintaining content in one
> > > location, cross-link from related sections to ensure discoverability
> > > without duplication.
> > >
> > > > One quick question, what are the different purposes of Learn Flink
> and
> > > Concepts. It seems Learn Flink also contains some api-agnostic
> concepts,
> > as
> > > well as introductions to different api sets. What is the boundary?
> > >
> > > Good question. The intended distinction is like this:
> > >
> > > - Learn Flink: Learning-oriented tutorials where you follow along and
> > build
> > > something. The focus is on either snippet-based or guided, hands-on
> > > learning paths.
> > > - Concepts: Explanation-oriented content about foundational ideas and
> > > mental models. The focus is on understanding how things work.
> > >
> > > The boundary isn't perfectly clean in the current proposal - for
> example,
> > > "Streaming Analytics" in Learn Flink has conceptual elements. The
> guiding
> > > principle is: if content is primarily "follow these steps to learn X",
> it
> > > belongs in Learn Flink. If it's primarily "here's how X works under the
> > > hood", it belongs in Concepts.
> > >
> > > We could refine this further, but the core distinction is (either
> snippet
> > > or hands-on) learning vs. theoretical understanding.
> > >
> > > Best regards,
> > >
> > > Martijn
> > >
> > > On Thu, Jan 8, 2026 at 4:58 AM Xintong Song <to...@gmail.com>
> > wrote:
> > >
> > > > Big +1! Thanks for putting this together, Martijn. This sounds great.
> > > >
> > > > One quick question, what are the different purposes of Learn Flink
> and
> > > > Concepts. It seems Learn Flink also contains some api-agnostic
> > concepts,
> > > as
> > > > well as introductions to different api sets. What is the boundary?
> > > >
> > > > Best,
> > > >
> > > > Xintong
> > > >
> > > >
> > > >
> > > > On Thu, Jan 8, 2026 at 10:43 AM dylanhz <dy...@163.com> wrote:
> > > >
> > > > > Hi Martin,
> > > > >
> > > > >
> > > > > This proposal looks great to me. I think the new structure is well
> > > > > organized and easy to understand.
> > > > >
> > > > >
> > > > > I have only one small suggestion:
> > > > > We might need a "Built-in Functions" section under "Flink APIs -
> > Table
> > > > API
> > > > > - Functions" that redirects to "Flink SQL - Functions - Built-in
> > > > > Functions". This would help users navigate more easily.
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Best regards,
> > > > > dylanhz
> > > > >
> > > > >
> > > > >
> > > > > At 2026-01-07 18:01:32, "Martijn Visser" <martijnvisser@apache.org
> >
> > > > wrote:
> > > > > >Hi everyone,
> > > > > >
> > > > > >I'd like to start a discussion on FLIP-561 [1] to restructure the
> > > Apache
> > > > > >Flink documentation to improve discoverability and usability of
> > them.
> > > > The
> > > > > >Flink documentation has evolved organically over many years. While
> > > > > >comprehensive, users face several navigation challenges:
> > > > > >
> > > > > >- The distinction between "Try Flink", "Learn Flink", and
> "Concepts"
> > > is
> > > > > >unclear (these blur the Diátaxis categories of tutorials,
> > explanation,
> > > > and
> > > > > >reference)
> > > > > >- SQL is bundled with Table API, forcing SQL-only users through
> > > > > >programmatic content
> > > > > >- Streaming concepts like Dynamic Tables are buried in Table API,
> > > though
> > > > > >they are general relational streaming concepts
> > > > > >- Python documentation duplicates the structure of both Table API
> > and
> > > > > >DataStream API
> > > > > >- Connector documentation is fragmented by API rather than by
> system
> > > > > >
> > > > > >This FLIP also refers to previously discussed and/or voted on
> > FLIP-60
> > > > [2]
> > > > > >and FLIP-541 [3]
> > > > > >
> > > > > >The FLIP contains the new proposed information architecture and a
> > > couple
> > > > > of
> > > > > >screenshots from my local changes. You can also use this live
> > website
> > > > [4]
> > > > > >that contains mostly the refactored menu structure, content that
> has
> > > > been
> > > > > >moved to a new location and some new content (like First Steps).
> > > > > >
> > > > > >I'd appreciate feedback on the proposed structure and rest of the
> > > FLIP.
> > > > > >
> > > > > >Thanks,
> > > > > >
> > > > > >Martijn
> > > > > >
> > > > > >[1]
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
> > > > > >[2]
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
> > > > > >[3]
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
> > > > > >[4]
> > > > > >
> > > > >
> > > >
> > >
> >
> https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] FLIP-561: Restructure Flink documentation
Posted by David Anderson <da...@apache.org>.
Martijn, thanks for taking this on. This looks great.
I think it's smart to limit the scope to something manageable and concrete
(as you've done).
Maybe we can add a couple more bullet points to the Future Work section. A
reorg of the Deployment and Operations sections could be listed there,
along with a reorg of some of the explanatory and learning content to
better follow the diataxis model. I have some ideas, and look forward to
helping out there in the future.
David
On Thu, Jan 8, 2026 at 7:33 AM Ryan van Huuksloot via dev <
dev@flink.apache.org> wrote:
> >Absolutely. I intentionally scoped Connectors restructuring as a separate
> follow-up FLIP (noted in the Future Work section), because it requires a
> bit more reorganization given the docs are in the external connector repos
> too and I have some additional ideas too. The vision is to organize by
> system (Kafka, JDBC, etc.) with tabs showing SQL/Table API/DataStream usage
> where applicable, plus a dedicated Formats subsection with proper context.
>
> Perfect, I missed that. Sounds like a good plan.
> Thanks for leading this effort!
>
> Ryan van Huuksloot
> Staff Engineer, Infrastructure | Streaming Platform
> [image: Shopify]
> <https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email>
>
>
> On Thu, Jan 8, 2026 at 10:14 AM Martijn Visser <ma...@apache.org>
> wrote:
>
> > Hi all,
> >
> > Thanks for the feedback! Let me go through them one-by-one:
> >
> > > I like the way that the data types are currently documented showing how
> > each are implemented in SQL and the APIs, this is useful for Flink
> > contributors.Is there a way to keep this view in the structure somewhere
> > maybe in a contributor section?
> >
> > The Data Types content (including the SQL/Java/Scala type mappings) is
> > moving to SQL Reference, not being removed. The contributor-friendly view
> > will be preserved. We could add add a cross-link from Table API
> > documentation to ensure discoverability from both entry points.
> >
> > > You mention the Diátaxis documentation framework - I wondered about
> the
> > how-to guides (task-oriented), and the reference material. Do we think
> > there is no how to guides?
> >
> > That's correct indeed. The current Flink docs are primarily tutorials and
> > reference material. True task-oriented how-to guides are largely missing
> or
> > scattered across blog posts. Since this is mostly a restructuring of
> > existing content, it won't immediately solve that problem. I'd rather
> first
> > focus on this effort and the others that are currently out of scope of
> the
> > FLIP, and then work on this one.
> >
> > > In the past I have used snippets for documentation that is duplicated,
> do
> > we plan to use something like that for the duplicated content in SQL and
> > table APIs? So we only have information authored once?
> >
> > Can you give an example of where you used snippets? Hugo does support
> > content reuse via shortcodes and partials, so when needed we can leverage
> > it. For most cases, cross-linking may be simpler to maintain.
> >
> > > One additional thing that may be worth evaluating changing is the
> > Connectors section.
> >
> > Absolutely. I intentionally scoped Connectors restructuring as a separate
> > follow-up FLIP (noted in the Future Work section), because it requires a
> > bit more reorganization given the docs are in the external connector
> repos
> > too and I have some additional ideas too. The vision is to organize by
> > system (Kafka, JDBC, etc.) with tabs showing SQL/Table API/DataStream
> usage
> > where applicable, plus a dedicated Formats subsection with proper
> context.
> >
> > > We might need a "Built-in Functions" section under "Flink APIs - Table
> > API - Functions" that redirects to "Flink SQL - Functions - Built-in
> > Functions". This would help users navigate more easily.
> >
> > Good suggestion. I'll add a link or redirect under "Flink APIs → Table
> API
> > → Functions" pointing to "Flink SQL → Functions → Built-in Functions".
> This
> > follows the same pattern as Data Types, by maintaining content in one
> > location, cross-link from related sections to ensure discoverability
> > without duplication.
> >
> > > One quick question, what are the different purposes of Learn Flink and
> > Concepts. It seems Learn Flink also contains some api-agnostic concepts,
> as
> > well as introductions to different api sets. What is the boundary?
> >
> > Good question. The intended distinction is like this:
> >
> > - Learn Flink: Learning-oriented tutorials where you follow along and
> build
> > something. The focus is on either snippet-based or guided, hands-on
> > learning paths.
> > - Concepts: Explanation-oriented content about foundational ideas and
> > mental models. The focus is on understanding how things work.
> >
> > The boundary isn't perfectly clean in the current proposal - for example,
> > "Streaming Analytics" in Learn Flink has conceptual elements. The guiding
> > principle is: if content is primarily "follow these steps to learn X", it
> > belongs in Learn Flink. If it's primarily "here's how X works under the
> > hood", it belongs in Concepts.
> >
> > We could refine this further, but the core distinction is (either snippet
> > or hands-on) learning vs. theoretical understanding.
> >
> > Best regards,
> >
> > Martijn
> >
> > On Thu, Jan 8, 2026 at 4:58 AM Xintong Song <to...@gmail.com>
> wrote:
> >
> > > Big +1! Thanks for putting this together, Martijn. This sounds great.
> > >
> > > One quick question, what are the different purposes of Learn Flink and
> > > Concepts. It seems Learn Flink also contains some api-agnostic
> concepts,
> > as
> > > well as introductions to different api sets. What is the boundary?
> > >
> > > Best,
> > >
> > > Xintong
> > >
> > >
> > >
> > > On Thu, Jan 8, 2026 at 10:43 AM dylanhz <dy...@163.com> wrote:
> > >
> > > > Hi Martin,
> > > >
> > > >
> > > > This proposal looks great to me. I think the new structure is well
> > > > organized and easy to understand.
> > > >
> > > >
> > > > I have only one small suggestion:
> > > > We might need a "Built-in Functions" section under "Flink APIs -
> Table
> > > API
> > > > - Functions" that redirects to "Flink SQL - Functions - Built-in
> > > > Functions". This would help users navigate more easily.
> > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Best regards,
> > > > dylanhz
> > > >
> > > >
> > > >
> > > > At 2026-01-07 18:01:32, "Martijn Visser" <ma...@apache.org>
> > > wrote:
> > > > >Hi everyone,
> > > > >
> > > > >I'd like to start a discussion on FLIP-561 [1] to restructure the
> > Apache
> > > > >Flink documentation to improve discoverability and usability of
> them.
> > > The
> > > > >Flink documentation has evolved organically over many years. While
> > > > >comprehensive, users face several navigation challenges:
> > > > >
> > > > >- The distinction between "Try Flink", "Learn Flink", and "Concepts"
> > is
> > > > >unclear (these blur the Diátaxis categories of tutorials,
> explanation,
> > > and
> > > > >reference)
> > > > >- SQL is bundled with Table API, forcing SQL-only users through
> > > > >programmatic content
> > > > >- Streaming concepts like Dynamic Tables are buried in Table API,
> > though
> > > > >they are general relational streaming concepts
> > > > >- Python documentation duplicates the structure of both Table API
> and
> > > > >DataStream API
> > > > >- Connector documentation is fragmented by API rather than by system
> > > > >
> > > > >This FLIP also refers to previously discussed and/or voted on
> FLIP-60
> > > [2]
> > > > >and FLIP-541 [3]
> > > > >
> > > > >The FLIP contains the new proposed information architecture and a
> > couple
> > > > of
> > > > >screenshots from my local changes. You can also use this live
> website
> > > [4]
> > > > >that contains mostly the refactored menu structure, content that has
> > > been
> > > > >moved to a new location and some new content (like First Steps).
> > > > >
> > > > >I'd appreciate feedback on the proposed structure and rest of the
> > FLIP.
> > > > >
> > > > >Thanks,
> > > > >
> > > > >Martijn
> > > > >
> > > > >[1]
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
> > > > >[2]
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
> > > > >[3]
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
> > > > >[4]
> > > > >
> > > >
> > >
> >
> https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
> > > >
> > >
> >
>
Re: [DISCUSS] FLIP-561: Restructure Flink documentation
Posted by Ryan van Huuksloot via dev <de...@flink.apache.org>.
>Absolutely. I intentionally scoped Connectors restructuring as a separate
follow-up FLIP (noted in the Future Work section), because it requires a
bit more reorganization given the docs are in the external connector repos
too and I have some additional ideas too. The vision is to organize by
system (Kafka, JDBC, etc.) with tabs showing SQL/Table API/DataStream usage
where applicable, plus a dedicated Formats subsection with proper context.
Perfect, I missed that. Sounds like a good plan.
Thanks for leading this effort!
Ryan van Huuksloot
Staff Engineer, Infrastructure | Streaming Platform
[image: Shopify]
<https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email>
On Thu, Jan 8, 2026 at 10:14 AM Martijn Visser <ma...@apache.org>
wrote:
> Hi all,
>
> Thanks for the feedback! Let me go through them one-by-one:
>
> > I like the way that the data types are currently documented showing how
> each are implemented in SQL and the APIs, this is useful for Flink
> contributors.Is there a way to keep this view in the structure somewhere
> maybe in a contributor section?
>
> The Data Types content (including the SQL/Java/Scala type mappings) is
> moving to SQL Reference, not being removed. The contributor-friendly view
> will be preserved. We could add add a cross-link from Table API
> documentation to ensure discoverability from both entry points.
>
> > You mention the Diátaxis documentation framework - I wondered about the
> how-to guides (task-oriented), and the reference material. Do we think
> there is no how to guides?
>
> That's correct indeed. The current Flink docs are primarily tutorials and
> reference material. True task-oriented how-to guides are largely missing or
> scattered across blog posts. Since this is mostly a restructuring of
> existing content, it won't immediately solve that problem. I'd rather first
> focus on this effort and the others that are currently out of scope of the
> FLIP, and then work on this one.
>
> > In the past I have used snippets for documentation that is duplicated, do
> we plan to use something like that for the duplicated content in SQL and
> table APIs? So we only have information authored once?
>
> Can you give an example of where you used snippets? Hugo does support
> content reuse via shortcodes and partials, so when needed we can leverage
> it. For most cases, cross-linking may be simpler to maintain.
>
> > One additional thing that may be worth evaluating changing is the
> Connectors section.
>
> Absolutely. I intentionally scoped Connectors restructuring as a separate
> follow-up FLIP (noted in the Future Work section), because it requires a
> bit more reorganization given the docs are in the external connector repos
> too and I have some additional ideas too. The vision is to organize by
> system (Kafka, JDBC, etc.) with tabs showing SQL/Table API/DataStream usage
> where applicable, plus a dedicated Formats subsection with proper context.
>
> > We might need a "Built-in Functions" section under "Flink APIs - Table
> API - Functions" that redirects to "Flink SQL - Functions - Built-in
> Functions". This would help users navigate more easily.
>
> Good suggestion. I'll add a link or redirect under "Flink APIs → Table API
> → Functions" pointing to "Flink SQL → Functions → Built-in Functions". This
> follows the same pattern as Data Types, by maintaining content in one
> location, cross-link from related sections to ensure discoverability
> without duplication.
>
> > One quick question, what are the different purposes of Learn Flink and
> Concepts. It seems Learn Flink also contains some api-agnostic concepts, as
> well as introductions to different api sets. What is the boundary?
>
> Good question. The intended distinction is like this:
>
> - Learn Flink: Learning-oriented tutorials where you follow along and build
> something. The focus is on either snippet-based or guided, hands-on
> learning paths.
> - Concepts: Explanation-oriented content about foundational ideas and
> mental models. The focus is on understanding how things work.
>
> The boundary isn't perfectly clean in the current proposal - for example,
> "Streaming Analytics" in Learn Flink has conceptual elements. The guiding
> principle is: if content is primarily "follow these steps to learn X", it
> belongs in Learn Flink. If it's primarily "here's how X works under the
> hood", it belongs in Concepts.
>
> We could refine this further, but the core distinction is (either snippet
> or hands-on) learning vs. theoretical understanding.
>
> Best regards,
>
> Martijn
>
> On Thu, Jan 8, 2026 at 4:58 AM Xintong Song <to...@gmail.com> wrote:
>
> > Big +1! Thanks for putting this together, Martijn. This sounds great.
> >
> > One quick question, what are the different purposes of Learn Flink and
> > Concepts. It seems Learn Flink also contains some api-agnostic concepts,
> as
> > well as introductions to different api sets. What is the boundary?
> >
> > Best,
> >
> > Xintong
> >
> >
> >
> > On Thu, Jan 8, 2026 at 10:43 AM dylanhz <dy...@163.com> wrote:
> >
> > > Hi Martin,
> > >
> > >
> > > This proposal looks great to me. I think the new structure is well
> > > organized and easy to understand.
> > >
> > >
> > > I have only one small suggestion:
> > > We might need a "Built-in Functions" section under "Flink APIs - Table
> > API
> > > - Functions" that redirects to "Flink SQL - Functions - Built-in
> > > Functions". This would help users navigate more easily.
> > >
> > >
> > >
> > > --
> > >
> > > Best regards,
> > > dylanhz
> > >
> > >
> > >
> > > At 2026-01-07 18:01:32, "Martijn Visser" <ma...@apache.org>
> > wrote:
> > > >Hi everyone,
> > > >
> > > >I'd like to start a discussion on FLIP-561 [1] to restructure the
> Apache
> > > >Flink documentation to improve discoverability and usability of them.
> > The
> > > >Flink documentation has evolved organically over many years. While
> > > >comprehensive, users face several navigation challenges:
> > > >
> > > >- The distinction between "Try Flink", "Learn Flink", and "Concepts"
> is
> > > >unclear (these blur the Diátaxis categories of tutorials, explanation,
> > and
> > > >reference)
> > > >- SQL is bundled with Table API, forcing SQL-only users through
> > > >programmatic content
> > > >- Streaming concepts like Dynamic Tables are buried in Table API,
> though
> > > >they are general relational streaming concepts
> > > >- Python documentation duplicates the structure of both Table API and
> > > >DataStream API
> > > >- Connector documentation is fragmented by API rather than by system
> > > >
> > > >This FLIP also refers to previously discussed and/or voted on FLIP-60
> > [2]
> > > >and FLIP-541 [3]
> > > >
> > > >The FLIP contains the new proposed information architecture and a
> couple
> > > of
> > > >screenshots from my local changes. You can also use this live website
> > [4]
> > > >that contains mostly the refactored menu structure, content that has
> > been
> > > >moved to a new location and some new content (like First Steps).
> > > >
> > > >I'd appreciate feedback on the proposed structure and rest of the
> FLIP.
> > > >
> > > >Thanks,
> > > >
> > > >Martijn
> > > >
> > > >[1]
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
> > > >[2]
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
> > > >[3]
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
> > > >[4]
> > > >
> > >
> >
> https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
> > >
> >
>
Re: [DISCUSS] FLIP-561: Restructure Flink documentation
Posted by Martijn Visser <ma...@apache.org>.
Hi all,
Thanks for the feedback! Let me go through them one-by-one:
> I like the way that the data types are currently documented showing how
each are implemented in SQL and the APIs, this is useful for Flink
contributors.Is there a way to keep this view in the structure somewhere
maybe in a contributor section?
The Data Types content (including the SQL/Java/Scala type mappings) is
moving to SQL Reference, not being removed. The contributor-friendly view
will be preserved. We could add add a cross-link from Table API
documentation to ensure discoverability from both entry points.
> You mention the Diátaxis documentation framework - I wondered about the
how-to guides (task-oriented), and the reference material. Do we think
there is no how to guides?
That's correct indeed. The current Flink docs are primarily tutorials and
reference material. True task-oriented how-to guides are largely missing or
scattered across blog posts. Since this is mostly a restructuring of
existing content, it won't immediately solve that problem. I'd rather first
focus on this effort and the others that are currently out of scope of the
FLIP, and then work on this one.
> In the past I have used snippets for documentation that is duplicated, do
we plan to use something like that for the duplicated content in SQL and
table APIs? So we only have information authored once?
Can you give an example of where you used snippets? Hugo does support
content reuse via shortcodes and partials, so when needed we can leverage
it. For most cases, cross-linking may be simpler to maintain.
> One additional thing that may be worth evaluating changing is the
Connectors section.
Absolutely. I intentionally scoped Connectors restructuring as a separate
follow-up FLIP (noted in the Future Work section), because it requires a
bit more reorganization given the docs are in the external connector repos
too and I have some additional ideas too. The vision is to organize by
system (Kafka, JDBC, etc.) with tabs showing SQL/Table API/DataStream usage
where applicable, plus a dedicated Formats subsection with proper context.
> We might need a "Built-in Functions" section under "Flink APIs - Table
API - Functions" that redirects to "Flink SQL - Functions - Built-in
Functions". This would help users navigate more easily.
Good suggestion. I'll add a link or redirect under "Flink APIs → Table API
→ Functions" pointing to "Flink SQL → Functions → Built-in Functions". This
follows the same pattern as Data Types, by maintaining content in one
location, cross-link from related sections to ensure discoverability
without duplication.
> One quick question, what are the different purposes of Learn Flink and
Concepts. It seems Learn Flink also contains some api-agnostic concepts, as
well as introductions to different api sets. What is the boundary?
Good question. The intended distinction is like this:
- Learn Flink: Learning-oriented tutorials where you follow along and build
something. The focus is on either snippet-based or guided, hands-on
learning paths.
- Concepts: Explanation-oriented content about foundational ideas and
mental models. The focus is on understanding how things work.
The boundary isn't perfectly clean in the current proposal - for example,
"Streaming Analytics" in Learn Flink has conceptual elements. The guiding
principle is: if content is primarily "follow these steps to learn X", it
belongs in Learn Flink. If it's primarily "here's how X works under the
hood", it belongs in Concepts.
We could refine this further, but the core distinction is (either snippet
or hands-on) learning vs. theoretical understanding.
Best regards,
Martijn
On Thu, Jan 8, 2026 at 4:58 AM Xintong Song <to...@gmail.com> wrote:
> Big +1! Thanks for putting this together, Martijn. This sounds great.
>
> One quick question, what are the different purposes of Learn Flink and
> Concepts. It seems Learn Flink also contains some api-agnostic concepts, as
> well as introductions to different api sets. What is the boundary?
>
> Best,
>
> Xintong
>
>
>
> On Thu, Jan 8, 2026 at 10:43 AM dylanhz <dy...@163.com> wrote:
>
> > Hi Martin,
> >
> >
> > This proposal looks great to me. I think the new structure is well
> > organized and easy to understand.
> >
> >
> > I have only one small suggestion:
> > We might need a "Built-in Functions" section under "Flink APIs - Table
> API
> > - Functions" that redirects to "Flink SQL - Functions - Built-in
> > Functions". This would help users navigate more easily.
> >
> >
> >
> > --
> >
> > Best regards,
> > dylanhz
> >
> >
> >
> > At 2026-01-07 18:01:32, "Martijn Visser" <ma...@apache.org>
> wrote:
> > >Hi everyone,
> > >
> > >I'd like to start a discussion on FLIP-561 [1] to restructure the Apache
> > >Flink documentation to improve discoverability and usability of them.
> The
> > >Flink documentation has evolved organically over many years. While
> > >comprehensive, users face several navigation challenges:
> > >
> > >- The distinction between "Try Flink", "Learn Flink", and "Concepts" is
> > >unclear (these blur the Diátaxis categories of tutorials, explanation,
> and
> > >reference)
> > >- SQL is bundled with Table API, forcing SQL-only users through
> > >programmatic content
> > >- Streaming concepts like Dynamic Tables are buried in Table API, though
> > >they are general relational streaming concepts
> > >- Python documentation duplicates the structure of both Table API and
> > >DataStream API
> > >- Connector documentation is fragmented by API rather than by system
> > >
> > >This FLIP also refers to previously discussed and/or voted on FLIP-60
> [2]
> > >and FLIP-541 [3]
> > >
> > >The FLIP contains the new proposed information architecture and a couple
> > of
> > >screenshots from my local changes. You can also use this live website
> [4]
> > >that contains mostly the refactored menu structure, content that has
> been
> > >moved to a new location and some new content (like First Steps).
> > >
> > >I'd appreciate feedback on the proposed structure and rest of the FLIP.
> > >
> > >Thanks,
> > >
> > >Martijn
> > >
> > >[1]
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
> > >[2]
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
> > >[3]
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
> > >[4]
> > >
> >
> https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
> >
>
Re: [DISCUSS] FLIP-561: Restructure Flink documentation
Posted by Xintong Song <to...@gmail.com>.
Big +1! Thanks for putting this together, Martijn. This sounds great.
One quick question, what are the different purposes of Learn Flink and
Concepts. It seems Learn Flink also contains some api-agnostic concepts, as
well as introductions to different api sets. What is the boundary?
Best,
Xintong
On Thu, Jan 8, 2026 at 10:43 AM dylanhz <dy...@163.com> wrote:
> Hi Martin,
>
>
> This proposal looks great to me. I think the new structure is well
> organized and easy to understand.
>
>
> I have only one small suggestion:
> We might need a "Built-in Functions" section under "Flink APIs - Table API
> - Functions" that redirects to "Flink SQL - Functions - Built-in
> Functions". This would help users navigate more easily.
>
>
>
> --
>
> Best regards,
> dylanhz
>
>
>
> At 2026-01-07 18:01:32, "Martijn Visser" <ma...@apache.org> wrote:
> >Hi everyone,
> >
> >I'd like to start a discussion on FLIP-561 [1] to restructure the Apache
> >Flink documentation to improve discoverability and usability of them. The
> >Flink documentation has evolved organically over many years. While
> >comprehensive, users face several navigation challenges:
> >
> >- The distinction between "Try Flink", "Learn Flink", and "Concepts" is
> >unclear (these blur the Diátaxis categories of tutorials, explanation, and
> >reference)
> >- SQL is bundled with Table API, forcing SQL-only users through
> >programmatic content
> >- Streaming concepts like Dynamic Tables are buried in Table API, though
> >they are general relational streaming concepts
> >- Python documentation duplicates the structure of both Table API and
> >DataStream API
> >- Connector documentation is fragmented by API rather than by system
> >
> >This FLIP also refers to previously discussed and/or voted on FLIP-60 [2]
> >and FLIP-541 [3]
> >
> >The FLIP contains the new proposed information architecture and a couple
> of
> >screenshots from my local changes. You can also use this live website [4]
> >that contains mostly the refactored menu structure, content that has been
> >moved to a new location and some new content (like First Steps).
> >
> >I'd appreciate feedback on the proposed structure and rest of the FLIP.
> >
> >Thanks,
> >
> >Martijn
> >
> >[1]
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
> >[2]
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
> >[3]
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
> >[4]
> >
> https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
>
Re:[DISCUSS] FLIP-561: Restructure Flink documentation
Posted by dylanhz <dy...@163.com>.
Hi Martin,
This proposal looks great to me. I think the new structure is well organized and easy to understand.
I have only one small suggestion:
We might need a "Built-in Functions" section under "Flink APIs - Table API - Functions" that redirects to "Flink SQL - Functions - Built-in Functions". This would help users navigate more easily.
--
Best regards,
dylanhz
At 2026-01-07 18:01:32, "Martijn Visser" <ma...@apache.org> wrote:
>Hi everyone,
>
>I'd like to start a discussion on FLIP-561 [1] to restructure the Apache
>Flink documentation to improve discoverability and usability of them. The
>Flink documentation has evolved organically over many years. While
>comprehensive, users face several navigation challenges:
>
>- The distinction between "Try Flink", "Learn Flink", and "Concepts" is
>unclear (these blur the Diátaxis categories of tutorials, explanation, and
>reference)
>- SQL is bundled with Table API, forcing SQL-only users through
>programmatic content
>- Streaming concepts like Dynamic Tables are buried in Table API, though
>they are general relational streaming concepts
>- Python documentation duplicates the structure of both Table API and
>DataStream API
>- Connector documentation is fragmented by API rather than by system
>
>This FLIP also refers to previously discussed and/or voted on FLIP-60 [2]
>and FLIP-541 [3]
>
>The FLIP contains the new proposed information architecture and a couple of
>screenshots from my local changes. You can also use this live website [4]
>that contains mostly the refactored menu structure, content that has been
>moved to a new location and some new content (like First Steps).
>
>I'd appreciate feedback on the proposed structure and rest of the FLIP.
>
>Thanks,
>
>Martijn
>
>[1]
>https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
>[2]
>https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
>[3]
>https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
>[4]
>https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
Re: [DISCUSS] FLIP-561: Restructure Flink documentation
Posted by Robin Moffatt via user <us...@flink.apache.org>.
Hi Martijn,
Thanks for putting this together. I think it's an excellent idea, and I
agree with the proposal.
Robin.
On Wed, 7 Jan 2026 at 10:02, Martijn Visser <ma...@apache.org>
wrote:
> Hi everyone,
>
> I'd like to start a discussion on FLIP-561 [1] to restructure the Apache
> Flink documentation to improve discoverability and usability of them. The
> Flink documentation has evolved organically over many years. While
> comprehensive, users face several navigation challenges:
>
> - The distinction between "Try Flink", "Learn Flink", and "Concepts" is
> unclear (these blur the Diátaxis categories of tutorials, explanation, and
> reference)
> - SQL is bundled with Table API, forcing SQL-only users through
> programmatic content
> - Streaming concepts like Dynamic Tables are buried in Table API, though
> they are general relational streaming concepts
> - Python documentation duplicates the structure of both Table API and
> DataStream API
> - Connector documentation is fragmented by API rather than by system
>
> This FLIP also refers to previously discussed and/or voted on FLIP-60 [2]
> and FLIP-541 [3]
>
> The FLIP contains the new proposed information architecture and a couple
> of screenshots from my local changes. You can also use this live website
> [4] that contains mostly the refactored menu structure, content that has
> been moved to a new location and some new content (like First Steps).
>
> I'd appreciate feedback on the proposed structure and rest of the FLIP.
>
> Thanks,
>
> Martijn
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
> [2]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
> [3]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
> [4]
> https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
>
--
*Robin Moffatt*
*Sr. Principal Advisor, Streaming Data Technologies*
RE: [DISCUSS] FLIP-561: Restructure Flink documentation
Posted by David Radley <da...@uk.ibm.com>.
Hi,
I agree this sounds like a good idea. Some thoughts (not show stoppers - but I wanted to share):
I like the way that the data types are currently documented [1] showing how each are implemented in SQL and the APIs, this is useful for Flink contributors.Is there a way to keep this view in the structure somewhere maybe in a contributor section?
You mention the Diátaxis documentation framework<https://diataxis.fr/> - I wondered about the how-to guides (task-oriented), and the reference material. Do we think there is no how to guides?
In the past I have used snippets for documentation that is duplicated, do we plan to use something like that for the duplicated content in SQL and table APIs? So we only have information authored once?
[1]https://nightlies.apache.org/flink/flink-docs-release-2.2/docs/dev/table/types/
Kind regards, David.
From: Rion Williams <ri...@gmail.com>
Date: Wednesday, 7 January 2026 at 12:04
To: user@flink.apache.org <us...@flink.apache.org>
Subject: [EXTERNAL] Re: [DISCUSS] FLIP-561: Restructure Flink documentation
This Message Is From an Untrusted Sender
You have not previously corresponded with this sender.
Report Suspicious<https://us-phishalarm-ewt.proofpoint.com/EWT/v1/AdhS1Rd-!9_FVM1i8MyHVtRbX4KWBko41WO_lJ-CjQ7ezmVhcDkH1VhZBHBA27diuIpptfvjmXTTOg0NQX18AKZDnl8JUilkiMt3Lst_UdoTv$>
+1, this seems like a great idea! Thanks for putting it together Martijn!
On Jan 7, 2026, at 4:03 AM, Martijn Visser <ma...@apache.org> wrote:
Hi everyone,
I'd like to start a discussion on FLIP-561 [1] to restructure the Apache Flink documentation to improve discoverability and usability of them. The Flink documentation has evolved organically over many years. While comprehensive, users face several navigation challenges:
- The distinction between "Try Flink", "Learn Flink", and "Concepts" is unclear (these blur the Diátaxis categories of tutorials, explanation, and reference)
- SQL is bundled with Table API, forcing SQL-only users through programmatic content
- Streaming concepts like Dynamic Tables are buried in Table API, though they are general relational streaming concepts
- Python documentation duplicates the structure of both Table API and DataStream API
- Connector documentation is fragmented by API rather than by system
This FLIP also refers to previously discussed and/or voted on FLIP-60 [2] and FLIP-541 [3]
The FLIP contains the new proposed information architecture and a couple of screenshots from my local changes. You can also use this live website [4] that contains mostly the refactored menu structure, content that has been moved to a new location and some new content (like First Steps).
I'd appreciate feedback on the proposed structure and rest of the FLIP.
Thanks,
Martijn
[1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
[2] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
[3] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
[4] https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
Unless otherwise stated above:
IBM United Kingdom Limited
Registered in England and Wales with number 741598
Registered office: Building C, IBM Hursley Office, Hursley Park Road, Winchester, Hampshire SO21 2JN
Re: [DISCUSS] FLIP-561: Restructure Flink documentation
Posted by Rion Williams <ri...@gmail.com>.
+1, this seems like a great idea! Thanks for putting it together Martijn!
> On Jan 7, 2026, at 4:03 AM, Martijn Visser <ma...@apache.org> wrote:
>
>
> Hi everyone,
>
> I'd like to start a discussion on FLIP-561 [1] to restructure the Apache Flink documentation to improve discoverability and usability of them. The Flink documentation has evolved organically over many years. While comprehensive, users face several navigation challenges:
>
> - The distinction between "Try Flink", "Learn Flink", and "Concepts" is unclear (these blur the Diátaxis categories of tutorials, explanation, and reference)
> - SQL is bundled with Table API, forcing SQL-only users through programmatic content
> - Streaming concepts like Dynamic Tables are buried in Table API, though they are general relational streaming concepts
> - Python documentation duplicates the structure of both Table API and DataStream API
> - Connector documentation is fragmented by API rather than by system
>
> This FLIP also refers to previously discussed and/or voted on FLIP-60 [2] and FLIP-541 [3]
>
> The FLIP contains the new proposed information architecture and a couple of screenshots from my local changes. You can also use this live website [4] that contains mostly the refactored menu structure, content that has been moved to a new location and some new content (like First Steps).
>
> I'd appreciate feedback on the proposed structure and rest of the FLIP.
>
> Thanks,
>
> Martijn
>
> [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
> [2] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
> [3] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
> [4] https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
Re: [DISCUSS] FLIP-561: Restructure Flink documentation
Posted by Zander Matheson <a....@gmail.com>.
Thanks for putting this together, Martijn. I really like the proposed
structure.
I think it would make sense to link out to the existing PyDocs site(
https://nightlies.apache.org/flink/flink-docs-release-2.2/api/python/) as a
sub tab under the proposed `Flink APIs` tab and we can move the
documentation in the current Python sub tab to this single source. This
would loosely follow how PySpark and PyIcergerg docs are organized.
PyIceberg Specific Documentation
[image: Screenshot 2026-01-07 at 2.10.40 PM.png]
Iceberg Main Documentation
[image: Screenshot 2026-01-07 at 2.10.25 PM.png]
PySpark Specific Documentation
[image: Screenshot 2026-01-07 at 1.30.27 PM.png]
Spark Main Documentation
[image: Screenshot 2026-01-07 at 1.30.16 PM.png]
I am happy to contribute to the restructuring of the Python documentation!
On Wed, Jan 7, 2026 at 7:26 AM Ryan van Huuksloot via dev <
dev@flink.apache.org> wrote:
> Hi Martijn,
>
> +1 overall. I like the overall new structure better.
>
> One additional thing that may be worth evaluating changing
> is the Connectors section. A few things come to mind:
> - Hybrid Source under Datastream Connectors. It is more of a concept.
> - Hive under Table Connectors blurs the line between a connector, catalog
> and built in functions.
> - The "Connectors" header, from my perspective, should represent just the
> Data Source and Data Sink connectors. We've colocated anything that is an
> external connection under Connectors. I wonder if we need a different high
> level name so things like "Models" make more sense under that header.
> - Similar to above, the "Formats" under the Connectors feel out of place. I
> don't have a recommendation here but they seem to be put there because they
> are coupled to Connectors but there isn't much there to understand why they
> are there unless you know what Formats are and how they are used.
> - The "Overview" in each of the Formats is out of date.
> - The "Download" in Table Connectors is out of date - and could likely be
> removed.
>
> Ryan van Huuksloot
> Staff Engineer, Infrastructure | Streaming Platform
> [image: Shopify]
> <https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email>
>
>
> On Wed, Jan 7, 2026 at 6:20 AM Leonard Xu <xb...@gmail.com> wrote:
>
> > +1 from my side, the new structure looks good and compatibility for
> > existing urls makes sense to me, thanks Martijn for improving our
> > documentations.
> >
> > Best,
> > Leonard
> >
> > > 2026 1月 7 18:01，Martijn Visser <ma...@apache.org> 写道：
> > >
> > > Hi everyone,
> > >
> > > I'd like to start a discussion on FLIP-561 [1] to restructure the
> Apache
> > Flink documentation to improve discoverability and usability of them. The
> > Flink documentation has evolved organically over many years. While
> > comprehensive, users face several navigation challenges:
> > >
> > > - The distinction between "Try Flink", "Learn Flink", and "Concepts" is
> > unclear (these blur the Diátaxis categories of tutorials, explanation,
> and
> > reference)
> > > - SQL is bundled with Table API, forcing SQL-only users through
> > programmatic content
> > > - Streaming concepts like Dynamic Tables are buried in Table API,
> though
> > they are general relational streaming concepts
> > > - Python documentation duplicates the structure of both Table API and
> > DataStream API
> > > - Connector documentation is fragmented by API rather than by system
> > >
> > > This FLIP also refers to previously discussed and/or voted on FLIP-60
> > [2] and FLIP-541 [3]
> > >
> > > The FLIP contains the new proposed information architecture and a
> couple
> > of screenshots from my local changes. You can also use this live website
> > [4] that contains mostly the refactored menu structure, content that has
> > been moved to a new location and some new content (like First Steps).
> > >
> > > I'd appreciate feedback on the proposed structure and rest of the FLIP.
> > >
> > > Thanks,
> > >
> > > Martijn
> > >
> > > [1]
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
> > > [2]
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
> > > [3]
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
> > > [4]
> >
> https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
> >
> >
>
Re: [DISCUSS] FLIP-561: Restructure Flink documentation
Posted by Ryan van Huuksloot via dev <de...@flink.apache.org>.
Hi Martijn,
+1 overall. I like the overall new structure better.
One additional thing that may be worth evaluating changing
is the Connectors section. A few things come to mind:
- Hybrid Source under Datastream Connectors. It is more of a concept.
- Hive under Table Connectors blurs the line between a connector, catalog
and built in functions.
- The "Connectors" header, from my perspective, should represent just the
Data Source and Data Sink connectors. We've colocated anything that is an
external connection under Connectors. I wonder if we need a different high
level name so things like "Models" make more sense under that header.
- Similar to above, the "Formats" under the Connectors feel out of place. I
don't have a recommendation here but they seem to be put there because they
are coupled to Connectors but there isn't much there to understand why they
are there unless you know what Formats are and how they are used.
- The "Overview" in each of the Formats is out of date.
- The "Download" in Table Connectors is out of date - and could likely be
removed.
Ryan van Huuksloot
Staff Engineer, Infrastructure | Streaming Platform
[image: Shopify]
<https://www.shopify.com/?utm_medium=salessignatures&utm_source=hs_email>
On Wed, Jan 7, 2026 at 6:20 AM Leonard Xu <xb...@gmail.com> wrote:
> +1 from my side, the new structure looks good and compatibility for
> existing urls makes sense to me, thanks Martijn for improving our
> documentations.
>
> Best,
> Leonard
>
> > 2026 1月 7 18:01，Martijn Visser <ma...@apache.org> 写道：
> >
> > Hi everyone,
> >
> > I'd like to start a discussion on FLIP-561 [1] to restructure the Apache
> Flink documentation to improve discoverability and usability of them. The
> Flink documentation has evolved organically over many years. While
> comprehensive, users face several navigation challenges:
> >
> > - The distinction between "Try Flink", "Learn Flink", and "Concepts" is
> unclear (these blur the Diátaxis categories of tutorials, explanation, and
> reference)
> > - SQL is bundled with Table API, forcing SQL-only users through
> programmatic content
> > - Streaming concepts like Dynamic Tables are buried in Table API, though
> they are general relational streaming concepts
> > - Python documentation duplicates the structure of both Table API and
> DataStream API
> > - Connector documentation is fragmented by API rather than by system
> >
> > This FLIP also refers to previously discussed and/or voted on FLIP-60
> [2] and FLIP-541 [3]
> >
> > The FLIP contains the new proposed information architecture and a couple
> of screenshots from my local changes. You can also use this live website
> [4] that contains mostly the refactored menu structure, content that has
> been moved to a new location and some new content (like First Steps).
> >
> > I'd appreciate feedback on the proposed structure and rest of the FLIP.
> >
> > Thanks,
> >
> > Martijn
> >
> > [1]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
> > [2]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
> > [3]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
> > [4]
> https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
>
>
Re: [DISCUSS] FLIP-561: Restructure Flink documentation
Posted by Leonard Xu <xb...@gmail.com>.
+1 from my side, the new structure looks good and compatibility for existing urls makes sense to me, thanks Martijn for improving our documentations.
Best,
Leonard
> 2026 1月 7 18:01，Martijn Visser <ma...@apache.org> 写道：
>
> Hi everyone,
>
> I'd like to start a discussion on FLIP-561 [1] to restructure the Apache Flink documentation to improve discoverability and usability of them. The Flink documentation has evolved organically over many years. While comprehensive, users face several navigation challenges:
>
> - The distinction between "Try Flink", "Learn Flink", and "Concepts" is unclear (these blur the Diátaxis categories of tutorials, explanation, and reference)
> - SQL is bundled with Table API, forcing SQL-only users through programmatic content
> - Streaming concepts like Dynamic Tables are buried in Table API, though they are general relational streaming concepts
> - Python documentation duplicates the structure of both Table API and DataStream API
> - Connector documentation is fragmented by API rather than by system
>
> This FLIP also refers to previously discussed and/or voted on FLIP-60 [2] and FLIP-541 [3]
>
> The FLIP contains the new proposed information architecture and a couple of screenshots from my local changes. You can also use this live website [4] that contains mostly the refactored menu structure, content that has been moved to a new location and some new content (like First Steps).
>
> I'd appreciate feedback on the proposed structure and rest of the FLIP.
>
> Thanks,
>
> Martijn
>
> [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
> [2] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
> [3] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
> [4] https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
Re: [DISCUSS] FLIP-561: Restructure Flink documentation
Posted by Robin Moffatt via dev <de...@flink.apache.org>.
Hi Martijn,
Thanks for putting this together. I think it's an excellent idea, and I
agree with the proposal.
Robin.
On Wed, 7 Jan 2026 at 10:02, Martijn Visser <ma...@apache.org>
wrote:
> Hi everyone,
>
> I'd like to start a discussion on FLIP-561 [1] to restructure the Apache
> Flink documentation to improve discoverability and usability of them. The
> Flink documentation has evolved organically over many years. While
> comprehensive, users face several navigation challenges:
>
> - The distinction between "Try Flink", "Learn Flink", and "Concepts" is
> unclear (these blur the Diátaxis categories of tutorials, explanation, and
> reference)
> - SQL is bundled with Table API, forcing SQL-only users through
> programmatic content
> - Streaming concepts like Dynamic Tables are buried in Table API, though
> they are general relational streaming concepts
> - Python documentation duplicates the structure of both Table API and
> DataStream API
> - Connector documentation is fragmented by API rather than by system
>
> This FLIP also refers to previously discussed and/or voted on FLIP-60 [2]
> and FLIP-541 [3]
>
> The FLIP contains the new proposed information architecture and a couple
> of screenshots from my local changes. You can also use this live website
> [4] that contains mostly the refactored menu structure, content that has
> been moved to a new location and some new content (like First Steps).
>
> I'd appreciate feedback on the proposed structure and rest of the FLIP.
>
> Thanks,
>
> Martijn
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
> [2]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
> [3]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
> [4]
> https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/
>
--
*Robin Moffatt*
*Sr. Principal Advisor, Streaming Data Technologies*
Re: [DISCUSS] FLIP-561: Restructure Flink documentation
Posted by Leonard Xu <xb...@gmail.com>.
+1 from my side, the new structure looks good and compatibility for existing urls makes sense to me, thanks Martijn for improving our documentations.
Best,
Leonard
> 2026 1月 7 18:01，Martijn Visser <ma...@apache.org> 写道：
>
> Hi everyone,
>
> I'd like to start a discussion on FLIP-561 [1] to restructure the Apache Flink documentation to improve discoverability and usability of them. The Flink documentation has evolved organically over many years. While comprehensive, users face several navigation challenges:
>
> - The distinction between "Try Flink", "Learn Flink", and "Concepts" is unclear (these blur the Diátaxis categories of tutorials, explanation, and reference)
> - SQL is bundled with Table API, forcing SQL-only users through programmatic content
> - Streaming concepts like Dynamic Tables are buried in Table API, though they are general relational streaming concepts
> - Python documentation duplicates the structure of both Table API and DataStream API
> - Connector documentation is fragmented by API rather than by system
>
> This FLIP also refers to previously discussed and/or voted on FLIP-60 [2] and FLIP-541 [3]
>
> The FLIP contains the new proposed information architecture and a couple of screenshots from my local changes. You can also use this live website [4] that contains mostly the refactored menu structure, content that has been moved to a new location and some new content (like First Steps).
>
> I'd appreciate feedback on the proposed structure and rest of the FLIP.
>
> Thanks,
>
> Martijn
>
> [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-561%3A+Restructure+Flink+documentation
> [2] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405685
> [3] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=378473217
> [4] https://apache-flink-doc-refactoring.netlify.app/docs/getting-started/local_installation/