[DISCUSS] Integration of training materials into Apache Flink

[David Anderson <da...@ververica.com>]

Thank you all for the very positive response to our proposal to contribute
the training materials that have been at training.ververica.com to the
Apache Flink project. Now I’d like to begin the more detailed discussion of
how to go about this.

In that earlier thread I mentioned that we were thinking of merging the
markdown-based web pages into flink.apache.org, and to add the exercises to
flink-playgrounds. This was based on thinking that it would be something of
a maintenance headache to add the website content into the docs, where it
would have to be versioned.

Since then, a better approach has been suggested:

We already have quite a bit of “getting started” material in the docs: Code
Walkthroughs, Docker Playgrounds, Tutorials, and Examples. Having a second
location (namely flink.apache.org) where this kind of content could be
found doesn’t seem ideal. So let’s go ahead and add the expository content
from the training materials to the documentation, with pointers into the
rest of the docs (which are already present in the training), and with
pointers to the exercises (rather than including the exercise descriptions
in the docs). This will keep the content that will need more frequent
revision out of the documentation.

Then let’s create a new repo -- named flink-training -- that contains the
exercises, the solutions, and the tests that go with them, PLUS all of the
material that describes how to get setup to do the exercises, the
explanations for each exercise, and accompanying discussion material that
should be read after doing each exercise. Note that the exercise solutions
already have tests, and Travis is already being used for CI on the existing
project, so CI shouldn’t be an issue.

Action Item: would a committer or PMC member kindly volunteer to help with
creating this new flink-training repo?

With the content refactored in this way, I believe ongoing maintenance
won’t be much trouble. With each new Flink release I’ve been updating the
exercises to build against the latest release, and to avoid any newly
deprecated parts of the API. But since these exercises are focused on the
most basic parts of the API, that hasn’t been difficult. As for content
from the training website that would move into the docs, this content is
much more stable, and has only needed a gentle revision every year or two.

Regards,
David

Re: [DISCUSS] Integration of training materials into Apache Flink

[Seth Wiesman <sj...@gmail.com>]

Aljoscha, I know you've been working on the docs recently. Are you working
on the concepts section? I don't want to just drop it if someone has work
in the pipeline.

Seth


On Wed, Apr 15, 2020 at 2:19 PM Seth Wiesman <sj...@gmail.com> wrote:

> Hi David,
>
> I am happy to get the repo created for you.
>
> If we go with the documentation (vs flink.apache.org) do you think we
> should remove any of the existing content? There is already a getting
> started section with quickstarts and walkthroughs and a concepts section.
> In particular, the concepts section today is not complete and almost every
> page on master contains multiple TODOs. I don't believe anyone is working
> on these.  What do you think about replacing the current concepts section
> with the training material? I just re-examined the training site and I
> believe it covers the same material as concepts but better. Of course, we
> would salvage anything worth keeping, like the glossary.
>
> Seth
>
> On Wed, Apr 15, 2020 at 2:02 PM David Anderson <da...@ververica.com>
> wrote:
>
>> Thank you all for the very positive response to our proposal to contribute
>> the training materials that have been at training.ververica.com to the
>> Apache Flink project. Now I’d like to begin the more detailed discussion
>> of
>> how to go about this.
>>
>> In that earlier thread I mentioned that we were thinking of merging the
>> markdown-based web pages into flink.apache.org, and to add the exercises
>> to
>> flink-playgrounds. This was based on thinking that it would be something
>> of
>> a maintenance headache to add the website content into the docs, where it
>> would have to be versioned.
>>
>> Since then, a better approach has been suggested:
>>
>> We already have quite a bit of “getting started” material in the docs:
>> Code
>> Walkthroughs, Docker Playgrounds, Tutorials, and Examples. Having a second
>> location (namely flink.apache.org) where this kind of content could be
>> found doesn’t seem ideal. So let’s go ahead and add the expository content
>> from the training materials to the documentation, with pointers into the
>> rest of the docs (which are already present in the training), and with
>> pointers to the exercises (rather than including the exercise descriptions
>> in the docs). This will keep the content that will need more frequent
>> revision out of the documentation.
>>
>> Then let’s create a new repo -- named flink-training -- that contains the
>> exercises, the solutions, and the tests that go with them, PLUS all of the
>> material that describes how to get setup to do the exercises, the
>> explanations for each exercise, and accompanying discussion material that
>> should be read after doing each exercise. Note that the exercise solutions
>> already have tests, and Travis is already being used for CI on the
>> existing
>> project, so CI shouldn’t be an issue.
>>
>> Action Item: would a committer or PMC member kindly volunteer to help with
>> creating this new flink-training repo?
>>
>> With the content refactored in this way, I believe ongoing maintenance
>> won’t be much trouble. With each new Flink release I’ve been updating the
>> exercises to build against the latest release, and to avoid any newly
>> deprecated parts of the API. But since these exercises are focused on the
>> most basic parts of the API, that hasn’t been difficult. As for content
>> from the training website that would move into the docs, this content is
>> much more stable, and has only needed a gentle revision every year or two.
>>
>> Regards,
>> David
>>
>

Re: [DISCUSS] Integration of training materials into Apache Flink

[Robert Metzger <rm...@apache.org>]

The Jira component has been created!


On Mon, Apr 20, 2020 at 12:04 PM Tzu-Li (Gordon) Tai <tz...@apache.org>
wrote:

> The flink-training repository has been created:
> https://github.com/apache/flink-training
>
> On Mon, Apr 20, 2020 at 12:47 AM David Anderson <da...@ververica.com>
> wrote:
>
>> How about calling the component "Documentation / Training"?
>>
>>
>> Sounds good. Thanks!
>>
>> *David Anderson* | Training Coordinator
>>
>> Follow us @VervericaData
>> --
>> Join Flink Forward - The Apache Flink Conference
>> Stream Processing | Event Driven | Real Time
>>
>>
>> On Sun, Apr 19, 2020 at 10:57 AM Robert Metzger <rm...@apache.org>
>> wrote:
>>
>>> @David: I'm happy to create a component in JIRA.
>>>
>>> How about calling the component "Documentation / Training"?
>>>
>>> On Sat, Apr 18, 2020 at 11:14 AM Tzu-Li (Gordon) Tai <
>>> tzulitai@apache.org> wrote:
>>>
>>>> I can help create the flink-training repository.
>>>>
>>>> Before I do that though, I'd like to wait a bit on [1] over the next
>>>> few days.
>>>>
>>>> There's a similar case there, where we want to create a new repo to
>>>> host existing codebases,
>>>> but may not necessarily need a vote because the contribution is not a
>>>> direction-shifting change to the project.
>>>>
>>>> If consensus passes there, I'll create the repo for flink-training as
>>>> well.
>>>>
>>>> [1]
>>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Creating-a-new-repo-to-host-Flink-benchmarks-td40037.html
>>>>
>>>> On Sat, Apr 18, 2020 at 4:09 AM Seth Wiesman <sj...@gmail.com>
>>>> wrote:
>>>>
>>>>> I'm not able to create the repository. INFRA requests that new repos
>>>>> are
>>>>> created using https://selfserve.apache.org/ and it requires a PMC
>>>>> member.
>>>>> It looks like it should take someone 30 seconds.
>>>>>
>>>>> Seth
>>>>>
>>>>> On Fri, Apr 17, 2020 at 9:07 AM David Anderson <da...@ververica.com>
>>>>> wrote:
>>>>>
>>>>> > @Robert, do you think we could create a new flink-training component
>>>>> in
>>>>> > Jira for tracking tickets related to the flink-training content?
>>>>> >
>>>>> > David
>>>>> >
>>>>> > On Thu, Apr 16, 2020 at 10:54 AM David Anderson <david@ververica.com
>>>>> >
>>>>> > wrote:
>>>>> >
>>>>> >> > I am happy to get the repo created for you.
>>>>> >>
>>>>> >> Thank you, @seth. I think we are also going to want a new
>>>>> flink-training
>>>>> >> component in Jira. Maybe you can help there too?
>>>>> >>
>>>>> >> > If we go with the documentation (vs flink.apache.org) do you
>>>>> think we
>>>>> >> > should remove any of the existing content? There is already a
>>>>> getting
>>>>> >> > started section with quickstarts and walkthroughs and a concepts
>>>>> >> section.
>>>>> >> > In particular, the concepts section today is not complete and
>>>>> almost
>>>>> >> every
>>>>> >> > page on master contains multiple TODOs.
>>>>> >>
>>>>> >> I'll look at this, and also coordinate with what Aljoscha is doing
>>>>> there.
>>>>> >> But yes, there is room for improvement in this part of the docs, so
>>>>> I'm
>>>>> >> expecting to be able to help with that.
>>>>> >>
>>>>> >> David
>>>>> >>
>>>>> >> On Wed, Apr 15, 2020 at 9:20 PM Seth Wiesman <sj...@gmail.com>
>>>>> wrote:
>>>>> >>
>>>>> >>> Hi David,
>>>>> >>>
>>>>> >>> I am happy to get the repo created for you.
>>>>> >>>
>>>>> >>> If we go with the documentation (vs flink.apache.org) do you
>>>>> think we
>>>>> >>> should remove any of the existing content? There is already a
>>>>> getting
>>>>> >>> started section with quickstarts and walkthroughs and a concepts
>>>>> section.
>>>>> >>> In particular, the concepts section today is not complete and
>>>>> almost
>>>>> >>> every
>>>>> >>> page on master contains multiple TODOs. I don't believe anyone is
>>>>> working
>>>>> >>> on these.  What do you think about replacing the current concepts
>>>>> section
>>>>> >>> with the training material? I just re-examined the training site
>>>>> and I
>>>>> >>> believe it covers the same material as concepts but better. Of
>>>>> course, we
>>>>> >>> would salvage anything worth keeping, like the glossary.
>>>>> >>>
>>>>> >>> Seth
>>>>> >>>
>>>>> >>> On Wed, Apr 15, 2020 at 2:02 PM David Anderson <
>>>>> david@ververica.com>
>>>>> >>> wrote:
>>>>> >>>
>>>>> >>> > Thank you all for the very positive response to our proposal to
>>>>> >>> contribute
>>>>> >>> > the training materials that have been at training.ververica.com
>>>>> to the
>>>>> >>> > Apache Flink project. Now I’d like to begin the more detailed
>>>>> >>> discussion of
>>>>> >>> > how to go about this.
>>>>> >>> >
>>>>> >>> > In that earlier thread I mentioned that we were thinking of
>>>>> merging the
>>>>> >>> > markdown-based web pages into flink.apache.org, and to add the
>>>>> >>> exercises
>>>>> >>> > to
>>>>> >>> > flink-playgrounds. This was based on thinking that it would be
>>>>> >>> something of
>>>>> >>> > a maintenance headache to add the website content into the docs,
>>>>> where
>>>>> >>> it
>>>>> >>> > would have to be versioned.
>>>>> >>> >
>>>>> >>> > Since then, a better approach has been suggested:
>>>>> >>> >
>>>>> >>> > We already have quite a bit of “getting started” material in the
>>>>> docs:
>>>>> >>> Code
>>>>> >>> > Walkthroughs, Docker Playgrounds, Tutorials, and Examples.
>>>>> Having a
>>>>> >>> second
>>>>> >>> > location (namely flink.apache.org) where this kind of content
>>>>> could be
>>>>> >>> > found doesn’t seem ideal. So let’s go ahead and add the
>>>>> expository
>>>>> >>> content
>>>>> >>> > from the training materials to the documentation, with pointers
>>>>> into
>>>>> >>> the
>>>>> >>> > rest of the docs (which are already present in the training),
>>>>> and with
>>>>> >>> > pointers to the exercises (rather than including the exercise
>>>>> >>> descriptions
>>>>> >>> > in the docs). This will keep the content that will need more
>>>>> frequent
>>>>> >>> > revision out of the documentation.
>>>>> >>> >
>>>>> >>> > Then let’s create a new repo -- named flink-training -- that
>>>>> contains
>>>>> >>> the
>>>>> >>> > exercises, the solutions, and the tests that go with them, PLUS
>>>>> all of
>>>>> >>> the
>>>>> >>> > material that describes how to get setup to do the exercises, the
>>>>> >>> > explanations for each exercise, and accompanying discussion
>>>>> material
>>>>> >>> that
>>>>> >>> > should be read after doing each exercise. Note that the exercise
>>>>> >>> solutions
>>>>> >>> > already have tests, and Travis is already being used for CI on
>>>>> the
>>>>> >>> existing
>>>>> >>> > project, so CI shouldn’t be an issue.
>>>>> >>> >
>>>>> >>> > Action Item: would a committer or PMC member kindly volunteer to
>>>>> help
>>>>> >>> with
>>>>> >>> > creating this new flink-training repo?
>>>>> >>> >
>>>>> >>> > With the content refactored in this way, I believe ongoing
>>>>> maintenance
>>>>> >>> > won’t be much trouble. With each new Flink release I’ve been
>>>>> updating
>>>>> >>> the
>>>>> >>> > exercises to build against the latest release, and to avoid any
>>>>> newly
>>>>> >>> > deprecated parts of the API. But since these exercises are
>>>>> focused on
>>>>> >>> the
>>>>> >>> > most basic parts of the API, that hasn’t been difficult. As for
>>>>> content
>>>>> >>> > from the training website that would move into the docs, this
>>>>> content
>>>>> >>> is
>>>>> >>> > much more stable, and has only needed a gentle revision every
>>>>> year or
>>>>> >>> two.
>>>>> >>> >
>>>>> >>> > Regards,
>>>>> >>> > David
>>>>> >>> >
>>>>> >>>
>>>>> >>
>>>>>
>>>>

Re: [DISCUSS] Integration of training materials into Apache Flink

["Tzu-Li (Gordon) Tai" <tz...@apache.org>]

The flink-training repository has been created:
https://github.com/apache/flink-training

On Mon, Apr 20, 2020 at 12:47 AM David Anderson <da...@ververica.com> wrote:

> How about calling the component "Documentation / Training"?
>
>
> Sounds good. Thanks!
>
> *David Anderson* | Training Coordinator
>
> Follow us @VervericaData
> --
> Join Flink Forward - The Apache Flink Conference
> Stream Processing | Event Driven | Real Time
>
>
> On Sun, Apr 19, 2020 at 10:57 AM Robert Metzger <rm...@apache.org>
> wrote:
>
>> @David: I'm happy to create a component in JIRA.
>>
>> How about calling the component "Documentation / Training"?
>>
>> On Sat, Apr 18, 2020 at 11:14 AM Tzu-Li (Gordon) Tai <tz...@apache.org>
>> wrote:
>>
>>> I can help create the flink-training repository.
>>>
>>> Before I do that though, I'd like to wait a bit on [1] over the next few
>>> days.
>>>
>>> There's a similar case there, where we want to create a new repo to host
>>> existing codebases,
>>> but may not necessarily need a vote because the contribution is not a
>>> direction-shifting change to the project.
>>>
>>> If consensus passes there, I'll create the repo for flink-training as
>>> well.
>>>
>>> [1]
>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Creating-a-new-repo-to-host-Flink-benchmarks-td40037.html
>>>
>>> On Sat, Apr 18, 2020 at 4:09 AM Seth Wiesman <sj...@gmail.com>
>>> wrote:
>>>
>>>> I'm not able to create the repository. INFRA requests that new repos are
>>>> created using https://selfserve.apache.org/ and it requires a PMC
>>>> member.
>>>> It looks like it should take someone 30 seconds.
>>>>
>>>> Seth
>>>>
>>>> On Fri, Apr 17, 2020 at 9:07 AM David Anderson <da...@ververica.com>
>>>> wrote:
>>>>
>>>> > @Robert, do you think we could create a new flink-training component
>>>> in
>>>> > Jira for tracking tickets related to the flink-training content?
>>>> >
>>>> > David
>>>> >
>>>> > On Thu, Apr 16, 2020 at 10:54 AM David Anderson <da...@ververica.com>
>>>> > wrote:
>>>> >
>>>> >> > I am happy to get the repo created for you.
>>>> >>
>>>> >> Thank you, @seth. I think we are also going to want a new
>>>> flink-training
>>>> >> component in Jira. Maybe you can help there too?
>>>> >>
>>>> >> > If we go with the documentation (vs flink.apache.org) do you
>>>> think we
>>>> >> > should remove any of the existing content? There is already a
>>>> getting
>>>> >> > started section with quickstarts and walkthroughs and a concepts
>>>> >> section.
>>>> >> > In particular, the concepts section today is not complete and
>>>> almost
>>>> >> every
>>>> >> > page on master contains multiple TODOs.
>>>> >>
>>>> >> I'll look at this, and also coordinate with what Aljoscha is doing
>>>> there.
>>>> >> But yes, there is room for improvement in this part of the docs, so
>>>> I'm
>>>> >> expecting to be able to help with that.
>>>> >>
>>>> >> David
>>>> >>
>>>> >> On Wed, Apr 15, 2020 at 9:20 PM Seth Wiesman <sj...@gmail.com>
>>>> wrote:
>>>> >>
>>>> >>> Hi David,
>>>> >>>
>>>> >>> I am happy to get the repo created for you.
>>>> >>>
>>>> >>> If we go with the documentation (vs flink.apache.org) do you think
>>>> we
>>>> >>> should remove any of the existing content? There is already a
>>>> getting
>>>> >>> started section with quickstarts and walkthroughs and a concepts
>>>> section.
>>>> >>> In particular, the concepts section today is not complete and almost
>>>> >>> every
>>>> >>> page on master contains multiple TODOs. I don't believe anyone is
>>>> working
>>>> >>> on these.  What do you think about replacing the current concepts
>>>> section
>>>> >>> with the training material? I just re-examined the training site
>>>> and I
>>>> >>> believe it covers the same material as concepts but better. Of
>>>> course, we
>>>> >>> would salvage anything worth keeping, like the glossary.
>>>> >>>
>>>> >>> Seth
>>>> >>>
>>>> >>> On Wed, Apr 15, 2020 at 2:02 PM David Anderson <david@ververica.com
>>>> >
>>>> >>> wrote:
>>>> >>>
>>>> >>> > Thank you all for the very positive response to our proposal to
>>>> >>> contribute
>>>> >>> > the training materials that have been at training.ververica.com
>>>> to the
>>>> >>> > Apache Flink project. Now I’d like to begin the more detailed
>>>> >>> discussion of
>>>> >>> > how to go about this.
>>>> >>> >
>>>> >>> > In that earlier thread I mentioned that we were thinking of
>>>> merging the
>>>> >>> > markdown-based web pages into flink.apache.org, and to add the
>>>> >>> exercises
>>>> >>> > to
>>>> >>> > flink-playgrounds. This was based on thinking that it would be
>>>> >>> something of
>>>> >>> > a maintenance headache to add the website content into the docs,
>>>> where
>>>> >>> it
>>>> >>> > would have to be versioned.
>>>> >>> >
>>>> >>> > Since then, a better approach has been suggested:
>>>> >>> >
>>>> >>> > We already have quite a bit of “getting started” material in the
>>>> docs:
>>>> >>> Code
>>>> >>> > Walkthroughs, Docker Playgrounds, Tutorials, and Examples. Having
>>>> a
>>>> >>> second
>>>> >>> > location (namely flink.apache.org) where this kind of content
>>>> could be
>>>> >>> > found doesn’t seem ideal. So let’s go ahead and add the expository
>>>> >>> content
>>>> >>> > from the training materials to the documentation, with pointers
>>>> into
>>>> >>> the
>>>> >>> > rest of the docs (which are already present in the training), and
>>>> with
>>>> >>> > pointers to the exercises (rather than including the exercise
>>>> >>> descriptions
>>>> >>> > in the docs). This will keep the content that will need more
>>>> frequent
>>>> >>> > revision out of the documentation.
>>>> >>> >
>>>> >>> > Then let’s create a new repo -- named flink-training -- that
>>>> contains
>>>> >>> the
>>>> >>> > exercises, the solutions, and the tests that go with them, PLUS
>>>> all of
>>>> >>> the
>>>> >>> > material that describes how to get setup to do the exercises, the
>>>> >>> > explanations for each exercise, and accompanying discussion
>>>> material
>>>> >>> that
>>>> >>> > should be read after doing each exercise. Note that the exercise
>>>> >>> solutions
>>>> >>> > already have tests, and Travis is already being used for CI on the
>>>> >>> existing
>>>> >>> > project, so CI shouldn’t be an issue.
>>>> >>> >
>>>> >>> > Action Item: would a committer or PMC member kindly volunteer to
>>>> help
>>>> >>> with
>>>> >>> > creating this new flink-training repo?
>>>> >>> >
>>>> >>> > With the content refactored in this way, I believe ongoing
>>>> maintenance
>>>> >>> > won’t be much trouble. With each new Flink release I’ve been
>>>> updating
>>>> >>> the
>>>> >>> > exercises to build against the latest release, and to avoid any
>>>> newly
>>>> >>> > deprecated parts of the API. But since these exercises are
>>>> focused on
>>>> >>> the
>>>> >>> > most basic parts of the API, that hasn’t been difficult. As for
>>>> content
>>>> >>> > from the training website that would move into the docs, this
>>>> content
>>>> >>> is
>>>> >>> > much more stable, and has only needed a gentle revision every
>>>> year or
>>>> >>> two.
>>>> >>> >
>>>> >>> > Regards,
>>>> >>> > David
>>>> >>> >
>>>> >>>
>>>> >>
>>>>
>>>

Re: [DISCUSS] Integration of training materials into Apache Flink

[David Anderson <da...@ververica.com>]

>
> How about calling the component "Documentation / Training"?


Sounds good. Thanks!

*David Anderson* | Training Coordinator

Follow us @VervericaData
--
Join Flink Forward - The Apache Flink Conference
Stream Processing | Event Driven | Real Time


On Sun, Apr 19, 2020 at 10:57 AM Robert Metzger <rm...@apache.org> wrote:

> @David: I'm happy to create a component in JIRA.
>
> How about calling the component "Documentation / Training"?
>
> On Sat, Apr 18, 2020 at 11:14 AM Tzu-Li (Gordon) Tai <tz...@apache.org>
> wrote:
>
>> I can help create the flink-training repository.
>>
>> Before I do that though, I'd like to wait a bit on [1] over the next few
>> days.
>>
>> There's a similar case there, where we want to create a new repo to host
>> existing codebases,
>> but may not necessarily need a vote because the contribution is not a
>> direction-shifting change to the project.
>>
>> If consensus passes there, I'll create the repo for flink-training as
>> well.
>>
>> [1]
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Creating-a-new-repo-to-host-Flink-benchmarks-td40037.html
>>
>> On Sat, Apr 18, 2020 at 4:09 AM Seth Wiesman <sj...@gmail.com> wrote:
>>
>>> I'm not able to create the repository. INFRA requests that new repos are
>>> created using https://selfserve.apache.org/ and it requires a PMC
>>> member.
>>> It looks like it should take someone 30 seconds.
>>>
>>> Seth
>>>
>>> On Fri, Apr 17, 2020 at 9:07 AM David Anderson <da...@ververica.com>
>>> wrote:
>>>
>>> > @Robert, do you think we could create a new flink-training component in
>>> > Jira for tracking tickets related to the flink-training content?
>>> >
>>> > David
>>> >
>>> > On Thu, Apr 16, 2020 at 10:54 AM David Anderson <da...@ververica.com>
>>> > wrote:
>>> >
>>> >> > I am happy to get the repo created for you.
>>> >>
>>> >> Thank you, @seth. I think we are also going to want a new
>>> flink-training
>>> >> component in Jira. Maybe you can help there too?
>>> >>
>>> >> > If we go with the documentation (vs flink.apache.org) do you think
>>> we
>>> >> > should remove any of the existing content? There is already a
>>> getting
>>> >> > started section with quickstarts and walkthroughs and a concepts
>>> >> section.
>>> >> > In particular, the concepts section today is not complete and almost
>>> >> every
>>> >> > page on master contains multiple TODOs.
>>> >>
>>> >> I'll look at this, and also coordinate with what Aljoscha is doing
>>> there.
>>> >> But yes, there is room for improvement in this part of the docs, so
>>> I'm
>>> >> expecting to be able to help with that.
>>> >>
>>> >> David
>>> >>
>>> >> On Wed, Apr 15, 2020 at 9:20 PM Seth Wiesman <sj...@gmail.com>
>>> wrote:
>>> >>
>>> >>> Hi David,
>>> >>>
>>> >>> I am happy to get the repo created for you.
>>> >>>
>>> >>> If we go with the documentation (vs flink.apache.org) do you think
>>> we
>>> >>> should remove any of the existing content? There is already a getting
>>> >>> started section with quickstarts and walkthroughs and a concepts
>>> section.
>>> >>> In particular, the concepts section today is not complete and almost
>>> >>> every
>>> >>> page on master contains multiple TODOs. I don't believe anyone is
>>> working
>>> >>> on these.  What do you think about replacing the current concepts
>>> section
>>> >>> with the training material? I just re-examined the training site and
>>> I
>>> >>> believe it covers the same material as concepts but better. Of
>>> course, we
>>> >>> would salvage anything worth keeping, like the glossary.
>>> >>>
>>> >>> Seth
>>> >>>
>>> >>> On Wed, Apr 15, 2020 at 2:02 PM David Anderson <da...@ververica.com>
>>> >>> wrote:
>>> >>>
>>> >>> > Thank you all for the very positive response to our proposal to
>>> >>> contribute
>>> >>> > the training materials that have been at training.ververica.com
>>> to the
>>> >>> > Apache Flink project. Now I’d like to begin the more detailed
>>> >>> discussion of
>>> >>> > how to go about this.
>>> >>> >
>>> >>> > In that earlier thread I mentioned that we were thinking of
>>> merging the
>>> >>> > markdown-based web pages into flink.apache.org, and to add the
>>> >>> exercises
>>> >>> > to
>>> >>> > flink-playgrounds. This was based on thinking that it would be
>>> >>> something of
>>> >>> > a maintenance headache to add the website content into the docs,
>>> where
>>> >>> it
>>> >>> > would have to be versioned.
>>> >>> >
>>> >>> > Since then, a better approach has been suggested:
>>> >>> >
>>> >>> > We already have quite a bit of “getting started” material in the
>>> docs:
>>> >>> Code
>>> >>> > Walkthroughs, Docker Playgrounds, Tutorials, and Examples. Having a
>>> >>> second
>>> >>> > location (namely flink.apache.org) where this kind of content
>>> could be
>>> >>> > found doesn’t seem ideal. So let’s go ahead and add the expository
>>> >>> content
>>> >>> > from the training materials to the documentation, with pointers
>>> into
>>> >>> the
>>> >>> > rest of the docs (which are already present in the training), and
>>> with
>>> >>> > pointers to the exercises (rather than including the exercise
>>> >>> descriptions
>>> >>> > in the docs). This will keep the content that will need more
>>> frequent
>>> >>> > revision out of the documentation.
>>> >>> >
>>> >>> > Then let’s create a new repo -- named flink-training -- that
>>> contains
>>> >>> the
>>> >>> > exercises, the solutions, and the tests that go with them, PLUS
>>> all of
>>> >>> the
>>> >>> > material that describes how to get setup to do the exercises, the
>>> >>> > explanations for each exercise, and accompanying discussion
>>> material
>>> >>> that
>>> >>> > should be read after doing each exercise. Note that the exercise
>>> >>> solutions
>>> >>> > already have tests, and Travis is already being used for CI on the
>>> >>> existing
>>> >>> > project, so CI shouldn’t be an issue.
>>> >>> >
>>> >>> > Action Item: would a committer or PMC member kindly volunteer to
>>> help
>>> >>> with
>>> >>> > creating this new flink-training repo?
>>> >>> >
>>> >>> > With the content refactored in this way, I believe ongoing
>>> maintenance
>>> >>> > won’t be much trouble. With each new Flink release I’ve been
>>> updating
>>> >>> the
>>> >>> > exercises to build against the latest release, and to avoid any
>>> newly
>>> >>> > deprecated parts of the API. But since these exercises are focused
>>> on
>>> >>> the
>>> >>> > most basic parts of the API, that hasn’t been difficult. As for
>>> content
>>> >>> > from the training website that would move into the docs, this
>>> content
>>> >>> is
>>> >>> > much more stable, and has only needed a gentle revision every year
>>> or
>>> >>> two.
>>> >>> >
>>> >>> > Regards,
>>> >>> > David
>>> >>> >
>>> >>>
>>> >>
>>>
>>

Re: [DISCUSS] Integration of training materials into Apache Flink

[Robert Metzger <rm...@apache.org>]

@David: I'm happy to create a component in JIRA.

How about calling the component "Documentation / Training"?

On Sat, Apr 18, 2020 at 11:14 AM Tzu-Li (Gordon) Tai <tz...@apache.org>
wrote:

> I can help create the flink-training repository.
>
> Before I do that though, I'd like to wait a bit on [1] over the next few
> days.
>
> There's a similar case there, where we want to create a new repo to host
> existing codebases,
> but may not necessarily need a vote because the contribution is not a
> direction-shifting change to the project.
>
> If consensus passes there, I'll create the repo for flink-training as well.
>
> [1]
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Creating-a-new-repo-to-host-Flink-benchmarks-td40037.html
>
> On Sat, Apr 18, 2020 at 4:09 AM Seth Wiesman <sj...@gmail.com> wrote:
>
>> I'm not able to create the repository. INFRA requests that new repos are
>> created using https://selfserve.apache.org/ and it requires a PMC member.
>> It looks like it should take someone 30 seconds.
>>
>> Seth
>>
>> On Fri, Apr 17, 2020 at 9:07 AM David Anderson <da...@ververica.com>
>> wrote:
>>
>> > @Robert, do you think we could create a new flink-training component in
>> > Jira for tracking tickets related to the flink-training content?
>> >
>> > David
>> >
>> > On Thu, Apr 16, 2020 at 10:54 AM David Anderson <da...@ververica.com>
>> > wrote:
>> >
>> >> > I am happy to get the repo created for you.
>> >>
>> >> Thank you, @seth. I think we are also going to want a new
>> flink-training
>> >> component in Jira. Maybe you can help there too?
>> >>
>> >> > If we go with the documentation (vs flink.apache.org) do you think
>> we
>> >> > should remove any of the existing content? There is already a getting
>> >> > started section with quickstarts and walkthroughs and a concepts
>> >> section.
>> >> > In particular, the concepts section today is not complete and almost
>> >> every
>> >> > page on master contains multiple TODOs.
>> >>
>> >> I'll look at this, and also coordinate with what Aljoscha is doing
>> there.
>> >> But yes, there is room for improvement in this part of the docs, so I'm
>> >> expecting to be able to help with that.
>> >>
>> >> David
>> >>
>> >> On Wed, Apr 15, 2020 at 9:20 PM Seth Wiesman <sj...@gmail.com>
>> wrote:
>> >>
>> >>> Hi David,
>> >>>
>> >>> I am happy to get the repo created for you.
>> >>>
>> >>> If we go with the documentation (vs flink.apache.org) do you think we
>> >>> should remove any of the existing content? There is already a getting
>> >>> started section with quickstarts and walkthroughs and a concepts
>> section.
>> >>> In particular, the concepts section today is not complete and almost
>> >>> every
>> >>> page on master contains multiple TODOs. I don't believe anyone is
>> working
>> >>> on these.  What do you think about replacing the current concepts
>> section
>> >>> with the training material? I just re-examined the training site and I
>> >>> believe it covers the same material as concepts but better. Of
>> course, we
>> >>> would salvage anything worth keeping, like the glossary.
>> >>>
>> >>> Seth
>> >>>
>> >>> On Wed, Apr 15, 2020 at 2:02 PM David Anderson <da...@ververica.com>
>> >>> wrote:
>> >>>
>> >>> > Thank you all for the very positive response to our proposal to
>> >>> contribute
>> >>> > the training materials that have been at training.ververica.com to
>> the
>> >>> > Apache Flink project. Now I’d like to begin the more detailed
>> >>> discussion of
>> >>> > how to go about this.
>> >>> >
>> >>> > In that earlier thread I mentioned that we were thinking of merging
>> the
>> >>> > markdown-based web pages into flink.apache.org, and to add the
>> >>> exercises
>> >>> > to
>> >>> > flink-playgrounds. This was based on thinking that it would be
>> >>> something of
>> >>> > a maintenance headache to add the website content into the docs,
>> where
>> >>> it
>> >>> > would have to be versioned.
>> >>> >
>> >>> > Since then, a better approach has been suggested:
>> >>> >
>> >>> > We already have quite a bit of “getting started” material in the
>> docs:
>> >>> Code
>> >>> > Walkthroughs, Docker Playgrounds, Tutorials, and Examples. Having a
>> >>> second
>> >>> > location (namely flink.apache.org) where this kind of content
>> could be
>> >>> > found doesn’t seem ideal. So let’s go ahead and add the expository
>> >>> content
>> >>> > from the training materials to the documentation, with pointers into
>> >>> the
>> >>> > rest of the docs (which are already present in the training), and
>> with
>> >>> > pointers to the exercises (rather than including the exercise
>> >>> descriptions
>> >>> > in the docs). This will keep the content that will need more
>> frequent
>> >>> > revision out of the documentation.
>> >>> >
>> >>> > Then let’s create a new repo -- named flink-training -- that
>> contains
>> >>> the
>> >>> > exercises, the solutions, and the tests that go with them, PLUS all
>> of
>> >>> the
>> >>> > material that describes how to get setup to do the exercises, the
>> >>> > explanations for each exercise, and accompanying discussion material
>> >>> that
>> >>> > should be read after doing each exercise. Note that the exercise
>> >>> solutions
>> >>> > already have tests, and Travis is already being used for CI on the
>> >>> existing
>> >>> > project, so CI shouldn’t be an issue.
>> >>> >
>> >>> > Action Item: would a committer or PMC member kindly volunteer to
>> help
>> >>> with
>> >>> > creating this new flink-training repo?
>> >>> >
>> >>> > With the content refactored in this way, I believe ongoing
>> maintenance
>> >>> > won’t be much trouble. With each new Flink release I’ve been
>> updating
>> >>> the
>> >>> > exercises to build against the latest release, and to avoid any
>> newly
>> >>> > deprecated parts of the API. But since these exercises are focused
>> on
>> >>> the
>> >>> > most basic parts of the API, that hasn’t been difficult. As for
>> content
>> >>> > from the training website that would move into the docs, this
>> content
>> >>> is
>> >>> > much more stable, and has only needed a gentle revision every year
>> or
>> >>> two.
>> >>> >
>> >>> > Regards,
>> >>> > David
>> >>> >
>> >>>
>> >>
>>
>

Re: [DISCUSS] Integration of training materials into Apache Flink

["Tzu-Li (Gordon) Tai" <tz...@apache.org>]

I can help create the flink-training repository.

Before I do that though, I'd like to wait a bit on [1] over the next few
days.

There's a similar case there, where we want to create a new repo to host
existing codebases,
but may not necessarily need a vote because the contribution is not a
direction-shifting change to the project.

If consensus passes there, I'll create the repo for flink-training as well.

[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Creating-a-new-repo-to-host-Flink-benchmarks-td40037.html

On Sat, Apr 18, 2020 at 4:09 AM Seth Wiesman <sj...@gmail.com> wrote:

> I'm not able to create the repository. INFRA requests that new repos are
> created using https://selfserve.apache.org/ and it requires a PMC member.
> It looks like it should take someone 30 seconds.
>
> Seth
>
> On Fri, Apr 17, 2020 at 9:07 AM David Anderson <da...@ververica.com>
> wrote:
>
> > @Robert, do you think we could create a new flink-training component in
> > Jira for tracking tickets related to the flink-training content?
> >
> > David
> >
> > On Thu, Apr 16, 2020 at 10:54 AM David Anderson <da...@ververica.com>
> > wrote:
> >
> >> > I am happy to get the repo created for you.
> >>
> >> Thank you, @seth. I think we are also going to want a new flink-training
> >> component in Jira. Maybe you can help there too?
> >>
> >> > If we go with the documentation (vs flink.apache.org) do you think we
> >> > should remove any of the existing content? There is already a getting
> >> > started section with quickstarts and walkthroughs and a concepts
> >> section.
> >> > In particular, the concepts section today is not complete and almost
> >> every
> >> > page on master contains multiple TODOs.
> >>
> >> I'll look at this, and also coordinate with what Aljoscha is doing
> there.
> >> But yes, there is room for improvement in this part of the docs, so I'm
> >> expecting to be able to help with that.
> >>
> >> David
> >>
> >> On Wed, Apr 15, 2020 at 9:20 PM Seth Wiesman <sj...@gmail.com>
> wrote:
> >>
> >>> Hi David,
> >>>
> >>> I am happy to get the repo created for you.
> >>>
> >>> If we go with the documentation (vs flink.apache.org) do you think we
> >>> should remove any of the existing content? There is already a getting
> >>> started section with quickstarts and walkthroughs and a concepts
> section.
> >>> In particular, the concepts section today is not complete and almost
> >>> every
> >>> page on master contains multiple TODOs. I don't believe anyone is
> working
> >>> on these.  What do you think about replacing the current concepts
> section
> >>> with the training material? I just re-examined the training site and I
> >>> believe it covers the same material as concepts but better. Of course,
> we
> >>> would salvage anything worth keeping, like the glossary.
> >>>
> >>> Seth
> >>>
> >>> On Wed, Apr 15, 2020 at 2:02 PM David Anderson <da...@ververica.com>
> >>> wrote:
> >>>
> >>> > Thank you all for the very positive response to our proposal to
> >>> contribute
> >>> > the training materials that have been at training.ververica.com to
> the
> >>> > Apache Flink project. Now I’d like to begin the more detailed
> >>> discussion of
> >>> > how to go about this.
> >>> >
> >>> > In that earlier thread I mentioned that we were thinking of merging
> the
> >>> > markdown-based web pages into flink.apache.org, and to add the
> >>> exercises
> >>> > to
> >>> > flink-playgrounds. This was based on thinking that it would be
> >>> something of
> >>> > a maintenance headache to add the website content into the docs,
> where
> >>> it
> >>> > would have to be versioned.
> >>> >
> >>> > Since then, a better approach has been suggested:
> >>> >
> >>> > We already have quite a bit of “getting started” material in the
> docs:
> >>> Code
> >>> > Walkthroughs, Docker Playgrounds, Tutorials, and Examples. Having a
> >>> second
> >>> > location (namely flink.apache.org) where this kind of content could
> be
> >>> > found doesn’t seem ideal. So let’s go ahead and add the expository
> >>> content
> >>> > from the training materials to the documentation, with pointers into
> >>> the
> >>> > rest of the docs (which are already present in the training), and
> with
> >>> > pointers to the exercises (rather than including the exercise
> >>> descriptions
> >>> > in the docs). This will keep the content that will need more frequent
> >>> > revision out of the documentation.
> >>> >
> >>> > Then let’s create a new repo -- named flink-training -- that contains
> >>> the
> >>> > exercises, the solutions, and the tests that go with them, PLUS all
> of
> >>> the
> >>> > material that describes how to get setup to do the exercises, the
> >>> > explanations for each exercise, and accompanying discussion material
> >>> that
> >>> > should be read after doing each exercise. Note that the exercise
> >>> solutions
> >>> > already have tests, and Travis is already being used for CI on the
> >>> existing
> >>> > project, so CI shouldn’t be an issue.
> >>> >
> >>> > Action Item: would a committer or PMC member kindly volunteer to help
> >>> with
> >>> > creating this new flink-training repo?
> >>> >
> >>> > With the content refactored in this way, I believe ongoing
> maintenance
> >>> > won’t be much trouble. With each new Flink release I’ve been updating
> >>> the
> >>> > exercises to build against the latest release, and to avoid any newly
> >>> > deprecated parts of the API. But since these exercises are focused on
> >>> the
> >>> > most basic parts of the API, that hasn’t been difficult. As for
> content
> >>> > from the training website that would move into the docs, this content
> >>> is
> >>> > much more stable, and has only needed a gentle revision every year or
> >>> two.
> >>> >
> >>> > Regards,
> >>> > David
> >>> >
> >>>
> >>
>

Re: [DISCUSS] Integration of training materials into Apache Flink

[Seth Wiesman <sj...@gmail.com>]

I'm not able to create the repository. INFRA requests that new repos are
created using https://selfserve.apache.org/ and it requires a PMC member.
It looks like it should take someone 30 seconds.

Seth

On Fri, Apr 17, 2020 at 9:07 AM David Anderson <da...@ververica.com> wrote:

> @Robert, do you think we could create a new flink-training component in
> Jira for tracking tickets related to the flink-training content?
>
> David
>
> On Thu, Apr 16, 2020 at 10:54 AM David Anderson <da...@ververica.com>
> wrote:
>
>> > I am happy to get the repo created for you.
>>
>> Thank you, @seth. I think we are also going to want a new flink-training
>> component in Jira. Maybe you can help there too?
>>
>> > If we go with the documentation (vs flink.apache.org) do you think we
>> > should remove any of the existing content? There is already a getting
>> > started section with quickstarts and walkthroughs and a concepts
>> section.
>> > In particular, the concepts section today is not complete and almost
>> every
>> > page on master contains multiple TODOs.
>>
>> I'll look at this, and also coordinate with what Aljoscha is doing there.
>> But yes, there is room for improvement in this part of the docs, so I'm
>> expecting to be able to help with that.
>>
>> David
>>
>> On Wed, Apr 15, 2020 at 9:20 PM Seth Wiesman <sj...@gmail.com> wrote:
>>
>>> Hi David,
>>>
>>> I am happy to get the repo created for you.
>>>
>>> If we go with the documentation (vs flink.apache.org) do you think we
>>> should remove any of the existing content? There is already a getting
>>> started section with quickstarts and walkthroughs and a concepts section.
>>> In particular, the concepts section today is not complete and almost
>>> every
>>> page on master contains multiple TODOs. I don't believe anyone is working
>>> on these.  What do you think about replacing the current concepts section
>>> with the training material? I just re-examined the training site and I
>>> believe it covers the same material as concepts but better. Of course, we
>>> would salvage anything worth keeping, like the glossary.
>>>
>>> Seth
>>>
>>> On Wed, Apr 15, 2020 at 2:02 PM David Anderson <da...@ververica.com>
>>> wrote:
>>>
>>> > Thank you all for the very positive response to our proposal to
>>> contribute
>>> > the training materials that have been at training.ververica.com to the
>>> > Apache Flink project. Now I’d like to begin the more detailed
>>> discussion of
>>> > how to go about this.
>>> >
>>> > In that earlier thread I mentioned that we were thinking of merging the
>>> > markdown-based web pages into flink.apache.org, and to add the
>>> exercises
>>> > to
>>> > flink-playgrounds. This was based on thinking that it would be
>>> something of
>>> > a maintenance headache to add the website content into the docs, where
>>> it
>>> > would have to be versioned.
>>> >
>>> > Since then, a better approach has been suggested:
>>> >
>>> > We already have quite a bit of “getting started” material in the docs:
>>> Code
>>> > Walkthroughs, Docker Playgrounds, Tutorials, and Examples. Having a
>>> second
>>> > location (namely flink.apache.org) where this kind of content could be
>>> > found doesn’t seem ideal. So let’s go ahead and add the expository
>>> content
>>> > from the training materials to the documentation, with pointers into
>>> the
>>> > rest of the docs (which are already present in the training), and with
>>> > pointers to the exercises (rather than including the exercise
>>> descriptions
>>> > in the docs). This will keep the content that will need more frequent
>>> > revision out of the documentation.
>>> >
>>> > Then let’s create a new repo -- named flink-training -- that contains
>>> the
>>> > exercises, the solutions, and the tests that go with them, PLUS all of
>>> the
>>> > material that describes how to get setup to do the exercises, the
>>> > explanations for each exercise, and accompanying discussion material
>>> that
>>> > should be read after doing each exercise. Note that the exercise
>>> solutions
>>> > already have tests, and Travis is already being used for CI on the
>>> existing
>>> > project, so CI shouldn’t be an issue.
>>> >
>>> > Action Item: would a committer or PMC member kindly volunteer to help
>>> with
>>> > creating this new flink-training repo?
>>> >
>>> > With the content refactored in this way, I believe ongoing maintenance
>>> > won’t be much trouble. With each new Flink release I’ve been updating
>>> the
>>> > exercises to build against the latest release, and to avoid any newly
>>> > deprecated parts of the API. But since these exercises are focused on
>>> the
>>> > most basic parts of the API, that hasn’t been difficult. As for content
>>> > from the training website that would move into the docs, this content
>>> is
>>> > much more stable, and has only needed a gentle revision every year or
>>> two.
>>> >
>>> > Regards,
>>> > David
>>> >
>>>
>>

Re: [DISCUSS] Integration of training materials into Apache Flink

[David Anderson <da...@ververica.com>]

@Robert, do you think we could create a new flink-training component in
Jira for tracking tickets related to the flink-training content?

David

On Thu, Apr 16, 2020 at 10:54 AM David Anderson <da...@ververica.com> wrote:

> > I am happy to get the repo created for you.
>
> Thank you, @seth. I think we are also going to want a new flink-training
> component in Jira. Maybe you can help there too?
>
> > If we go with the documentation (vs flink.apache.org) do you think we
> > should remove any of the existing content? There is already a getting
> > started section with quickstarts and walkthroughs and a concepts section.
> > In particular, the concepts section today is not complete and almost
> every
> > page on master contains multiple TODOs.
>
> I'll look at this, and also coordinate with what Aljoscha is doing there.
> But yes, there is room for improvement in this part of the docs, so I'm
> expecting to be able to help with that.
>
> David
>
> On Wed, Apr 15, 2020 at 9:20 PM Seth Wiesman <sj...@gmail.com> wrote:
>
>> Hi David,
>>
>> I am happy to get the repo created for you.
>>
>> If we go with the documentation (vs flink.apache.org) do you think we
>> should remove any of the existing content? There is already a getting
>> started section with quickstarts and walkthroughs and a concepts section.
>> In particular, the concepts section today is not complete and almost every
>> page on master contains multiple TODOs. I don't believe anyone is working
>> on these.  What do you think about replacing the current concepts section
>> with the training material? I just re-examined the training site and I
>> believe it covers the same material as concepts but better. Of course, we
>> would salvage anything worth keeping, like the glossary.
>>
>> Seth
>>
>> On Wed, Apr 15, 2020 at 2:02 PM David Anderson <da...@ververica.com>
>> wrote:
>>
>> > Thank you all for the very positive response to our proposal to
>> contribute
>> > the training materials that have been at training.ververica.com to the
>> > Apache Flink project. Now I’d like to begin the more detailed
>> discussion of
>> > how to go about this.
>> >
>> > In that earlier thread I mentioned that we were thinking of merging the
>> > markdown-based web pages into flink.apache.org, and to add the
>> exercises
>> > to
>> > flink-playgrounds. This was based on thinking that it would be
>> something of
>> > a maintenance headache to add the website content into the docs, where
>> it
>> > would have to be versioned.
>> >
>> > Since then, a better approach has been suggested:
>> >
>> > We already have quite a bit of “getting started” material in the docs:
>> Code
>> > Walkthroughs, Docker Playgrounds, Tutorials, and Examples. Having a
>> second
>> > location (namely flink.apache.org) where this kind of content could be
>> > found doesn’t seem ideal. So let’s go ahead and add the expository
>> content
>> > from the training materials to the documentation, with pointers into the
>> > rest of the docs (which are already present in the training), and with
>> > pointers to the exercises (rather than including the exercise
>> descriptions
>> > in the docs). This will keep the content that will need more frequent
>> > revision out of the documentation.
>> >
>> > Then let’s create a new repo -- named flink-training -- that contains
>> the
>> > exercises, the solutions, and the tests that go with them, PLUS all of
>> the
>> > material that describes how to get setup to do the exercises, the
>> > explanations for each exercise, and accompanying discussion material
>> that
>> > should be read after doing each exercise. Note that the exercise
>> solutions
>> > already have tests, and Travis is already being used for CI on the
>> existing
>> > project, so CI shouldn’t be an issue.
>> >
>> > Action Item: would a committer or PMC member kindly volunteer to help
>> with
>> > creating this new flink-training repo?
>> >
>> > With the content refactored in this way, I believe ongoing maintenance
>> > won’t be much trouble. With each new Flink release I’ve been updating
>> the
>> > exercises to build against the latest release, and to avoid any newly
>> > deprecated parts of the API. But since these exercises are focused on
>> the
>> > most basic parts of the API, that hasn’t been difficult. As for content
>> > from the training website that would move into the docs, this content is
>> > much more stable, and has only needed a gentle revision every year or
>> two.
>> >
>> > Regards,
>> > David
>> >
>>
>

Re: [DISCUSS] Integration of training materials into Apache Flink

[Aljoscha Krettek <al...@apache.org>]

I'd be very happy if someone took over that part of the documentation! 
There are open issues for the TODOs in the concepts section here: 
https://issues.apache.org/jira/browse/FLINK-12639. But feel free to 
comment there/close/re-arrange as you see fit. Maybe we use this thread 
and Jira to coordinate the efforts.

Aljoscha

On 16.04.20 10:54, David Anderson wrote:
>> I am happy to get the repo created for you.
> 
> Thank you, @seth. I think we are also going to want a new flink-training
> component in Jira. Maybe you can help there too?
> 
>> If we go with the documentation (vs flink.apache.org) do you think we
>> should remove any of the existing content? There is already a getting
>> started section with quickstarts and walkthroughs and a concepts section.
>> In particular, the concepts section today is not complete and almost every
>> page on master contains multiple TODOs.
> 
> I'll look at this, and also coordinate with what Aljoscha is doing there.
> But yes, there is room for improvement in this part of the docs, so I'm
> expecting to be able to help with that.
> 
> David
> 
> On Wed, Apr 15, 2020 at 9:20 PM Seth Wiesman <sj...@gmail.com> wrote:
> 
>> Hi David,
>>
>> I am happy to get the repo created for you.
>>
>> If we go with the documentation (vs flink.apache.org) do you think we
>> should remove any of the existing content? There is already a getting
>> started section with quickstarts and walkthroughs and a concepts section.
>> In particular, the concepts section today is not complete and almost every
>> page on master contains multiple TODOs. I don't believe anyone is working
>> on these.  What do you think about replacing the current concepts section
>> with the training material? I just re-examined the training site and I
>> believe it covers the same material as concepts but better. Of course, we
>> would salvage anything worth keeping, like the glossary.
>>
>> Seth
>>
>> On Wed, Apr 15, 2020 at 2:02 PM David Anderson <da...@ververica.com>
>> wrote:
>>
>>> Thank you all for the very positive response to our proposal to
>> contribute
>>> the training materials that have been at training.ververica.com to the
>>> Apache Flink project. Now I’d like to begin the more detailed discussion
>> of
>>> how to go about this.
>>>
>>> In that earlier thread I mentioned that we were thinking of merging the
>>> markdown-based web pages into flink.apache.org, and to add the exercises
>>> to
>>> flink-playgrounds. This was based on thinking that it would be something
>> of
>>> a maintenance headache to add the website content into the docs, where it
>>> would have to be versioned.
>>>
>>> Since then, a better approach has been suggested:
>>>
>>> We already have quite a bit of “getting started” material in the docs:
>> Code
>>> Walkthroughs, Docker Playgrounds, Tutorials, and Examples. Having a
>> second
>>> location (namely flink.apache.org) where this kind of content could be
>>> found doesn’t seem ideal. So let’s go ahead and add the expository
>> content
>>> from the training materials to the documentation, with pointers into the
>>> rest of the docs (which are already present in the training), and with
>>> pointers to the exercises (rather than including the exercise
>> descriptions
>>> in the docs). This will keep the content that will need more frequent
>>> revision out of the documentation.
>>>
>>> Then let’s create a new repo -- named flink-training -- that contains the
>>> exercises, the solutions, and the tests that go with them, PLUS all of
>> the
>>> material that describes how to get setup to do the exercises, the
>>> explanations for each exercise, and accompanying discussion material that
>>> should be read after doing each exercise. Note that the exercise
>> solutions
>>> already have tests, and Travis is already being used for CI on the
>> existing
>>> project, so CI shouldn’t be an issue.
>>>
>>> Action Item: would a committer or PMC member kindly volunteer to help
>> with
>>> creating this new flink-training repo?
>>>
>>> With the content refactored in this way, I believe ongoing maintenance
>>> won’t be much trouble. With each new Flink release I’ve been updating the
>>> exercises to build against the latest release, and to avoid any newly
>>> deprecated parts of the API. But since these exercises are focused on the
>>> most basic parts of the API, that hasn’t been difficult. As for content
>>> from the training website that would move into the docs, this content is
>>> much more stable, and has only needed a gentle revision every year or
>> two.
>>>
>>> Regards,
>>> David
>>>
>>
> 

Re: [DISCUSS] Integration of training materials into Apache Flink

[David Anderson <da...@ververica.com>]

> I am happy to get the repo created for you.

Thank you, @seth. I think we are also going to want a new flink-training
component in Jira. Maybe you can help there too?

> If we go with the documentation (vs flink.apache.org) do you think we
> should remove any of the existing content? There is already a getting
> started section with quickstarts and walkthroughs and a concepts section.
> In particular, the concepts section today is not complete and almost every
> page on master contains multiple TODOs.

I'll look at this, and also coordinate with what Aljoscha is doing there.
But yes, there is room for improvement in this part of the docs, so I'm
expecting to be able to help with that.

David

On Wed, Apr 15, 2020 at 9:20 PM Seth Wiesman <sj...@gmail.com> wrote:

> Hi David,
>
> I am happy to get the repo created for you.
>
> If we go with the documentation (vs flink.apache.org) do you think we
> should remove any of the existing content? There is already a getting
> started section with quickstarts and walkthroughs and a concepts section.
> In particular, the concepts section today is not complete and almost every
> page on master contains multiple TODOs. I don't believe anyone is working
> on these.  What do you think about replacing the current concepts section
> with the training material? I just re-examined the training site and I
> believe it covers the same material as concepts but better. Of course, we
> would salvage anything worth keeping, like the glossary.
>
> Seth
>
> On Wed, Apr 15, 2020 at 2:02 PM David Anderson <da...@ververica.com>
> wrote:
>
> > Thank you all for the very positive response to our proposal to
> contribute
> > the training materials that have been at training.ververica.com to the
> > Apache Flink project. Now I’d like to begin the more detailed discussion
> of
> > how to go about this.
> >
> > In that earlier thread I mentioned that we were thinking of merging the
> > markdown-based web pages into flink.apache.org, and to add the exercises
> > to
> > flink-playgrounds. This was based on thinking that it would be something
> of
> > a maintenance headache to add the website content into the docs, where it
> > would have to be versioned.
> >
> > Since then, a better approach has been suggested:
> >
> > We already have quite a bit of “getting started” material in the docs:
> Code
> > Walkthroughs, Docker Playgrounds, Tutorials, and Examples. Having a
> second
> > location (namely flink.apache.org) where this kind of content could be
> > found doesn’t seem ideal. So let’s go ahead and add the expository
> content
> > from the training materials to the documentation, with pointers into the
> > rest of the docs (which are already present in the training), and with
> > pointers to the exercises (rather than including the exercise
> descriptions
> > in the docs). This will keep the content that will need more frequent
> > revision out of the documentation.
> >
> > Then let’s create a new repo -- named flink-training -- that contains the
> > exercises, the solutions, and the tests that go with them, PLUS all of
> the
> > material that describes how to get setup to do the exercises, the
> > explanations for each exercise, and accompanying discussion material that
> > should be read after doing each exercise. Note that the exercise
> solutions
> > already have tests, and Travis is already being used for CI on the
> existing
> > project, so CI shouldn’t be an issue.
> >
> > Action Item: would a committer or PMC member kindly volunteer to help
> with
> > creating this new flink-training repo?
> >
> > With the content refactored in this way, I believe ongoing maintenance
> > won’t be much trouble. With each new Flink release I’ve been updating the
> > exercises to build against the latest release, and to avoid any newly
> > deprecated parts of the API. But since these exercises are focused on the
> > most basic parts of the API, that hasn’t been difficult. As for content
> > from the training website that would move into the docs, this content is
> > much more stable, and has only needed a gentle revision every year or
> two.
> >
> > Regards,
> > David
> >
>

Re: [DISCUSS] Integration of training materials into Apache Flink

[Seth Wiesman <sj...@gmail.com>]

Hi David,

I am happy to get the repo created for you.

If we go with the documentation (vs flink.apache.org) do you think we
should remove any of the existing content? There is already a getting
started section with quickstarts and walkthroughs and a concepts section.
In particular, the concepts section today is not complete and almost every
page on master contains multiple TODOs. I don't believe anyone is working
on these.  What do you think about replacing the current concepts section
with the training material? I just re-examined the training site and I
believe it covers the same material as concepts but better. Of course, we
would salvage anything worth keeping, like the glossary.

Seth

On Wed, Apr 15, 2020 at 2:02 PM David Anderson <da...@ververica.com> wrote:

> Thank you all for the very positive response to our proposal to contribute
> the training materials that have been at training.ververica.com to the
> Apache Flink project. Now I’d like to begin the more detailed discussion of
> how to go about this.
>
> In that earlier thread I mentioned that we were thinking of merging the
> markdown-based web pages into flink.apache.org, and to add the exercises
> to
> flink-playgrounds. This was based on thinking that it would be something of
> a maintenance headache to add the website content into the docs, where it
> would have to be versioned.
>
> Since then, a better approach has been suggested:
>
> We already have quite a bit of “getting started” material in the docs: Code
> Walkthroughs, Docker Playgrounds, Tutorials, and Examples. Having a second
> location (namely flink.apache.org) where this kind of content could be
> found doesn’t seem ideal. So let’s go ahead and add the expository content
> from the training materials to the documentation, with pointers into the
> rest of the docs (which are already present in the training), and with
> pointers to the exercises (rather than including the exercise descriptions
> in the docs). This will keep the content that will need more frequent
> revision out of the documentation.
>
> Then let’s create a new repo -- named flink-training -- that contains the
> exercises, the solutions, and the tests that go with them, PLUS all of the
> material that describes how to get setup to do the exercises, the
> explanations for each exercise, and accompanying discussion material that
> should be read after doing each exercise. Note that the exercise solutions
> already have tests, and Travis is already being used for CI on the existing
> project, so CI shouldn’t be an issue.
>
> Action Item: would a committer or PMC member kindly volunteer to help with
> creating this new flink-training repo?
>
> With the content refactored in this way, I believe ongoing maintenance
> won’t be much trouble. With each new Flink release I’ve been updating the
> exercises to build against the latest release, and to avoid any newly
> deprecated parts of the API. But since these exercises are focused on the
> most basic parts of the API, that hasn’t been difficult. As for content
> from the training website that would move into the docs, this content is
> much more stable, and has only needed a gentle revision every year or two.
>
> Regards,
> David
>