[DISCUSS] FLIP 1 - Flink Improvement Proposal

[Aljoscha Krettek <al...@apache.org>]

I'm proposing to add a formal process for how we deal with (major)
improvements to Flink and design docs. This has been mentioned several
times recently but we never took any decisive action to actually implement
such a process so here we go.

Right now, we have Jira issues and we sometimes we have design docs that we
keep in Google Docs. Jamie recently added links to those that he could find
on the mailing list to the Flink wiki:
https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home. The
problem with these is that a) the comments on the Google Docs are not
reflected in Jira and the mailing list. There has been some very active
discussion on some of the docs that most people would never notice. The
community therefore might seem less active than it actually is. b) the
documents are not very discoverable, if we had a clearly defined place
where we put them and also prominently link to this on the Flink homepage
this would greatly help people that try to find out about current
developments.

Kafka has a process like this:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals.
They call it KIP, for Kafka Improvement Proposal. We could either adapt
this for Flink or come up with our own process. Doing the former would save
us a lot of time and I don't think the Kafka community would mind us
copying their process. The subject also hints at this, our process could be
called FLIP, for Flink Improvement Proposal.

What do you think? Feedback is highly welcome. :-)

Cheers,
Aljoscha

Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal

[Maximilian Michels <mx...@apache.org>]

Great initiative! I think mailing list discussions can be very hard to
follow in retrospective. The Kafka community does a great job in
summing up the discussions on individual wiki pages. We should do that
as well!

On Tue, Jul 12, 2016 at 9:55 PM, Aljoscha Krettek <al...@apache.org> wrote:
> I'll make the changes to the list of things that we consider public "API"
> and also add these to the FLIP Template. Then we should be good to go. I'll
> convert the design docs I started to FLIPs in the next few days and will
> also ask other people that have authored design docs to move them to FLIPs.
>
> On Fri, 8 Jul 2016 at 17:44 Aljoscha Krettek <al...@apache.org> wrote:
>
>> I was planning to ask people to do that once the process is in place, yes.
>>
>> Wouldn't it be "forward porting", though? :-)
>>
>> On Fri, 8 Jul 2016 at 16:38 Robert Metzger <rm...@apache.org> wrote:
>>
>>> What about "backporting" some of the currently ongoing FLIPs to the new
>>> process?
>>> In particular those which are not yet fully implemented (Security, Mesos,
>>> Key Groups, ...) ?
>>>
>>> On Fri, Jul 8, 2016 at 4:21 PM, Aljoscha Krettek <al...@apache.org>
>>> wrote:
>>>
>>> > Of course, this would make the list look like this:
>>> >
>>> > All of the following are public interfaces that people build around:
>>> >  - DataStream and DataSet API, including classes related to that, such
>>> as
>>> > StreamExecutionEnvironment
>>> >  - Classes marked with the @Public annotation
>>> >  - On-disk binary formats, such as checkpoints/savepoints
>>> >  - User-facing scripts, i.e. bin/flink, Yarn scripts, Mesos scripts
>>> >  - Configuration
>>> >  - Monitoring
>>> >
>>> > On Fri, 8 Jul 2016 at 16:00 Ufuk Celebi <uc...@apache.org> wrote:
>>> >
>>> > > What about the scripts?
>>> > >
>>> > > On Fri, Jul 8, 2016 at 2:47 PM, Aljoscha Krettek <aljoscha@apache.org
>>> >
>>> > > wrote:
>>> > > > Thanks a lot, Matthias! I created an actual "FLIP-Template" based on
>>> > the
>>> > > > template you copied over.
>>> > > >
>>> > > > I propose to change the list of public interfaces to this:
>>> > > >
>>> > > > All of the following are public interfaces that people build around:
>>> > > >  - DataStream and DataSet API, including classes related to that,
>>> such
>>> > as
>>> > > > StreamExecutionEnvironment
>>> > > >  - Classes marked with the @Public annotation
>>> > > >  - On-disk binary formats, such as checkpoints/savepoints
>>> > > >  - Configuration
>>> > > >  - Monitoring
>>> > > >
>>> > > > Cheers,
>>> > > > Aljoscha
>>> > > >
>>> > > > On Fri, 8 Jul 2016 at 13:49 Matthias J. Sax <mj...@apache.org>
>>> wrote:
>>> > > >
>>> > > >> Done.
>>> > > >>
>>> > > >> I replaced KIP -> FLIP and Kafka -> Flink. Still there are some
>>> things
>>> > > >> we need to rework. Please have a look and propose changes.
>>> > > >>
>>> > > >>
>>> > > >>
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
>>> > > >>
>>> > > >> Same for FLIP Template
>>> > > >>
>>> > > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP+Template
>>> > > >>
>>> > > >>
>>> > > >> -Matthias
>>> > > >>
>>> > > >> On 07/08/2016 09:56 AM, Aljoscha Krettek wrote:
>>> > > >> > Hi,
>>> > > >> > I got this reply from one of the Kafka committers:
>>> > > >> >
>>> > > >> > Thanks for sharing your intention to use a process similar to our
>>> > KIP
>>> > > >> >
>>> > > >> > process. You are more than welcome to copy the structures and
>>> docs
>>> > > that
>>> > > >> we
>>> > > >> >
>>> > > >> > have for the KIP process. :)
>>> > > >> >
>>> > > >> > Ismael
>>> > > >> >
>>> > > >> >
>>> > > >> > So it seems we're good to go. @Matthias, since you are a Kafka
>>> > > >> contributor,
>>> > > >> > could you maybe copy the relevant docs from the Kafka wiki to our
>>> > > wiki?
>>> > > >> The
>>> > > >> > source for this page:
>>> > > >> >
>>> > > >>
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>>> > > >> > and
>>> > > >> > the KIP template that is referenced there. If you want, I can
>>> take
>>> > it
>>> > > >> from
>>> > > >> > there and adapt it to Flink and then let everyone discuss it
>>> again.
>>> > > >> >
>>> > > >> >
>>> > > >> > Cheers,
>>> > > >> >
>>> > > >> > Aljoscha
>>> > > >> >
>>> > > >> > On Thu, 7 Jul 2016 at 11:38 Aljoscha Krettek <
>>> aljoscha@apache.org>
>>> > > >> wrote:
>>> > > >> >
>>> > > >> >> I'll reach out to the Kafka community and ask if it's ok for us
>>> to
>>> > > >> "steal"
>>> > > >> >> their process for this.
>>> > > >> >>
>>> > > >> >> On Thu, 7 Jul 2016 at 11:36 Aljoscha Krettek <
>>> aljoscha@apache.org>
>>> > > >> wrote:
>>> > > >> >>
>>> > > >> >>> @Matthias: Yes, this is the reason why I like the KIP process
>>> and
>>> > > why I
>>> > > >> >>> said "The problem with these is that a) the comments on the
>>> Google
>>> > > >> Docs are
>>> > > >> >>> not reflected in Jira and the mailing list. There has been some
>>> > very
>>> > > >> active
>>> > > >> >>> discussion on some of the docs that most people would never
>>> > > notice.".
>>> > > >> >>>
>>> > > >> >>> On Thu, 7 Jul 2016 at 11:28 Robert Metzger <
>>> rmetzger@apache.org>
>>> > > >> wrote:
>>> > > >> >>>
>>> > > >> >>>> I also like the proposal. I think its an issue that Google
>>> Docs
>>> > > >> comments
>>> > > >> >>>> are not reflected within ASF infra. Therefore, I'm +1 on
>>> > discussing
>>> > > >> the
>>> > > >> >>>> proposals on the mailing list.
>>> > > >> >>>>
>>> > > >> >>>> I agree that we need to clean up our wiki.
>>> > > >> >>>>
>>> > > >> >>>> On Thu, Jul 7, 2016 at 10:58 AM, Matthias J. Sax <
>>> > mjsax@apache.org
>>> > > >
>>> > > >> >>>> wrote:
>>> > > >> >>>>
>>> > > >> >>>>> Just to point out one thing about Kafka KIPs and using the
>>> > project
>>> > > >> >>>> wiki:
>>> > > >> >>>>>
>>> > > >> >>>>> The wiki contains the current state of the proposal, while
>>> the
>>> > > >> >>>>> discussion is covered over the dev-mailing list. IMHO, this
>>> > makes
>>> > > a
>>> > > >> lot
>>> > > >> >>>>> of sense, as people tend to follow the mailing list but not
>>> wiki
>>> > > >> >>>>> changes. Furthermore, the mailing list tracks the whole
>>> > discussion
>>> > > >> >>>>> history, while the proposal is kept in a clean state and thus
>>> > > easy to
>>> > > >> >>>> read.
>>> > > >> >>>>>
>>> > > >> >>>>> -Matthias
>>> > > >> >>>>>
>>> > > >> >>>>>
>>> > > >> >>>>> On 07/06/2016 10:09 PM, Aljoscha Krettek wrote:
>>> > > >> >>>>>> Jip, that's why I referenced the Kafka process which is
>>> also in
>>> > > >> their
>>> > > >> >>>>> wiki.
>>> > > >> >>>>>>
>>> > > >> >>>>>> On Wed, 6 Jul 2016 at 21:01 Stephan Ewen <se...@apache.org>
>>> > > wrote:
>>> > > >> >>>>>>
>>> > > >> >>>>>>> Yes, big +1
>>> > > >> >>>>>>>
>>> > > >> >>>>>>> I had actually talked about the same thing with some
>>> people as
>>> > > >> well.
>>> > > >> >>>>>>>
>>> > > >> >>>>>>> I am currently sketching a few FLIPs for things, like
>>> > > improvements
>>> > > >> >>>> to
>>> > > >> >>>>> the
>>> > > >> >>>>>>> Yarn/Mesos/Kubernetes integration
>>> > > >> >>>>>>>
>>> > > >> >>>>>>>
>>> > > >> >>>>>>> One thing we should do here is to actually structure the
>>> wiki
>>> > a
>>> > > bit
>>> > > >> >>>> to
>>> > > >> >>>>> make
>>> > > >> >>>>>>> it easier to find information and proposals.
>>> > > >> >>>>>>>
>>> > > >> >>>>>>>
>>> > > >> >>>>>>>
>>> > > >> >>>>>>>
>>> > > >> >>>>>>> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk Celebi <
>>> uce@apache.org>
>>> > > >> wrote:
>>> > > >> >>>>>>>
>>> > > >> >>>>>>>> Hey Aljoscha,
>>> > > >> >>>>>>>>
>>> > > >> >>>>>>>> thanks for this proposal. I've somehow missed it last
>>> week. I
>>> > > like
>>> > > >> >>>> the
>>> > > >> >>>>>>>> idea very much and agree with your assessment about the
>>> > > problems
>>> > > >> >>>> with
>>> > > >> >>>>>>>> the Google Doc approach.
>>> > > >> >>>>>>>>
>>> > > >> >>>>>>>> Regarding the process: I'm also in favour of adopting it
>>> from
>>> > > >> >>>> Kafka. I
>>> > > >> >>>>>>>> would not expect any problems with this, but we can post a
>>> > > quick
>>> > > >> >>>> note
>>> > > >> >>>>>>>> to their ML.
>>> > > >> >>>>>>>>
>>> > > >> >>>>>>>> @Matthias: The name works for me. ;-)
>>> > > >> >>>>>>>>
>>> > > >> >>>>>>>> – Ufuk
>>> > > >> >>>>>>>>
>>> > > >> >>>>>>>> On Tue, Jun 28, 2016 at 10:19 PM, Matthias J. Sax <
>>> > > >> >>>> mjsax@apache.org>
>>> > > >> >>>>>>>> wrote:
>>> > > >> >>>>>>>>> FLIP ?? Really? :D
>>> > > >> >>>>>>>>>
>>> > > >> >>>>>>>>> http://www.maya.tv/en/character/flip
>>> > > >> >>>>>>>>>
>>> > > >> >>>>>>>>> -Matthias
>>> > > >> >>>>>>>>>
>>> > > >> >>>>>>>>>
>>> > > >> >>>>>>>>> On 06/28/2016 06:26 PM, Aljoscha Krettek wrote:
>>> > > >> >>>>>>>>>> I'm proposing to add a formal process for how we deal
>>> with
>>> > > >> >>>> (major)
>>> > > >> >>>>>>>>>> improvements to Flink and design docs. This has been
>>> > > mentioned
>>> > > >> >>>>> several
>>> > > >> >>>>>>>>>> times recently but we never took any decisive action to
>>> > > actually
>>> > > >> >>>>>>>> implement
>>> > > >> >>>>>>>>>> such a process so here we go.
>>> > > >> >>>>>>>>>>
>>> > > >> >>>>>>>>>> Right now, we have Jira issues and we sometimes we have
>>> > > design
>>> > > >> >>>> docs
>>> > > >> >>>>>>>> that we
>>> > > >> >>>>>>>>>> keep in Google Docs. Jamie recently added links to those
>>> > > that he
>>> > > >> >>>>> could
>>> > > >> >>>>>>>> find
>>> > > >> >>>>>>>>>> on the mailing list to the Flink wiki:
>>> > > >> >>>>>>>>>>
>>> > > >> >>>>
>>> > > https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
>>> > > >> >>>>>>>> The
>>> > > >> >>>>>>>>>> problem with these is that a) the comments on the Google
>>> > Docs
>>> > > >> >>>> are not
>>> > > >> >>>>>>>>>> reflected in Jira and the mailing list. There has been
>>> some
>>> > > very
>>> > > >> >>>>>>> active
>>> > > >> >>>>>>>>>> discussion on some of the docs that most people would
>>> never
>>> > > >> >>>> notice.
>>> > > >> >>>>>>> The
>>> > > >> >>>>>>>>>> community therefore might seem less active than it
>>> actually
>>> > > is.
>>> > > >> >>>> b)
>>> > > >> >>>>> the
>>> > > >> >>>>>>>>>> documents are not very discoverable, if we had a clearly
>>> > > defined
>>> > > >> >>>>> place
>>> > > >> >>>>>>>>>> where we put them and also prominently link to this on
>>> the
>>> > > Flink
>>> > > >> >>>>>>>> homepage
>>> > > >> >>>>>>>>>> this would greatly help people that try to find out
>>> about
>>> > > >> current
>>> > > >> >>>>>>>>>> developments.
>>> > > >> >>>>>>>>>>
>>> > > >> >>>>>>>>>> Kafka has a process like this:
>>> > > >> >>>>>>>>>>
>>> > > >> >>>>>>>>
>>> > > >> >>>>>>>
>>> > > >> >>>>>
>>> > > >> >>>>
>>> > > >>
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>>> > > >> >>>>>>>> .
>>> > > >> >>>>>>>>>> They call it KIP, for Kafka Improvement Proposal. We
>>> could
>>> > > >> either
>>> > > >> >>>>>>> adapt
>>> > > >> >>>>>>>>>> this for Flink or come up with our own process. Doing
>>> the
>>> > > former
>>> > > >> >>>>> would
>>> > > >> >>>>>>>> save
>>> > > >> >>>>>>>>>> us a lot of time and I don't think the Kafka community
>>> > would
>>> > > >> >>>> mind us
>>> > > >> >>>>>>>>>> copying their process. The subject also hints at this,
>>> our
>>> > > >> >>>> process
>>> > > >> >>>>>>>> could be
>>> > > >> >>>>>>>>>> called FLIP, for Flink Improvement Proposal.
>>> > > >> >>>>>>>>>>
>>> > > >> >>>>>>>>>> What do you think? Feedback is highly welcome. :-)
>>> > > >> >>>>>>>>>>
>>> > > >> >>>>>>>>>> Cheers,
>>> > > >> >>>>>>>>>> Aljoscha
>>> > > >> >>>>>>>>>>
>>> > > >> >>>>>>>>>
>>> > > >> >>>>>>>>
>>> > > >> >>>>>>>
>>> > > >> >>>>>>
>>> > > >> >>>>>
>>> > > >> >>>>>
>>> > > >> >>>>
>>> > > >> >>>
>>> > > >> >
>>> > > >>
>>> > > >>
>>> > >
>>> >
>>>
>>

Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal

[Aljoscha Krettek <al...@apache.org>]

I'll make the changes to the list of things that we consider public "API"
and also add these to the FLIP Template. Then we should be good to go. I'll
convert the design docs I started to FLIPs in the next few days and will
also ask other people that have authored design docs to move them to FLIPs.

On Fri, 8 Jul 2016 at 17:44 Aljoscha Krettek <al...@apache.org> wrote:

> I was planning to ask people to do that once the process is in place, yes.
>
> Wouldn't it be "forward porting", though? :-)
>
> On Fri, 8 Jul 2016 at 16:38 Robert Metzger <rm...@apache.org> wrote:
>
>> What about "backporting" some of the currently ongoing FLIPs to the new
>> process?
>> In particular those which are not yet fully implemented (Security, Mesos,
>> Key Groups, ...) ?
>>
>> On Fri, Jul 8, 2016 at 4:21 PM, Aljoscha Krettek <al...@apache.org>
>> wrote:
>>
>> > Of course, this would make the list look like this:
>> >
>> > All of the following are public interfaces that people build around:
>> >  - DataStream and DataSet API, including classes related to that, such
>> as
>> > StreamExecutionEnvironment
>> >  - Classes marked with the @Public annotation
>> >  - On-disk binary formats, such as checkpoints/savepoints
>> >  - User-facing scripts, i.e. bin/flink, Yarn scripts, Mesos scripts
>> >  - Configuration
>> >  - Monitoring
>> >
>> > On Fri, 8 Jul 2016 at 16:00 Ufuk Celebi <uc...@apache.org> wrote:
>> >
>> > > What about the scripts?
>> > >
>> > > On Fri, Jul 8, 2016 at 2:47 PM, Aljoscha Krettek <aljoscha@apache.org
>> >
>> > > wrote:
>> > > > Thanks a lot, Matthias! I created an actual "FLIP-Template" based on
>> > the
>> > > > template you copied over.
>> > > >
>> > > > I propose to change the list of public interfaces to this:
>> > > >
>> > > > All of the following are public interfaces that people build around:
>> > > >  - DataStream and DataSet API, including classes related to that,
>> such
>> > as
>> > > > StreamExecutionEnvironment
>> > > >  - Classes marked with the @Public annotation
>> > > >  - On-disk binary formats, such as checkpoints/savepoints
>> > > >  - Configuration
>> > > >  - Monitoring
>> > > >
>> > > > Cheers,
>> > > > Aljoscha
>> > > >
>> > > > On Fri, 8 Jul 2016 at 13:49 Matthias J. Sax <mj...@apache.org>
>> wrote:
>> > > >
>> > > >> Done.
>> > > >>
>> > > >> I replaced KIP -> FLIP and Kafka -> Flink. Still there are some
>> things
>> > > >> we need to rework. Please have a look and propose changes.
>> > > >>
>> > > >>
>> > > >>
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
>> > > >>
>> > > >> Same for FLIP Template
>> > > >>
>> > > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP+Template
>> > > >>
>> > > >>
>> > > >> -Matthias
>> > > >>
>> > > >> On 07/08/2016 09:56 AM, Aljoscha Krettek wrote:
>> > > >> > Hi,
>> > > >> > I got this reply from one of the Kafka committers:
>> > > >> >
>> > > >> > Thanks for sharing your intention to use a process similar to our
>> > KIP
>> > > >> >
>> > > >> > process. You are more than welcome to copy the structures and
>> docs
>> > > that
>> > > >> we
>> > > >> >
>> > > >> > have for the KIP process. :)
>> > > >> >
>> > > >> > Ismael
>> > > >> >
>> > > >> >
>> > > >> > So it seems we're good to go. @Matthias, since you are a Kafka
>> > > >> contributor,
>> > > >> > could you maybe copy the relevant docs from the Kafka wiki to our
>> > > wiki?
>> > > >> The
>> > > >> > source for this page:
>> > > >> >
>> > > >>
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>> > > >> > and
>> > > >> > the KIP template that is referenced there. If you want, I can
>> take
>> > it
>> > > >> from
>> > > >> > there and adapt it to Flink and then let everyone discuss it
>> again.
>> > > >> >
>> > > >> >
>> > > >> > Cheers,
>> > > >> >
>> > > >> > Aljoscha
>> > > >> >
>> > > >> > On Thu, 7 Jul 2016 at 11:38 Aljoscha Krettek <
>> aljoscha@apache.org>
>> > > >> wrote:
>> > > >> >
>> > > >> >> I'll reach out to the Kafka community and ask if it's ok for us
>> to
>> > > >> "steal"
>> > > >> >> their process for this.
>> > > >> >>
>> > > >> >> On Thu, 7 Jul 2016 at 11:36 Aljoscha Krettek <
>> aljoscha@apache.org>
>> > > >> wrote:
>> > > >> >>
>> > > >> >>> @Matthias: Yes, this is the reason why I like the KIP process
>> and
>> > > why I
>> > > >> >>> said "The problem with these is that a) the comments on the
>> Google
>> > > >> Docs are
>> > > >> >>> not reflected in Jira and the mailing list. There has been some
>> > very
>> > > >> active
>> > > >> >>> discussion on some of the docs that most people would never
>> > > notice.".
>> > > >> >>>
>> > > >> >>> On Thu, 7 Jul 2016 at 11:28 Robert Metzger <
>> rmetzger@apache.org>
>> > > >> wrote:
>> > > >> >>>
>> > > >> >>>> I also like the proposal. I think its an issue that Google
>> Docs
>> > > >> comments
>> > > >> >>>> are not reflected within ASF infra. Therefore, I'm +1 on
>> > discussing
>> > > >> the
>> > > >> >>>> proposals on the mailing list.
>> > > >> >>>>
>> > > >> >>>> I agree that we need to clean up our wiki.
>> > > >> >>>>
>> > > >> >>>> On Thu, Jul 7, 2016 at 10:58 AM, Matthias J. Sax <
>> > mjsax@apache.org
>> > > >
>> > > >> >>>> wrote:
>> > > >> >>>>
>> > > >> >>>>> Just to point out one thing about Kafka KIPs and using the
>> > project
>> > > >> >>>> wiki:
>> > > >> >>>>>
>> > > >> >>>>> The wiki contains the current state of the proposal, while
>> the
>> > > >> >>>>> discussion is covered over the dev-mailing list. IMHO, this
>> > makes
>> > > a
>> > > >> lot
>> > > >> >>>>> of sense, as people tend to follow the mailing list but not
>> wiki
>> > > >> >>>>> changes. Furthermore, the mailing list tracks the whole
>> > discussion
>> > > >> >>>>> history, while the proposal is kept in a clean state and thus
>> > > easy to
>> > > >> >>>> read.
>> > > >> >>>>>
>> > > >> >>>>> -Matthias
>> > > >> >>>>>
>> > > >> >>>>>
>> > > >> >>>>> On 07/06/2016 10:09 PM, Aljoscha Krettek wrote:
>> > > >> >>>>>> Jip, that's why I referenced the Kafka process which is
>> also in
>> > > >> their
>> > > >> >>>>> wiki.
>> > > >> >>>>>>
>> > > >> >>>>>> On Wed, 6 Jul 2016 at 21:01 Stephan Ewen <se...@apache.org>
>> > > wrote:
>> > > >> >>>>>>
>> > > >> >>>>>>> Yes, big +1
>> > > >> >>>>>>>
>> > > >> >>>>>>> I had actually talked about the same thing with some
>> people as
>> > > >> well.
>> > > >> >>>>>>>
>> > > >> >>>>>>> I am currently sketching a few FLIPs for things, like
>> > > improvements
>> > > >> >>>> to
>> > > >> >>>>> the
>> > > >> >>>>>>> Yarn/Mesos/Kubernetes integration
>> > > >> >>>>>>>
>> > > >> >>>>>>>
>> > > >> >>>>>>> One thing we should do here is to actually structure the
>> wiki
>> > a
>> > > bit
>> > > >> >>>> to
>> > > >> >>>>> make
>> > > >> >>>>>>> it easier to find information and proposals.
>> > > >> >>>>>>>
>> > > >> >>>>>>>
>> > > >> >>>>>>>
>> > > >> >>>>>>>
>> > > >> >>>>>>> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk Celebi <
>> uce@apache.org>
>> > > >> wrote:
>> > > >> >>>>>>>
>> > > >> >>>>>>>> Hey Aljoscha,
>> > > >> >>>>>>>>
>> > > >> >>>>>>>> thanks for this proposal. I've somehow missed it last
>> week. I
>> > > like
>> > > >> >>>> the
>> > > >> >>>>>>>> idea very much and agree with your assessment about the
>> > > problems
>> > > >> >>>> with
>> > > >> >>>>>>>> the Google Doc approach.
>> > > >> >>>>>>>>
>> > > >> >>>>>>>> Regarding the process: I'm also in favour of adopting it
>> from
>> > > >> >>>> Kafka. I
>> > > >> >>>>>>>> would not expect any problems with this, but we can post a
>> > > quick
>> > > >> >>>> note
>> > > >> >>>>>>>> to their ML.
>> > > >> >>>>>>>>
>> > > >> >>>>>>>> @Matthias: The name works for me. ;-)
>> > > >> >>>>>>>>
>> > > >> >>>>>>>> – Ufuk
>> > > >> >>>>>>>>
>> > > >> >>>>>>>> On Tue, Jun 28, 2016 at 10:19 PM, Matthias J. Sax <
>> > > >> >>>> mjsax@apache.org>
>> > > >> >>>>>>>> wrote:
>> > > >> >>>>>>>>> FLIP ?? Really? :D
>> > > >> >>>>>>>>>
>> > > >> >>>>>>>>> http://www.maya.tv/en/character/flip
>> > > >> >>>>>>>>>
>> > > >> >>>>>>>>> -Matthias
>> > > >> >>>>>>>>>
>> > > >> >>>>>>>>>
>> > > >> >>>>>>>>> On 06/28/2016 06:26 PM, Aljoscha Krettek wrote:
>> > > >> >>>>>>>>>> I'm proposing to add a formal process for how we deal
>> with
>> > > >> >>>> (major)
>> > > >> >>>>>>>>>> improvements to Flink and design docs. This has been
>> > > mentioned
>> > > >> >>>>> several
>> > > >> >>>>>>>>>> times recently but we never took any decisive action to
>> > > actually
>> > > >> >>>>>>>> implement
>> > > >> >>>>>>>>>> such a process so here we go.
>> > > >> >>>>>>>>>>
>> > > >> >>>>>>>>>> Right now, we have Jira issues and we sometimes we have
>> > > design
>> > > >> >>>> docs
>> > > >> >>>>>>>> that we
>> > > >> >>>>>>>>>> keep in Google Docs. Jamie recently added links to those
>> > > that he
>> > > >> >>>>> could
>> > > >> >>>>>>>> find
>> > > >> >>>>>>>>>> on the mailing list to the Flink wiki:
>> > > >> >>>>>>>>>>
>> > > >> >>>>
>> > > https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
>> > > >> >>>>>>>> The
>> > > >> >>>>>>>>>> problem with these is that a) the comments on the Google
>> > Docs
>> > > >> >>>> are not
>> > > >> >>>>>>>>>> reflected in Jira and the mailing list. There has been
>> some
>> > > very
>> > > >> >>>>>>> active
>> > > >> >>>>>>>>>> discussion on some of the docs that most people would
>> never
>> > > >> >>>> notice.
>> > > >> >>>>>>> The
>> > > >> >>>>>>>>>> community therefore might seem less active than it
>> actually
>> > > is.
>> > > >> >>>> b)
>> > > >> >>>>> the
>> > > >> >>>>>>>>>> documents are not very discoverable, if we had a clearly
>> > > defined
>> > > >> >>>>> place
>> > > >> >>>>>>>>>> where we put them and also prominently link to this on
>> the
>> > > Flink
>> > > >> >>>>>>>> homepage
>> > > >> >>>>>>>>>> this would greatly help people that try to find out
>> about
>> > > >> current
>> > > >> >>>>>>>>>> developments.
>> > > >> >>>>>>>>>>
>> > > >> >>>>>>>>>> Kafka has a process like this:
>> > > >> >>>>>>>>>>
>> > > >> >>>>>>>>
>> > > >> >>>>>>>
>> > > >> >>>>>
>> > > >> >>>>
>> > > >>
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>> > > >> >>>>>>>> .
>> > > >> >>>>>>>>>> They call it KIP, for Kafka Improvement Proposal. We
>> could
>> > > >> either
>> > > >> >>>>>>> adapt
>> > > >> >>>>>>>>>> this for Flink or come up with our own process. Doing
>> the
>> > > former
>> > > >> >>>>> would
>> > > >> >>>>>>>> save
>> > > >> >>>>>>>>>> us a lot of time and I don't think the Kafka community
>> > would
>> > > >> >>>> mind us
>> > > >> >>>>>>>>>> copying their process. The subject also hints at this,
>> our
>> > > >> >>>> process
>> > > >> >>>>>>>> could be
>> > > >> >>>>>>>>>> called FLIP, for Flink Improvement Proposal.
>> > > >> >>>>>>>>>>
>> > > >> >>>>>>>>>> What do you think? Feedback is highly welcome. :-)
>> > > >> >>>>>>>>>>
>> > > >> >>>>>>>>>> Cheers,
>> > > >> >>>>>>>>>> Aljoscha
>> > > >> >>>>>>>>>>
>> > > >> >>>>>>>>>
>> > > >> >>>>>>>>
>> > > >> >>>>>>>
>> > > >> >>>>>>
>> > > >> >>>>>
>> > > >> >>>>>
>> > > >> >>>>
>> > > >> >>>
>> > > >> >
>> > > >>
>> > > >>
>> > >
>> >
>>
>

Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal

[Aljoscha Krettek <al...@apache.org>]

I was planning to ask people to do that once the process is in place, yes.

Wouldn't it be "forward porting", though? :-)

On Fri, 8 Jul 2016 at 16:38 Robert Metzger <rm...@apache.org> wrote:

> What about "backporting" some of the currently ongoing FLIPs to the new
> process?
> In particular those which are not yet fully implemented (Security, Mesos,
> Key Groups, ...) ?
>
> On Fri, Jul 8, 2016 at 4:21 PM, Aljoscha Krettek <al...@apache.org>
> wrote:
>
> > Of course, this would make the list look like this:
> >
> > All of the following are public interfaces that people build around:
> >  - DataStream and DataSet API, including classes related to that, such as
> > StreamExecutionEnvironment
> >  - Classes marked with the @Public annotation
> >  - On-disk binary formats, such as checkpoints/savepoints
> >  - User-facing scripts, i.e. bin/flink, Yarn scripts, Mesos scripts
> >  - Configuration
> >  - Monitoring
> >
> > On Fri, 8 Jul 2016 at 16:00 Ufuk Celebi <uc...@apache.org> wrote:
> >
> > > What about the scripts?
> > >
> > > On Fri, Jul 8, 2016 at 2:47 PM, Aljoscha Krettek <al...@apache.org>
> > > wrote:
> > > > Thanks a lot, Matthias! I created an actual "FLIP-Template" based on
> > the
> > > > template you copied over.
> > > >
> > > > I propose to change the list of public interfaces to this:
> > > >
> > > > All of the following are public interfaces that people build around:
> > > >  - DataStream and DataSet API, including classes related to that,
> such
> > as
> > > > StreamExecutionEnvironment
> > > >  - Classes marked with the @Public annotation
> > > >  - On-disk binary formats, such as checkpoints/savepoints
> > > >  - Configuration
> > > >  - Monitoring
> > > >
> > > > Cheers,
> > > > Aljoscha
> > > >
> > > > On Fri, 8 Jul 2016 at 13:49 Matthias J. Sax <mj...@apache.org>
> wrote:
> > > >
> > > >> Done.
> > > >>
> > > >> I replaced KIP -> FLIP and Kafka -> Flink. Still there are some
> things
> > > >> we need to rework. Please have a look and propose changes.
> > > >>
> > > >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
> > > >>
> > > >> Same for FLIP Template
> > > >>
> > > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP+Template
> > > >>
> > > >>
> > > >> -Matthias
> > > >>
> > > >> On 07/08/2016 09:56 AM, Aljoscha Krettek wrote:
> > > >> > Hi,
> > > >> > I got this reply from one of the Kafka committers:
> > > >> >
> > > >> > Thanks for sharing your intention to use a process similar to our
> > KIP
> > > >> >
> > > >> > process. You are more than welcome to copy the structures and docs
> > > that
> > > >> we
> > > >> >
> > > >> > have for the KIP process. :)
> > > >> >
> > > >> > Ismael
> > > >> >
> > > >> >
> > > >> > So it seems we're good to go. @Matthias, since you are a Kafka
> > > >> contributor,
> > > >> > could you maybe copy the relevant docs from the Kafka wiki to our
> > > wiki?
> > > >> The
> > > >> > source for this page:
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> > > >> > and
> > > >> > the KIP template that is referenced there. If you want, I can take
> > it
> > > >> from
> > > >> > there and adapt it to Flink and then let everyone discuss it
> again.
> > > >> >
> > > >> >
> > > >> > Cheers,
> > > >> >
> > > >> > Aljoscha
> > > >> >
> > > >> > On Thu, 7 Jul 2016 at 11:38 Aljoscha Krettek <aljoscha@apache.org
> >
> > > >> wrote:
> > > >> >
> > > >> >> I'll reach out to the Kafka community and ask if it's ok for us
> to
> > > >> "steal"
> > > >> >> their process for this.
> > > >> >>
> > > >> >> On Thu, 7 Jul 2016 at 11:36 Aljoscha Krettek <
> aljoscha@apache.org>
> > > >> wrote:
> > > >> >>
> > > >> >>> @Matthias: Yes, this is the reason why I like the KIP process
> and
> > > why I
> > > >> >>> said "The problem with these is that a) the comments on the
> Google
> > > >> Docs are
> > > >> >>> not reflected in Jira and the mailing list. There has been some
> > very
> > > >> active
> > > >> >>> discussion on some of the docs that most people would never
> > > notice.".
> > > >> >>>
> > > >> >>> On Thu, 7 Jul 2016 at 11:28 Robert Metzger <rmetzger@apache.org
> >
> > > >> wrote:
> > > >> >>>
> > > >> >>>> I also like the proposal. I think its an issue that Google Docs
> > > >> comments
> > > >> >>>> are not reflected within ASF infra. Therefore, I'm +1 on
> > discussing
> > > >> the
> > > >> >>>> proposals on the mailing list.
> > > >> >>>>
> > > >> >>>> I agree that we need to clean up our wiki.
> > > >> >>>>
> > > >> >>>> On Thu, Jul 7, 2016 at 10:58 AM, Matthias J. Sax <
> > mjsax@apache.org
> > > >
> > > >> >>>> wrote:
> > > >> >>>>
> > > >> >>>>> Just to point out one thing about Kafka KIPs and using the
> > project
> > > >> >>>> wiki:
> > > >> >>>>>
> > > >> >>>>> The wiki contains the current state of the proposal, while the
> > > >> >>>>> discussion is covered over the dev-mailing list. IMHO, this
> > makes
> > > a
> > > >> lot
> > > >> >>>>> of sense, as people tend to follow the mailing list but not
> wiki
> > > >> >>>>> changes. Furthermore, the mailing list tracks the whole
> > discussion
> > > >> >>>>> history, while the proposal is kept in a clean state and thus
> > > easy to
> > > >> >>>> read.
> > > >> >>>>>
> > > >> >>>>> -Matthias
> > > >> >>>>>
> > > >> >>>>>
> > > >> >>>>> On 07/06/2016 10:09 PM, Aljoscha Krettek wrote:
> > > >> >>>>>> Jip, that's why I referenced the Kafka process which is also
> in
> > > >> their
> > > >> >>>>> wiki.
> > > >> >>>>>>
> > > >> >>>>>> On Wed, 6 Jul 2016 at 21:01 Stephan Ewen <se...@apache.org>
> > > wrote:
> > > >> >>>>>>
> > > >> >>>>>>> Yes, big +1
> > > >> >>>>>>>
> > > >> >>>>>>> I had actually talked about the same thing with some people
> as
> > > >> well.
> > > >> >>>>>>>
> > > >> >>>>>>> I am currently sketching a few FLIPs for things, like
> > > improvements
> > > >> >>>> to
> > > >> >>>>> the
> > > >> >>>>>>> Yarn/Mesos/Kubernetes integration
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>> One thing we should do here is to actually structure the
> wiki
> > a
> > > bit
> > > >> >>>> to
> > > >> >>>>> make
> > > >> >>>>>>> it easier to find information and proposals.
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk Celebi <uce@apache.org
> >
> > > >> wrote:
> > > >> >>>>>>>
> > > >> >>>>>>>> Hey Aljoscha,
> > > >> >>>>>>>>
> > > >> >>>>>>>> thanks for this proposal. I've somehow missed it last
> week. I
> > > like
> > > >> >>>> the
> > > >> >>>>>>>> idea very much and agree with your assessment about the
> > > problems
> > > >> >>>> with
> > > >> >>>>>>>> the Google Doc approach.
> > > >> >>>>>>>>
> > > >> >>>>>>>> Regarding the process: I'm also in favour of adopting it
> from
> > > >> >>>> Kafka. I
> > > >> >>>>>>>> would not expect any problems with this, but we can post a
> > > quick
> > > >> >>>> note
> > > >> >>>>>>>> to their ML.
> > > >> >>>>>>>>
> > > >> >>>>>>>> @Matthias: The name works for me. ;-)
> > > >> >>>>>>>>
> > > >> >>>>>>>> – Ufuk
> > > >> >>>>>>>>
> > > >> >>>>>>>> On Tue, Jun 28, 2016 at 10:19 PM, Matthias J. Sax <
> > > >> >>>> mjsax@apache.org>
> > > >> >>>>>>>> wrote:
> > > >> >>>>>>>>> FLIP ?? Really? :D
> > > >> >>>>>>>>>
> > > >> >>>>>>>>> http://www.maya.tv/en/character/flip
> > > >> >>>>>>>>>
> > > >> >>>>>>>>> -Matthias
> > > >> >>>>>>>>>
> > > >> >>>>>>>>>
> > > >> >>>>>>>>> On 06/28/2016 06:26 PM, Aljoscha Krettek wrote:
> > > >> >>>>>>>>>> I'm proposing to add a formal process for how we deal
> with
> > > >> >>>> (major)
> > > >> >>>>>>>>>> improvements to Flink and design docs. This has been
> > > mentioned
> > > >> >>>>> several
> > > >> >>>>>>>>>> times recently but we never took any decisive action to
> > > actually
> > > >> >>>>>>>> implement
> > > >> >>>>>>>>>> such a process so here we go.
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>> Right now, we have Jira issues and we sometimes we have
> > > design
> > > >> >>>> docs
> > > >> >>>>>>>> that we
> > > >> >>>>>>>>>> keep in Google Docs. Jamie recently added links to those
> > > that he
> > > >> >>>>> could
> > > >> >>>>>>>> find
> > > >> >>>>>>>>>> on the mailing list to the Flink wiki:
> > > >> >>>>>>>>>>
> > > >> >>>>
> > > https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
> > > >> >>>>>>>> The
> > > >> >>>>>>>>>> problem with these is that a) the comments on the Google
> > Docs
> > > >> >>>> are not
> > > >> >>>>>>>>>> reflected in Jira and the mailing list. There has been
> some
> > > very
> > > >> >>>>>>> active
> > > >> >>>>>>>>>> discussion on some of the docs that most people would
> never
> > > >> >>>> notice.
> > > >> >>>>>>> The
> > > >> >>>>>>>>>> community therefore might seem less active than it
> actually
> > > is.
> > > >> >>>> b)
> > > >> >>>>> the
> > > >> >>>>>>>>>> documents are not very discoverable, if we had a clearly
> > > defined
> > > >> >>>>> place
> > > >> >>>>>>>>>> where we put them and also prominently link to this on
> the
> > > Flink
> > > >> >>>>>>>> homepage
> > > >> >>>>>>>>>> this would greatly help people that try to find out about
> > > >> current
> > > >> >>>>>>>>>> developments.
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>> Kafka has a process like this:
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>
> > > >> >>>>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> > > >> >>>>>>>> .
> > > >> >>>>>>>>>> They call it KIP, for Kafka Improvement Proposal. We
> could
> > > >> either
> > > >> >>>>>>> adapt
> > > >> >>>>>>>>>> this for Flink or come up with our own process. Doing the
> > > former
> > > >> >>>>> would
> > > >> >>>>>>>> save
> > > >> >>>>>>>>>> us a lot of time and I don't think the Kafka community
> > would
> > > >> >>>> mind us
> > > >> >>>>>>>>>> copying their process. The subject also hints at this,
> our
> > > >> >>>> process
> > > >> >>>>>>>> could be
> > > >> >>>>>>>>>> called FLIP, for Flink Improvement Proposal.
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>> What do you think? Feedback is highly welcome. :-)
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>> Cheers,
> > > >> >>>>>>>>>> Aljoscha
> > > >> >>>>>>>>>>
> > > >> >>>>>>>>>
> > > >> >>>>>>>>
> > > >> >>>>>>>
> > > >> >>>>>>
> > > >> >>>>>
> > > >> >>>>>
> > > >> >>>>
> > > >> >>>
> > > >> >
> > > >>
> > > >>
> > >
> >
>

Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal

[Robert Metzger <rm...@apache.org>]

What about "backporting" some of the currently ongoing FLIPs to the new
process?
In particular those which are not yet fully implemented (Security, Mesos,
Key Groups, ...) ?

On Fri, Jul 8, 2016 at 4:21 PM, Aljoscha Krettek <al...@apache.org>
wrote:

> Of course, this would make the list look like this:
>
> All of the following are public interfaces that people build around:
>  - DataStream and DataSet API, including classes related to that, such as
> StreamExecutionEnvironment
>  - Classes marked with the @Public annotation
>  - On-disk binary formats, such as checkpoints/savepoints
>  - User-facing scripts, i.e. bin/flink, Yarn scripts, Mesos scripts
>  - Configuration
>  - Monitoring
>
> On Fri, 8 Jul 2016 at 16:00 Ufuk Celebi <uc...@apache.org> wrote:
>
> > What about the scripts?
> >
> > On Fri, Jul 8, 2016 at 2:47 PM, Aljoscha Krettek <al...@apache.org>
> > wrote:
> > > Thanks a lot, Matthias! I created an actual "FLIP-Template" based on
> the
> > > template you copied over.
> > >
> > > I propose to change the list of public interfaces to this:
> > >
> > > All of the following are public interfaces that people build around:
> > >  - DataStream and DataSet API, including classes related to that, such
> as
> > > StreamExecutionEnvironment
> > >  - Classes marked with the @Public annotation
> > >  - On-disk binary formats, such as checkpoints/savepoints
> > >  - Configuration
> > >  - Monitoring
> > >
> > > Cheers,
> > > Aljoscha
> > >
> > > On Fri, 8 Jul 2016 at 13:49 Matthias J. Sax <mj...@apache.org> wrote:
> > >
> > >> Done.
> > >>
> > >> I replaced KIP -> FLIP and Kafka -> Flink. Still there are some things
> > >> we need to rework. Please have a look and propose changes.
> > >>
> > >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
> > >>
> > >> Same for FLIP Template
> > >>
> > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP+Template
> > >>
> > >>
> > >> -Matthias
> > >>
> > >> On 07/08/2016 09:56 AM, Aljoscha Krettek wrote:
> > >> > Hi,
> > >> > I got this reply from one of the Kafka committers:
> > >> >
> > >> > Thanks for sharing your intention to use a process similar to our
> KIP
> > >> >
> > >> > process. You are more than welcome to copy the structures and docs
> > that
> > >> we
> > >> >
> > >> > have for the KIP process. :)
> > >> >
> > >> > Ismael
> > >> >
> > >> >
> > >> > So it seems we're good to go. @Matthias, since you are a Kafka
> > >> contributor,
> > >> > could you maybe copy the relevant docs from the Kafka wiki to our
> > wiki?
> > >> The
> > >> > source for this page:
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> > >> > and
> > >> > the KIP template that is referenced there. If you want, I can take
> it
> > >> from
> > >> > there and adapt it to Flink and then let everyone discuss it again.
> > >> >
> > >> >
> > >> > Cheers,
> > >> >
> > >> > Aljoscha
> > >> >
> > >> > On Thu, 7 Jul 2016 at 11:38 Aljoscha Krettek <al...@apache.org>
> > >> wrote:
> > >> >
> > >> >> I'll reach out to the Kafka community and ask if it's ok for us to
> > >> "steal"
> > >> >> their process for this.
> > >> >>
> > >> >> On Thu, 7 Jul 2016 at 11:36 Aljoscha Krettek <al...@apache.org>
> > >> wrote:
> > >> >>
> > >> >>> @Matthias: Yes, this is the reason why I like the KIP process and
> > why I
> > >> >>> said "The problem with these is that a) the comments on the Google
> > >> Docs are
> > >> >>> not reflected in Jira and the mailing list. There has been some
> very
> > >> active
> > >> >>> discussion on some of the docs that most people would never
> > notice.".
> > >> >>>
> > >> >>> On Thu, 7 Jul 2016 at 11:28 Robert Metzger <rm...@apache.org>
> > >> wrote:
> > >> >>>
> > >> >>>> I also like the proposal. I think its an issue that Google Docs
> > >> comments
> > >> >>>> are not reflected within ASF infra. Therefore, I'm +1 on
> discussing
> > >> the
> > >> >>>> proposals on the mailing list.
> > >> >>>>
> > >> >>>> I agree that we need to clean up our wiki.
> > >> >>>>
> > >> >>>> On Thu, Jul 7, 2016 at 10:58 AM, Matthias J. Sax <
> mjsax@apache.org
> > >
> > >> >>>> wrote:
> > >> >>>>
> > >> >>>>> Just to point out one thing about Kafka KIPs and using the
> project
> > >> >>>> wiki:
> > >> >>>>>
> > >> >>>>> The wiki contains the current state of the proposal, while the
> > >> >>>>> discussion is covered over the dev-mailing list. IMHO, this
> makes
> > a
> > >> lot
> > >> >>>>> of sense, as people tend to follow the mailing list but not wiki
> > >> >>>>> changes. Furthermore, the mailing list tracks the whole
> discussion
> > >> >>>>> history, while the proposal is kept in a clean state and thus
> > easy to
> > >> >>>> read.
> > >> >>>>>
> > >> >>>>> -Matthias
> > >> >>>>>
> > >> >>>>>
> > >> >>>>> On 07/06/2016 10:09 PM, Aljoscha Krettek wrote:
> > >> >>>>>> Jip, that's why I referenced the Kafka process which is also in
> > >> their
> > >> >>>>> wiki.
> > >> >>>>>>
> > >> >>>>>> On Wed, 6 Jul 2016 at 21:01 Stephan Ewen <se...@apache.org>
> > wrote:
> > >> >>>>>>
> > >> >>>>>>> Yes, big +1
> > >> >>>>>>>
> > >> >>>>>>> I had actually talked about the same thing with some people as
> > >> well.
> > >> >>>>>>>
> > >> >>>>>>> I am currently sketching a few FLIPs for things, like
> > improvements
> > >> >>>> to
> > >> >>>>> the
> > >> >>>>>>> Yarn/Mesos/Kubernetes integration
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> One thing we should do here is to actually structure the wiki
> a
> > bit
> > >> >>>> to
> > >> >>>>> make
> > >> >>>>>>> it easier to find information and proposals.
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk Celebi <uc...@apache.org>
> > >> wrote:
> > >> >>>>>>>
> > >> >>>>>>>> Hey Aljoscha,
> > >> >>>>>>>>
> > >> >>>>>>>> thanks for this proposal. I've somehow missed it last week. I
> > like
> > >> >>>> the
> > >> >>>>>>>> idea very much and agree with your assessment about the
> > problems
> > >> >>>> with
> > >> >>>>>>>> the Google Doc approach.
> > >> >>>>>>>>
> > >> >>>>>>>> Regarding the process: I'm also in favour of adopting it from
> > >> >>>> Kafka. I
> > >> >>>>>>>> would not expect any problems with this, but we can post a
> > quick
> > >> >>>> note
> > >> >>>>>>>> to their ML.
> > >> >>>>>>>>
> > >> >>>>>>>> @Matthias: The name works for me. ;-)
> > >> >>>>>>>>
> > >> >>>>>>>> – Ufuk
> > >> >>>>>>>>
> > >> >>>>>>>> On Tue, Jun 28, 2016 at 10:19 PM, Matthias J. Sax <
> > >> >>>> mjsax@apache.org>
> > >> >>>>>>>> wrote:
> > >> >>>>>>>>> FLIP ?? Really? :D
> > >> >>>>>>>>>
> > >> >>>>>>>>> http://www.maya.tv/en/character/flip
> > >> >>>>>>>>>
> > >> >>>>>>>>> -Matthias
> > >> >>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>> On 06/28/2016 06:26 PM, Aljoscha Krettek wrote:
> > >> >>>>>>>>>> I'm proposing to add a formal process for how we deal with
> > >> >>>> (major)
> > >> >>>>>>>>>> improvements to Flink and design docs. This has been
> > mentioned
> > >> >>>>> several
> > >> >>>>>>>>>> times recently but we never took any decisive action to
> > actually
> > >> >>>>>>>> implement
> > >> >>>>>>>>>> such a process so here we go.
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> Right now, we have Jira issues and we sometimes we have
> > design
> > >> >>>> docs
> > >> >>>>>>>> that we
> > >> >>>>>>>>>> keep in Google Docs. Jamie recently added links to those
> > that he
> > >> >>>>> could
> > >> >>>>>>>> find
> > >> >>>>>>>>>> on the mailing list to the Flink wiki:
> > >> >>>>>>>>>>
> > >> >>>>
> > https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
> > >> >>>>>>>> The
> > >> >>>>>>>>>> problem with these is that a) the comments on the Google
> Docs
> > >> >>>> are not
> > >> >>>>>>>>>> reflected in Jira and the mailing list. There has been some
> > very
> > >> >>>>>>> active
> > >> >>>>>>>>>> discussion on some of the docs that most people would never
> > >> >>>> notice.
> > >> >>>>>>> The
> > >> >>>>>>>>>> community therefore might seem less active than it actually
> > is.
> > >> >>>> b)
> > >> >>>>> the
> > >> >>>>>>>>>> documents are not very discoverable, if we had a clearly
> > defined
> > >> >>>>> place
> > >> >>>>>>>>>> where we put them and also prominently link to this on the
> > Flink
> > >> >>>>>>>> homepage
> > >> >>>>>>>>>> this would greatly help people that try to find out about
> > >> current
> > >> >>>>>>>>>> developments.
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> Kafka has a process like this:
> > >> >>>>>>>>>>
> > >> >>>>>>>>
> > >> >>>>>>>
> > >> >>>>>
> > >> >>>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> > >> >>>>>>>> .
> > >> >>>>>>>>>> They call it KIP, for Kafka Improvement Proposal. We could
> > >> either
> > >> >>>>>>> adapt
> > >> >>>>>>>>>> this for Flink or come up with our own process. Doing the
> > former
> > >> >>>>> would
> > >> >>>>>>>> save
> > >> >>>>>>>>>> us a lot of time and I don't think the Kafka community
> would
> > >> >>>> mind us
> > >> >>>>>>>>>> copying their process. The subject also hints at this, our
> > >> >>>> process
> > >> >>>>>>>> could be
> > >> >>>>>>>>>> called FLIP, for Flink Improvement Proposal.
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> What do you think? Feedback is highly welcome. :-)
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> Cheers,
> > >> >>>>>>>>>> Aljoscha
> > >> >>>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>
> > >> >>>>>>>
> > >> >>>>>>
> > >> >>>>>
> > >> >>>>>
> > >> >>>>
> > >> >>>
> > >> >
> > >>
> > >>
> >
>

Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal

[Aljoscha Krettek <al...@apache.org>]

Of course, this would make the list look like this:

All of the following are public interfaces that people build around:
 - DataStream and DataSet API, including classes related to that, such as
StreamExecutionEnvironment
 - Classes marked with the @Public annotation
 - On-disk binary formats, such as checkpoints/savepoints
 - User-facing scripts, i.e. bin/flink, Yarn scripts, Mesos scripts
 - Configuration
 - Monitoring

On Fri, 8 Jul 2016 at 16:00 Ufuk Celebi <uc...@apache.org> wrote:

> What about the scripts?
>
> On Fri, Jul 8, 2016 at 2:47 PM, Aljoscha Krettek <al...@apache.org>
> wrote:
> > Thanks a lot, Matthias! I created an actual "FLIP-Template" based on the
> > template you copied over.
> >
> > I propose to change the list of public interfaces to this:
> >
> > All of the following are public interfaces that people build around:
> >  - DataStream and DataSet API, including classes related to that, such as
> > StreamExecutionEnvironment
> >  - Classes marked with the @Public annotation
> >  - On-disk binary formats, such as checkpoints/savepoints
> >  - Configuration
> >  - Monitoring
> >
> > Cheers,
> > Aljoscha
> >
> > On Fri, 8 Jul 2016 at 13:49 Matthias J. Sax <mj...@apache.org> wrote:
> >
> >> Done.
> >>
> >> I replaced KIP -> FLIP and Kafka -> Flink. Still there are some things
> >> we need to rework. Please have a look and propose changes.
> >>
> >>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
> >>
> >> Same for FLIP Template
> >>
> >> https://cwiki.apache.org/confluence/display/FLINK/FLIP+Template
> >>
> >>
> >> -Matthias
> >>
> >> On 07/08/2016 09:56 AM, Aljoscha Krettek wrote:
> >> > Hi,
> >> > I got this reply from one of the Kafka committers:
> >> >
> >> > Thanks for sharing your intention to use a process similar to our KIP
> >> >
> >> > process. You are more than welcome to copy the structures and docs
> that
> >> we
> >> >
> >> > have for the KIP process. :)
> >> >
> >> > Ismael
> >> >
> >> >
> >> > So it seems we're good to go. @Matthias, since you are a Kafka
> >> contributor,
> >> > could you maybe copy the relevant docs from the Kafka wiki to our
> wiki?
> >> The
> >> > source for this page:
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> >> > and
> >> > the KIP template that is referenced there. If you want, I can take it
> >> from
> >> > there and adapt it to Flink and then let everyone discuss it again.
> >> >
> >> >
> >> > Cheers,
> >> >
> >> > Aljoscha
> >> >
> >> > On Thu, 7 Jul 2016 at 11:38 Aljoscha Krettek <al...@apache.org>
> >> wrote:
> >> >
> >> >> I'll reach out to the Kafka community and ask if it's ok for us to
> >> "steal"
> >> >> their process for this.
> >> >>
> >> >> On Thu, 7 Jul 2016 at 11:36 Aljoscha Krettek <al...@apache.org>
> >> wrote:
> >> >>
> >> >>> @Matthias: Yes, this is the reason why I like the KIP process and
> why I
> >> >>> said "The problem with these is that a) the comments on the Google
> >> Docs are
> >> >>> not reflected in Jira and the mailing list. There has been some very
> >> active
> >> >>> discussion on some of the docs that most people would never
> notice.".
> >> >>>
> >> >>> On Thu, 7 Jul 2016 at 11:28 Robert Metzger <rm...@apache.org>
> >> wrote:
> >> >>>
> >> >>>> I also like the proposal. I think its an issue that Google Docs
> >> comments
> >> >>>> are not reflected within ASF infra. Therefore, I'm +1 on discussing
> >> the
> >> >>>> proposals on the mailing list.
> >> >>>>
> >> >>>> I agree that we need to clean up our wiki.
> >> >>>>
> >> >>>> On Thu, Jul 7, 2016 at 10:58 AM, Matthias J. Sax <mjsax@apache.org
> >
> >> >>>> wrote:
> >> >>>>
> >> >>>>> Just to point out one thing about Kafka KIPs and using the project
> >> >>>> wiki:
> >> >>>>>
> >> >>>>> The wiki contains the current state of the proposal, while the
> >> >>>>> discussion is covered over the dev-mailing list. IMHO, this makes
> a
> >> lot
> >> >>>>> of sense, as people tend to follow the mailing list but not wiki
> >> >>>>> changes. Furthermore, the mailing list tracks the whole discussion
> >> >>>>> history, while the proposal is kept in a clean state and thus
> easy to
> >> >>>> read.
> >> >>>>>
> >> >>>>> -Matthias
> >> >>>>>
> >> >>>>>
> >> >>>>> On 07/06/2016 10:09 PM, Aljoscha Krettek wrote:
> >> >>>>>> Jip, that's why I referenced the Kafka process which is also in
> >> their
> >> >>>>> wiki.
> >> >>>>>>
> >> >>>>>> On Wed, 6 Jul 2016 at 21:01 Stephan Ewen <se...@apache.org>
> wrote:
> >> >>>>>>
> >> >>>>>>> Yes, big +1
> >> >>>>>>>
> >> >>>>>>> I had actually talked about the same thing with some people as
> >> well.
> >> >>>>>>>
> >> >>>>>>> I am currently sketching a few FLIPs for things, like
> improvements
> >> >>>> to
> >> >>>>> the
> >> >>>>>>> Yarn/Mesos/Kubernetes integration
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> One thing we should do here is to actually structure the wiki a
> bit
> >> >>>> to
> >> >>>>> make
> >> >>>>>>> it easier to find information and proposals.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk Celebi <uc...@apache.org>
> >> wrote:
> >> >>>>>>>
> >> >>>>>>>> Hey Aljoscha,
> >> >>>>>>>>
> >> >>>>>>>> thanks for this proposal. I've somehow missed it last week. I
> like
> >> >>>> the
> >> >>>>>>>> idea very much and agree with your assessment about the
> problems
> >> >>>> with
> >> >>>>>>>> the Google Doc approach.
> >> >>>>>>>>
> >> >>>>>>>> Regarding the process: I'm also in favour of adopting it from
> >> >>>> Kafka. I
> >> >>>>>>>> would not expect any problems with this, but we can post a
> quick
> >> >>>> note
> >> >>>>>>>> to their ML.
> >> >>>>>>>>
> >> >>>>>>>> @Matthias: The name works for me. ;-)
> >> >>>>>>>>
> >> >>>>>>>> – Ufuk
> >> >>>>>>>>
> >> >>>>>>>> On Tue, Jun 28, 2016 at 10:19 PM, Matthias J. Sax <
> >> >>>> mjsax@apache.org>
> >> >>>>>>>> wrote:
> >> >>>>>>>>> FLIP ?? Really? :D
> >> >>>>>>>>>
> >> >>>>>>>>> http://www.maya.tv/en/character/flip
> >> >>>>>>>>>
> >> >>>>>>>>> -Matthias
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>> On 06/28/2016 06:26 PM, Aljoscha Krettek wrote:
> >> >>>>>>>>>> I'm proposing to add a formal process for how we deal with
> >> >>>> (major)
> >> >>>>>>>>>> improvements to Flink and design docs. This has been
> mentioned
> >> >>>>> several
> >> >>>>>>>>>> times recently but we never took any decisive action to
> actually
> >> >>>>>>>> implement
> >> >>>>>>>>>> such a process so here we go.
> >> >>>>>>>>>>
> >> >>>>>>>>>> Right now, we have Jira issues and we sometimes we have
> design
> >> >>>> docs
> >> >>>>>>>> that we
> >> >>>>>>>>>> keep in Google Docs. Jamie recently added links to those
> that he
> >> >>>>> could
> >> >>>>>>>> find
> >> >>>>>>>>>> on the mailing list to the Flink wiki:
> >> >>>>>>>>>>
> >> >>>>
> https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
> >> >>>>>>>> The
> >> >>>>>>>>>> problem with these is that a) the comments on the Google Docs
> >> >>>> are not
> >> >>>>>>>>>> reflected in Jira and the mailing list. There has been some
> very
> >> >>>>>>> active
> >> >>>>>>>>>> discussion on some of the docs that most people would never
> >> >>>> notice.
> >> >>>>>>> The
> >> >>>>>>>>>> community therefore might seem less active than it actually
> is.
> >> >>>> b)
> >> >>>>> the
> >> >>>>>>>>>> documents are not very discoverable, if we had a clearly
> defined
> >> >>>>> place
> >> >>>>>>>>>> where we put them and also prominently link to this on the
> Flink
> >> >>>>>>>> homepage
> >> >>>>>>>>>> this would greatly help people that try to find out about
> >> current
> >> >>>>>>>>>> developments.
> >> >>>>>>>>>>
> >> >>>>>>>>>> Kafka has a process like this:
> >> >>>>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>
> >> >>>>>
> >> >>>>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> >> >>>>>>>> .
> >> >>>>>>>>>> They call it KIP, for Kafka Improvement Proposal. We could
> >> either
> >> >>>>>>> adapt
> >> >>>>>>>>>> this for Flink or come up with our own process. Doing the
> former
> >> >>>>> would
> >> >>>>>>>> save
> >> >>>>>>>>>> us a lot of time and I don't think the Kafka community would
> >> >>>> mind us
> >> >>>>>>>>>> copying their process. The subject also hints at this, our
> >> >>>> process
> >> >>>>>>>> could be
> >> >>>>>>>>>> called FLIP, for Flink Improvement Proposal.
> >> >>>>>>>>>>
> >> >>>>>>>>>> What do you think? Feedback is highly welcome. :-)
> >> >>>>>>>>>>
> >> >>>>>>>>>> Cheers,
> >> >>>>>>>>>> Aljoscha
> >> >>>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>
> >> >>>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>
> >> >>>
> >> >
> >>
> >>
>

Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal

[Ufuk Celebi <uc...@apache.org>]

What about the scripts?

On Fri, Jul 8, 2016 at 2:47 PM, Aljoscha Krettek <al...@apache.org> wrote:
> Thanks a lot, Matthias! I created an actual "FLIP-Template" based on the
> template you copied over.
>
> I propose to change the list of public interfaces to this:
>
> All of the following are public interfaces that people build around:
>  - DataStream and DataSet API, including classes related to that, such as
> StreamExecutionEnvironment
>  - Classes marked with the @Public annotation
>  - On-disk binary formats, such as checkpoints/savepoints
>  - Configuration
>  - Monitoring
>
> Cheers,
> Aljoscha
>
> On Fri, 8 Jul 2016 at 13:49 Matthias J. Sax <mj...@apache.org> wrote:
>
>> Done.
>>
>> I replaced KIP -> FLIP and Kafka -> Flink. Still there are some things
>> we need to rework. Please have a look and propose changes.
>>
>>
>> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
>>
>> Same for FLIP Template
>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP+Template
>>
>>
>> -Matthias
>>
>> On 07/08/2016 09:56 AM, Aljoscha Krettek wrote:
>> > Hi,
>> > I got this reply from one of the Kafka committers:
>> >
>> > Thanks for sharing your intention to use a process similar to our KIP
>> >
>> > process. You are more than welcome to copy the structures and docs that
>> we
>> >
>> > have for the KIP process. :)
>> >
>> > Ismael
>> >
>> >
>> > So it seems we're good to go. @Matthias, since you are a Kafka
>> contributor,
>> > could you maybe copy the relevant docs from the Kafka wiki to our wiki?
>> The
>> > source for this page:
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>> > and
>> > the KIP template that is referenced there. If you want, I can take it
>> from
>> > there and adapt it to Flink and then let everyone discuss it again.
>> >
>> >
>> > Cheers,
>> >
>> > Aljoscha
>> >
>> > On Thu, 7 Jul 2016 at 11:38 Aljoscha Krettek <al...@apache.org>
>> wrote:
>> >
>> >> I'll reach out to the Kafka community and ask if it's ok for us to
>> "steal"
>> >> their process for this.
>> >>
>> >> On Thu, 7 Jul 2016 at 11:36 Aljoscha Krettek <al...@apache.org>
>> wrote:
>> >>
>> >>> @Matthias: Yes, this is the reason why I like the KIP process and why I
>> >>> said "The problem with these is that a) the comments on the Google
>> Docs are
>> >>> not reflected in Jira and the mailing list. There has been some very
>> active
>> >>> discussion on some of the docs that most people would never notice.".
>> >>>
>> >>> On Thu, 7 Jul 2016 at 11:28 Robert Metzger <rm...@apache.org>
>> wrote:
>> >>>
>> >>>> I also like the proposal. I think its an issue that Google Docs
>> comments
>> >>>> are not reflected within ASF infra. Therefore, I'm +1 on discussing
>> the
>> >>>> proposals on the mailing list.
>> >>>>
>> >>>> I agree that we need to clean up our wiki.
>> >>>>
>> >>>> On Thu, Jul 7, 2016 at 10:58 AM, Matthias J. Sax <mj...@apache.org>
>> >>>> wrote:
>> >>>>
>> >>>>> Just to point out one thing about Kafka KIPs and using the project
>> >>>> wiki:
>> >>>>>
>> >>>>> The wiki contains the current state of the proposal, while the
>> >>>>> discussion is covered over the dev-mailing list. IMHO, this makes a
>> lot
>> >>>>> of sense, as people tend to follow the mailing list but not wiki
>> >>>>> changes. Furthermore, the mailing list tracks the whole discussion
>> >>>>> history, while the proposal is kept in a clean state and thus easy to
>> >>>> read.
>> >>>>>
>> >>>>> -Matthias
>> >>>>>
>> >>>>>
>> >>>>> On 07/06/2016 10:09 PM, Aljoscha Krettek wrote:
>> >>>>>> Jip, that's why I referenced the Kafka process which is also in
>> their
>> >>>>> wiki.
>> >>>>>>
>> >>>>>> On Wed, 6 Jul 2016 at 21:01 Stephan Ewen <se...@apache.org> wrote:
>> >>>>>>
>> >>>>>>> Yes, big +1
>> >>>>>>>
>> >>>>>>> I had actually talked about the same thing with some people as
>> well.
>> >>>>>>>
>> >>>>>>> I am currently sketching a few FLIPs for things, like improvements
>> >>>> to
>> >>>>> the
>> >>>>>>> Yarn/Mesos/Kubernetes integration
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> One thing we should do here is to actually structure the wiki a bit
>> >>>> to
>> >>>>> make
>> >>>>>>> it easier to find information and proposals.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk Celebi <uc...@apache.org>
>> wrote:
>> >>>>>>>
>> >>>>>>>> Hey Aljoscha,
>> >>>>>>>>
>> >>>>>>>> thanks for this proposal. I've somehow missed it last week. I like
>> >>>> the
>> >>>>>>>> idea very much and agree with your assessment about the problems
>> >>>> with
>> >>>>>>>> the Google Doc approach.
>> >>>>>>>>
>> >>>>>>>> Regarding the process: I'm also in favour of adopting it from
>> >>>> Kafka. I
>> >>>>>>>> would not expect any problems with this, but we can post a quick
>> >>>> note
>> >>>>>>>> to their ML.
>> >>>>>>>>
>> >>>>>>>> @Matthias: The name works for me. ;-)
>> >>>>>>>>
>> >>>>>>>> – Ufuk
>> >>>>>>>>
>> >>>>>>>> On Tue, Jun 28, 2016 at 10:19 PM, Matthias J. Sax <
>> >>>> mjsax@apache.org>
>> >>>>>>>> wrote:
>> >>>>>>>>> FLIP ?? Really? :D
>> >>>>>>>>>
>> >>>>>>>>> http://www.maya.tv/en/character/flip
>> >>>>>>>>>
>> >>>>>>>>> -Matthias
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On 06/28/2016 06:26 PM, Aljoscha Krettek wrote:
>> >>>>>>>>>> I'm proposing to add a formal process for how we deal with
>> >>>> (major)
>> >>>>>>>>>> improvements to Flink and design docs. This has been mentioned
>> >>>>> several
>> >>>>>>>>>> times recently but we never took any decisive action to actually
>> >>>>>>>> implement
>> >>>>>>>>>> such a process so here we go.
>> >>>>>>>>>>
>> >>>>>>>>>> Right now, we have Jira issues and we sometimes we have design
>> >>>> docs
>> >>>>>>>> that we
>> >>>>>>>>>> keep in Google Docs. Jamie recently added links to those that he
>> >>>>> could
>> >>>>>>>> find
>> >>>>>>>>>> on the mailing list to the Flink wiki:
>> >>>>>>>>>>
>> >>>> https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
>> >>>>>>>> The
>> >>>>>>>>>> problem with these is that a) the comments on the Google Docs
>> >>>> are not
>> >>>>>>>>>> reflected in Jira and the mailing list. There has been some very
>> >>>>>>> active
>> >>>>>>>>>> discussion on some of the docs that most people would never
>> >>>> notice.
>> >>>>>>> The
>> >>>>>>>>>> community therefore might seem less active than it actually is.
>> >>>> b)
>> >>>>> the
>> >>>>>>>>>> documents are not very discoverable, if we had a clearly defined
>> >>>>> place
>> >>>>>>>>>> where we put them and also prominently link to this on the Flink
>> >>>>>>>> homepage
>> >>>>>>>>>> this would greatly help people that try to find out about
>> current
>> >>>>>>>>>> developments.
>> >>>>>>>>>>
>> >>>>>>>>>> Kafka has a process like this:
>> >>>>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>> >>>>>>>> .
>> >>>>>>>>>> They call it KIP, for Kafka Improvement Proposal. We could
>> either
>> >>>>>>> adapt
>> >>>>>>>>>> this for Flink or come up with our own process. Doing the former
>> >>>>> would
>> >>>>>>>> save
>> >>>>>>>>>> us a lot of time and I don't think the Kafka community would
>> >>>> mind us
>> >>>>>>>>>> copying their process. The subject also hints at this, our
>> >>>> process
>> >>>>>>>> could be
>> >>>>>>>>>> called FLIP, for Flink Improvement Proposal.
>> >>>>>>>>>>
>> >>>>>>>>>> What do you think? Feedback is highly welcome. :-)
>> >>>>>>>>>>
>> >>>>>>>>>> Cheers,
>> >>>>>>>>>> Aljoscha
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>
>> >>>
>> >
>>
>>

Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal

[Aljoscha Krettek <al...@apache.org>]

Thanks a lot, Matthias! I created an actual "FLIP-Template" based on the
template you copied over.

I propose to change the list of public interfaces to this:

All of the following are public interfaces that people build around:
 - DataStream and DataSet API, including classes related to that, such as
StreamExecutionEnvironment
 - Classes marked with the @Public annotation
 - On-disk binary formats, such as checkpoints/savepoints
 - Configuration
 - Monitoring

Cheers,
Aljoscha

On Fri, 8 Jul 2016 at 13:49 Matthias J. Sax <mj...@apache.org> wrote:

> Done.
>
> I replaced KIP -> FLIP and Kafka -> Flink. Still there are some things
> we need to rework. Please have a look and propose changes.
>
>
> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
>
> Same for FLIP Template
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP+Template
>
>
> -Matthias
>
> On 07/08/2016 09:56 AM, Aljoscha Krettek wrote:
> > Hi,
> > I got this reply from one of the Kafka committers:
> >
> > Thanks for sharing your intention to use a process similar to our KIP
> >
> > process. You are more than welcome to copy the structures and docs that
> we
> >
> > have for the KIP process. :)
> >
> > Ismael
> >
> >
> > So it seems we're good to go. @Matthias, since you are a Kafka
> contributor,
> > could you maybe copy the relevant docs from the Kafka wiki to our wiki?
> The
> > source for this page:
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> > and
> > the KIP template that is referenced there. If you want, I can take it
> from
> > there and adapt it to Flink and then let everyone discuss it again.
> >
> >
> > Cheers,
> >
> > Aljoscha
> >
> > On Thu, 7 Jul 2016 at 11:38 Aljoscha Krettek <al...@apache.org>
> wrote:
> >
> >> I'll reach out to the Kafka community and ask if it's ok for us to
> "steal"
> >> their process for this.
> >>
> >> On Thu, 7 Jul 2016 at 11:36 Aljoscha Krettek <al...@apache.org>
> wrote:
> >>
> >>> @Matthias: Yes, this is the reason why I like the KIP process and why I
> >>> said "The problem with these is that a) the comments on the Google
> Docs are
> >>> not reflected in Jira and the mailing list. There has been some very
> active
> >>> discussion on some of the docs that most people would never notice.".
> >>>
> >>> On Thu, 7 Jul 2016 at 11:28 Robert Metzger <rm...@apache.org>
> wrote:
> >>>
> >>>> I also like the proposal. I think its an issue that Google Docs
> comments
> >>>> are not reflected within ASF infra. Therefore, I'm +1 on discussing
> the
> >>>> proposals on the mailing list.
> >>>>
> >>>> I agree that we need to clean up our wiki.
> >>>>
> >>>> On Thu, Jul 7, 2016 at 10:58 AM, Matthias J. Sax <mj...@apache.org>
> >>>> wrote:
> >>>>
> >>>>> Just to point out one thing about Kafka KIPs and using the project
> >>>> wiki:
> >>>>>
> >>>>> The wiki contains the current state of the proposal, while the
> >>>>> discussion is covered over the dev-mailing list. IMHO, this makes a
> lot
> >>>>> of sense, as people tend to follow the mailing list but not wiki
> >>>>> changes. Furthermore, the mailing list tracks the whole discussion
> >>>>> history, while the proposal is kept in a clean state and thus easy to
> >>>> read.
> >>>>>
> >>>>> -Matthias
> >>>>>
> >>>>>
> >>>>> On 07/06/2016 10:09 PM, Aljoscha Krettek wrote:
> >>>>>> Jip, that's why I referenced the Kafka process which is also in
> their
> >>>>> wiki.
> >>>>>>
> >>>>>> On Wed, 6 Jul 2016 at 21:01 Stephan Ewen <se...@apache.org> wrote:
> >>>>>>
> >>>>>>> Yes, big +1
> >>>>>>>
> >>>>>>> I had actually talked about the same thing with some people as
> well.
> >>>>>>>
> >>>>>>> I am currently sketching a few FLIPs for things, like improvements
> >>>> to
> >>>>> the
> >>>>>>> Yarn/Mesos/Kubernetes integration
> >>>>>>>
> >>>>>>>
> >>>>>>> One thing we should do here is to actually structure the wiki a bit
> >>>> to
> >>>>> make
> >>>>>>> it easier to find information and proposals.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk Celebi <uc...@apache.org>
> wrote:
> >>>>>>>
> >>>>>>>> Hey Aljoscha,
> >>>>>>>>
> >>>>>>>> thanks for this proposal. I've somehow missed it last week. I like
> >>>> the
> >>>>>>>> idea very much and agree with your assessment about the problems
> >>>> with
> >>>>>>>> the Google Doc approach.
> >>>>>>>>
> >>>>>>>> Regarding the process: I'm also in favour of adopting it from
> >>>> Kafka. I
> >>>>>>>> would not expect any problems with this, but we can post a quick
> >>>> note
> >>>>>>>> to their ML.
> >>>>>>>>
> >>>>>>>> @Matthias: The name works for me. ;-)
> >>>>>>>>
> >>>>>>>> – Ufuk
> >>>>>>>>
> >>>>>>>> On Tue, Jun 28, 2016 at 10:19 PM, Matthias J. Sax <
> >>>> mjsax@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>> FLIP ?? Really? :D
> >>>>>>>>>
> >>>>>>>>> http://www.maya.tv/en/character/flip
> >>>>>>>>>
> >>>>>>>>> -Matthias
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On 06/28/2016 06:26 PM, Aljoscha Krettek wrote:
> >>>>>>>>>> I'm proposing to add a formal process for how we deal with
> >>>> (major)
> >>>>>>>>>> improvements to Flink and design docs. This has been mentioned
> >>>>> several
> >>>>>>>>>> times recently but we never took any decisive action to actually
> >>>>>>>> implement
> >>>>>>>>>> such a process so here we go.
> >>>>>>>>>>
> >>>>>>>>>> Right now, we have Jira issues and we sometimes we have design
> >>>> docs
> >>>>>>>> that we
> >>>>>>>>>> keep in Google Docs. Jamie recently added links to those that he
> >>>>> could
> >>>>>>>> find
> >>>>>>>>>> on the mailing list to the Flink wiki:
> >>>>>>>>>>
> >>>> https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
> >>>>>>>> The
> >>>>>>>>>> problem with these is that a) the comments on the Google Docs
> >>>> are not
> >>>>>>>>>> reflected in Jira and the mailing list. There has been some very
> >>>>>>> active
> >>>>>>>>>> discussion on some of the docs that most people would never
> >>>> notice.
> >>>>>>> The
> >>>>>>>>>> community therefore might seem less active than it actually is.
> >>>> b)
> >>>>> the
> >>>>>>>>>> documents are not very discoverable, if we had a clearly defined
> >>>>> place
> >>>>>>>>>> where we put them and also prominently link to this on the Flink
> >>>>>>>> homepage
> >>>>>>>>>> this would greatly help people that try to find out about
> current
> >>>>>>>>>> developments.
> >>>>>>>>>>
> >>>>>>>>>> Kafka has a process like this:
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> >>>>>>>> .
> >>>>>>>>>> They call it KIP, for Kafka Improvement Proposal. We could
> either
> >>>>>>> adapt
> >>>>>>>>>> this for Flink or come up with our own process. Doing the former
> >>>>> would
> >>>>>>>> save
> >>>>>>>>>> us a lot of time and I don't think the Kafka community would
> >>>> mind us
> >>>>>>>>>> copying their process. The subject also hints at this, our
> >>>> process
> >>>>>>>> could be
> >>>>>>>>>> called FLIP, for Flink Improvement Proposal.
> >>>>>>>>>>
> >>>>>>>>>> What do you think? Feedback is highly welcome. :-)
> >>>>>>>>>>
> >>>>>>>>>> Cheers,
> >>>>>>>>>> Aljoscha
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >
>
>

Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal

["Matthias J. Sax" <mj...@apache.org>]

Done.

I replaced KIP -> FLIP and Kafka -> Flink. Still there are some things
we need to rework. Please have a look and propose changes.

https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals

Same for FLIP Template

https://cwiki.apache.org/confluence/display/FLINK/FLIP+Template


-Matthias

On 07/08/2016 09:56 AM, Aljoscha Krettek wrote:
> Hi,
> I got this reply from one of the Kafka committers:
> 
> Thanks for sharing your intention to use a process similar to our KIP
> 
> process. You are more than welcome to copy the structures and docs that we
> 
> have for the KIP process. :)
> 
> Ismael
> 
> 
> So it seems we're good to go. @Matthias, since you are a Kafka contributor,
> could you maybe copy the relevant docs from the Kafka wiki to our wiki? The
> source for this page:
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> and
> the KIP template that is referenced there. If you want, I can take it from
> there and adapt it to Flink and then let everyone discuss it again.
> 
> 
> Cheers,
> 
> Aljoscha
> 
> On Thu, 7 Jul 2016 at 11:38 Aljoscha Krettek <al...@apache.org> wrote:
> 
>> I'll reach out to the Kafka community and ask if it's ok for us to "steal"
>> their process for this.
>>
>> On Thu, 7 Jul 2016 at 11:36 Aljoscha Krettek <al...@apache.org> wrote:
>>
>>> @Matthias: Yes, this is the reason why I like the KIP process and why I
>>> said "The problem with these is that a) the comments on the Google Docs are
>>> not reflected in Jira and the mailing list. There has been some very active
>>> discussion on some of the docs that most people would never notice.".
>>>
>>> On Thu, 7 Jul 2016 at 11:28 Robert Metzger <rm...@apache.org> wrote:
>>>
>>>> I also like the proposal. I think its an issue that Google Docs comments
>>>> are not reflected within ASF infra. Therefore, I'm +1 on discussing the
>>>> proposals on the mailing list.
>>>>
>>>> I agree that we need to clean up our wiki.
>>>>
>>>> On Thu, Jul 7, 2016 at 10:58 AM, Matthias J. Sax <mj...@apache.org>
>>>> wrote:
>>>>
>>>>> Just to point out one thing about Kafka KIPs and using the project
>>>> wiki:
>>>>>
>>>>> The wiki contains the current state of the proposal, while the
>>>>> discussion is covered over the dev-mailing list. IMHO, this makes a lot
>>>>> of sense, as people tend to follow the mailing list but not wiki
>>>>> changes. Furthermore, the mailing list tracks the whole discussion
>>>>> history, while the proposal is kept in a clean state and thus easy to
>>>> read.
>>>>>
>>>>> -Matthias
>>>>>
>>>>>
>>>>> On 07/06/2016 10:09 PM, Aljoscha Krettek wrote:
>>>>>> Jip, that's why I referenced the Kafka process which is also in their
>>>>> wiki.
>>>>>>
>>>>>> On Wed, 6 Jul 2016 at 21:01 Stephan Ewen <se...@apache.org> wrote:
>>>>>>
>>>>>>> Yes, big +1
>>>>>>>
>>>>>>> I had actually talked about the same thing with some people as well.
>>>>>>>
>>>>>>> I am currently sketching a few FLIPs for things, like improvements
>>>> to
>>>>> the
>>>>>>> Yarn/Mesos/Kubernetes integration
>>>>>>>
>>>>>>>
>>>>>>> One thing we should do here is to actually structure the wiki a bit
>>>> to
>>>>> make
>>>>>>> it easier to find information and proposals.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk Celebi <uc...@apache.org> wrote:
>>>>>>>
>>>>>>>> Hey Aljoscha,
>>>>>>>>
>>>>>>>> thanks for this proposal. I've somehow missed it last week. I like
>>>> the
>>>>>>>> idea very much and agree with your assessment about the problems
>>>> with
>>>>>>>> the Google Doc approach.
>>>>>>>>
>>>>>>>> Regarding the process: I'm also in favour of adopting it from
>>>> Kafka. I
>>>>>>>> would not expect any problems with this, but we can post a quick
>>>> note
>>>>>>>> to their ML.
>>>>>>>>
>>>>>>>> @Matthias: The name works for me. ;-)
>>>>>>>>
>>>>>>>> – Ufuk
>>>>>>>>
>>>>>>>> On Tue, Jun 28, 2016 at 10:19 PM, Matthias J. Sax <
>>>> mjsax@apache.org>
>>>>>>>> wrote:
>>>>>>>>> FLIP ?? Really? :D
>>>>>>>>>
>>>>>>>>> http://www.maya.tv/en/character/flip
>>>>>>>>>
>>>>>>>>> -Matthias
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 06/28/2016 06:26 PM, Aljoscha Krettek wrote:
>>>>>>>>>> I'm proposing to add a formal process for how we deal with
>>>> (major)
>>>>>>>>>> improvements to Flink and design docs. This has been mentioned
>>>>> several
>>>>>>>>>> times recently but we never took any decisive action to actually
>>>>>>>> implement
>>>>>>>>>> such a process so here we go.
>>>>>>>>>>
>>>>>>>>>> Right now, we have Jira issues and we sometimes we have design
>>>> docs
>>>>>>>> that we
>>>>>>>>>> keep in Google Docs. Jamie recently added links to those that he
>>>>> could
>>>>>>>> find
>>>>>>>>>> on the mailing list to the Flink wiki:
>>>>>>>>>>
>>>> https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
>>>>>>>> The
>>>>>>>>>> problem with these is that a) the comments on the Google Docs
>>>> are not
>>>>>>>>>> reflected in Jira and the mailing list. There has been some very
>>>>>>> active
>>>>>>>>>> discussion on some of the docs that most people would never
>>>> notice.
>>>>>>> The
>>>>>>>>>> community therefore might seem less active than it actually is.
>>>> b)
>>>>> the
>>>>>>>>>> documents are not very discoverable, if we had a clearly defined
>>>>> place
>>>>>>>>>> where we put them and also prominently link to this on the Flink
>>>>>>>> homepage
>>>>>>>>>> this would greatly help people that try to find out about current
>>>>>>>>>> developments.
>>>>>>>>>>
>>>>>>>>>> Kafka has a process like this:
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>>>>>>>> .
>>>>>>>>>> They call it KIP, for Kafka Improvement Proposal. We could either
>>>>>>> adapt
>>>>>>>>>> this for Flink or come up with our own process. Doing the former
>>>>> would
>>>>>>>> save
>>>>>>>>>> us a lot of time and I don't think the Kafka community would
>>>> mind us
>>>>>>>>>> copying their process. The subject also hints at this, our
>>>> process
>>>>>>>> could be
>>>>>>>>>> called FLIP, for Flink Improvement Proposal.
>>>>>>>>>>
>>>>>>>>>> What do you think? Feedback is highly welcome. :-)
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Aljoscha
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
> 

Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal

[Aljoscha Krettek <al...@apache.org>]

Hi,
I got this reply from one of the Kafka committers:

Thanks for sharing your intention to use a process similar to our KIP

process. You are more than welcome to copy the structures and docs that we

have for the KIP process. :)

Ismael


So it seems we're good to go. @Matthias, since you are a Kafka contributor,
could you maybe copy the relevant docs from the Kafka wiki to our wiki? The
source for this page:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
and
the KIP template that is referenced there. If you want, I can take it from
there and adapt it to Flink and then let everyone discuss it again.


Cheers,

Aljoscha

On Thu, 7 Jul 2016 at 11:38 Aljoscha Krettek <al...@apache.org> wrote:

> I'll reach out to the Kafka community and ask if it's ok for us to "steal"
> their process for this.
>
> On Thu, 7 Jul 2016 at 11:36 Aljoscha Krettek <al...@apache.org> wrote:
>
>> @Matthias: Yes, this is the reason why I like the KIP process and why I
>> said "The problem with these is that a) the comments on the Google Docs are
>> not reflected in Jira and the mailing list. There has been some very active
>> discussion on some of the docs that most people would never notice.".
>>
>> On Thu, 7 Jul 2016 at 11:28 Robert Metzger <rm...@apache.org> wrote:
>>
>>> I also like the proposal. I think its an issue that Google Docs comments
>>> are not reflected within ASF infra. Therefore, I'm +1 on discussing the
>>> proposals on the mailing list.
>>>
>>> I agree that we need to clean up our wiki.
>>>
>>> On Thu, Jul 7, 2016 at 10:58 AM, Matthias J. Sax <mj...@apache.org>
>>> wrote:
>>>
>>> > Just to point out one thing about Kafka KIPs and using the project
>>> wiki:
>>> >
>>> > The wiki contains the current state of the proposal, while the
>>> > discussion is covered over the dev-mailing list. IMHO, this makes a lot
>>> > of sense, as people tend to follow the mailing list but not wiki
>>> > changes. Furthermore, the mailing list tracks the whole discussion
>>> > history, while the proposal is kept in a clean state and thus easy to
>>> read.
>>> >
>>> > -Matthias
>>> >
>>> >
>>> > On 07/06/2016 10:09 PM, Aljoscha Krettek wrote:
>>> > > Jip, that's why I referenced the Kafka process which is also in their
>>> > wiki.
>>> > >
>>> > > On Wed, 6 Jul 2016 at 21:01 Stephan Ewen <se...@apache.org> wrote:
>>> > >
>>> > >> Yes, big +1
>>> > >>
>>> > >> I had actually talked about the same thing with some people as well.
>>> > >>
>>> > >> I am currently sketching a few FLIPs for things, like improvements
>>> to
>>> > the
>>> > >> Yarn/Mesos/Kubernetes integration
>>> > >>
>>> > >>
>>> > >> One thing we should do here is to actually structure the wiki a bit
>>> to
>>> > make
>>> > >> it easier to find information and proposals.
>>> > >>
>>> > >>
>>> > >>
>>> > >>
>>> > >> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk Celebi <uc...@apache.org> wrote:
>>> > >>
>>> > >>> Hey Aljoscha,
>>> > >>>
>>> > >>> thanks for this proposal. I've somehow missed it last week. I like
>>> the
>>> > >>> idea very much and agree with your assessment about the problems
>>> with
>>> > >>> the Google Doc approach.
>>> > >>>
>>> > >>> Regarding the process: I'm also in favour of adopting it from
>>> Kafka. I
>>> > >>> would not expect any problems with this, but we can post a quick
>>> note
>>> > >>> to their ML.
>>> > >>>
>>> > >>> @Matthias: The name works for me. ;-)
>>> > >>>
>>> > >>> – Ufuk
>>> > >>>
>>> > >>> On Tue, Jun 28, 2016 at 10:19 PM, Matthias J. Sax <
>>> mjsax@apache.org>
>>> > >>> wrote:
>>> > >>>> FLIP ?? Really? :D
>>> > >>>>
>>> > >>>> http://www.maya.tv/en/character/flip
>>> > >>>>
>>> > >>>> -Matthias
>>> > >>>>
>>> > >>>>
>>> > >>>> On 06/28/2016 06:26 PM, Aljoscha Krettek wrote:
>>> > >>>>> I'm proposing to add a formal process for how we deal with
>>> (major)
>>> > >>>>> improvements to Flink and design docs. This has been mentioned
>>> > several
>>> > >>>>> times recently but we never took any decisive action to actually
>>> > >>> implement
>>> > >>>>> such a process so here we go.
>>> > >>>>>
>>> > >>>>> Right now, we have Jira issues and we sometimes we have design
>>> docs
>>> > >>> that we
>>> > >>>>> keep in Google Docs. Jamie recently added links to those that he
>>> > could
>>> > >>> find
>>> > >>>>> on the mailing list to the Flink wiki:
>>> > >>>>>
>>> https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
>>> > >>> The
>>> > >>>>> problem with these is that a) the comments on the Google Docs
>>> are not
>>> > >>>>> reflected in Jira and the mailing list. There has been some very
>>> > >> active
>>> > >>>>> discussion on some of the docs that most people would never
>>> notice.
>>> > >> The
>>> > >>>>> community therefore might seem less active than it actually is.
>>> b)
>>> > the
>>> > >>>>> documents are not very discoverable, if we had a clearly defined
>>> > place
>>> > >>>>> where we put them and also prominently link to this on the Flink
>>> > >>> homepage
>>> > >>>>> this would greatly help people that try to find out about current
>>> > >>>>> developments.
>>> > >>>>>
>>> > >>>>> Kafka has a process like this:
>>> > >>>>>
>>> > >>>
>>> > >>
>>> >
>>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>>> > >>> .
>>> > >>>>> They call it KIP, for Kafka Improvement Proposal. We could either
>>> > >> adapt
>>> > >>>>> this for Flink or come up with our own process. Doing the former
>>> > would
>>> > >>> save
>>> > >>>>> us a lot of time and I don't think the Kafka community would
>>> mind us
>>> > >>>>> copying their process. The subject also hints at this, our
>>> process
>>> > >>> could be
>>> > >>>>> called FLIP, for Flink Improvement Proposal.
>>> > >>>>>
>>> > >>>>> What do you think? Feedback is highly welcome. :-)
>>> > >>>>>
>>> > >>>>> Cheers,
>>> > >>>>> Aljoscha
>>> > >>>>>
>>> > >>>>
>>> > >>>
>>> > >>
>>> > >
>>> >
>>> >
>>>
>>

Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal

[Aljoscha Krettek <al...@apache.org>]

I'll reach out to the Kafka community and ask if it's ok for us to "steal"
their process for this.

On Thu, 7 Jul 2016 at 11:36 Aljoscha Krettek <al...@apache.org> wrote:

> @Matthias: Yes, this is the reason why I like the KIP process and why I
> said "The problem with these is that a) the comments on the Google Docs are
> not reflected in Jira and the mailing list. There has been some very active
> discussion on some of the docs that most people would never notice.".
>
> On Thu, 7 Jul 2016 at 11:28 Robert Metzger <rm...@apache.org> wrote:
>
>> I also like the proposal. I think its an issue that Google Docs comments
>> are not reflected within ASF infra. Therefore, I'm +1 on discussing the
>> proposals on the mailing list.
>>
>> I agree that we need to clean up our wiki.
>>
>> On Thu, Jul 7, 2016 at 10:58 AM, Matthias J. Sax <mj...@apache.org>
>> wrote:
>>
>> > Just to point out one thing about Kafka KIPs and using the project wiki:
>> >
>> > The wiki contains the current state of the proposal, while the
>> > discussion is covered over the dev-mailing list. IMHO, this makes a lot
>> > of sense, as people tend to follow the mailing list but not wiki
>> > changes. Furthermore, the mailing list tracks the whole discussion
>> > history, while the proposal is kept in a clean state and thus easy to
>> read.
>> >
>> > -Matthias
>> >
>> >
>> > On 07/06/2016 10:09 PM, Aljoscha Krettek wrote:
>> > > Jip, that's why I referenced the Kafka process which is also in their
>> > wiki.
>> > >
>> > > On Wed, 6 Jul 2016 at 21:01 Stephan Ewen <se...@apache.org> wrote:
>> > >
>> > >> Yes, big +1
>> > >>
>> > >> I had actually talked about the same thing with some people as well.
>> > >>
>> > >> I am currently sketching a few FLIPs for things, like improvements to
>> > the
>> > >> Yarn/Mesos/Kubernetes integration
>> > >>
>> > >>
>> > >> One thing we should do here is to actually structure the wiki a bit
>> to
>> > make
>> > >> it easier to find information and proposals.
>> > >>
>> > >>
>> > >>
>> > >>
>> > >> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk Celebi <uc...@apache.org> wrote:
>> > >>
>> > >>> Hey Aljoscha,
>> > >>>
>> > >>> thanks for this proposal. I've somehow missed it last week. I like
>> the
>> > >>> idea very much and agree with your assessment about the problems
>> with
>> > >>> the Google Doc approach.
>> > >>>
>> > >>> Regarding the process: I'm also in favour of adopting it from
>> Kafka. I
>> > >>> would not expect any problems with this, but we can post a quick
>> note
>> > >>> to their ML.
>> > >>>
>> > >>> @Matthias: The name works for me. ;-)
>> > >>>
>> > >>> – Ufuk
>> > >>>
>> > >>> On Tue, Jun 28, 2016 at 10:19 PM, Matthias J. Sax <mjsax@apache.org
>> >
>> > >>> wrote:
>> > >>>> FLIP ?? Really? :D
>> > >>>>
>> > >>>> http://www.maya.tv/en/character/flip
>> > >>>>
>> > >>>> -Matthias
>> > >>>>
>> > >>>>
>> > >>>> On 06/28/2016 06:26 PM, Aljoscha Krettek wrote:
>> > >>>>> I'm proposing to add a formal process for how we deal with (major)
>> > >>>>> improvements to Flink and design docs. This has been mentioned
>> > several
>> > >>>>> times recently but we never took any decisive action to actually
>> > >>> implement
>> > >>>>> such a process so here we go.
>> > >>>>>
>> > >>>>> Right now, we have Jira issues and we sometimes we have design
>> docs
>> > >>> that we
>> > >>>>> keep in Google Docs. Jamie recently added links to those that he
>> > could
>> > >>> find
>> > >>>>> on the mailing list to the Flink wiki:
>> > >>>>>
>> https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
>> > >>> The
>> > >>>>> problem with these is that a) the comments on the Google Docs are
>> not
>> > >>>>> reflected in Jira and the mailing list. There has been some very
>> > >> active
>> > >>>>> discussion on some of the docs that most people would never
>> notice.
>> > >> The
>> > >>>>> community therefore might seem less active than it actually is. b)
>> > the
>> > >>>>> documents are not very discoverable, if we had a clearly defined
>> > place
>> > >>>>> where we put them and also prominently link to this on the Flink
>> > >>> homepage
>> > >>>>> this would greatly help people that try to find out about current
>> > >>>>> developments.
>> > >>>>>
>> > >>>>> Kafka has a process like this:
>> > >>>>>
>> > >>>
>> > >>
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>> > >>> .
>> > >>>>> They call it KIP, for Kafka Improvement Proposal. We could either
>> > >> adapt
>> > >>>>> this for Flink or come up with our own process. Doing the former
>> > would
>> > >>> save
>> > >>>>> us a lot of time and I don't think the Kafka community would mind
>> us
>> > >>>>> copying their process. The subject also hints at this, our process
>> > >>> could be
>> > >>>>> called FLIP, for Flink Improvement Proposal.
>> > >>>>>
>> > >>>>> What do you think? Feedback is highly welcome. :-)
>> > >>>>>
>> > >>>>> Cheers,
>> > >>>>> Aljoscha
>> > >>>>>
>> > >>>>
>> > >>>
>> > >>
>> > >
>> >
>> >
>>
>

Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal

[Aljoscha Krettek <al...@apache.org>]

@Matthias: Yes, this is the reason why I like the KIP process and why I
said "The problem with these is that a) the comments on the Google Docs are
not reflected in Jira and the mailing list. There has been some very active
discussion on some of the docs that most people would never notice.".

On Thu, 7 Jul 2016 at 11:28 Robert Metzger <rm...@apache.org> wrote:

> I also like the proposal. I think its an issue that Google Docs comments
> are not reflected within ASF infra. Therefore, I'm +1 on discussing the
> proposals on the mailing list.
>
> I agree that we need to clean up our wiki.
>
> On Thu, Jul 7, 2016 at 10:58 AM, Matthias J. Sax <mj...@apache.org> wrote:
>
> > Just to point out one thing about Kafka KIPs and using the project wiki:
> >
> > The wiki contains the current state of the proposal, while the
> > discussion is covered over the dev-mailing list. IMHO, this makes a lot
> > of sense, as people tend to follow the mailing list but not wiki
> > changes. Furthermore, the mailing list tracks the whole discussion
> > history, while the proposal is kept in a clean state and thus easy to
> read.
> >
> > -Matthias
> >
> >
> > On 07/06/2016 10:09 PM, Aljoscha Krettek wrote:
> > > Jip, that's why I referenced the Kafka process which is also in their
> > wiki.
> > >
> > > On Wed, 6 Jul 2016 at 21:01 Stephan Ewen <se...@apache.org> wrote:
> > >
> > >> Yes, big +1
> > >>
> > >> I had actually talked about the same thing with some people as well.
> > >>
> > >> I am currently sketching a few FLIPs for things, like improvements to
> > the
> > >> Yarn/Mesos/Kubernetes integration
> > >>
> > >>
> > >> One thing we should do here is to actually structure the wiki a bit to
> > make
> > >> it easier to find information and proposals.
> > >>
> > >>
> > >>
> > >>
> > >> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk Celebi <uc...@apache.org> wrote:
> > >>
> > >>> Hey Aljoscha,
> > >>>
> > >>> thanks for this proposal. I've somehow missed it last week. I like
> the
> > >>> idea very much and agree with your assessment about the problems with
> > >>> the Google Doc approach.
> > >>>
> > >>> Regarding the process: I'm also in favour of adopting it from Kafka.
> I
> > >>> would not expect any problems with this, but we can post a quick note
> > >>> to their ML.
> > >>>
> > >>> @Matthias: The name works for me. ;-)
> > >>>
> > >>> – Ufuk
> > >>>
> > >>> On Tue, Jun 28, 2016 at 10:19 PM, Matthias J. Sax <mj...@apache.org>
> > >>> wrote:
> > >>>> FLIP ?? Really? :D
> > >>>>
> > >>>> http://www.maya.tv/en/character/flip
> > >>>>
> > >>>> -Matthias
> > >>>>
> > >>>>
> > >>>> On 06/28/2016 06:26 PM, Aljoscha Krettek wrote:
> > >>>>> I'm proposing to add a formal process for how we deal with (major)
> > >>>>> improvements to Flink and design docs. This has been mentioned
> > several
> > >>>>> times recently but we never took any decisive action to actually
> > >>> implement
> > >>>>> such a process so here we go.
> > >>>>>
> > >>>>> Right now, we have Jira issues and we sometimes we have design docs
> > >>> that we
> > >>>>> keep in Google Docs. Jamie recently added links to those that he
> > could
> > >>> find
> > >>>>> on the mailing list to the Flink wiki:
> > >>>>>
> https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
> > >>> The
> > >>>>> problem with these is that a) the comments on the Google Docs are
> not
> > >>>>> reflected in Jira and the mailing list. There has been some very
> > >> active
> > >>>>> discussion on some of the docs that most people would never notice.
> > >> The
> > >>>>> community therefore might seem less active than it actually is. b)
> > the
> > >>>>> documents are not very discoverable, if we had a clearly defined
> > place
> > >>>>> where we put them and also prominently link to this on the Flink
> > >>> homepage
> > >>>>> this would greatly help people that try to find out about current
> > >>>>> developments.
> > >>>>>
> > >>>>> Kafka has a process like this:
> > >>>>>
> > >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> > >>> .
> > >>>>> They call it KIP, for Kafka Improvement Proposal. We could either
> > >> adapt
> > >>>>> this for Flink or come up with our own process. Doing the former
> > would
> > >>> save
> > >>>>> us a lot of time and I don't think the Kafka community would mind
> us
> > >>>>> copying their process. The subject also hints at this, our process
> > >>> could be
> > >>>>> called FLIP, for Flink Improvement Proposal.
> > >>>>>
> > >>>>> What do you think? Feedback is highly welcome. :-)
> > >>>>>
> > >>>>> Cheers,
> > >>>>> Aljoscha
> > >>>>>
> > >>>>
> > >>>
> > >>
> > >
> >
> >
>

Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal

[Robert Metzger <rm...@apache.org>]

I also like the proposal. I think its an issue that Google Docs comments
are not reflected within ASF infra. Therefore, I'm +1 on discussing the
proposals on the mailing list.

I agree that we need to clean up our wiki.

On Thu, Jul 7, 2016 at 10:58 AM, Matthias J. Sax <mj...@apache.org> wrote:

> Just to point out one thing about Kafka KIPs and using the project wiki:
>
> The wiki contains the current state of the proposal, while the
> discussion is covered over the dev-mailing list. IMHO, this makes a lot
> of sense, as people tend to follow the mailing list but not wiki
> changes. Furthermore, the mailing list tracks the whole discussion
> history, while the proposal is kept in a clean state and thus easy to read.
>
> -Matthias
>
>
> On 07/06/2016 10:09 PM, Aljoscha Krettek wrote:
> > Jip, that's why I referenced the Kafka process which is also in their
> wiki.
> >
> > On Wed, 6 Jul 2016 at 21:01 Stephan Ewen <se...@apache.org> wrote:
> >
> >> Yes, big +1
> >>
> >> I had actually talked about the same thing with some people as well.
> >>
> >> I am currently sketching a few FLIPs for things, like improvements to
> the
> >> Yarn/Mesos/Kubernetes integration
> >>
> >>
> >> One thing we should do here is to actually structure the wiki a bit to
> make
> >> it easier to find information and proposals.
> >>
> >>
> >>
> >>
> >> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk Celebi <uc...@apache.org> wrote:
> >>
> >>> Hey Aljoscha,
> >>>
> >>> thanks for this proposal. I've somehow missed it last week. I like the
> >>> idea very much and agree with your assessment about the problems with
> >>> the Google Doc approach.
> >>>
> >>> Regarding the process: I'm also in favour of adopting it from Kafka. I
> >>> would not expect any problems with this, but we can post a quick note
> >>> to their ML.
> >>>
> >>> @Matthias: The name works for me. ;-)
> >>>
> >>> – Ufuk
> >>>
> >>> On Tue, Jun 28, 2016 at 10:19 PM, Matthias J. Sax <mj...@apache.org>
> >>> wrote:
> >>>> FLIP ?? Really? :D
> >>>>
> >>>> http://www.maya.tv/en/character/flip
> >>>>
> >>>> -Matthias
> >>>>
> >>>>
> >>>> On 06/28/2016 06:26 PM, Aljoscha Krettek wrote:
> >>>>> I'm proposing to add a formal process for how we deal with (major)
> >>>>> improvements to Flink and design docs. This has been mentioned
> several
> >>>>> times recently but we never took any decisive action to actually
> >>> implement
> >>>>> such a process so here we go.
> >>>>>
> >>>>> Right now, we have Jira issues and we sometimes we have design docs
> >>> that we
> >>>>> keep in Google Docs. Jamie recently added links to those that he
> could
> >>> find
> >>>>> on the mailing list to the Flink wiki:
> >>>>> https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
> >>> The
> >>>>> problem with these is that a) the comments on the Google Docs are not
> >>>>> reflected in Jira and the mailing list. There has been some very
> >> active
> >>>>> discussion on some of the docs that most people would never notice.
> >> The
> >>>>> community therefore might seem less active than it actually is. b)
> the
> >>>>> documents are not very discoverable, if we had a clearly defined
> place
> >>>>> where we put them and also prominently link to this on the Flink
> >>> homepage
> >>>>> this would greatly help people that try to find out about current
> >>>>> developments.
> >>>>>
> >>>>> Kafka has a process like this:
> >>>>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> >>> .
> >>>>> They call it KIP, for Kafka Improvement Proposal. We could either
> >> adapt
> >>>>> this for Flink or come up with our own process. Doing the former
> would
> >>> save
> >>>>> us a lot of time and I don't think the Kafka community would mind us
> >>>>> copying their process. The subject also hints at this, our process
> >>> could be
> >>>>> called FLIP, for Flink Improvement Proposal.
> >>>>>
> >>>>> What do you think? Feedback is highly welcome. :-)
> >>>>>
> >>>>> Cheers,
> >>>>> Aljoscha
> >>>>>
> >>>>
> >>>
> >>
> >
>
>

Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal

["Matthias J. Sax" <mj...@apache.org>]

Just to point out one thing about Kafka KIPs and using the project wiki:

The wiki contains the current state of the proposal, while the
discussion is covered over the dev-mailing list. IMHO, this makes a lot
of sense, as people tend to follow the mailing list but not wiki
changes. Furthermore, the mailing list tracks the whole discussion
history, while the proposal is kept in a clean state and thus easy to read.

-Matthias


On 07/06/2016 10:09 PM, Aljoscha Krettek wrote:
> Jip, that's why I referenced the Kafka process which is also in their wiki.
> 
> On Wed, 6 Jul 2016 at 21:01 Stephan Ewen <se...@apache.org> wrote:
> 
>> Yes, big +1
>>
>> I had actually talked about the same thing with some people as well.
>>
>> I am currently sketching a few FLIPs for things, like improvements to the
>> Yarn/Mesos/Kubernetes integration
>>
>>
>> One thing we should do here is to actually structure the wiki a bit to make
>> it easier to find information and proposals.
>>
>>
>>
>>
>> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk Celebi <uc...@apache.org> wrote:
>>
>>> Hey Aljoscha,
>>>
>>> thanks for this proposal. I've somehow missed it last week. I like the
>>> idea very much and agree with your assessment about the problems with
>>> the Google Doc approach.
>>>
>>> Regarding the process: I'm also in favour of adopting it from Kafka. I
>>> would not expect any problems with this, but we can post a quick note
>>> to their ML.
>>>
>>> @Matthias: The name works for me. ;-)
>>>
>>> – Ufuk
>>>
>>> On Tue, Jun 28, 2016 at 10:19 PM, Matthias J. Sax <mj...@apache.org>
>>> wrote:
>>>> FLIP ?? Really? :D
>>>>
>>>> http://www.maya.tv/en/character/flip
>>>>
>>>> -Matthias
>>>>
>>>>
>>>> On 06/28/2016 06:26 PM, Aljoscha Krettek wrote:
>>>>> I'm proposing to add a formal process for how we deal with (major)
>>>>> improvements to Flink and design docs. This has been mentioned several
>>>>> times recently but we never took any decisive action to actually
>>> implement
>>>>> such a process so here we go.
>>>>>
>>>>> Right now, we have Jira issues and we sometimes we have design docs
>>> that we
>>>>> keep in Google Docs. Jamie recently added links to those that he could
>>> find
>>>>> on the mailing list to the Flink wiki:
>>>>> https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
>>> The
>>>>> problem with these is that a) the comments on the Google Docs are not
>>>>> reflected in Jira and the mailing list. There has been some very
>> active
>>>>> discussion on some of the docs that most people would never notice.
>> The
>>>>> community therefore might seem less active than it actually is. b) the
>>>>> documents are not very discoverable, if we had a clearly defined place
>>>>> where we put them and also prominently link to this on the Flink
>>> homepage
>>>>> this would greatly help people that try to find out about current
>>>>> developments.
>>>>>
>>>>> Kafka has a process like this:
>>>>>
>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>>> .
>>>>> They call it KIP, for Kafka Improvement Proposal. We could either
>> adapt
>>>>> this for Flink or come up with our own process. Doing the former would
>>> save
>>>>> us a lot of time and I don't think the Kafka community would mind us
>>>>> copying their process. The subject also hints at this, our process
>>> could be
>>>>> called FLIP, for Flink Improvement Proposal.
>>>>>
>>>>> What do you think? Feedback is highly welcome. :-)
>>>>>
>>>>> Cheers,
>>>>> Aljoscha
>>>>>
>>>>
>>>
>>
> 

Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal

[Aljoscha Krettek <al...@apache.org>]

Jip, that's why I referenced the Kafka process which is also in their wiki.

On Wed, 6 Jul 2016 at 21:01 Stephan Ewen <se...@apache.org> wrote:

> Yes, big +1
>
> I had actually talked about the same thing with some people as well.
>
> I am currently sketching a few FLIPs for things, like improvements to the
> Yarn/Mesos/Kubernetes integration
>
>
> One thing we should do here is to actually structure the wiki a bit to make
> it easier to find information and proposals.
>
>
>
>
> On Wed, Jul 6, 2016 at 4:24 PM, Ufuk Celebi <uc...@apache.org> wrote:
>
> > Hey Aljoscha,
> >
> > thanks for this proposal. I've somehow missed it last week. I like the
> > idea very much and agree with your assessment about the problems with
> > the Google Doc approach.
> >
> > Regarding the process: I'm also in favour of adopting it from Kafka. I
> > would not expect any problems with this, but we can post a quick note
> > to their ML.
> >
> > @Matthias: The name works for me. ;-)
> >
> > – Ufuk
> >
> > On Tue, Jun 28, 2016 at 10:19 PM, Matthias J. Sax <mj...@apache.org>
> > wrote:
> > > FLIP ?? Really? :D
> > >
> > > http://www.maya.tv/en/character/flip
> > >
> > > -Matthias
> > >
> > >
> > > On 06/28/2016 06:26 PM, Aljoscha Krettek wrote:
> > >> I'm proposing to add a formal process for how we deal with (major)
> > >> improvements to Flink and design docs. This has been mentioned several
> > >> times recently but we never took any decisive action to actually
> > implement
> > >> such a process so here we go.
> > >>
> > >> Right now, we have Jira issues and we sometimes we have design docs
> > that we
> > >> keep in Google Docs. Jamie recently added links to those that he could
> > find
> > >> on the mailing list to the Flink wiki:
> > >> https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
> > The
> > >> problem with these is that a) the comments on the Google Docs are not
> > >> reflected in Jira and the mailing list. There has been some very
> active
> > >> discussion on some of the docs that most people would never notice.
> The
> > >> community therefore might seem less active than it actually is. b) the
> > >> documents are not very discoverable, if we had a clearly defined place
> > >> where we put them and also prominently link to this on the Flink
> > homepage
> > >> this would greatly help people that try to find out about current
> > >> developments.
> > >>
> > >> Kafka has a process like this:
> > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> > .
> > >> They call it KIP, for Kafka Improvement Proposal. We could either
> adapt
> > >> this for Flink or come up with our own process. Doing the former would
> > save
> > >> us a lot of time and I don't think the Kafka community would mind us
> > >> copying their process. The subject also hints at this, our process
> > could be
> > >> called FLIP, for Flink Improvement Proposal.
> > >>
> > >> What do you think? Feedback is highly welcome. :-)
> > >>
> > >> Cheers,
> > >> Aljoscha
> > >>
> > >
> >
>

Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal

[Stephan Ewen <se...@apache.org>]

Yes, big +1

I had actually talked about the same thing with some people as well.

I am currently sketching a few FLIPs for things, like improvements to the
Yarn/Mesos/Kubernetes integration


One thing we should do here is to actually structure the wiki a bit to make
it easier to find information and proposals.




On Wed, Jul 6, 2016 at 4:24 PM, Ufuk Celebi <uc...@apache.org> wrote:

> Hey Aljoscha,
>
> thanks for this proposal. I've somehow missed it last week. I like the
> idea very much and agree with your assessment about the problems with
> the Google Doc approach.
>
> Regarding the process: I'm also in favour of adopting it from Kafka. I
> would not expect any problems with this, but we can post a quick note
> to their ML.
>
> @Matthias: The name works for me. ;-)
>
> – Ufuk
>
> On Tue, Jun 28, 2016 at 10:19 PM, Matthias J. Sax <mj...@apache.org>
> wrote:
> > FLIP ?? Really? :D
> >
> > http://www.maya.tv/en/character/flip
> >
> > -Matthias
> >
> >
> > On 06/28/2016 06:26 PM, Aljoscha Krettek wrote:
> >> I'm proposing to add a formal process for how we deal with (major)
> >> improvements to Flink and design docs. This has been mentioned several
> >> times recently but we never took any decisive action to actually
> implement
> >> such a process so here we go.
> >>
> >> Right now, we have Jira issues and we sometimes we have design docs
> that we
> >> keep in Google Docs. Jamie recently added links to those that he could
> find
> >> on the mailing list to the Flink wiki:
> >> https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home.
> The
> >> problem with these is that a) the comments on the Google Docs are not
> >> reflected in Jira and the mailing list. There has been some very active
> >> discussion on some of the docs that most people would never notice. The
> >> community therefore might seem less active than it actually is. b) the
> >> documents are not very discoverable, if we had a clearly defined place
> >> where we put them and also prominently link to this on the Flink
> homepage
> >> this would greatly help people that try to find out about current
> >> developments.
> >>
> >> Kafka has a process like this:
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
> .
> >> They call it KIP, for Kafka Improvement Proposal. We could either adapt
> >> this for Flink or come up with our own process. Doing the former would
> save
> >> us a lot of time and I don't think the Kafka community would mind us
> >> copying their process. The subject also hints at this, our process
> could be
> >> called FLIP, for Flink Improvement Proposal.
> >>
> >> What do you think? Feedback is highly welcome. :-)
> >>
> >> Cheers,
> >> Aljoscha
> >>
> >
>

Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal

[Ufuk Celebi <uc...@apache.org>]

Hey Aljoscha,

thanks for this proposal. I've somehow missed it last week. I like the
idea very much and agree with your assessment about the problems with
the Google Doc approach.

Regarding the process: I'm also in favour of adopting it from Kafka. I
would not expect any problems with this, but we can post a quick note
to their ML.

@Matthias: The name works for me. ;-)

– Ufuk

On Tue, Jun 28, 2016 at 10:19 PM, Matthias J. Sax <mj...@apache.org> wrote:
> FLIP ?? Really? :D
>
> http://www.maya.tv/en/character/flip
>
> -Matthias
>
>
> On 06/28/2016 06:26 PM, Aljoscha Krettek wrote:
>> I'm proposing to add a formal process for how we deal with (major)
>> improvements to Flink and design docs. This has been mentioned several
>> times recently but we never took any decisive action to actually implement
>> such a process so here we go.
>>
>> Right now, we have Jira issues and we sometimes we have design docs that we
>> keep in Google Docs. Jamie recently added links to those that he could find
>> on the mailing list to the Flink wiki:
>> https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home. The
>> problem with these is that a) the comments on the Google Docs are not
>> reflected in Jira and the mailing list. There has been some very active
>> discussion on some of the docs that most people would never notice. The
>> community therefore might seem less active than it actually is. b) the
>> documents are not very discoverable, if we had a clearly defined place
>> where we put them and also prominently link to this on the Flink homepage
>> this would greatly help people that try to find out about current
>> developments.
>>
>> Kafka has a process like this:
>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals.
>> They call it KIP, for Kafka Improvement Proposal. We could either adapt
>> this for Flink or come up with our own process. Doing the former would save
>> us a lot of time and I don't think the Kafka community would mind us
>> copying their process. The subject also hints at this, our process could be
>> called FLIP, for Flink Improvement Proposal.
>>
>> What do you think? Feedback is highly welcome. :-)
>>
>> Cheers,
>> Aljoscha
>>
>

Re: [DISCUSS] FLIP 1 - Flink Improvement Proposal

["Matthias J. Sax" <mj...@apache.org>]

FLIP ?? Really? :D

http://www.maya.tv/en/character/flip

-Matthias


On 06/28/2016 06:26 PM, Aljoscha Krettek wrote:
> I'm proposing to add a formal process for how we deal with (major)
> improvements to Flink and design docs. This has been mentioned several
> times recently but we never took any decisive action to actually implement
> such a process so here we go.
> 
> Right now, we have Jira issues and we sometimes we have design docs that we
> keep in Google Docs. Jamie recently added links to those that he could find
> on the mailing list to the Flink wiki:
> https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home. The
> problem with these is that a) the comments on the Google Docs are not
> reflected in Jira and the mailing list. There has been some very active
> discussion on some of the docs that most people would never notice. The
> community therefore might seem less active than it actually is. b) the
> documents are not very discoverable, if we had a clearly defined place
> where we put them and also prominently link to this on the Flink homepage
> this would greatly help people that try to find out about current
> developments.
> 
> Kafka has a process like this:
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals.
> They call it KIP, for Kafka Improvement Proposal. We could either adapt
> this for Flink or come up with our own process. Doing the former would save
> us a lot of time and I don't think the Kafka community would mind us
> copying their process. The subject also hints at this, our process could be
> called FLIP, for Flink Improvement Proposal.
> 
> What do you think? Feedback is highly welcome. :-)
> 
> Cheers,
> Aljoscha
>