Re: Hazelcast Jet Runner

[Kenneth Knowles <ke...@apache.org>]

On Wed, Jun 12, 2019 at 2:32 AM Ismaël Mejía <ie...@gmail.com> wrote:

> Seems the discussion moved a bit of my original intent that was to
> make the Jet runner directory to be just called runners/jet in the
> directory and mark the 'experimental' part of it in documentation as
> we do for all other things in Beam.
>

Thanks for returning to the one question at hand. We don't have to make an
overall decision about all "experimental" things.


> Can we do this or is there still any considerable argument to not do it?
>

I think we actually have some competing goals:

I agree 100% on the arguments, but let’s think in the reverse terms,
> highlighting lack of maturity can play against the intended goal of
> use and adoption even if for a noble reason. It is basic priming 101
> [1].


_My_ goal is exactly to highlight lack of maturity so that users are not
harmed by either (1) necessary breaking changes or (2) permanent low
quality. Only users who are willing to follow along with the project and
update their own code regularly should use experimental features.

Evaluating the Jet runner I am convinced by your arguments, because looking
at the two dangers:
(1) necessary breaking changes -- runners don't really have their own APIs
to break, except their own small set of APIs and pipeline options
(2) permanent low quality -- because there is no API design possible,
there's no risk of permanent low quality except by fundamental mismatches.
Plus as you mention the testing is already quite good.

So I am OK to not call it experimental. But I have a slight remaining
concern that it did not really go through what other runners went through.
I hope this just means it is more mature. I hope it does not indicate that
we are reducing rigor.

Kenn


> On Wed, May 29, 2019 at 3:02 PM Reza Rokni <re...@google.com> wrote:
> >
> > Hi,
> >
> > Over 800 usages under java, might be worth doing a few PR...
> >
> > Also suggest we use a very light review process: First round go for low
> hanging fruit, if anyone does a -1 against a change then we leave that for
> round two.
> >
> > Thoughts?
> >
> > Cheers
> >
> > Reza
> >
> > On Wed, 29 May 2019 at 12:05, Kenneth Knowles <ke...@apache.org> wrote:
> >>
> >>
> >>
> >> On Mon, May 27, 2019 at 4:05 PM Reza Rokni <re...@google.com> wrote:
> >>>
> >>> "Many APIs that have been in place for years and are used by most Beam
> users are still marked Experimental."
> >>>
> >>> Should there be a formal process in place to start 'graduating'
> features out of @Experimental? Perhaps even target an up coming release
> with a PR to remove the annotation from well established API's?
> >>
> >>
> >> Good idea. I think a PR like this would be an opportunity to discuss
> whether the feature is non-experimental. Probably many of them are ready.
> It would help to address Ismael's very good point that this new practice
> could make users think the old Experimental stuff is not experimental.
> Maybe it is true that it is not really still Experimental.
> >>
> >> Kenn
> >>
> >>
> >>>
> >>> On Tue, 28 May 2019 at 06:44, Reuven Lax <re...@google.com> wrote:
> >>>>
> >>>> We generally use Experimental for two different things, which leads
> to confusion.
> >>>>   1. Features that work stably, but where we think we might still
> make some changes to the API.
> >>>>   2. New features that we think might not yet be stable.
> >>>>
> >>>> This dual usage leads to a lot of confusion IMO. The fact that we
> tend to forget to remove the @Experimental tag also makes it somewhat
> useless. Many APIs that have been in place for years and are used by most
> Beam users are still marked Experimental.
> >>>>
> >>>> Reuven
> >>>>
> >>>> On Mon, May 27, 2019 at 2:16 PM Ismaël Mejía <ie...@gmail.com>
> wrote:
> >>>>>
> >>>>> > Personally, I think that it is good that moving from experimental
> to non-experimental is a breaking change in the dependency - one has
> backwards-incompatible changes and the other does not. If artifacts had
> separate versioning we could use 0.x for this.
> >>>>>
> >>>>> In theory it seems so, but in practice it is an annoyance to an end
> >>>>> user that already took the ‘risk’ of using an experimental feature.
> >>>>> Awareness is probably not the most important reason to break existing
> >>>>> code (even if it could be easily fixed). The alternative of doing
> this
> >>>>> with version numbers at least seems less impacting but can be
> >>>>> confusing.
> >>>>>
> >>>>> > But biggest motivation for me are these:
> >>>>> >
> >>>>> >  - using experimental features should be opt-in
> >>>>> >  - should be impossible to use an experimental feature without
> knowing it (so "opt-in" to a normal-looking feature is not enough)
> >>>>> > - developers of an experimental feature should be motivated to
> "graduate" it
> >>>>>
> >>>>> The fundamental problem of this approach is inconsistency with our
> >>>>> present/past. So far we have ‘Experimental’ features everywhere. So
> >>>>> suddenly becoming opt-in let us in an inconsistent state. For example
> >>>>> all IOs are marked internally as Experimental but not at the level of
> >>>>> directories/artifacts. Adding this suffix in a new IO apart of adding
> >>>>> fear of use to the end users may also give the fake impression that
> >>>>> the older ones not explicitly marked are not experimental.
> >>>>>
> >>>>> What will be the state for example in the case of runner modules that
> >>>>> contain both mature and well tested runners like old Flink and Spark
> >>>>> runners vs the more experimental new translations for Portability,
> >>>>> again more confusion.
> >>>>>
> >>>>> > FWIW I don't think "experimental" should be viewed as a bad thing.
> It just means you are able to make backwards-incompatible changes, and that
> users should be aware that they will need to adjust APIs (probably only a
> little) with new releases. Most software is not very good until it has been
> around for a long time, and in my experience the problem is missing the
> mark on abstractions, so backwards compatibility *must* be broken to
> achieve quality. Freezing it early dooms it to never achieving high
> quality. I know of projects where the users explicitly requested that the
> developers not freeze the API but instead prioritize speed and quality.
> >>>>>
> >>>>> I agree 100% on the arguments, but let’s think in the reverse terms,
> >>>>> highlighting lack of maturity can play against the intended goal of
> >>>>> use and adoption even if for a noble reason. It is basic priming 101
> >>>>> [1].
> >>>>>
> >>>>> > Maybe the word is just too negative-sounding? Alternatives might
> be "unstable" or "incubating".
> >>>>>
> >>>>> Yes! “experimental” should not be viewed as a bad thing unless you
> are
> >>>>> a company that has less resources and is trying to protect its
> >>>>> investment so in that case they may doubt to use it. In this case
> >>>>> probably incubating is a better term because it has less of the
> >>>>> ‘tentative’ dimension associated with Experimental.
> >>>>>
> >>>>> > Now, for the Jet runner, most runners sit on a branch for a while,
> not being released at all, and move to master as their "graduation". I
> think releasing under an "experimental" name is an improvement, making it
> available to users to try out. But we probably should have discussed before
> doing something different than all the other runners.
> >>>>>
> >>>>> There is something I don’t get in the case of Jet runner. From the
> >>>>> discussion in this thread it seems it has everything required to not
> >>>>> be ‘experimental’. It passes ValidatesRunner and can even run Nexmark
> >>>>> that’s more that some runners already merged in master, so I still
> >>>>> don’t get why we want to give it a different connotation.
> >>>>>
> >>>>> [1] https://en.wikipedia.org/wiki/Priming_(psychology)
> >>>>>
> >>>>> On Sun, May 26, 2019 at 4:43 AM Kenneth Knowles <ke...@apache.org>
> wrote:
> >>>>> >
> >>>>> > Personally, I think that it is good that moving from experimental
> to non-experimental is a breaking change in the dependency - one has
> backwards-incompatible changes and the other does not. If artifacts had
> separate versioning we could use 0.x for this.
> >>>>> >
> >>>>> > But biggest motivation for me are these:
> >>>>> >
> >>>>> >  - using experimental features should be opt-in
> >>>>> >  - should be impossible to use an experimental feature without
> knowing it (so "opt-in" to a normal-looking feature is not enough)
> >>>>> >  - developers of an experimental feature should be motivated to
> "graduate" it
> >>>>> >
> >>>>> > So I think a user of an experimental feature should have to
> actually type the word "experimental" either on the command line or in
> their dependencies. That's just my opinion. In the thread [1] myself and
> Robert were the ones that went in this direction of opt-in. But it was
> mostly lazy consensus, plus the review on the pull request, that got us to
> this state. Definitely worth discussing more.
> >>>>> >
> >>>>> > FWIW I don't think "experimental" should be viewed as a bad thing.
> It just means you are able to make backwards-incompatible changes, and that
> users should be aware that they will need to adjust APIs (probably only a
> little) with new releases. Most software is not very good until it has been
> around for a long time, and in my experience the problem is missing the
> mark on abstractions, so backwards compatibility *must* be broken to
> achieve quality. Freezing it early dooms it to never achieving high
> quality. I know of projects where the users explicitly requested that the
> developers not freeze the API but instead prioritize speed and quality.
> >>>>> >
> >>>>> > Maybe the word is just too negative-sounding? Alternatives might
> be "unstable" or "incubating".
> >>>>> >
> >>>>> > Now, for the Jet runner, most runners sit on a branch for a while,
> not being released at all, and move to master as their "graduation". I
> think releasing under an "experimental" name is an improvement, making it
> available to users to try out. But we probably should have discussed before
> doing something different than all the other runners.
> >>>>> >
> >>>>> > Kenn
> >>>>> >
> >>>>> > [1]
> https://lists.apache.org/thread.html/302bd51c77feb5c9ce39882316d391535a0fc92e7608a623d9139160@%3Cdev.beam.apache.org%3E
> >>>>> >
> >>>>> > On Sat, May 25, 2019 at 1:03 AM Ismaël Mejía <ie...@gmail.com>
> wrote:
> >>>>> >>
> >>>>> >> Including the experimental suffix in artifact names is not a good
> idea
> >>>>> >> either because once we decide that it is not experimantal anymore
> this
> >>>>> >> will be a breaking change for users who will need then to update
> its
> >>>>> >> dependencies code. Also it is error-prone to use different
> mappings
> >>>>> >> for directories and artifacts (even if possible).
> >>>>> >>
> >>>>> >> May we reconsider this Kenn? I understand the motivation but I
> hardly
> >>>>> >> see this making things better or more clear. Any runner user will
> end
> >>>>> >> up reading the runner documentation and capability matrix so he
> will
> >>>>> >> catch the current status that way.
> >>>>> >>
> >>>>> >>
> >>>>> >>
> >>>>> >> On Sat, May 25, 2019 at 8:35 AM Jozsef Bartok <
> jozsi@hazelcast.com> wrote:
> >>>>> >> >
> >>>>> >> > I missed Ken's input when writing my previous mail. Sorry.
> >>>>> >> > So, to recap: I should remove "experimental" from any directory
> names, but find an other way of configuring the artifact so that it still
> has "experimental" in it's name.
> >>>>> >> > Right?
> >>>>> >> >
> >>>>> >> > On Sat, May 25, 2019 at 9:32 AM Jozsef Bartok <
> jozsi@hazelcast.com> wrote:
> >>>>> >> >>
> >>>>> >> >> Yes, I'll gladly fix it, we aren't particularly keen to be
> labeled as experimental either..
> >>>>> >> >>
> >>>>> >> >> Btw. initially the "experimental" word was only in the Gradle
> module name, but then there was some change
> >>>>> >> >> ([BEAM-4046] decouple gradle project names and maven artifact
> ids - 4/2/19) which kind of ended up
> >>>>> >> >> putting it in the directory name. Maybe I should have merged
> with that differently, but this is how
> >>>>> >> >> it seemed consistent.
> >>>>> >> >>
> >>>>> >> >> Anyways, will fix it in my next PR.
> >>>>> >> >>
> >>>>> >> >> On Fri, May 24, 2019 at 5:53 PM Ismaël Mejía <
> iemejia@gmail.com> wrote:
> >>>>> >> >>>
> >>>>> >> >>> I see thanks Jozsef, marking things as Experimental was
> discussed but
> >>>>> >> >>> we never agreed on doing this at the directory level. We can
> cover the
> >>>>> >> >>> same ground by putting an annotation in the classes (in
> particular the
> >>>>> >> >>> JetRunner and JetPipelineOptions classes which are the real
> public
> >>>>> >> >>> interface, or in the documentation (in particular website), I
> do not
> >>>>> >> >>> see how putting this in the directory name helps and if so we
> may need
> >>>>> >> >>> to put this in many other directories which is far from
> ideal. Any
> >>>>> >> >>> chance this can be fixed (jet-experimental -> jet) ?
> >>>>> >> >>>
> >>>>> >> >>> On Fri, May 24, 2019 at 9:08 AM Jozsef Bartok <
> jozsi@hazelcast.com> wrote:
> >>>>> >> >>> >
> >>>>> >> >>> > Hi Ismaël!
> >>>>> >> >>> >
> >>>>> >> >>> > Quoting Kenn (from PR-8410): "We discussed on list that it
> would be better to have new things always start as experimental in a way
> that clearly distinguishes them from the core."
> >>>>> >> >>> >
> >>>>> >> >>> > Rgds
> >>>>> >> >>> >
> >>>>> >> >>> > On Thu, May 23, 2019 at 10:44 PM Ismaël Mejía <
> iemejia@gmail.com> wrote:
> >>>>> >> >>> >>
> >>>>> >> >>> >> I saw that the runner was merged but I don’t get why the
> foler is
> >>>>> >> >>> >> called ‘runners/jet experimental’ and not simply
> ‘runners/jet’. Is it
> >>>>> >> >>> >> because the runner does not pass ValidatesRunner? Or
> because the
> >>>>> >> >>> >> contributors are few? I don’t really see any reason behind
> this
> >>>>> >> >>> >> suffix. And even if the status is not mature that’s not
> different from
> >>>>> >> >>> >> other already merged runners.
> >>>>> >> >>> >>
> >>>>> >> >>> >> On Fri, Apr 26, 2019 at 9:43 PM Kenneth Knowles <
> kenn@apache.org> wrote:
> >>>>> >> >>> >> >
> >>>>> >> >>> >> > Nice! That is *way* more than the PR I was looking for.
> I just meant that you could update the website/ directory. It is fine to
> keep the runner in your own repository if you want.
> >>>>> >> >>> >> >
> >>>>> >> >>> >> > But I think it is great if you want to contribute it to
> Apache Beam (hence donate it to the Apache Software Foundation). The
> benefits include: low-latency testing, free updates when someone does a
> refactor. Things to consider are: subject to ASF / Beam governance, PMC,
> commiters, subject to Beam's release cadence (and we might exclude from
> Beam releases for a little bit). Typically, we have kept runners on a
> branch until they are somewhat stable. I don't feel strongly about this for
> disjoint codebases that can easily be excluded from releases. We might want
> to suffix `-experimental` to the artifacts for some time.
> >>>>> >> >>> >> >
> >>>>> >> >>> >> > I commented on the PR about the necessary i.p. clearance
> steps.
> >>>>> >> >>> >> >
> >>>>> >> >>> >> > Kenn
> >>>>> >> >>> >> >the probl
>  >>>>> >> >>> >> > On Fri, Apr 26, 2019 at 3:59 AM jozsi@hazelcast.com <
> jozsi@hazelcast.com> wrote:
> >>>>> >> >>> >> >>
> >>>>> >> >>> >> >> Hi Kenn.
> >>>>> >> >>> >> >>
> >>>>> >> >>> >> >> It took me a while to migrate our code to the Beam
> repo, but I finally have been able to create the Pull Request you asked
> for, this is it: https://github.com/apache/beam/pull/8410
> >>>>> >> >>> >> >>
> >>>>> >> >>> >> >> Looking forward to your feedback!
> >>>>> >> >>> >> >>
> >>>>> >> >>> >> >> Best regards,
> >>>>> >> >>> >> >> Jozsef
> >>>>> >> >>> >> >>
> >>>>> >> >>> >> >> On 2019/04/19 20:52:42, Kenneth Knowles <
> kenn@apache.org> wrote:
> >>>>> >> >>> >> >> > The ValidatesRunner tests are the best source we have
> for knowing the
> >>>>> >> >>> >> >> > capabilities of a runner. Are there instructions for
> running the tests?
> >>>>> >> >>> >> >> >
> >>>>> >> >>> >> >> > Assuming we can check it out, then just open a PR to
> the website with the
> >>>>> >> >>> >> >> > current capabilities and caveats. Since it is a big
> deal and could use lots
> >>>>> >> >>> >> >> > of eyes, I would share the PR link on this thread.
> >>>>> >> >>> >> >> >
> >>>>> >> >>> >> >> > Kenn
> >>>>> >> >>> >> >> >
> >>>>> >> >>> >> >> > On Thu, Apr 18, 2019 at 11:53 AM Jozsef Bartok <
> jozsi@hazelcast.com> wrote:
> >>>>> >> >>> >> >> >
> >>>>> >> >>> >> >> > > Hi. We at Hazelcast Jet have been working for a
> while now to implement a
> >>>>> >> >>> >> >> > > Java Beam Runner (non-portable) based on Hazelcast
> Jet (
> >>>>> >> >>> >> >> > > https://jet.hazelcast.org/). The process is still
> ongoing (
> >>>>> >> >>> >> >> > >
> https://github.com/hazelcast/hazelcast-jet-beam-runner), but we are
> >>>>> >> >>> >> >> > > aiming for a fully functional, reliable Runner
> which can proudly join the
> >>>>> >> >>> >> >> > > Capability Matrix. For that purpose I would like to
> ask what’s your process
> >>>>> >> >>> >> >> > > of validating runners? We are already running the
> @ValidatesRunner tests
> >>>>> >> >>> >> >> > > and the Nexmark test suite, but beyond that what
> other steps do we need to
> >>>>> >> >>> >> >> > > take to get our Runner to the level it needs to be
> at?
> >>>>> >> >>> >> >> > >
> >>>>> >> >>> >> >> >
> >>>
> >>>
> >>>
> >>> --
> >>>
> >>> This email may be confidential and privileged. If you received this
> communication by mistake, please don't forward it to anyone else, please
> erase all copies and attachments, and please let me know that it has gone
> to the wrong person.
> >>>
> >>> The above terms reflect a potential business arrangement, are provided
> solely as a basis for further discussion, and are not intended to be and do
> not constitute a legally binding obligation. No legally binding obligations
> will be created, implied, or inferred until an agreement in final form is
> executed in writing by all parties involved.
> >
> >
> >
> > --
> >
> > This email may be confidential and privileged. If you received this
> communication by mistake, please don't forward it to anyone else, please
> erase all copies and attachments, and please let me know that it has gone
> to the wrong person.
> >
> > The above terms reflect a potential business arrangement, are provided
> solely as a basis for further discussion, and are not intended to be and do
> not constitute a legally binding obligation. No legally binding obligations
> will be created, implied, or inferred until an agreement in final form is
> executed in writing by all parties involved.
>

>

Re: Hazelcast Jet Runner

[Maximilian Michels <mx...@apache.org>]

I believe that is the case. Thanks Kenn.

On 10.07.19 21:35, Ismaël Mejía wrote:
> Yes please!
>
> On Wed, Jul 10, 2019 at 8:38 PM Kenneth Knowles <ke...@apache.org> wrote:
> >
> > Just to make sure we have closed on the Jet runner, my understanding is: I was the main person asking for "runners-jet-experimental" but I am convinced to go with plain "runners-jet". It seems everyone else is already fine with this, so go ahead?
> >
> > On Tue, Jul 9, 2019 at 1:23 PM Maximilian Michels <mx...@apache.org> wrote:
> >>
> >> We should fork the discussion around removing instances of @Experimental, but it was good to mention it here.
> >>
> >> As for the Jet runner, I can only second Ismael: The Jet runner is the first runner I can think of that came with ValidatesRunner and Nexmark out of the box. Of course that doesn't mean the runner is "battled-tested", but we do not have other means to test its maturity.
> >>
> >> For the future, we could come up with other criteria, e.g. a "probation period", but enforcing this now seems arbitrary.
> >>
> >> If the authors of the Runners decide that it is experimental, so be it. Otherwise I would leave it to the user to decide (it might be helpful to list the inception date of each runner). That said, I value your concern Kenn. I can see that we establish a consistent onboarding of new runners which may involve marking them experimental for a while.
> >>
> >> -Max
> >>
> >> On 01.07.19 22:20, Kenneth Knowles wrote:
> >>>
> >>>
> >>> On Wed, Jun 12, 2019 at 2:32 AM Ismaël Mejía <iemejia@gmail.com
> >>> <ma...@gmail.com>> wrote:
> >>>
> >>>     Seems the discussion moved a bit of my original intent that was to
> >>>     make the Jet runner directory to be just called runners/jet in the
> >>>     directory and mark the 'experimental' part of it in documentation as
> >>>     we do for all other things in Beam.
> >>>
> >>>
> >>> Thanks for returning to the one question at hand. We don't have to make
> >>> an overall decision about all "experimental" things.
> >>>
> >>>
> >>>     Can we do this or is there still any considerable argument to not do it?
> >>>
> >>>
> >>> I think we actually have some competing goals:
> >>>
> >>>     I agree 100% on the arguments, but let’s think in the reverse terms,
> >>>     highlighting lack of maturity can play against the intended goal of
> >>>     use and adoption even if for a noble reason. It is basic priming 101
> >>>     [1].
> >>>
> >>>
> >>> _My_ goal is exactly to highlight lack of maturity so that users are not
> >>> harmed by either (1) necessary breaking changes or (2) permanent low
> >>> quality. Only users who are willing to follow along with the project and
> >>> update their own code regularly should use experimental features.
> >>>
> >>> Evaluating the Jet runner I am convinced by your arguments, because
> >>> looking at the two dangers:
> >>> (1) necessary breaking changes -- runners don't really have their own
> >>> APIs to break, except their own small set of APIs and pipeline options
> >>> (2) permanent low quality -- because there is no API design possible,
> >>> there's no risk of permanent low quality except by fundamental
> >>> mismatches. Plus as you mention the testing is already quite good.
> >>>
> >>> So I am OK to not call it experimental. But I have a slight remaining
> >>> concern that it did not really go through what other runners went
> >>> through. I hope this just means it is more mature. I hope it does not
> >>> indicate that we are reducing rigor.
> >>>
> >>> Kenn
> >>>
> >>>
> >>>     On Wed, May 29, 2019 at 3:02 PM Reza Rokni <rez@google.com
> >>>     <ma...@google.com>> wrote:
> >>>     >
> >>>     > Hi,
> >>>     >
> >>>     > Over 800 usages under java, might be worth doing a few PR...
> >>>     >
> >>>     > Also suggest we use a very light review process: First round go
> >>>     for low hanging fruit, if anyone does a -1 against a change then we
> >>>     leave that for round two.
> >>>     >
> >>>     > Thoughts?
> >>>     >
> >>>     > Cheers
> >>>     >
> >>>     > Reza
> >>>     >
> >>>     > On Wed, 29 May 2019 at 12:05, Kenneth Knowles <kenn@apache.org
> >>>     <ma...@apache.org>> wrote:
> >>>     >>
> >>>     >>
> >>>     >>
> >>>     >> On Mon, May 27, 2019 at 4:05 PM Reza Rokni <rez@google.com
> >>>     <ma...@google.com>> wrote:
> >>>     >>>
> >>>     >>> "Many APIs that have been in place for years and are used by
> >>>     most Beam users are still marked Experimental."
> >>>     >>>
> >>>     >>> Should there be a formal process in place to start 'graduating'
> >>>     features out of @Experimental? Perhaps even target an up coming
> >>>     release with a PR to remove the annotation from well established API's?
> >>>     >>
> >>>     >>
> >>>     >> Good idea. I think a PR like this would be an opportunity to
> >>>     discuss whether the feature is non-experimental. Probably many of
> >>>     them are ready. It would help to address Ismael's very good point
> >>>     that this new practice could make users think the old Experimental
> >>>     stuff is not experimental. Maybe it is true that it is not really
> >>>     still Experimental.
> >>>     >>
> >>>     >> Kenn
> >>>     >>
> >>>     >>
> >>>     >>>
> >>>     >>> On Tue, 28 May 2019 at 06:44, Reuven Lax <relax@google.com
> >>>     <ma...@google.com>> wrote:
> >>>     >>>>
> >>>     >>>> We generally use Experimental for two different things, which
> >>>     leads to confusion.
> >>>     >>>>   1. Features that work stably, but where we think we might
> >>>     still make some changes to the API.
> >>>     >>>>   2. New features that we think might not yet be stable.
> >>>     >>>>
> >>>     >>>> This dual usage leads to a lot of confusion IMO. The fact that
> >>>     we tend to forget to remove the @Experimental tag also makes it
> >>>     somewhat useless. Many APIs that have been in place for years and
> >>>     are used by most Beam users are still marked Experimental.
> >>>     >>>>
> >>>     >>>> Reuven
> >>>     >>>>
> >>>     >>>> On Mon, May 27, 2019 at 2:16 PM Ismaël Mejía <iemejia@gmail.com
> >>>     <ma...@gmail.com>> wrote:
> >>>     >>>>>
> >>>     >>>>> > Personally, I think that it is good that moving from
> >>>     experimental to non-experimental is a breaking change in the
> >>>     dependency - one has backwards-incompatible changes and the other
> >>>     does not. If artifacts had separate versioning we could use 0.x for
> >>>     this.
> >>>     >>>>>
> >>>     >>>>> In theory it seems so, but in practice it is an annoyance to
> >>>     an end
> >>>     >>>>> user that already took the ‘risk’ of using an experimental
> >>>     feature.
> >>>     >>>>> Awareness is probably not the most important reason to break
> >>>     existing
> >>>     >>>>> code (even if it could be easily fixed). The alternative of
> >>>     doing this
> >>>     >>>>> with version numbers at least seems less impacting but can be
> >>>     >>>>> confusing.
> >>>     >>>>>
> >>>     >>>>> > But biggest motivation for me are these:
> >>>     >>>>> >
> >>>     >>>>> >  - using experimental features should be opt-in
> >>>     >>>>> >  - should be impossible to use an experimental feature
> >>>     without knowing it (so "opt-in" to a normal-looking feature is not
> >>>     enough)
> >>>     >>>>> > - developers of an experimental feature should be motivated
> >>>     to "graduate" it
> >>>     >>>>>
> >>>     >>>>> The fundamental problem of this approach is inconsistency with our
> >>>     >>>>> present/past. So far we have ‘Experimental’ features
> >>>     everywhere. So
> >>>     >>>>> suddenly becoming opt-in let us in an inconsistent state. For
> >>>     example
> >>>     >>>>> all IOs are marked internally as Experimental but not at the
> >>>     level of
> >>>     >>>>> directories/artifacts. Adding this suffix in a new IO apart of
> >>>     adding
> >>>     >>>>> fear of use to the end users may also give the fake impression
> >>>     that
> >>>     >>>>> the older ones not explicitly marked are not experimental.
> >>>     >>>>>
> >>>     >>>>> What will be the state for example in the case of runner
> >>>     modules that
> >>>     >>>>> contain both mature and well tested runners like old Flink and
> >>>     Spark
> >>>     >>>>> runners vs the more experimental new translations for Portability,
> >>>     >>>>> again more confusion.
> >>>     >>>>>
> >>>     >>>>> > FWIW I don't think "experimental" should be viewed as a bad
> >>>     thing. It just means you are able to make backwards-incompatible
> >>>     changes, and that users should be aware that they will need to
> >>>     adjust APIs (probably only a little) with new releases. Most
> >>>     software is not very good until it has been around for a long time,
> >>>     and in my experience the problem is missing the mark on
> >>>     abstractions, so backwards compatibility *must* be broken to achieve
> >>>     quality. Freezing it early dooms it to never achieving high quality.
> >>>     I know of projects where the users explicitly requested that the
> >>>     developers not freeze the API but instead prioritize speed and quality.
> >>>     >>>>>
> >>>     >>>>> I agree 100% on the arguments, but let’s think in the reverse
> >>>     terms,
> >>>     >>>>> highlighting lack of maturity can play against the intended
> >>>     goal of
> >>>     >>>>> use and adoption even if for a noble reason. It is basic
> >>>     priming 101
> >>>     >>>>> [1].
> >>>     >>>>>
> >>>     >>>>> > Maybe the word is just too negative-sounding? Alternatives
> >>>     might be "unstable" or "incubating".
> >>>     >>>>>
> >>>     >>>>> Yes! “experimental” should not be viewed as a bad thing unless
> >>>     you are
> >>>     >>>>> a company that has less resources and is trying to protect its
> >>>     >>>>> investment so in that case they may doubt to use it. In this case
> >>>     >>>>> probably incubating is a better term because it has less of the
> >>>     >>>>> ‘tentative’ dimension associated with Experimental.
> >>>     >>>>>
> >>>     >>>>> > Now, for the Jet runner, most runners sit on a branch for a
> >>>     while, not being released at all, and move to master as their
> >>>     "graduation". I think releasing under an "experimental" name is an
> >>>     improvement, making it available to users to try out. But we
> >>>     probably should have discussed before doing something different than
> >>>     all the other runners.
> >>>     >>>>>
> >>>     >>>>> There is something I don’t get in the case of Jet runner. From the
> >>>     >>>>> discussion in this thread it seems it has everything required
> >>>     to not
> >>>     >>>>> be ‘experimental’. It passes ValidatesRunner and can even run
> >>>     Nexmark
> >>>     >>>>> that’s more that some runners already merged in master, so I still
> >>>     >>>>> don’t get why we want to give it a different connotation.
> >>>     >>>>>
> >>>     >>>>> [1] https://en.wikipedia.org/wiki/Priming_(psychology)
> >>>     >>>>>
> >>>     >>>>> On Sun, May 26, 2019 at 4:43 AM Kenneth Knowles
> >>>     <kenn@apache.org <ma...@apache.org>> wrote:
> >>>     >>>>> >
> >>>     >>>>> > Personally, I think that it is good that moving from
> >>>     experimental to non-experimental is a breaking change in the
> >>>     dependency - one has backwards-incompatible changes and the other
> >>>     does not. If artifacts had separate versioning we could use 0.x for
> >>>     this.
> >>>     >>>>> >
> >>>     >>>>> > But biggest motivation for me are these:
> >>>     >>>>> >
> >>>     >>>>> >  - using experimental features should be opt-in
> >>>     >>>>> >  - should be impossible to use an experimental feature
> >>>     without knowing it (so "opt-in" to a normal-looking feature is not
> >>>     enough)
> >>>     >>>>> >  - developers of an experimental feature should be motivated
> >>>     to "graduate" it
> >>>     >>>>> >
> >>>     >>>>> > So I think a user of an experimental feature should have to
> >>>     actually type the word "experimental" either on the command line or
> >>>     in their dependencies. That's just my opinion. In the thread [1]
> >>>     myself and Robert were the ones that went in this direction of
> >>>     opt-in. But it was mostly lazy consensus, plus the review on the
> >>>     pull request, that got us to this state. Definitely worth discussing
> >>>     more.
> >>>     >>>>> >
> >>>     >>>>> > FWIW I don't think "experimental" should be viewed as a bad
> >>>     thing. It just means you are able to make backwards-incompatible
> >>>     changes, and that users should be aware that they will need to
> >>>     adjust APIs (probably only a little) with new releases. Most
> >>>     software is not very good until it has been around for a long time,
> >>>     and in my experience the problem is missing the mark on
> >>>     abstractions, so backwards compatibility *must* be broken to achieve
> >>>     quality. Freezing it early dooms it to never achieving high quality.
> >>>     I know of projects where the users explicitly requested that the
> >>>     developers not freeze the API but instead prioritize speed and quality.
> >>>     >>>>> >
> >>>     >>>>> > Maybe the word is just too negative-sounding? Alternatives
> >>>     might be "unstable" or "incubating".
> >>>     >>>>> >
> >>>     >>>>> > Now, for the Jet runner, most runners sit on a branch for a
> >>>     while, not being released at all, and move to master as their
> >>>     "graduation". I think releasing under an "experimental" name is an
> >>>     improvement, making it available to users to try out. But we
> >>>     probably should have discussed before doing something different than
> >>>     all the other runners.
> >>>     >>>>> >
> >>>     >>>>> > Kenn
> >>>     >>>>> >
> >>>     >>>>> > [1]
> >>>     https://lists.apache.org/thread.html/302bd51c77feb5c9ce39882316d391535a0fc92e7608a623d9139160@%3Cdev.beam.apache.org%3E
> >>>     >>>>> >
> >>>     >>>>> > On Sat, May 25, 2019 at 1:03 AM Ismaël Mejía
> >>>     <iemejia@gmail.com <ma...@gmail.com>> wrote:
> >>>     >>>>> >>
> >>>     >>>>> >> Including the experimental suffix in artifact names is not
> >>>     a good idea
> >>>     >>>>> >> either because once we decide that it is not experimantal
> >>>     anymore this
> >>>     >>>>> >> will be a breaking change for users who will need then to
> >>>     update its
> >>>     >>>>> >> dependencies code. Also it is error-prone to use different
> >>>     mappings
> >>>     >>>>> >> for directories and artifacts (even if possible).
> >>>     >>>>> >>
> >>>     >>>>> >> May we reconsider this Kenn? I understand the motivation
> >>>     but I hardly
> >>>     >>>>> >> see this making things better or more clear. Any runner
> >>>     user will end
> >>>     >>>>> >> up reading the runner documentation and capability matrix
> >>>     so he will
> >>>     >>>>> >> catch the current status that way.
> >>>     >>>>> >>
> >>>     >>>>> >>
> >>>     >>>>> >>
> >>>     >>>>> >> On Sat, May 25, 2019 at 8:35 AM Jozsef Bartok
> >>>     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
> >>>     >>>>> >> >
> >>>     >>>>> >> > I missed Ken's input when writing my previous mail. Sorry.
> >>>     >>>>> >> > So, to recap: I should remove "experimental" from any
> >>>     directory names, but find an other way of configuring the artifact
> >>>     so that it still has "experimental" in it's name.
> >>>     >>>>> >> > Right?
> >>>     >>>>> >> >
> >>>     >>>>> >> > On Sat, May 25, 2019 at 9:32 AM Jozsef Bartok
> >>>     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
> >>>     >>>>> >> >>
> >>>     >>>>> >> >> Yes, I'll gladly fix it, we aren't particularly keen to
> >>>     be labeled as experimental either..
> >>>     >>>>> >> >>
> >>>     >>>>> >> >> Btw. initially the "experimental" word was only in the
> >>>     Gradle module name, but then there was some change
> >>>     >>>>> >> >> ([BEAM-4046] decouple gradle project names and maven
> >>>     artifact ids - 4/2/19) which kind of ended up
> >>>     >>>>> >> >> putting it in the directory name. Maybe I should have
> >>>     merged with that differently, but this is how
> >>>     >>>>> >> >> it seemed consistent.
> >>>     >>>>> >> >>
> >>>     >>>>> >> >> Anyways, will fix it in my next PR.
> >>>     >>>>> >> >>
> >>>     >>>>> >> >> On Fri, May 24, 2019 at 5:53 PM Ismaël Mejía
> >>>     <iemejia@gmail.com <ma...@gmail.com>> wrote:
> >>>     >>>>> >> >>>
> >>>     >>>>> >> >>> I see thanks Jozsef, marking things as Experimental was
> >>>     discussed but
> >>>     >>>>> >> >>> we never agreed on doing this at the directory level.
> >>>     We can cover the
> >>>     >>>>> >> >>> same ground by putting an annotation in the classes (in
> >>>     particular the
> >>>     >>>>> >> >>> JetRunner and JetPipelineOptions classes which are the
> >>>     real public
> >>>     >>>>> >> >>> interface, or in the documentation (in particular
> >>>     website), I do not
> >>>     >>>>> >> >>> see how putting this in the directory name helps and if
> >>>     so we may need
> >>>     >>>>> >> >>> to put this in many other directories which is far from
> >>>     ideal. Any
> >>>     >>>>> >> >>> chance this can be fixed (jet-experimental -> jet) ?
> >>>     >>>>> >> >>>
> >>>     >>>>> >> >>> On Fri, May 24, 2019 at 9:08 AM Jozsef Bartok
> >>>     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
> >>>     >>>>> >> >>> >
> >>>     >>>>> >> >>> > Hi Ismaël!
> >>>     >>>>> >> >>> >
> >>>     >>>>> >> >>> > Quoting Kenn (from PR-8410): "We discussed on list
> >>>     that it would be better to have new things always start as
> >>>     experimental in a way that clearly distinguishes them from the core."
> >>>     >>>>> >> >>> >
> >>>     >>>>> >> >>> > Rgds
> >>>     >>>>> >> >>> >
> >>>     >>>>> >> >>> > On Thu, May 23, 2019 at 10:44 PM Ismaël Mejía
> >>>     <iemejia@gmail.com <ma...@gmail.com>> wrote:
> >>>     >>>>> >> >>> >>
> >>>     >>>>> >> >>> >> I saw that the runner was merged but I don’t get why
> >>>     the foler is
> >>>     >>>>> >> >>> >> called ‘runners/jet experimental’ and not simply
> >>>     ‘runners/jet’. Is it
> >>>     >>>>> >> >>> >> because the runner does not pass ValidatesRunner? Or
> >>>     because the
> >>>     >>>>> >> >>> >> contributors are few? I don’t really see any reason
> >>>     behind this
> >>>     >>>>> >> >>> >> suffix. And even if the status is not mature that’s
> >>>     not different from
> >>>     >>>>> >> >>> >> other already merged runners.
> >>>     >>>>> >> >>> >>
> >>>     >>>>> >> >>> >> On Fri, Apr 26, 2019 at 9:43 PM Kenneth Knowles
> >>>     <kenn@apache.org <ma...@apache.org>> wrote:
> >>>     >>>>> >> >>> >> >
> >>>     >>>>> >> >>> >> > Nice! That is *way* more than the PR I was looking
> >>>     for. I just meant that you could update the website/ directory. It
> >>>     is fine to keep the runner in your own repository if you want.
> >>>     >>>>> >> >>> >> >
> >>>     >>>>> >> >>> >> > But I think it is great if you want to contribute
> >>>     it to Apache Beam (hence donate it to the Apache Software
> >>>     Foundation). The benefits include: low-latency testing, free updates
> >>>     when someone does a refactor. Things to consider are: subject to ASF
> >>>     / Beam governance, PMC, commiters, subject to Beam's release cadence
> >>>     (and we might exclude from Beam releases for a little bit).
> >>>     Typically, we have kept runners on a branch until they are somewhat
> >>>     stable. I don't feel strongly about this for disjoint codebases that
> >>>     can easily be excluded from releases. We might want to suffix
> >>>     `-experimental` to the artifacts for some time.
> >>>     >>>>> >> >>> >> >
> >>>     >>>>> >> >>> >> > I commented on the PR about the necessary i.p.
> >>>     clearance steps.
> >>>     >>>>> >> >>> >> >
> >>>     >>>>> >> >>> >> > Kenn
> >>>     >>>>> >> >>> >> >the probl
> >>>      >>>>> >> >>> >> > On Fri, Apr 26, 2019 at 3:59 AM
> >>>     jozsi@hazelcast.com <ma...@hazelcast.com>
> >>>     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
> >>>     >>>>> >> >>> >> >>
> >>>     >>>>> >> >>> >> >> Hi Kenn.
> >>>     >>>>> >> >>> >> >>
> >>>     >>>>> >> >>> >> >> It took me a while to migrate our code to the
> >>>     Beam repo, but I finally have been able to create the Pull Request
> >>>     you asked for, this is it: https://github.com/apache/beam/pull/8410
> >>>     >>>>> >> >>> >> >>
> >>>     >>>>> >> >>> >> >> Looking forward to your feedback!
> >>>     >>>>> >> >>> >> >>
> >>>     >>>>> >> >>> >> >> Best regards,
> >>>     >>>>> >> >>> >> >> Jozsef
> >>>     >>>>> >> >>> >> >>
> >>>     >>>>> >> >>> >> >> On 2019/04/19 20:52:42, Kenneth Knowles
> >>>     <kenn@apache.org <ma...@apache.org>> wrote:
> >>>     >>>>> >> >>> >> >> > The ValidatesRunner tests are the best source
> >>>     we have for knowing the
> >>>     >>>>> >> >>> >> >> > capabilities of a runner. Are there
> >>>     instructions for running the tests?
> >>>     >>>>> >> >>> >> >> >
> >>>     >>>>> >> >>> >> >> > Assuming we can check it out, then just open a
> >>>     PR to the website with the
> >>>     >>>>> >> >>> >> >> > current capabilities and caveats. Since it is a
> >>>     big deal and could use lots
> >>>     >>>>> >> >>> >> >> > of eyes, I would share the PR link on this thread.
> >>>     >>>>> >> >>> >> >> >
> >>>     >>>>> >> >>> >> >> > Kenn
> >>>     >>>>> >> >>> >> >> >
> >>>     >>>>> >> >>> >> >> > On Thu, Apr 18, 2019 at 11:53 AM Jozsef Bartok
> >>>     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
> >>>     >>>>> >> >>> >> >> >
> >>>     >>>>> >> >>> >> >> > > Hi. We at Hazelcast Jet have been working for
> >>>     a while now to implement a
> >>>     >>>>> >> >>> >> >> > > Java Beam Runner (non-portable) based on
> >>>     Hazelcast Jet (
> >>>     >>>>> >> >>> >> >> > > https://jet.hazelcast.org/). The process is
> >>>     still ongoing (
> >>>     >>>>> >> >>> >> >> > >
> >>>     https://github.com/hazelcast/hazelcast-jet-beam-runner), but we are
> >>>     >>>>> >> >>> >> >> > > aiming for a fully functional, reliable
> >>>     Runner which can proudly join the
> >>>     >>>>> >> >>> >> >> > > Capability Matrix. For that purpose I would
> >>>     like to ask what’s your process
> >>>     >>>>> >> >>> >> >> > > of validating runners? We are already running
> >>>     the @ValidatesRunner tests
> >>>     >>>>> >> >>> >> >> > > and the Nexmark test suite, but beyond that
> >>>     what other steps do we need to
> >>>     >>>>> >> >>> >> >> > > take to get our Runner to the level it needs
> >>>     to be at?
> >>>     >>>>> >> >>> >> >> > >
> >>>     >>>>> >> >>> >> >> >
> >>>     >>>
> >>>     >>>
> >>>     >>>
> >>>     >>> --
> >>>     >>>
> >>>     >>> This email may be confidential and privileged. If you received
> >>>     this communication by mistake, please don't forward it to anyone
> >>>     else, please erase all copies and attachments, and please let me
> >>>     know that it has gone to the wrong person.
> >>>     >>>
> >>>     >>> The above terms reflect a potential business arrangement, are
> >>>     provided solely as a basis for further discussion, and are not
> >>>     intended to be and do not constitute a legally binding obligation.
> >>>     No legally binding obligations will be created, implied, or inferred
> >>>     until an agreement in final form is executed in writing by all
> >>>     parties involved.
> >>>     >
> >>>     >
> >>>     >
> >>>     > --
> >>>     >
> >>>     > This email may be confidential and privileged. If you received
> >>>     this communication by mistake, please don't forward it to anyone
> >>>     else, please erase all copies and attachments, and please let me
> >>>     know that it has gone to the wrong person.
> >>>     >
> >>>     > The above terms reflect a potential business arrangement, are
> >>>     provided solely as a basis for further discussion, and are not
> >>>     intended to be and do not constitute a legally binding obligation.
> >>>     No legally binding obligations will be created, implied, or inferred
> >>>     until an agreement in final form is executed in writing by all
> >>>     parties involved.
> >>>
> >>>
> >>

Re: Hazelcast Jet Runner

[Ismaël Mejía <ie...@gmail.com>]

Yes please!

On Wed, Jul 10, 2019 at 8:38 PM Kenneth Knowles <ke...@apache.org> wrote:
>
> Just to make sure we have closed on the Jet runner, my understanding is: I was the main person asking for "runners-jet-experimental" but I am convinced to go with plain "runners-jet". It seems everyone else is already fine with this, so go ahead?
>
> On Tue, Jul 9, 2019 at 1:23 PM Maximilian Michels <mx...@apache.org> wrote:
>>
>> We should fork the discussion around removing instances of @Experimental, but it was good to mention it here.
>>
>> As for the Jet runner, I can only second Ismael: The Jet runner is the first runner I can think of that came with ValidatesRunner and Nexmark out of the box. Of course that doesn't mean the runner is "battled-tested", but we do not have other means to test its maturity.
>>
>> For the future, we could come up with other criteria, e.g. a "probation period", but enforcing this now seems arbitrary.
>>
>> If the authors of the Runners decide that it is experimental, so be it. Otherwise I would leave it to the user to decide (it might be helpful to list the inception date of each runner). That said, I value your concern Kenn. I can see that we establish a consistent onboarding of new runners which may involve marking them experimental for a while.
>>
>> -Max
>>
>> On 01.07.19 22:20, Kenneth Knowles wrote:
>> >
>> >
>> > On Wed, Jun 12, 2019 at 2:32 AM Ismaël Mejía <iemejia@gmail.com
>> > <ma...@gmail.com>> wrote:
>> >
>> >     Seems the discussion moved a bit of my original intent that was to
>> >     make the Jet runner directory to be just called runners/jet in the
>> >     directory and mark the 'experimental' part of it in documentation as
>> >     we do for all other things in Beam.
>> >
>> >
>> > Thanks for returning to the one question at hand. We don't have to make
>> > an overall decision about all "experimental" things.
>> >
>> >
>> >     Can we do this or is there still any considerable argument to not do it?
>> >
>> >
>> > I think we actually have some competing goals:
>> >
>> >     I agree 100% on the arguments, but let’s think in the reverse terms,
>> >     highlighting lack of maturity can play against the intended goal of
>> >     use and adoption even if for a noble reason. It is basic priming 101
>> >     [1].
>> >
>> >
>> > _My_ goal is exactly to highlight lack of maturity so that users are not
>> > harmed by either (1) necessary breaking changes or (2) permanent low
>> > quality. Only users who are willing to follow along with the project and
>> > update their own code regularly should use experimental features.
>> >
>> > Evaluating the Jet runner I am convinced by your arguments, because
>> > looking at the two dangers:
>> > (1) necessary breaking changes -- runners don't really have their own
>> > APIs to break, except their own small set of APIs and pipeline options
>> > (2) permanent low quality -- because there is no API design possible,
>> > there's no risk of permanent low quality except by fundamental
>> > mismatches. Plus as you mention the testing is already quite good.
>> >
>> > So I am OK to not call it experimental. But I have a slight remaining
>> > concern that it did not really go through what other runners went
>> > through. I hope this just means it is more mature. I hope it does not
>> > indicate that we are reducing rigor.
>> >
>> > Kenn
>> >
>> >
>> >     On Wed, May 29, 2019 at 3:02 PM Reza Rokni <rez@google.com
>> >     <ma...@google.com>> wrote:
>> >     >
>> >     > Hi,
>> >     >
>> >     > Over 800 usages under java, might be worth doing a few PR...
>> >     >
>> >     > Also suggest we use a very light review process: First round go
>> >     for low hanging fruit, if anyone does a -1 against a change then we
>> >     leave that for round two.
>> >     >
>> >     > Thoughts?
>> >     >
>> >     > Cheers
>> >     >
>> >     > Reza
>> >     >
>> >     > On Wed, 29 May 2019 at 12:05, Kenneth Knowles <kenn@apache.org
>> >     <ma...@apache.org>> wrote:
>> >     >>
>> >     >>
>> >     >>
>> >     >> On Mon, May 27, 2019 at 4:05 PM Reza Rokni <rez@google.com
>> >     <ma...@google.com>> wrote:
>> >     >>>
>> >     >>> "Many APIs that have been in place for years and are used by
>> >     most Beam users are still marked Experimental."
>> >     >>>
>> >     >>> Should there be a formal process in place to start 'graduating'
>> >     features out of @Experimental? Perhaps even target an up coming
>> >     release with a PR to remove the annotation from well established API's?
>> >     >>
>> >     >>
>> >     >> Good idea. I think a PR like this would be an opportunity to
>> >     discuss whether the feature is non-experimental. Probably many of
>> >     them are ready. It would help to address Ismael's very good point
>> >     that this new practice could make users think the old Experimental
>> >     stuff is not experimental. Maybe it is true that it is not really
>> >     still Experimental.
>> >     >>
>> >     >> Kenn
>> >     >>
>> >     >>
>> >     >>>
>> >     >>> On Tue, 28 May 2019 at 06:44, Reuven Lax <relax@google.com
>> >     <ma...@google.com>> wrote:
>> >     >>>>
>> >     >>>> We generally use Experimental for two different things, which
>> >     leads to confusion.
>> >     >>>>   1. Features that work stably, but where we think we might
>> >     still make some changes to the API.
>> >     >>>>   2. New features that we think might not yet be stable.
>> >     >>>>
>> >     >>>> This dual usage leads to a lot of confusion IMO. The fact that
>> >     we tend to forget to remove the @Experimental tag also makes it
>> >     somewhat useless. Many APIs that have been in place for years and
>> >     are used by most Beam users are still marked Experimental.
>> >     >>>>
>> >     >>>> Reuven
>> >     >>>>
>> >     >>>> On Mon, May 27, 2019 at 2:16 PM Ismaël Mejía <iemejia@gmail.com
>> >     <ma...@gmail.com>> wrote:
>> >     >>>>>
>> >     >>>>> > Personally, I think that it is good that moving from
>> >     experimental to non-experimental is a breaking change in the
>> >     dependency - one has backwards-incompatible changes and the other
>> >     does not. If artifacts had separate versioning we could use 0.x for
>> >     this.
>> >     >>>>>
>> >     >>>>> In theory it seems so, but in practice it is an annoyance to
>> >     an end
>> >     >>>>> user that already took the ‘risk’ of using an experimental
>> >     feature.
>> >     >>>>> Awareness is probably not the most important reason to break
>> >     existing
>> >     >>>>> code (even if it could be easily fixed). The alternative of
>> >     doing this
>> >     >>>>> with version numbers at least seems less impacting but can be
>> >     >>>>> confusing.
>> >     >>>>>
>> >     >>>>> > But biggest motivation for me are these:
>> >     >>>>> >
>> >     >>>>> >  - using experimental features should be opt-in
>> >     >>>>> >  - should be impossible to use an experimental feature
>> >     without knowing it (so "opt-in" to a normal-looking feature is not
>> >     enough)
>> >     >>>>> > - developers of an experimental feature should be motivated
>> >     to "graduate" it
>> >     >>>>>
>> >     >>>>> The fundamental problem of this approach is inconsistency with our
>> >     >>>>> present/past. So far we have ‘Experimental’ features
>> >     everywhere. So
>> >     >>>>> suddenly becoming opt-in let us in an inconsistent state. For
>> >     example
>> >     >>>>> all IOs are marked internally as Experimental but not at the
>> >     level of
>> >     >>>>> directories/artifacts. Adding this suffix in a new IO apart of
>> >     adding
>> >     >>>>> fear of use to the end users may also give the fake impression
>> >     that
>> >     >>>>> the older ones not explicitly marked are not experimental.
>> >     >>>>>
>> >     >>>>> What will be the state for example in the case of runner
>> >     modules that
>> >     >>>>> contain both mature and well tested runners like old Flink and
>> >     Spark
>> >     >>>>> runners vs the more experimental new translations for Portability,
>> >     >>>>> again more confusion.
>> >     >>>>>
>> >     >>>>> > FWIW I don't think "experimental" should be viewed as a bad
>> >     thing. It just means you are able to make backwards-incompatible
>> >     changes, and that users should be aware that they will need to
>> >     adjust APIs (probably only a little) with new releases. Most
>> >     software is not very good until it has been around for a long time,
>> >     and in my experience the problem is missing the mark on
>> >     abstractions, so backwards compatibility *must* be broken to achieve
>> >     quality. Freezing it early dooms it to never achieving high quality.
>> >     I know of projects where the users explicitly requested that the
>> >     developers not freeze the API but instead prioritize speed and quality.
>> >     >>>>>
>> >     >>>>> I agree 100% on the arguments, but let’s think in the reverse
>> >     terms,
>> >     >>>>> highlighting lack of maturity can play against the intended
>> >     goal of
>> >     >>>>> use and adoption even if for a noble reason. It is basic
>> >     priming 101
>> >     >>>>> [1].
>> >     >>>>>
>> >     >>>>> > Maybe the word is just too negative-sounding? Alternatives
>> >     might be "unstable" or "incubating".
>> >     >>>>>
>> >     >>>>> Yes! “experimental” should not be viewed as a bad thing unless
>> >     you are
>> >     >>>>> a company that has less resources and is trying to protect its
>> >     >>>>> investment so in that case they may doubt to use it. In this case
>> >     >>>>> probably incubating is a better term because it has less of the
>> >     >>>>> ‘tentative’ dimension associated with Experimental.
>> >     >>>>>
>> >     >>>>> > Now, for the Jet runner, most runners sit on a branch for a
>> >     while, not being released at all, and move to master as their
>> >     "graduation". I think releasing under an "experimental" name is an
>> >     improvement, making it available to users to try out. But we
>> >     probably should have discussed before doing something different than
>> >     all the other runners.
>> >     >>>>>
>> >     >>>>> There is something I don’t get in the case of Jet runner. From the
>> >     >>>>> discussion in this thread it seems it has everything required
>> >     to not
>> >     >>>>> be ‘experimental’. It passes ValidatesRunner and can even run
>> >     Nexmark
>> >     >>>>> that’s more that some runners already merged in master, so I still
>> >     >>>>> don’t get why we want to give it a different connotation.
>> >     >>>>>
>> >     >>>>> [1] https://en.wikipedia.org/wiki/Priming_(psychology)
>> >     >>>>>
>> >     >>>>> On Sun, May 26, 2019 at 4:43 AM Kenneth Knowles
>> >     <kenn@apache.org <ma...@apache.org>> wrote:
>> >     >>>>> >
>> >     >>>>> > Personally, I think that it is good that moving from
>> >     experimental to non-experimental is a breaking change in the
>> >     dependency - one has backwards-incompatible changes and the other
>> >     does not. If artifacts had separate versioning we could use 0.x for
>> >     this.
>> >     >>>>> >
>> >     >>>>> > But biggest motivation for me are these:
>> >     >>>>> >
>> >     >>>>> >  - using experimental features should be opt-in
>> >     >>>>> >  - should be impossible to use an experimental feature
>> >     without knowing it (so "opt-in" to a normal-looking feature is not
>> >     enough)
>> >     >>>>> >  - developers of an experimental feature should be motivated
>> >     to "graduate" it
>> >     >>>>> >
>> >     >>>>> > So I think a user of an experimental feature should have to
>> >     actually type the word "experimental" either on the command line or
>> >     in their dependencies. That's just my opinion. In the thread [1]
>> >     myself and Robert were the ones that went in this direction of
>> >     opt-in. But it was mostly lazy consensus, plus the review on the
>> >     pull request, that got us to this state. Definitely worth discussing
>> >     more.
>> >     >>>>> >
>> >     >>>>> > FWIW I don't think "experimental" should be viewed as a bad
>> >     thing. It just means you are able to make backwards-incompatible
>> >     changes, and that users should be aware that they will need to
>> >     adjust APIs (probably only a little) with new releases. Most
>> >     software is not very good until it has been around for a long time,
>> >     and in my experience the problem is missing the mark on
>> >     abstractions, so backwards compatibility *must* be broken to achieve
>> >     quality. Freezing it early dooms it to never achieving high quality.
>> >     I know of projects where the users explicitly requested that the
>> >     developers not freeze the API but instead prioritize speed and quality.
>> >     >>>>> >
>> >     >>>>> > Maybe the word is just too negative-sounding? Alternatives
>> >     might be "unstable" or "incubating".
>> >     >>>>> >
>> >     >>>>> > Now, for the Jet runner, most runners sit on a branch for a
>> >     while, not being released at all, and move to master as their
>> >     "graduation". I think releasing under an "experimental" name is an
>> >     improvement, making it available to users to try out. But we
>> >     probably should have discussed before doing something different than
>> >     all the other runners.
>> >     >>>>> >
>> >     >>>>> > Kenn
>> >     >>>>> >
>> >     >>>>> > [1]
>> >     https://lists.apache.org/thread.html/302bd51c77feb5c9ce39882316d391535a0fc92e7608a623d9139160@%3Cdev.beam.apache.org%3E
>> >     >>>>> >
>> >     >>>>> > On Sat, May 25, 2019 at 1:03 AM Ismaël Mejía
>> >     <iemejia@gmail.com <ma...@gmail.com>> wrote:
>> >     >>>>> >>
>> >     >>>>> >> Including the experimental suffix in artifact names is not
>> >     a good idea
>> >     >>>>> >> either because once we decide that it is not experimantal
>> >     anymore this
>> >     >>>>> >> will be a breaking change for users who will need then to
>> >     update its
>> >     >>>>> >> dependencies code. Also it is error-prone to use different
>> >     mappings
>> >     >>>>> >> for directories and artifacts (even if possible).
>> >     >>>>> >>
>> >     >>>>> >> May we reconsider this Kenn? I understand the motivation
>> >     but I hardly
>> >     >>>>> >> see this making things better or more clear. Any runner
>> >     user will end
>> >     >>>>> >> up reading the runner documentation and capability matrix
>> >     so he will
>> >     >>>>> >> catch the current status that way.
>> >     >>>>> >>
>> >     >>>>> >>
>> >     >>>>> >>
>> >     >>>>> >> On Sat, May 25, 2019 at 8:35 AM Jozsef Bartok
>> >     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
>> >     >>>>> >> >
>> >     >>>>> >> > I missed Ken's input when writing my previous mail. Sorry.
>> >     >>>>> >> > So, to recap: I should remove "experimental" from any
>> >     directory names, but find an other way of configuring the artifact
>> >     so that it still has "experimental" in it's name.
>> >     >>>>> >> > Right?
>> >     >>>>> >> >
>> >     >>>>> >> > On Sat, May 25, 2019 at 9:32 AM Jozsef Bartok
>> >     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
>> >     >>>>> >> >>
>> >     >>>>> >> >> Yes, I'll gladly fix it, we aren't particularly keen to
>> >     be labeled as experimental either..
>> >     >>>>> >> >>
>> >     >>>>> >> >> Btw. initially the "experimental" word was only in the
>> >     Gradle module name, but then there was some change
>> >     >>>>> >> >> ([BEAM-4046] decouple gradle project names and maven
>> >     artifact ids - 4/2/19) which kind of ended up
>> >     >>>>> >> >> putting it in the directory name. Maybe I should have
>> >     merged with that differently, but this is how
>> >     >>>>> >> >> it seemed consistent.
>> >     >>>>> >> >>
>> >     >>>>> >> >> Anyways, will fix it in my next PR.
>> >     >>>>> >> >>
>> >     >>>>> >> >> On Fri, May 24, 2019 at 5:53 PM Ismaël Mejía
>> >     <iemejia@gmail.com <ma...@gmail.com>> wrote:
>> >     >>>>> >> >>>
>> >     >>>>> >> >>> I see thanks Jozsef, marking things as Experimental was
>> >     discussed but
>> >     >>>>> >> >>> we never agreed on doing this at the directory level.
>> >     We can cover the
>> >     >>>>> >> >>> same ground by putting an annotation in the classes (in
>> >     particular the
>> >     >>>>> >> >>> JetRunner and JetPipelineOptions classes which are the
>> >     real public
>> >     >>>>> >> >>> interface, or in the documentation (in particular
>> >     website), I do not
>> >     >>>>> >> >>> see how putting this in the directory name helps and if
>> >     so we may need
>> >     >>>>> >> >>> to put this in many other directories which is far from
>> >     ideal. Any
>> >     >>>>> >> >>> chance this can be fixed (jet-experimental -> jet) ?
>> >     >>>>> >> >>>
>> >     >>>>> >> >>> On Fri, May 24, 2019 at 9:08 AM Jozsef Bartok
>> >     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
>> >     >>>>> >> >>> >
>> >     >>>>> >> >>> > Hi Ismaël!
>> >     >>>>> >> >>> >
>> >     >>>>> >> >>> > Quoting Kenn (from PR-8410): "We discussed on list
>> >     that it would be better to have new things always start as
>> >     experimental in a way that clearly distinguishes them from the core."
>> >     >>>>> >> >>> >
>> >     >>>>> >> >>> > Rgds
>> >     >>>>> >> >>> >
>> >     >>>>> >> >>> > On Thu, May 23, 2019 at 10:44 PM Ismaël Mejía
>> >     <iemejia@gmail.com <ma...@gmail.com>> wrote:
>> >     >>>>> >> >>> >>
>> >     >>>>> >> >>> >> I saw that the runner was merged but I don’t get why
>> >     the foler is
>> >     >>>>> >> >>> >> called ‘runners/jet experimental’ and not simply
>> >     ‘runners/jet’. Is it
>> >     >>>>> >> >>> >> because the runner does not pass ValidatesRunner? Or
>> >     because the
>> >     >>>>> >> >>> >> contributors are few? I don’t really see any reason
>> >     behind this
>> >     >>>>> >> >>> >> suffix. And even if the status is not mature that’s
>> >     not different from
>> >     >>>>> >> >>> >> other already merged runners.
>> >     >>>>> >> >>> >>
>> >     >>>>> >> >>> >> On Fri, Apr 26, 2019 at 9:43 PM Kenneth Knowles
>> >     <kenn@apache.org <ma...@apache.org>> wrote:
>> >     >>>>> >> >>> >> >
>> >     >>>>> >> >>> >> > Nice! That is *way* more than the PR I was looking
>> >     for. I just meant that you could update the website/ directory. It
>> >     is fine to keep the runner in your own repository if you want.
>> >     >>>>> >> >>> >> >
>> >     >>>>> >> >>> >> > But I think it is great if you want to contribute
>> >     it to Apache Beam (hence donate it to the Apache Software
>> >     Foundation). The benefits include: low-latency testing, free updates
>> >     when someone does a refactor. Things to consider are: subject to ASF
>> >     / Beam governance, PMC, commiters, subject to Beam's release cadence
>> >     (and we might exclude from Beam releases for a little bit).
>> >     Typically, we have kept runners on a branch until they are somewhat
>> >     stable. I don't feel strongly about this for disjoint codebases that
>> >     can easily be excluded from releases. We might want to suffix
>> >     `-experimental` to the artifacts for some time.
>> >     >>>>> >> >>> >> >
>> >     >>>>> >> >>> >> > I commented on the PR about the necessary i.p.
>> >     clearance steps.
>> >     >>>>> >> >>> >> >
>> >     >>>>> >> >>> >> > Kenn
>> >     >>>>> >> >>> >> >the probl
>> >      >>>>> >> >>> >> > On Fri, Apr 26, 2019 at 3:59 AM
>> >     jozsi@hazelcast.com <ma...@hazelcast.com>
>> >     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
>> >     >>>>> >> >>> >> >>
>> >     >>>>> >> >>> >> >> Hi Kenn.
>> >     >>>>> >> >>> >> >>
>> >     >>>>> >> >>> >> >> It took me a while to migrate our code to the
>> >     Beam repo, but I finally have been able to create the Pull Request
>> >     you asked for, this is it: https://github.com/apache/beam/pull/8410
>> >     >>>>> >> >>> >> >>
>> >     >>>>> >> >>> >> >> Looking forward to your feedback!
>> >     >>>>> >> >>> >> >>
>> >     >>>>> >> >>> >> >> Best regards,
>> >     >>>>> >> >>> >> >> Jozsef
>> >     >>>>> >> >>> >> >>
>> >     >>>>> >> >>> >> >> On 2019/04/19 20:52:42, Kenneth Knowles
>> >     <kenn@apache.org <ma...@apache.org>> wrote:
>> >     >>>>> >> >>> >> >> > The ValidatesRunner tests are the best source
>> >     we have for knowing the
>> >     >>>>> >> >>> >> >> > capabilities of a runner. Are there
>> >     instructions for running the tests?
>> >     >>>>> >> >>> >> >> >
>> >     >>>>> >> >>> >> >> > Assuming we can check it out, then just open a
>> >     PR to the website with the
>> >     >>>>> >> >>> >> >> > current capabilities and caveats. Since it is a
>> >     big deal and could use lots
>> >     >>>>> >> >>> >> >> > of eyes, I would share the PR link on this thread.
>> >     >>>>> >> >>> >> >> >
>> >     >>>>> >> >>> >> >> > Kenn
>> >     >>>>> >> >>> >> >> >
>> >     >>>>> >> >>> >> >> > On Thu, Apr 18, 2019 at 11:53 AM Jozsef Bartok
>> >     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
>> >     >>>>> >> >>> >> >> >
>> >     >>>>> >> >>> >> >> > > Hi. We at Hazelcast Jet have been working for
>> >     a while now to implement a
>> >     >>>>> >> >>> >> >> > > Java Beam Runner (non-portable) based on
>> >     Hazelcast Jet (
>> >     >>>>> >> >>> >> >> > > https://jet.hazelcast.org/). The process is
>> >     still ongoing (
>> >     >>>>> >> >>> >> >> > >
>> >     https://github.com/hazelcast/hazelcast-jet-beam-runner), but we are
>> >     >>>>> >> >>> >> >> > > aiming for a fully functional, reliable
>> >     Runner which can proudly join the
>> >     >>>>> >> >>> >> >> > > Capability Matrix. For that purpose I would
>> >     like to ask what’s your process
>> >     >>>>> >> >>> >> >> > > of validating runners? We are already running
>> >     the @ValidatesRunner tests
>> >     >>>>> >> >>> >> >> > > and the Nexmark test suite, but beyond that
>> >     what other steps do we need to
>> >     >>>>> >> >>> >> >> > > take to get our Runner to the level it needs
>> >     to be at?
>> >     >>>>> >> >>> >> >> > >
>> >     >>>>> >> >>> >> >> >
>> >     >>>
>> >     >>>
>> >     >>>
>> >     >>> --
>> >     >>>
>> >     >>> This email may be confidential and privileged. If you received
>> >     this communication by mistake, please don't forward it to anyone
>> >     else, please erase all copies and attachments, and please let me
>> >     know that it has gone to the wrong person.
>> >     >>>
>> >     >>> The above terms reflect a potential business arrangement, are
>> >     provided solely as a basis for further discussion, and are not
>> >     intended to be and do not constitute a legally binding obligation.
>> >     No legally binding obligations will be created, implied, or inferred
>> >     until an agreement in final form is executed in writing by all
>> >     parties involved.
>> >     >
>> >     >
>> >     >
>> >     > --
>> >     >
>> >     > This email may be confidential and privileged. If you received
>> >     this communication by mistake, please don't forward it to anyone
>> >     else, please erase all copies and attachments, and please let me
>> >     know that it has gone to the wrong person.
>> >     >
>> >     > The above terms reflect a potential business arrangement, are
>> >     provided solely as a basis for further discussion, and are not
>> >     intended to be and do not constitute a legally binding obligation.
>> >     No legally binding obligations will be created, implied, or inferred
>> >     until an agreement in final form is executed in writing by all
>> >     parties involved.
>> >
>> >
>>

Re: Hazelcast Jet Runner

[Kenneth Knowles <ke...@apache.org>]

Just to make sure we have closed on the Jet runner, my understanding is: I
was the main person asking for "runners-jet-experimental" but I am
convinced to go with plain "runners-jet". It seems everyone else is already
fine with this, so go ahead?

On Tue, Jul 9, 2019 at 1:23 PM Maximilian Michels <mx...@apache.org> wrote:

> We should fork the discussion around removing instances of @Experimental,
> but it was good to mention it here.
>
> As for the Jet runner, I can only second Ismael: The Jet runner is the
> first runner I can think of that came with ValidatesRunner and Nexmark out
> of the box. Of course that doesn't mean the runner is "battled-tested", but
> we do not have other means to test its maturity.
>
> For the future, we could come up with other criteria, e.g. a "probation
> period", but enforcing this now seems arbitrary.
>
> If the authors of the Runners decide that it is experimental, so be it.
> Otherwise I would leave it to the user to decide (it might be helpful to
> list the inception date of each runner). That said, I value your concern
> Kenn. I can see that we establish a consistent onboarding of new runners
> which may involve marking them experimental for a while.
>
> -Max
>
> On 01.07.19 22:20, Kenneth Knowles wrote:
> >
> >
> > On Wed, Jun 12, 2019 at 2:32 AM Ismaël Mejía <iemejia@gmail.com
> > <ma...@gmail.com>> wrote:
> >
> >     Seems the discussion moved a bit of my original intent that was to
> >     make the Jet runner directory to be just called runners/jet in the
> >     directory and mark the 'experimental' part of it in documentation as
> >     we do for all other things in Beam.
> >
> >
> > Thanks for returning to the one question at hand. We don't have to make
> > an overall decision about all "experimental" things.
> >
> >
> >     Can we do this or is there still any considerable argument to not do
> it?
> >
> >
> > I think we actually have some competing goals:
> >
> >     I agree 100% on the arguments, but let’s think in the reverse terms,
> >     highlighting lack of maturity can play against the intended goal of
> >     use and adoption even if for a noble reason. It is basic priming 101
> >     [1].
> >
> >
> > _My_ goal is exactly to highlight lack of maturity so that users are not
> > harmed by either (1) necessary breaking changes or (2) permanent low
> > quality. Only users who are willing to follow along with the project and
> > update their own code regularly should use experimental features.
> >
> > Evaluating the Jet runner I am convinced by your arguments, because
> > looking at the two dangers:
> > (1) necessary breaking changes -- runners don't really have their own
> > APIs to break, except their own small set of APIs and pipeline options
> > (2) permanent low quality -- because there is no API design possible,
> > there's no risk of permanent low quality except by fundamental
> > mismatches. Plus as you mention the testing is already quite good.
> >
> > So I am OK to not call it experimental. But I have a slight remaining
> > concern that it did not really go through what other runners went
> > through. I hope this just means it is more mature. I hope it does not
> > indicate that we are reducing rigor.
> >
> > Kenn
> >
> >
> >     On Wed, May 29, 2019 at 3:02 PM Reza Rokni <rez@google.com
> >     <ma...@google.com>> wrote:
> >     >
> >     > Hi,
> >     >
> >     > Over 800 usages under java, might be worth doing a few PR...
> >     >
> >     > Also suggest we use a very light review process: First round go
> >     for low hanging fruit, if anyone does a -1 against a change then we
> >     leave that for round two.
> >     >
> >     > Thoughts?
> >     >
> >     > Cheers
> >     >
> >     > Reza
> >     >
> >     > On Wed, 29 May 2019 at 12:05, Kenneth Knowles <kenn@apache.org
> >     <ma...@apache.org>> wrote:
> >     >>
> >     >>
> >     >>
> >     >> On Mon, May 27, 2019 at 4:05 PM Reza Rokni <rez@google.com
> >     <ma...@google.com>> wrote:
> >     >>>
> >     >>> "Many APIs that have been in place for years and are used by
> >     most Beam users are still marked Experimental."
> >     >>>
> >     >>> Should there be a formal process in place to start 'graduating'
> >     features out of @Experimental? Perhaps even target an up coming
> >     release with a PR to remove the annotation from well established
> API's?
> >     >>
> >     >>
> >     >> Good idea. I think a PR like this would be an opportunity to
> >     discuss whether the feature is non-experimental. Probably many of
> >     them are ready. It would help to address Ismael's very good point
> >     that this new practice could make users think the old Experimental
> >     stuff is not experimental. Maybe it is true that it is not really
> >     still Experimental.
> >     >>
> >     >> Kenn
> >     >>
> >     >>
> >     >>>
> >     >>> On Tue, 28 May 2019 at 06:44, Reuven Lax <relax@google.com
> >     <ma...@google.com>> wrote:
> >     >>>>
> >     >>>> We generally use Experimental for two different things, which
> >     leads to confusion.
> >     >>>>   1. Features that work stably, but where we think we might
> >     still make some changes to the API.
> >     >>>>   2. New features that we think might not yet be stable.
> >     >>>>
> >     >>>> This dual usage leads to a lot of confusion IMO. The fact that
> >     we tend to forget to remove the @Experimental tag also makes it
> >     somewhat useless. Many APIs that have been in place for years and
> >     are used by most Beam users are still marked Experimental.
> >     >>>>
> >     >>>> Reuven
> >     >>>>
> >     >>>> On Mon, May 27, 2019 at 2:16 PM Ismaël Mejía <iemejia@gmail.com
> >     <ma...@gmail.com>> wrote:
> >     >>>>>
> >     >>>>> > Personally, I think that it is good that moving from
> >     experimental to non-experimental is a breaking change in the
> >     dependency - one has backwards-incompatible changes and the other
> >     does not. If artifacts had separate versioning we could use 0.x for
> >     this.
> >     >>>>>
> >     >>>>> In theory it seems so, but in practice it is an annoyance to
> >     an end
> >     >>>>> user that already took the ‘risk’ of using an experimental
> >     feature.
> >     >>>>> Awareness is probably not the most important reason to break
> >     existing
> >     >>>>> code (even if it could be easily fixed). The alternative of
> >     doing this
> >     >>>>> with version numbers at least seems less impacting but can be
> >     >>>>> confusing.
> >     >>>>>
> >     >>>>> > But biggest motivation for me are these:
> >     >>>>> >
> >     >>>>> >  - using experimental features should be opt-in
> >     >>>>> >  - should be impossible to use an experimental feature
> >     without knowing it (so "opt-in" to a normal-looking feature is not
> >     enough)
> >     >>>>> > - developers of an experimental feature should be motivated
> >     to "graduate" it
> >     >>>>>
> >     >>>>> The fundamental problem of this approach is inconsistency with
> our
> >     >>>>> present/past. So far we have ‘Experimental’ features
> >     everywhere. So
> >     >>>>> suddenly becoming opt-in let us in an inconsistent state. For
> >     example
> >     >>>>> all IOs are marked internally as Experimental but not at the
> >     level of
> >     >>>>> directories/artifacts. Adding this suffix in a new IO apart of
> >     adding
> >     >>>>> fear of use to the end users may also give the fake impression
> >     that
> >     >>>>> the older ones not explicitly marked are not experimental.
> >     >>>>>
> >     >>>>> What will be the state for example in the case of runner
> >     modules that
> >     >>>>> contain both mature and well tested runners like old Flink and
> >     Spark
> >     >>>>> runners vs the more experimental new translations for
> Portability,
> >     >>>>> again more confusion.
> >     >>>>>
> >     >>>>> > FWIW I don't think "experimental" should be viewed as a bad
> >     thing. It just means you are able to make backwards-incompatible
> >     changes, and that users should be aware that they will need to
> >     adjust APIs (probably only a little) with new releases. Most
> >     software is not very good until it has been around for a long time,
> >     and in my experience the problem is missing the mark on
> >     abstractions, so backwards compatibility *must* be broken to achieve
> >     quality. Freezing it early dooms it to never achieving high quality.
> >     I know of projects where the users explicitly requested that the
> >     developers not freeze the API but instead prioritize speed and
> quality.
> >     >>>>>
> >     >>>>> I agree 100% on the arguments, but let’s think in the reverse
> >     terms,
> >     >>>>> highlighting lack of maturity can play against the intended
> >     goal of
> >     >>>>> use and adoption even if for a noble reason. It is basic
> >     priming 101
> >     >>>>> [1].
> >     >>>>>
> >     >>>>> > Maybe the word is just too negative-sounding? Alternatives
> >     might be "unstable" or "incubating".
> >     >>>>>
> >     >>>>> Yes! “experimental” should not be viewed as a bad thing unless
> >     you are
> >     >>>>> a company that has less resources and is trying to protect its
> >     >>>>> investment so in that case they may doubt to use it. In this
> case
> >     >>>>> probably incubating is a better term because it has less of the
> >     >>>>> ‘tentative’ dimension associated with Experimental.
> >     >>>>>
> >     >>>>> > Now, for the Jet runner, most runners sit on a branch for a
> >     while, not being released at all, and move to master as their
> >     "graduation". I think releasing under an "experimental" name is an
> >     improvement, making it available to users to try out. But we
> >     probably should have discussed before doing something different than
> >     all the other runners.
> >     >>>>>
> >     >>>>> There is something I don’t get in the case of Jet runner. From
> the
> >     >>>>> discussion in this thread it seems it has everything required
> >     to not
> >     >>>>> be ‘experimental’. It passes ValidatesRunner and can even run
> >     Nexmark
> >     >>>>> that’s more that some runners already merged in master, so I
> still
> >     >>>>> don’t get why we want to give it a different connotation.
> >     >>>>>
> >     >>>>> [1] https://en.wikipedia.org/wiki/Priming_(psychology)
> >     >>>>>
> >     >>>>> On Sun, May 26, 2019 at 4:43 AM Kenneth Knowles
> >     <kenn@apache.org <ma...@apache.org>> wrote:
> >     >>>>> >
> >     >>>>> > Personally, I think that it is good that moving from
> >     experimental to non-experimental is a breaking change in the
> >     dependency - one has backwards-incompatible changes and the other
> >     does not. If artifacts had separate versioning we could use 0.x for
> >     this.
> >     >>>>> >
> >     >>>>> > But biggest motivation for me are these:
> >     >>>>> >
> >     >>>>> >  - using experimental features should be opt-in
> >     >>>>> >  - should be impossible to use an experimental feature
> >     without knowing it (so "opt-in" to a normal-looking feature is not
> >     enough)
> >     >>>>> >  - developers of an experimental feature should be motivated
> >     to "graduate" it
> >     >>>>> >
> >     >>>>> > So I think a user of an experimental feature should have to
> >     actually type the word "experimental" either on the command line or
> >     in their dependencies. That's just my opinion. In the thread [1]
> >     myself and Robert were the ones that went in this direction of
> >     opt-in. But it was mostly lazy consensus, plus the review on the
> >     pull request, that got us to this state. Definitely worth discussing
> >     more.
> >     >>>>> >
> >     >>>>> > FWIW I don't think "experimental" should be viewed as a bad
> >     thing. It just means you are able to make backwards-incompatible
> >     changes, and that users should be aware that they will need to
> >     adjust APIs (probably only a little) with new releases. Most
> >     software is not very good until it has been around for a long time,
> >     and in my experience the problem is missing the mark on
> >     abstractions, so backwards compatibility *must* be broken to achieve
> >     quality. Freezing it early dooms it to never achieving high quality.
> >     I know of projects where the users explicitly requested that the
> >     developers not freeze the API but instead prioritize speed and
> quality.
> >     >>>>> >
> >     >>>>> > Maybe the word is just too negative-sounding? Alternatives
> >     might be "unstable" or "incubating".
> >     >>>>> >
> >     >>>>> > Now, for the Jet runner, most runners sit on a branch for a
> >     while, not being released at all, and move to master as their
> >     "graduation". I think releasing under an "experimental" name is an
> >     improvement, making it available to users to try out. But we
> >     probably should have discussed before doing something different than
> >     all the other runners.
> >     >>>>> >
> >     >>>>> > Kenn
> >     >>>>> >
> >     >>>>> > [1]
> >
> https://lists.apache.org/thread.html/302bd51c77feb5c9ce39882316d391535a0fc92e7608a623d9139160@%3Cdev.beam.apache.org%3E
> >     >>>>> >
> >     >>>>> > On Sat, May 25, 2019 at 1:03 AM Ismaël Mejía
> >     <iemejia@gmail.com <ma...@gmail.com>> wrote:
> >     >>>>> >>
> >     >>>>> >> Including the experimental suffix in artifact names is not
> >     a good idea
> >     >>>>> >> either because once we decide that it is not experimantal
> >     anymore this
> >     >>>>> >> will be a breaking change for users who will need then to
> >     update its
> >     >>>>> >> dependencies code. Also it is error-prone to use different
> >     mappings
> >     >>>>> >> for directories and artifacts (even if possible).
> >     >>>>> >>
> >     >>>>> >> May we reconsider this Kenn? I understand the motivation
> >     but I hardly
> >     >>>>> >> see this making things better or more clear. Any runner
> >     user will end
> >     >>>>> >> up reading the runner documentation and capability matrix
> >     so he will
> >     >>>>> >> catch the current status that way.
> >     >>>>> >>
> >     >>>>> >>
> >     >>>>> >>
> >     >>>>> >> On Sat, May 25, 2019 at 8:35 AM Jozsef Bartok
> >     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
> >     >>>>> >> >
> >     >>>>> >> > I missed Ken's input when writing my previous mail. Sorry.
> >     >>>>> >> > So, to recap: I should remove "experimental" from any
> >     directory names, but find an other way of configuring the artifact
> >     so that it still has "experimental" in it's name.
> >     >>>>> >> > Right?
> >     >>>>> >> >
> >     >>>>> >> > On Sat, May 25, 2019 at 9:32 AM Jozsef Bartok
> >     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
> >     >>>>> >> >>
> >     >>>>> >> >> Yes, I'll gladly fix it, we aren't particularly keen to
> >     be labeled as experimental either..
> >     >>>>> >> >>
> >     >>>>> >> >> Btw. initially the "experimental" word was only in the
> >     Gradle module name, but then there was some change
> >     >>>>> >> >> ([BEAM-4046] decouple gradle project names and maven
> >     artifact ids - 4/2/19) which kind of ended up
> >     >>>>> >> >> putting it in the directory name. Maybe I should have
> >     merged with that differently, but this is how
> >     >>>>> >> >> it seemed consistent.
> >     >>>>> >> >>
> >     >>>>> >> >> Anyways, will fix it in my next PR.
> >     >>>>> >> >>
> >     >>>>> >> >> On Fri, May 24, 2019 at 5:53 PM Ismaël Mejía
> >     <iemejia@gmail.com <ma...@gmail.com>> wrote:
> >     >>>>> >> >>>
> >     >>>>> >> >>> I see thanks Jozsef, marking things as Experimental was
> >     discussed but
> >     >>>>> >> >>> we never agreed on doing this at the directory level.
> >     We can cover the
> >     >>>>> >> >>> same ground by putting an annotation in the classes (in
> >     particular the
> >     >>>>> >> >>> JetRunner and JetPipelineOptions classes which are the
> >     real public
> >     >>>>> >> >>> interface, or in the documentation (in particular
> >     website), I do not
> >     >>>>> >> >>> see how putting this in the directory name helps and if
> >     so we may need
> >     >>>>> >> >>> to put this in many other directories which is far from
> >     ideal. Any
> >     >>>>> >> >>> chance this can be fixed (jet-experimental -> jet) ?
> >     >>>>> >> >>>
> >     >>>>> >> >>> On Fri, May 24, 2019 at 9:08 AM Jozsef Bartok
> >     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
> >     >>>>> >> >>> >
> >     >>>>> >> >>> > Hi Ismaël!
> >     >>>>> >> >>> >
> >     >>>>> >> >>> > Quoting Kenn (from PR-8410): "We discussed on list
> >     that it would be better to have new things always start as
> >     experimental in a way that clearly distinguishes them from the core."
> >     >>>>> >> >>> >
> >     >>>>> >> >>> > Rgds
> >     >>>>> >> >>> >
> >     >>>>> >> >>> > On Thu, May 23, 2019 at 10:44 PM Ismaël Mejía
> >     <iemejia@gmail.com <ma...@gmail.com>> wrote:
> >     >>>>> >> >>> >>
> >     >>>>> >> >>> >> I saw that the runner was merged but I don’t get why
> >     the foler is
> >     >>>>> >> >>> >> called ‘runners/jet experimental’ and not simply
> >     ‘runners/jet’. Is it
> >     >>>>> >> >>> >> because the runner does not pass ValidatesRunner? Or
> >     because the
> >     >>>>> >> >>> >> contributors are few? I don’t really see any reason
> >     behind this
> >     >>>>> >> >>> >> suffix. And even if the status is not mature that’s
> >     not different from
> >     >>>>> >> >>> >> other already merged runners.
> >     >>>>> >> >>> >>
> >     >>>>> >> >>> >> On Fri, Apr 26, 2019 at 9:43 PM Kenneth Knowles
> >     <kenn@apache.org <ma...@apache.org>> wrote:
> >     >>>>> >> >>> >> >
> >     >>>>> >> >>> >> > Nice! That is *way* more than the PR I was looking
> >     for. I just meant that you could update the website/ directory. It
> >     is fine to keep the runner in your own repository if you want.
> >     >>>>> >> >>> >> >
> >     >>>>> >> >>> >> > But I think it is great if you want to contribute
> >     it to Apache Beam (hence donate it to the Apache Software
> >     Foundation). The benefits include: low-latency testing, free updates
> >     when someone does a refactor. Things to consider are: subject to ASF
> >     / Beam governance, PMC, commiters, subject to Beam's release cadence
> >     (and we might exclude from Beam releases for a little bit).
> >     Typically, we have kept runners on a branch until they are somewhat
> >     stable. I don't feel strongly about this for disjoint codebases that
> >     can easily be excluded from releases. We might want to suffix
> >     `-experimental` to the artifacts for some time.
> >     >>>>> >> >>> >> >
> >     >>>>> >> >>> >> > I commented on the PR about the necessary i.p.
> >     clearance steps.
> >     >>>>> >> >>> >> >
> >     >>>>> >> >>> >> > Kenn
> >     >>>>> >> >>> >> >the probl
> >      >>>>> >> >>> >> > On Fri, Apr 26, 2019 at 3:59 AM
> >     jozsi@hazelcast.com <ma...@hazelcast.com>
> >     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
> >     >>>>> >> >>> >> >>
> >     >>>>> >> >>> >> >> Hi Kenn.
> >     >>>>> >> >>> >> >>
> >     >>>>> >> >>> >> >> It took me a while to migrate our code to the
> >     Beam repo, but I finally have been able to create the Pull Request
> >     you asked for, this is it: https://github.com/apache/beam/pull/8410
> >     >>>>> >> >>> >> >>
> >     >>>>> >> >>> >> >> Looking forward to your feedback!
> >     >>>>> >> >>> >> >>
> >     >>>>> >> >>> >> >> Best regards,
> >     >>>>> >> >>> >> >> Jozsef
> >     >>>>> >> >>> >> >>
> >     >>>>> >> >>> >> >> On 2019/04/19 20:52:42, Kenneth Knowles
> >     <kenn@apache.org <ma...@apache.org>> wrote:
> >     >>>>> >> >>> >> >> > The ValidatesRunner tests are the best source
> >     we have for knowing the
> >     >>>>> >> >>> >> >> > capabilities of a runner. Are there
> >     instructions for running the tests?
> >     >>>>> >> >>> >> >> >
> >     >>>>> >> >>> >> >> > Assuming we can check it out, then just open a
> >     PR to the website with the
> >     >>>>> >> >>> >> >> > current capabilities and caveats. Since it is a
> >     big deal and could use lots
> >     >>>>> >> >>> >> >> > of eyes, I would share the PR link on this
> thread.
> >     >>>>> >> >>> >> >> >
> >     >>>>> >> >>> >> >> > Kenn
> >     >>>>> >> >>> >> >> >
> >     >>>>> >> >>> >> >> > On Thu, Apr 18, 2019 at 11:53 AM Jozsef Bartok
> >     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
> >     >>>>> >> >>> >> >> >
> >     >>>>> >> >>> >> >> > > Hi. We at Hazelcast Jet have been working for
> >     a while now to implement a
> >     >>>>> >> >>> >> >> > > Java Beam Runner (non-portable) based on
> >     Hazelcast Jet (
> >     >>>>> >> >>> >> >> > > https://jet.hazelcast.org/). The process is
> >     still ongoing (
> >     >>>>> >> >>> >> >> > >
> >     https://github.com/hazelcast/hazelcast-jet-beam-runner), but we are
> >     >>>>> >> >>> >> >> > > aiming for a fully functional, reliable
> >     Runner which can proudly join the
> >     >>>>> >> >>> >> >> > > Capability Matrix. For that purpose I would
> >     like to ask what’s your process
> >     >>>>> >> >>> >> >> > > of validating runners? We are already running
> >     the @ValidatesRunner tests
> >     >>>>> >> >>> >> >> > > and the Nexmark test suite, but beyond that
> >     what other steps do we need to
> >     >>>>> >> >>> >> >> > > take to get our Runner to the level it needs
> >     to be at?
> >     >>>>> >> >>> >> >> > >
> >     >>>>> >> >>> >> >> >
> >     >>>
> >     >>>
> >     >>>
> >     >>> --
> >     >>>
> >     >>> This email may be confidential and privileged. If you received
> >     this communication by mistake, please don't forward it to anyone
> >     else, please erase all copies and attachments, and please let me
> >     know that it has gone to the wrong person.
> >     >>>
> >     >>> The above terms reflect a potential business arrangement, are
> >     provided solely as a basis for further discussion, and are not
> >     intended to be and do not constitute a legally binding obligation.
> >     No legally binding obligations will be created, implied, or inferred
> >     until an agreement in final form is executed in writing by all
> >     parties involved.
> >     >
> >     >
> >     >
> >     > --
> >     >
> >     > This email may be confidential and privileged. If you received
> >     this communication by mistake, please don't forward it to anyone
> >     else, please erase all copies and attachments, and please let me
> >     know that it has gone to the wrong person.
> >     >
> >     > The above terms reflect a potential business arrangement, are
> >     provided solely as a basis for further discussion, and are not
> >     intended to be and do not constitute a legally binding obligation.
> >     No legally binding obligations will be created, implied, or inferred
> >     until an agreement in final form is executed in writing by all
> >     parties involved.
> >
> >
>
>

Re: Hazelcast Jet Runner

[Maximilian Michels <mx...@apache.org>]

We should fork the discussion around removing instances of @Experimental, but it was good to mention it here.

As for the Jet runner, I can only second Ismael: The Jet runner is the first runner I can think of that came with ValidatesRunner and Nexmark out of the box. Of course that doesn't mean the runner is "battled-tested", but we do not have other means to test its maturity.

For the future, we could come up with other criteria, e.g. a "probation period", but enforcing this now seems arbitrary.

If the authors of the Runners decide that it is experimental, so be it. Otherwise I would leave it to the user to decide (it might be helpful to list the inception date of each runner). That said, I value your concern Kenn. I can see that we establish a consistent onboarding of new runners which may involve marking them experimental for a while.

-Max

On 01.07.19 22:20, Kenneth Knowles wrote:
>
>
> On Wed, Jun 12, 2019 at 2:32 AM Ismaël Mejía <iemejia@gmail.com
> <ma...@gmail.com>> wrote:
>
>     Seems the discussion moved a bit of my original intent that was to
>     make the Jet runner directory to be just called runners/jet in the
>     directory and mark the 'experimental' part of it in documentation as
>     we do for all other things in Beam.
>
>
> Thanks for returning to the one question at hand. We don't have to make
> an overall decision about all "experimental" things.
>  
>
>     Can we do this or is there still any considerable argument to not do it?
>
>
> I think we actually have some competing goals:
>
>     I agree 100% on the arguments, but let’s think in the reverse terms,
>     highlighting lack of maturity can play against the intended goal of
>     use and adoption even if for a noble reason. It is basic priming 101
>     [1].
>
>
> _My_ goal is exactly to highlight lack of maturity so that users are not
> harmed by either (1) necessary breaking changes or (2) permanent low
> quality. Only users who are willing to follow along with the project and
> update their own code regularly should use experimental features.
>
> Evaluating the Jet runner I am convinced by your arguments, because
> looking at the two dangers:
> (1) necessary breaking changes -- runners don't really have their own
> APIs to break, except their own small set of APIs and pipeline options
> (2) permanent low quality -- because there is no API design possible,
> there's no risk of permanent low quality except by fundamental
> mismatches. Plus as you mention the testing is already quite good.
>
> So I am OK to not call it experimental. But I have a slight remaining
> concern that it did not really go through what other runners went
> through. I hope this just means it is more mature. I hope it does not
> indicate that we are reducing rigor.
>
> Kenn
>  
>
>     On Wed, May 29, 2019 at 3:02 PM Reza Rokni <rez@google.com
>     <ma...@google.com>> wrote:
>     >
>     > Hi,
>     >
>     > Over 800 usages under java, might be worth doing a few PR...
>     >
>     > Also suggest we use a very light review process: First round go
>     for low hanging fruit, if anyone does a -1 against a change then we
>     leave that for round two.
>     >
>     > Thoughts?
>     >
>     > Cheers
>     >
>     > Reza
>     >
>     > On Wed, 29 May 2019 at 12:05, Kenneth Knowles <kenn@apache.org
>     <ma...@apache.org>> wrote:
>     >>
>     >>
>     >>
>     >> On Mon, May 27, 2019 at 4:05 PM Reza Rokni <rez@google.com
>     <ma...@google.com>> wrote:
>     >>>
>     >>> "Many APIs that have been in place for years and are used by
>     most Beam users are still marked Experimental."
>     >>>
>     >>> Should there be a formal process in place to start 'graduating'
>     features out of @Experimental? Perhaps even target an up coming
>     release with a PR to remove the annotation from well established API's?
>     >>
>     >>
>     >> Good idea. I think a PR like this would be an opportunity to
>     discuss whether the feature is non-experimental. Probably many of
>     them are ready. It would help to address Ismael's very good point
>     that this new practice could make users think the old Experimental
>     stuff is not experimental. Maybe it is true that it is not really
>     still Experimental.
>     >>
>     >> Kenn
>     >>
>     >>
>     >>>
>     >>> On Tue, 28 May 2019 at 06:44, Reuven Lax <relax@google.com
>     <ma...@google.com>> wrote:
>     >>>>
>     >>>> We generally use Experimental for two different things, which
>     leads to confusion.
>     >>>>   1. Features that work stably, but where we think we might
>     still make some changes to the API.
>     >>>>   2. New features that we think might not yet be stable.
>     >>>>
>     >>>> This dual usage leads to a lot of confusion IMO. The fact that
>     we tend to forget to remove the @Experimental tag also makes it
>     somewhat useless. Many APIs that have been in place for years and
>     are used by most Beam users are still marked Experimental.
>     >>>>
>     >>>> Reuven
>     >>>>
>     >>>> On Mon, May 27, 2019 at 2:16 PM Ismaël Mejía <iemejia@gmail.com
>     <ma...@gmail.com>> wrote:
>     >>>>>
>     >>>>> > Personally, I think that it is good that moving from
>     experimental to non-experimental is a breaking change in the
>     dependency - one has backwards-incompatible changes and the other
>     does not. If artifacts had separate versioning we could use 0.x for
>     this.
>     >>>>>
>     >>>>> In theory it seems so, but in practice it is an annoyance to
>     an end
>     >>>>> user that already took the ‘risk’ of using an experimental
>     feature.
>     >>>>> Awareness is probably not the most important reason to break
>     existing
>     >>>>> code (even if it could be easily fixed). The alternative of
>     doing this
>     >>>>> with version numbers at least seems less impacting but can be
>     >>>>> confusing.
>     >>>>>
>     >>>>> > But biggest motivation for me are these:
>     >>>>> >
>     >>>>> >  - using experimental features should be opt-in
>     >>>>> >  - should be impossible to use an experimental feature
>     without knowing it (so "opt-in" to a normal-looking feature is not
>     enough)
>     >>>>> > - developers of an experimental feature should be motivated
>     to "graduate" it
>     >>>>>
>     >>>>> The fundamental problem of this approach is inconsistency with our
>     >>>>> present/past. So far we have ‘Experimental’ features
>     everywhere. So
>     >>>>> suddenly becoming opt-in let us in an inconsistent state. For
>     example
>     >>>>> all IOs are marked internally as Experimental but not at the
>     level of
>     >>>>> directories/artifacts. Adding this suffix in a new IO apart of
>     adding
>     >>>>> fear of use to the end users may also give the fake impression
>     that
>     >>>>> the older ones not explicitly marked are not experimental.
>     >>>>>
>     >>>>> What will be the state for example in the case of runner
>     modules that
>     >>>>> contain both mature and well tested runners like old Flink and
>     Spark
>     >>>>> runners vs the more experimental new translations for Portability,
>     >>>>> again more confusion.
>     >>>>>
>     >>>>> > FWIW I don't think "experimental" should be viewed as a bad
>     thing. It just means you are able to make backwards-incompatible
>     changes, and that users should be aware that they will need to
>     adjust APIs (probably only a little) with new releases. Most
>     software is not very good until it has been around for a long time,
>     and in my experience the problem is missing the mark on
>     abstractions, so backwards compatibility *must* be broken to achieve
>     quality. Freezing it early dooms it to never achieving high quality.
>     I know of projects where the users explicitly requested that the
>     developers not freeze the API but instead prioritize speed and quality.
>     >>>>>
>     >>>>> I agree 100% on the arguments, but let’s think in the reverse
>     terms,
>     >>>>> highlighting lack of maturity can play against the intended
>     goal of
>     >>>>> use and adoption even if for a noble reason. It is basic
>     priming 101
>     >>>>> [1].
>     >>>>>
>     >>>>> > Maybe the word is just too negative-sounding? Alternatives
>     might be "unstable" or "incubating".
>     >>>>>
>     >>>>> Yes! “experimental” should not be viewed as a bad thing unless
>     you are
>     >>>>> a company that has less resources and is trying to protect its
>     >>>>> investment so in that case they may doubt to use it. In this case
>     >>>>> probably incubating is a better term because it has less of the
>     >>>>> ‘tentative’ dimension associated with Experimental.
>     >>>>>
>     >>>>> > Now, for the Jet runner, most runners sit on a branch for a
>     while, not being released at all, and move to master as their
>     "graduation". I think releasing under an "experimental" name is an
>     improvement, making it available to users to try out. But we
>     probably should have discussed before doing something different than
>     all the other runners.
>     >>>>>
>     >>>>> There is something I don’t get in the case of Jet runner. From the
>     >>>>> discussion in this thread it seems it has everything required
>     to not
>     >>>>> be ‘experimental’. It passes ValidatesRunner and can even run
>     Nexmark
>     >>>>> that’s more that some runners already merged in master, so I still
>     >>>>> don’t get why we want to give it a different connotation.
>     >>>>>
>     >>>>> [1] https://en.wikipedia.org/wiki/Priming_(psychology)
>     >>>>>
>     >>>>> On Sun, May 26, 2019 at 4:43 AM Kenneth Knowles
>     <kenn@apache.org <ma...@apache.org>> wrote:
>     >>>>> >
>     >>>>> > Personally, I think that it is good that moving from
>     experimental to non-experimental is a breaking change in the
>     dependency - one has backwards-incompatible changes and the other
>     does not. If artifacts had separate versioning we could use 0.x for
>     this.
>     >>>>> >
>     >>>>> > But biggest motivation for me are these:
>     >>>>> >
>     >>>>> >  - using experimental features should be opt-in
>     >>>>> >  - should be impossible to use an experimental feature
>     without knowing it (so "opt-in" to a normal-looking feature is not
>     enough)
>     >>>>> >  - developers of an experimental feature should be motivated
>     to "graduate" it
>     >>>>> >
>     >>>>> > So I think a user of an experimental feature should have to
>     actually type the word "experimental" either on the command line or
>     in their dependencies. That's just my opinion. In the thread [1]
>     myself and Robert were the ones that went in this direction of
>     opt-in. But it was mostly lazy consensus, plus the review on the
>     pull request, that got us to this state. Definitely worth discussing
>     more.
>     >>>>> >
>     >>>>> > FWIW I don't think "experimental" should be viewed as a bad
>     thing. It just means you are able to make backwards-incompatible
>     changes, and that users should be aware that they will need to
>     adjust APIs (probably only a little) with new releases. Most
>     software is not very good until it has been around for a long time,
>     and in my experience the problem is missing the mark on
>     abstractions, so backwards compatibility *must* be broken to achieve
>     quality. Freezing it early dooms it to never achieving high quality.
>     I know of projects where the users explicitly requested that the
>     developers not freeze the API but instead prioritize speed and quality.
>     >>>>> >
>     >>>>> > Maybe the word is just too negative-sounding? Alternatives
>     might be "unstable" or "incubating".
>     >>>>> >
>     >>>>> > Now, for the Jet runner, most runners sit on a branch for a
>     while, not being released at all, and move to master as their
>     "graduation". I think releasing under an "experimental" name is an
>     improvement, making it available to users to try out. But we
>     probably should have discussed before doing something different than
>     all the other runners.
>     >>>>> >
>     >>>>> > Kenn
>     >>>>> >
>     >>>>> > [1]
>     https://lists.apache.org/thread.html/302bd51c77feb5c9ce39882316d391535a0fc92e7608a623d9139160@%3Cdev.beam.apache.org%3E
>     >>>>> >
>     >>>>> > On Sat, May 25, 2019 at 1:03 AM Ismaël Mejía
>     <iemejia@gmail.com <ma...@gmail.com>> wrote:
>     >>>>> >>
>     >>>>> >> Including the experimental suffix in artifact names is not
>     a good idea
>     >>>>> >> either because once we decide that it is not experimantal
>     anymore this
>     >>>>> >> will be a breaking change for users who will need then to
>     update its
>     >>>>> >> dependencies code. Also it is error-prone to use different
>     mappings
>     >>>>> >> for directories and artifacts (even if possible).
>     >>>>> >>
>     >>>>> >> May we reconsider this Kenn? I understand the motivation
>     but I hardly
>     >>>>> >> see this making things better or more clear. Any runner
>     user will end
>     >>>>> >> up reading the runner documentation and capability matrix
>     so he will
>     >>>>> >> catch the current status that way.
>     >>>>> >>
>     >>>>> >>
>     >>>>> >>
>     >>>>> >> On Sat, May 25, 2019 at 8:35 AM Jozsef Bartok
>     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
>     >>>>> >> >
>     >>>>> >> > I missed Ken's input when writing my previous mail. Sorry.
>     >>>>> >> > So, to recap: I should remove "experimental" from any
>     directory names, but find an other way of configuring the artifact
>     so that it still has "experimental" in it's name.
>     >>>>> >> > Right?
>     >>>>> >> >
>     >>>>> >> > On Sat, May 25, 2019 at 9:32 AM Jozsef Bartok
>     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
>     >>>>> >> >>
>     >>>>> >> >> Yes, I'll gladly fix it, we aren't particularly keen to
>     be labeled as experimental either..
>     >>>>> >> >>
>     >>>>> >> >> Btw. initially the "experimental" word was only in the
>     Gradle module name, but then there was some change
>     >>>>> >> >> ([BEAM-4046] decouple gradle project names and maven
>     artifact ids - 4/2/19) which kind of ended up
>     >>>>> >> >> putting it in the directory name. Maybe I should have
>     merged with that differently, but this is how
>     >>>>> >> >> it seemed consistent.
>     >>>>> >> >>
>     >>>>> >> >> Anyways, will fix it in my next PR.
>     >>>>> >> >>
>     >>>>> >> >> On Fri, May 24, 2019 at 5:53 PM Ismaël Mejía
>     <iemejia@gmail.com <ma...@gmail.com>> wrote:
>     >>>>> >> >>>
>     >>>>> >> >>> I see thanks Jozsef, marking things as Experimental was
>     discussed but
>     >>>>> >> >>> we never agreed on doing this at the directory level.
>     We can cover the
>     >>>>> >> >>> same ground by putting an annotation in the classes (in
>     particular the
>     >>>>> >> >>> JetRunner and JetPipelineOptions classes which are the
>     real public
>     >>>>> >> >>> interface, or in the documentation (in particular
>     website), I do not
>     >>>>> >> >>> see how putting this in the directory name helps and if
>     so we may need
>     >>>>> >> >>> to put this in many other directories which is far from
>     ideal. Any
>     >>>>> >> >>> chance this can be fixed (jet-experimental -> jet) ?
>     >>>>> >> >>>
>     >>>>> >> >>> On Fri, May 24, 2019 at 9:08 AM Jozsef Bartok
>     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
>     >>>>> >> >>> >
>     >>>>> >> >>> > Hi Ismaël!
>     >>>>> >> >>> >
>     >>>>> >> >>> > Quoting Kenn (from PR-8410): "We discussed on list
>     that it would be better to have new things always start as
>     experimental in a way that clearly distinguishes them from the core."
>     >>>>> >> >>> >
>     >>>>> >> >>> > Rgds
>     >>>>> >> >>> >
>     >>>>> >> >>> > On Thu, May 23, 2019 at 10:44 PM Ismaël Mejía
>     <iemejia@gmail.com <ma...@gmail.com>> wrote:
>     >>>>> >> >>> >>
>     >>>>> >> >>> >> I saw that the runner was merged but I don’t get why
>     the foler is
>     >>>>> >> >>> >> called ‘runners/jet experimental’ and not simply
>     ‘runners/jet’. Is it
>     >>>>> >> >>> >> because the runner does not pass ValidatesRunner? Or
>     because the
>     >>>>> >> >>> >> contributors are few? I don’t really see any reason
>     behind this
>     >>>>> >> >>> >> suffix. And even if the status is not mature that’s
>     not different from
>     >>>>> >> >>> >> other already merged runners.
>     >>>>> >> >>> >>
>     >>>>> >> >>> >> On Fri, Apr 26, 2019 at 9:43 PM Kenneth Knowles
>     <kenn@apache.org <ma...@apache.org>> wrote:
>     >>>>> >> >>> >> >
>     >>>>> >> >>> >> > Nice! That is *way* more than the PR I was looking
>     for. I just meant that you could update the website/ directory. It
>     is fine to keep the runner in your own repository if you want.
>     >>>>> >> >>> >> >
>     >>>>> >> >>> >> > But I think it is great if you want to contribute
>     it to Apache Beam (hence donate it to the Apache Software
>     Foundation). The benefits include: low-latency testing, free updates
>     when someone does a refactor. Things to consider are: subject to ASF
>     / Beam governance, PMC, commiters, subject to Beam's release cadence
>     (and we might exclude from Beam releases for a little bit).
>     Typically, we have kept runners on a branch until they are somewhat
>     stable. I don't feel strongly about this for disjoint codebases that
>     can easily be excluded from releases. We might want to suffix
>     `-experimental` to the artifacts for some time.
>     >>>>> >> >>> >> >
>     >>>>> >> >>> >> > I commented on the PR about the necessary i.p.
>     clearance steps.
>     >>>>> >> >>> >> >
>     >>>>> >> >>> >> > Kenn
>     >>>>> >> >>> >> >the probl
>      >>>>> >> >>> >> > On Fri, Apr 26, 2019 at 3:59 AM
>     jozsi@hazelcast.com <ma...@hazelcast.com>
>     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
>     >>>>> >> >>> >> >>
>     >>>>> >> >>> >> >> Hi Kenn.
>     >>>>> >> >>> >> >>
>     >>>>> >> >>> >> >> It took me a while to migrate our code to the
>     Beam repo, but I finally have been able to create the Pull Request
>     you asked for, this is it: https://github.com/apache/beam/pull/8410
>     >>>>> >> >>> >> >>
>     >>>>> >> >>> >> >> Looking forward to your feedback!
>     >>>>> >> >>> >> >>
>     >>>>> >> >>> >> >> Best regards,
>     >>>>> >> >>> >> >> Jozsef
>     >>>>> >> >>> >> >>
>     >>>>> >> >>> >> >> On 2019/04/19 20:52:42, Kenneth Knowles
>     <kenn@apache.org <ma...@apache.org>> wrote:
>     >>>>> >> >>> >> >> > The ValidatesRunner tests are the best source
>     we have for knowing the
>     >>>>> >> >>> >> >> > capabilities of a runner. Are there
>     instructions for running the tests?
>     >>>>> >> >>> >> >> >
>     >>>>> >> >>> >> >> > Assuming we can check it out, then just open a
>     PR to the website with the
>     >>>>> >> >>> >> >> > current capabilities and caveats. Since it is a
>     big deal and could use lots
>     >>>>> >> >>> >> >> > of eyes, I would share the PR link on this thread.
>     >>>>> >> >>> >> >> >
>     >>>>> >> >>> >> >> > Kenn
>     >>>>> >> >>> >> >> >
>     >>>>> >> >>> >> >> > On Thu, Apr 18, 2019 at 11:53 AM Jozsef Bartok
>     <jozsi@hazelcast.com <ma...@hazelcast.com>> wrote:
>     >>>>> >> >>> >> >> >
>     >>>>> >> >>> >> >> > > Hi. We at Hazelcast Jet have been working for
>     a while now to implement a
>     >>>>> >> >>> >> >> > > Java Beam Runner (non-portable) based on
>     Hazelcast Jet (
>     >>>>> >> >>> >> >> > > https://jet.hazelcast.org/). The process is
>     still ongoing (
>     >>>>> >> >>> >> >> > >
>     https://github.com/hazelcast/hazelcast-jet-beam-runner), but we are
>     >>>>> >> >>> >> >> > > aiming for a fully functional, reliable
>     Runner which can proudly join the
>     >>>>> >> >>> >> >> > > Capability Matrix. For that purpose I would
>     like to ask what’s your process
>     >>>>> >> >>> >> >> > > of validating runners? We are already running
>     the @ValidatesRunner tests
>     >>>>> >> >>> >> >> > > and the Nexmark test suite, but beyond that
>     what other steps do we need to
>     >>>>> >> >>> >> >> > > take to get our Runner to the level it needs
>     to be at?
>     >>>>> >> >>> >> >> > >
>     >>>>> >> >>> >> >> >
>     >>>
>     >>>
>     >>>
>     >>> --
>     >>>
>     >>> This email may be confidential and privileged. If you received
>     this communication by mistake, please don't forward it to anyone
>     else, please erase all copies and attachments, and please let me
>     know that it has gone to the wrong person.
>     >>>
>     >>> The above terms reflect a potential business arrangement, are
>     provided solely as a basis for further discussion, and are not
>     intended to be and do not constitute a legally binding obligation.
>     No legally binding obligations will be created, implied, or inferred
>     until an agreement in final form is executed in writing by all
>     parties involved.
>     >
>     >
>     >
>     > --
>     >
>     > This email may be confidential and privileged. If you received
>     this communication by mistake, please don't forward it to anyone
>     else, please erase all copies and attachments, and please let me
>     know that it has gone to the wrong person.
>     >
>     > The above terms reflect a potential business arrangement, are
>     provided solely as a basis for further discussion, and are not
>     intended to be and do not constitute a legally binding obligation.
>     No legally binding obligations will be created, implied, or inferred
>     until an agreement in final form is executed in writing by all
>     parties involved.
>
>