You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Jean-Baptiste Onofré <jb...@nanthrax.net> on 2018/06/12 19:32:50 UTC
[CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Hi all,
due to issues found by Udi and Pablo, I cancel RC1 to prepare a new release.
Thanks Udi and Pablo for catching up.
I will prepare a RC2 after cherry pick/fixed the found issues (tomorrow
morning my time).
Regards
JB
On 12/06/2018 21:04, Pablo Estrada wrote:
> Hello all,
> I've found another release blocker: The current findbugs dependency in
> build_rules.gradle has a GPL license. Here's the PR to depend on the
> correct findbugs: https://github.com/apache/beam/pull/5609
>
> Best
> -P.
>
> On Tue, Jun 12, 2018 at 11:03 AM Udi Meiri <ehudm@google.com
> <ma...@google.com>> wrote:
>
> Cherrypick created: https://github.com/apache/beam/pull/5607
> Tests still running.
>
> On Tue, Jun 12, 2018 at 9:59 AM Udi Meiri <ehudm@google.com
> <ma...@google.com>> wrote:
>
> -1: Would like to cherry pick a fix
> for https://issues.apache.org/jira/browse/BEAM-4536
>
> On Tue, Jun 12, 2018 at 9:48 AM Jean-Baptiste Onofré
> <jb@nanthrax.net <ma...@nanthrax.net>> wrote:
>
> Any update about your vote and fix ?
>
> Thanks !
> Regards
> JB
>
> On 12/06/2018 04:02, Udi Meiri wrote:
> > Another bug: reading from PubSub with_attributes=True is
> broken on
> > Python with Dataflow.
> > https://issues.apache.org/jira/browse/BEAM-4536
> >
> > JB, I'm making a PR that removes this keyword and I'd like
> to propose it
> > as a cherrypick to 2.5.0.
> > (feature should be fixed in the next release)
> >
> > On Mon, Jun 11, 2018 at 6:19 PM Chamikara Jayalath
> <chamikara@google.com <ma...@google.com>
> > <mailto:chamikara@google.com
> <ma...@google.com>>> wrote:
> >
> > FYI: looks like Python tests are failing for Windows. JIRA
> > is https://issues.apache.org/jira/browse/BEAM-4535.
> >
> > I don't think this is a release blocker but this
> should probably go
> > in release notes (for any user that tries to run tests
> on Python
> > source build). And we should try to incorporate a fix
> if we happen
> > to cut another release candidate for some reason.
> >
> > Thanks,
> > Cham
> >
> > On Mon, Jun 11, 2018 at 5:46 PM Pablo Estrada
> <pabloem@google.com <ma...@google.com>
> > <mailto:pabloem@google.com
> <ma...@google.com>>> wrote:
> >
> > Thanks everyone who has pitched in to validate the
> release!
> >
> > Boyuan Zhang and I have also run a few pipelines,
> and verified
> > that they work properly (see release validation
> spreadsheet[1]).
> >
> > We have also found that the Game Stats pipeline is
> failing in
> > Python Streaming Dataflow. I have filed
> BEAM-4534[2]. This is
> > not a blocker, since Python streaming is not yet
> fully supported.
> >
> > It seems that the uploaded artifacts look good.
> >
> > We have noticed that the Python artifacts are
> still missing
> > Python wheel files (compare [3] and [4]). JB,
> could you please
> > add the wheel files? Boyuan and I can try to help
> you prepare
> > them / upload them if necessary. Please let us know.
> >
> > Thanks again!
> > -P.
> >
> >
> [1] https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=152451807
> > [2] https://issues.apache.org/jira/browse/BEAM-4534
> > [3] https://dist.apache.org/repos/dist/dev/beam/2.4.0/
> > [4] https://dist.apache.org/repos/dist/dev/beam/2.5.0/
> >
> > On Mon, Jun 11, 2018 at 12:37 PM Alan Myrvold
> > <amyrvold@google.com <ma...@google.com>
> <mailto:amyrvold@google.com <ma...@google.com>>>
> wrote:
> >
> > +1 (non-binding)
> >
> > tested some of the quickstarts
> >
> > On Sun, Jun 10, 2018 at 1:39 AM Tim
> > <timrobertson100@gmail.com
> <ma...@gmail.com>
> > <mailto:timrobertson100@gmail.com
> <ma...@gmail.com>>> wrote:
> >
> > Tested by our team:
> > - mvn inclusion
> > - Avro, ES, Hadoop IF IO
> > - Pipelines run on Spark (Cloudera 5.12.0
> YARN cluster)
> > - Reviewed release notes
> >
> > +1
> >
> > Thanks also to everyone who helped get
> over the gradle
> > hurdle and in particular to JB.
> >
> > Tim
> >
> > > On 9 Jun 2018, at 05:56, Jean-Baptiste
> Onofré
> > <jb@nanthrax.net <ma...@nanthrax.net>
> <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> > >
> > > No problem Pablo.
> > >
> > > The vote period is a minimum, it can be
> extended as
> > requested or if we
> > > don't have the minimum of 3 binding votes.
> > >
> > > Regards
> > > JB
> > >
> > >> On 09/06/2018 01:54, Pablo Estrada wrote:
> > >> Hello all,
> > >> I'd like to request an extension of the
> voting period
> > until Monday
> > >> evening (US time, so later in other
> geographical
> > regions). This is
> > >> because we were only now able to
> publish Dataflow
> > Workers, and have not
> > >> had the chance to run release
> validation tests on
> > them. The extension
> > >> will allow us to validate and vote by
> Monday.
> > >> Is this acceptable to the community?
> > >>
> > >> Best
> > >> -P.
> > >>
> > >> On Fri, Jun 8, 2018 at 6:20 AM Alexey
> Romanenko
> > >> <aromanenko.dev@gmail.com
> <ma...@gmail.com>
> > <mailto:aromanenko.dev@gmail.com
> <ma...@gmail.com>>
> > <mailto:aromanenko.dev@gmail.com
> <ma...@gmail.com>
> > <mailto:aromanenko.dev@gmail.com
> <ma...@gmail.com>>>> wrote:
> > >>
> > >> Thank you JB for your work!
> > >>
> > >> I tested running simple streaming
> (/KafkaIO/) and
> > batch (/TextIO /
> > >> HDFS/) pipelines with SparkRunner on
> YARN cluster
> > - it works fine.
> > >>
> > >> WBR,
> > >> Alexey
> > >>
> > >>
> > >>> On 8 Jun 2018, at 10:00, Etienne
> Chauchot
> > <echauchot@apache.org
> <ma...@apache.org> <mailto:echauchot@apache.org
> <ma...@apache.org>>
> > >>> <mailto:echauchot@apache.org
> <ma...@apache.org>
> > <mailto:echauchot@apache.org
> <ma...@apache.org>>>> wrote:
> > >>>
> > >>> I forgot to vote:
> > >>> +1 (non binding).
> > >>> What I tested:
> > >>> - no functional or performance
> regression
> > comparing to v2.4
> > >>> - dependencies in the poms are ok
> > >>>
> > >>> Etienne
> > >>>> Le vendredi 08 juin 2018 à 08:27
> +0200, Romain
> > Manni-Bucau a écrit :
> > >>>> +1 (non-binding), mainstream usage
> is not broken
> > by the pom
> > >>>> changes and runtime has no known
> regression
> > compared to the 2.4.0
> > >>>>
> > >>>> (side note: kudo to JB for this
> build tool
> > change release, I know
> > >>>> how it can hurt ;))
> > >>>>
> > >>>> Romain Manni-Bucau
> > >>>> @rmannibucau
> <https://twitter.com/rmannibucau>
> > | Blog
> > >>>>
> <https://rmannibucau.metawerx.net/> | Old Blog
> > >>>>
> <http://rmannibucau.wordpress.com/> | Github
> > >>>> <https://github.com/rmannibucau> |
> LinkedIn
> > >>>>
> <https://www.linkedin.com/in/rmannibucau> | Book
> > >>>>
> >
> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
> > >>>>
> > >>>>
> > >>>> Le jeu. 7 juin 2018 à 16:17,
> Jean-Baptiste Onofré
> > >>>> <jb@nanthrax.net
> <ma...@nanthrax.net> <mailto:jb@nanthrax.net
> <ma...@nanthrax.net>>
> > <mailto:jb@nanthrax.net
> <ma...@nanthrax.net> <mailto:jb@nanthrax.net
> <ma...@nanthrax.net>>>> a écrit :
> > >>>>> Thanks for the details Etienne !
> > >>>>>
> > >>>>> The good news is that the
> artifacts seem OK and
> > the overall Nexmark
> > >>>>> results are consistent with the
> 2.4.0 release ones.
> > >>>>>
> > >>>>> I'm starting a complete review
> using the
> > beam-samples as well.
> > >>>>>
> > >>>>> Regards
> > >>>>> JB
> > >>>>>
> > >>>>>> On 07/06/2018 16:14, Etienne
> Chauchot wrote:
> > >>>>>> Hi,
> > >>>>>> I've just run the nexmark queries
> on v2.5.0-RC1 tag
> > >>>>>> What we can notice:
> > >>>>>> - query 3 (exercises CoGroupByKey,
> state and
> > timer) shows
> > >>>>> different
> > >>>>>> output with DR between batch and
> streaming and
> > with the other
> > >>>>> runners =>
> > >>>>>> I compared with v2.4 there were
> still these
> > differences but with
> > >>>>>> different output size numbers
> > >>>>>>
> > >>>>>> - query 6 (exercises specialized
> combiner) shows
> > different output
> > >>>>>> between the runners => the correct
> output is 401.
> > strange that
> > >>>>> in batch
> > >>>>>> mode some runners output les
> Sellers. I compared
> > with v2.4
> > >>>>> same output
> > >>>>>>
> > >>>>>> - response time of query 7
> (exercices Max
> > transform, fanout
> > >>>>> and side
> > >>>>>> input) is very slow on DR => I
> compared with v2.4
> > , comparable
> > >>>>> execution
> > >>>>>> times
> > >>>>>>
> > >>>>>> I'm not comparing q10 because it is
> a write to
> > GCS so it is
> > >>>>> very specific.
> > >>>>>>
> > >>>>>> => Basically no regression
> comparing to v2.4
> > >>>>>>
> > >>>>>> For the record here is the output
> (waiting for
> > ongoing perfkit
> > >>>>> integration):
> > >>>>>>
> > >>>>>>
> > >>>>>> 1. DR batch
> > >>>>>>
> > >>>>>> Performance:
> > >>>>>>
> > >>>>>>
> > >>>>> Conf Runtime(sec) (Baseline)
> Events(/sec)
> > (Baseline) Results (Baseline)
> > >>>>>>
> > >>>>>>
> > >>>>> 0000 5,8
> 17283,1
> > 100000
> > >>>>>>
> > >>>>>>
> > >>>>> 0001 3,2
> 31104,2
> > 92000
> > >>>>>>
> > >>>>>>
> > >>>>> 0002 1,2
> 82918,7
> > 351
> > >>>>>>
> > >>>>>>
> > >>>>> 0003 2,2
> 46210,7
> > 458
> > >>>>>>
> > >>>>>>
> > >>>>> 0004 1,2
> 8503,4
> > 40
> > >>>>>>
> > >>>>>>
> > >>>>> 0005 4,0
> 25220,7
> > 12
> > >>>>>>
> > >>>>>>
> > >>>>> 0006 0,9
> 11148,3
> > 401
> > >>>>>>
> > >>>>>>
> > >>>>> 0007 13,2
> 7580,9
> > 1
> > >>>>>>
> > >>>>>>
> > >>>>> 0008 1,5
> 67340,1
> > 6000
> > >>>>>>
> > >>>>>>
> > >>>>> 0009 0,7
> 14025,2
> > 298
> > >>>>>>
> > >>>>>>
> > >>>>> 0010 12,8
> 7793,0
> > 1
> > >>>>>>
> > >>>>>>
> > >>>>> 0011 2,4
> 42319,1
> > 1919
> > >>>>>>
> > >>>>>>
> > >>>>> 0012 1,6
> 61462,8
> > 1919
> > >>>>>>
> > >>>>>
> >
> ==========================================================================================
> > >>>>>>
> > >>>>>> 2. DR streaming
> > >>>>>>
> > >>>>>> Performance:
> > >>>>>>
> > >>>>>>
> > >>>>> Conf Runtime(sec) (Baseline)
> Events(/sec)
> > (Baseline) Results (Baseline)
> > >>>>>>
> > >>>>>>
> > >>>>> 0000 6,5
> 15285,8
> > 100000
> > >>>>>>
> > >>>>>>
> > >>>>> 0001 3,7
> 27397,3
> > 92000
> > >>>>>>
> > >>>>>>
> > >>>>> 0002 1,4
> 69108,5
> > 351
> > >>>>>>
> > >>>>>>
> > >>>>> 0003 3,2
> 31181,8
> > 447
> > >>>>>>
> > >>>>>>
> > >>>>> 0004 1,2
> 8361,2
> > 40
> > >>>>>>
> > >>>>>>
> > >>>>> 0005 5,3
> 18903,6
> > 12
> > >>>>>>
> > >>>>>>
> > >>>>> 0006 0,9
> 11111,1
> > 401
> > >>>>>>
> > >>>>>>
> > >>>>> 0007 82,5
> 1212,2
> > 1
> > >>>>>>
> > >>>>>>
> > >>>>> 0008 2,0
> 51072,5
> > 6000
> > >>>>>>
> > >>>>>>
> > >>>>> 0009 0,8
> 12903,2
> > 298
> > >>>>>>
> > >>>>>>
> > >>>>> 0010 49,5
> 2021,8
> > 1
> > >>>>>>
> > >>>>>>
> > >>>>> 0011 3,9
> 25667,4
> > 1919
> > >>>>>>
> > >>>>>>
> > >>>>> 0012 2,4
> 41067,8
> > 1919
> > >>>>>>
> > >>>>>
> >
> ==========================================================================================
> > >>>>>>
> > >>>>>> 3. Flink batch
> > >>>>>> Performance:
> > >>>>>>
> > >>>>>>
> > >>>>> Conf Runtime(sec) (Baseline)
> Events(/sec)
> > (Baseline) Results (Baseline)
> > >>>>>>
> > >>>>>>
> > >>>>> 0000 1,0
> 97656,3
> > 100000
> > >>>>>>
> > >>>>>>
> > >>>>> 0001 0,7
> 141643,1
> > 92000
> > >>>>>>
> > >>>>>>
> > >>>>> 0002 0,4
> 228310,5
> > 351
> > >>>>>>
> > >>>>>>
> > >>>>> 0003 1,6
> 64020,5
> > 580
> > >>>>>>
> > >>>>>>
> > >>>>> 0004 0,7
> 13831,3
> > 40
> > >>>>>>
> > >>>>>>
> > >>>>> 0005 1,4
> 72939,5
> > 12
> > >>>>>>
> > >>>>>>
> > >>>>> 0006 0,5
> 20491,8
> > 103
> > >>>>>>
> > >>>>>>
> > >>>>> 0007 1,3
> 74239,0
> > 1
> > >>>>>>
> > >>>>>>
> > >>>>> 0008 0,8
> 121506,7
> > 6000
> > >>>>>>
> > >>>>>>
> > >>>>> 0009 0,6
> 17953,3
> > 298
> > >>>>>>
> > >>>>>>
> > >>>>> 0010 1,3
> 74682,6
> > 1
> > >>>>>>
> > >>>>>>
> > >>>>> 0011 1,1
> 92936,8
> > 1919
> > >>>>>>
> > >>>>>>
> > >>>>> 0012 0,8
> 123001,2
> > 1919
> > >>>>>>
> > >>>>>
> >
> ==========================================================================================
> > >>>>>>
> > >>>>>> 4. Flink streaming
> > >>>>>> Performance:
> > >>>>>>
> > >>>>>>
> > >>>>> Conf Runtime(sec) (Baseline)
> Events(/sec)
> > (Baseline) Results (Baseline)
> > >>>>>>
> > >>>>>>
> > >>>>> 0000 5,4
> 18677,6
> > 100000
> > >>>>>>
> > >>>>>>
> > >>>>> 0001 2,8
> 35511,4
> > 92000
> > >>>>>>
> > >>>>>>
> > >>>>> 0002 1,8
> 54318,3
> > 351
> > >>>>>>
> > >>>>>>
> > >>>>> 0003 2,4
> 41614,6
> > 580
> > >>>>>>
> > >>>>>>
> > >>>>> 0004 1,0
> 10341,3
> > 40
> > >>>>>>
> > >>>>>>
> > >>>>> 0005 3,4
> 29568,3
> > 12
> > >>>>>>
> > >>>>>>
> > >>>>> 0006 0,7
> 13369,0
> > 401
> > >>>>>>
> > >>>>>>
> > >>>>> 0007 2,8
> 36192,5
> > 1
> > >>>>>>
> > >>>>>>
> > >>>>> 0008 1,8
> 54854,6
> > 6000
> > >>>>>>
> > >>>>>>
> > >>>>> 0009 0,7
> 13369,0
> > 298
> > >>>>>>
> > >>>>>>
> > >>>>> 0010 3,4
> 29841,8
> > 2
> > >>>>>>
> > >>>>>>
> > >>>>> 0011 5,0
> 19932,2
> > 1919
> > >>>>>>
> > >>>>>>
> > >>>>> 0012 2,6
> 38835,0
> > 1919
> > >>>>>>
> > >>>>>
> >
> ==========================================================================================
> > >>>>>>
> > >>>>>> 5. Spark batch
> > >>>>>> Performance:
> > >>>>>>
> > >>>>>>
> > >>>>> Conf Runtime(sec) (Baseline)
> Events(/sec)
> > (Baseline) Results (Baseline)
> > >>>>>>
> > >>>>>>
> > >>>>> 0000 1,5
> 65445,0
> > 100000
> > >>>>>>
> > >>>>>>
> > >>>>> 0001 1,3
> 79491,3
> > 92000
> > >>>>>>
> > >>>>>>
> > >>>>> 0002 0,9
> 112107,6
> > 351
> > >>>>>>
> > >>>>>>
> > >>>>> 0003 2,0
> 48804,3
> > 580
> > >>>>>>
> > >>>>>>
> > >>>>> 0004 1,2
> 8382,2
> > 40
> > >>>>>>
> > >>>>>>
> > >>>>> 0005 2,0
> 50838,8
> > 12
> > >>>>>>
> > >>>>>>
> > >>>>> 0006 1,0
> 9699,3
> > 103
> > >>>>>>
> > >>>>>>
> > >>>>> 0007 2,3
> 43308,8
> > 1
> > >>>>>>
> > >>>>>>
> > >>>>> 0008 2,1
> 46794,6
> > 6000
> > >>>>>>
> > >>>>>>
> > >>>>> 0009 1,1
> 8976,7
> > 298
> > >>>>>>
> > >>>>>>
> > >>>>> 0010 1,6
> 62111,8
> > 1
> > >>>>>>
> > >>>>>>
> > >>>>> 0011 2,1
> 46598,3
> > 1919
> > >>>>>>
> > >>>>>>
> > >>>>> 0012 2,3
> 43687,2
> > 1919
> > >>>>>>
> > >>>>>
> >
> ==========================================================================================
> > >>>>>>
> > >>>>>>> Le mercredi 06 juin 2018 à 10:50
> +0200, Etienne
> > Chauchot a écrit :
> > >>>>>>> Thanks JB for all your work ! I
> believe doing
> > the first
> > >>>>> gradle release
> > >>>>>>> must have been hard.
> > >>>>>>> I'll run Nexmark on the release
> and keep you posted.
> > >>>>>>>
> > >>>>>>> Best
> > >>>>>>> Etienne
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Le mercredi 06 juin 2018 à 10:44
> +0200,
> > Jean-Baptiste Onofré
> > >>>>> a écrit :
> > >>>>>>>> Hi everyone,
> > >>>>>>>>
> > >>>>>>>> Please review and vote on the
> release candidate
> > #1 for the
> > >>>>> version
> > >>>>>>>> 2.5.0, as follows:
> > >>>>>>>>
> > >>>>>>>> [ ] +1, Approve the release
> > >>>>>>>> [ ] -1, Do not approve the
> release (please
> > provide specific
> > >>>>> comments)
> > >>>>>>>>
> > >>>>>>>> NB: this is the first release
> using Gradle, so
> > don't be too
> > >>>>> harsh ;) A
> > >>>>>>>> PR about the release guide will
> follow thanks
> > to this release.
> > >>>>>>>>
> > >>>>>>>> The complete staging area is
> available for your
> > review,
> > >>>>> which includes:
> > >>>>>>>> * JIRA release notes [1],
> > >>>>>>>> * the official Apache source
> release to be
> > deployed to
> > >>>>> dist.apache.org
> <http://dist.apache.org> <http://dist.apache.org>
> > <http://dist.apache.org/>
> > >>>>>>>> [2], which is signed with the key
> with
> > fingerprint C8282E76 [3],
> > >>>>>>>> * all artifacts to be deployed to
> the Maven Central
> > >>>>> Repository [4],
> > >>>>>>>> * source code tag "v2.5.0-RC1" [5],
> > >>>>>>>> * website pull request listing
> the release and
> > publishing
> > >>>>> the API
> > >>>>>>>> reference manual [6].
> > >>>>>>>> * Java artifacts were built with
> Gradle 4.7
> > (wrapper) and
> > >>>>> OpenJDK/Oracle
> > >>>>>>>> JDK 1.8.0_172 (Oracle Corporation
> 25.172-b11).
> > >>>>>>>> * Python artifacts are deployed
> along with the
> > source
> > >>>>> release to the
> > >>>>>>>> dist.apache.org
> <http://dist.apache.org> <http://dist.apache.org>
> > <http://dist.apache.org/> [2].
> > >>>>>>>>
> > >>>>>>>> The vote will be open for at
> least 72 hours. It
> > is adopted
> > >>>>> by majority
> > >>>>>>>> approval, with at least 3 PMC
> affirmative votes.
> > >>>>>>>>
> > >>>>>>>> Thanks,
> > >>>>>>>> JB
> > >>>>>>>>
> > >>>>>>>> [1]
> > >>>>>>>>
> > >>>>>
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12342847
> > >>>>>>>> [2]
> >
> https://dist.apache.org/repos/dist/dev/beam/2.5.0/
> > >>>>>>>> [3]
> >
> https://dist.apache.org/repos/dist/release/beam/KEYS
> > >>>>>>>> [4]
> > >>>>>
> >
> https://repository.apache.org/content/repositories/orgapachebeam-1041/
> > >>>>>>>> [5]
> https://github.com/apache/beam/tree/v2.5.0-RC1
> > >>>>>>>> [6]
> https://github.com/apache/beam-site/pull/463
> > >>>>>>>>
> > >>>>>
> > >>
> > >> --
> > >> Got feedback? go/pabloem-feedback
> <https://goto.google.com/pabloem-feedback>
> > <https://goto.google.com/pabloem-feedback>
> > >
> > > --
> > > Jean-Baptiste Onofré
> > > jbonofre@apache.org
> <ma...@apache.org> <mailto:jbonofre@apache.org
> <ma...@apache.org>>
> > > http://blog.nanthrax.net
> > > Talend - http://www.talend.com
> >
> > --
> > Got feedback? go/pabloem-feedback
> <https://goto.google.com/pabloem-feedback>
> > <https://goto.google.com/pabloem-feedback>
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org <ma...@apache.org>
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
> --
> Got feedback? go/pabloem-feedback
Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
It looks good to me, I'm merging and moving forward.
Regards
JB
On 14/06/2018 00:45, Pablo Estrada wrote:
> Sent out https://github.com/apache/beam/pull/5640 to ignore the flaky
> test. As JB is the release manager, I'l let him make the call on what to
> do about it.
> Best
> -P.
>
> On Wed, Jun 13, 2018 at 3:34 PM Ahmet Altay <altay@google.com
> <ma...@google.com>> wrote:
>
> I would vote for second option, not a release blocker and disable
> the test in the release branch. My reasoning is:
> - ReferenceRunner is not yet the official alternative to existing
> direct runners.
> - It is bad to have flaky tests on the release branch, and we would
> not get good signal during validation.
>
> On Wed, Jun 13, 2018 at 3:14 PM, Pablo Estrada <pabloem@google.com
> <ma...@google.com>> wrote:
>
> Hello all,
> cherrypicks for the release branch seem to be going well, but
> thanks to them we were able to surface a flaky test in the
> release branch. JIRA is
> filed: https://issues.apache.org/jira/projects/BEAM/issues/BEAM-4558
>
> Given that test issue, I see the following options:
> - Consider that this test is not a release blocker. Go ahead
> with RC2 after cherrypicks are brought in, or
> - Consider that this test is not a release blocker, so we
> disable it before cutting RC2.
> - Consider this test a release blocker, and triage the bug for
> fixing.
>
> What do you think?
>
> Best
> -P.
>
> On Wed, Jun 13, 2018 at 9:54 AM Pablo Estrada
> <pabloem@google.com <ma...@google.com>> wrote:
>
> Precommits for
> PR https://github.com/apache/beam/pull/5609 are now passing.
> For now I've simply set failOnWarning to false to cherrypick
> into the release, and fix in master later on.
> Best
> -P.
>
> On Wed, Jun 13, 2018 at 9:08 AM Scott Wegner
> <swegner@google.com <ma...@google.com>> wrote:
>
> From my understanding, the @SuppressFBWarnings usage is
> in a dependency (ByteBuddy) rather than directly in our
> code; so we're not able to modify the usage.
>
> Pablo, feel free to disable failOnWarning for the
> sdks-java-core project temporarily. This isn't a major
> regression since we've only recently made the change to
> enable it [1]. We can work separately on figuring out
> how to resolve the warnings.
>
> [1] https://github.com/apache/beam/pull/5319
>
> On Tue, Jun 12, 2018 at 11:57 PM Tim Robertson
> <timrobertson100@gmail.com
> <ma...@gmail.com>> wrote:
>
> Hi Pablo,
>
> I'm afraid I couldn't find one either... there is an
> issue about it [1] which is old so it doesn't look
> likely to be resolved either.
>
> If you have time (sorry I am a bit busy) could you
> please verify the version does work if you install
> that version locally? I know the maven version of
> that [2] but not sure on the gradle equivalent. If
> we know it works, we can then find a repository that
> fits ok with Apache/Beam policy.
>
> Alternatively, we could consider using a fully
> qualified reference (i.e.
> @edu.umd.cs.findbugs.annotations.SuppressWarnings)
> to the deprecated version and leave the dependency
> at the 1.3.9-1. I believe our general direction is
> to remove findbugs when errorprone covers all
> aspects so I *expect* this should be considered
> reasonable.
>
> I hope this helps,
> Tim
>
> [1] https://github.com/stephenc/findbugs-annotations/issues/4
> [2] https://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html
>
> On Wed, Jun 13, 2018 at 8:39 AM, Pablo Estrada
> <pabloem@google.com <ma...@google.com>> wrote:
>
> Hi Tim,
> you're right. Thanks for pointing that out.
> There's just one problem that I'm running into
> now: The 3.0.1-1 version does not seem to be
> available in Maven Central[1]. Looking at the
> website, I am not quite sure if there's another
> repository where they do stage the newer
> versions?[2]
>
> -P
>
> [1] https://repo.maven.apache.org/maven2/com/github/stephenc/findbugs/findbugs-annotations/
> [2] http://stephenc.github.io/findbugs-annotations/
>
> On Tue, Jun 12, 2018 at 11:10 PM Tim Robertson
> <timrobertson100@gmail.com
> <ma...@gmail.com>> wrote:
>
> Hi Pablo,
>
> I took only a quick look.
>
> "- The JAR from the non-LGPL findbugs does
> not contain the SuppressFBWarnings annotation"
>
> Unless I misunderstand you it looks like
> SuppressFBWarnings was added in Stephen's
> version in this commit [1] which was
> introduced in version 2.0.3-1 - I've
> checked is in the 3.0.1-1 build [2]
> I notice in your commits [1] you've been
> exploring version 3.0.0 already though...
> what happens when you use 3.0.1-1? It sounds
> like the wrong version is coming in rather
> than the annotation being missing.
>
> Thanks,
> Tim
>
> [1] https://github.com/stephenc/findbugs-annotations/commits/master/src/main/java/edu/umd/cs/findbugs/annotations/SuppressWarnings.java
> [2] https://github.com/stephenc/findbugs-annotations/releases
> [3] https://github.com/apache/beam/pull/5609/commits/32c7df706e970557f154ff6bc521b2e00f9d09ab
>
>
>
>
>
>
>
> On Wed, Jun 13, 2018 at 2:37 AM, Pablo
> Estrada <pabloem@google.com
> <ma...@google.com>> wrote:
>
> Hi all,
> I'll humbly declare that after wrestling
> with he build to stop depending on the
> wrong findbugs_annotations, I feel
> somewhat lost. The issue is actually
> quite small:
>
> - The JAR from the non-LGPL findbugs
> does not contain the SuppressFBWarnings
> annotation. This means that when
> building, ByteBuddy produces a few
> warnings (nothing critical).
> - The easiest way to avoid this failure
> is to call
> applyJavaNature(failOnWarning: false),
> but this would be bad, since we want to
> keep a high standard for tasks like
> ErrorProne and FindBugs itself.
> - So I find myself lost: How do we
> suppress trivial warnings coming from
> missing annotations, and honor warnings
> coming from other plugins?
>
> Any help / a PR from someone more
> capable would be appreciated.
> Best
> -P.
>
> On Tue, Jun 12, 2018 at 3:02 PM Ismaël
> Mejía <iemejia@gmail.com
> <ma...@gmail.com>> wrote:
>
> Yes, ok I was not aware it was
> already being addressed, nice.
> On Tue, Jun 12, 2018 at 11:56 PM
> Ahmet Altay <altay@google.com
> <ma...@google.com>> wrote:
> >
> > Ismaël,
> >
> > I believe Pablo's
> https://github.com/apache/beam/pull/5609
> is fixing the issue by changing the
> findbugs back to
> "com.github.stephenc.findbugs". Is
> this what you are referring to?
> >
> > Ahmet
> >
> > On Tue, Jun 12, 2018 at 2:51 PM,
> Boyuan Zhang <boyuanz@google.com
> <ma...@google.com>> wrote:
> >>
> >> Hey JB,
> >>
> >> I added some instructions about
> how to create python wheels in this
> PR:
> https://github.com/apache/beam-site/pull/467
> . Hope it would be helpful.
> >>
> >> Boyuan
> >>
> >
>
> --
> Got feedback? go/pabloem-feedback
> <https://goto.google.com/pabloem-feedback>
>
>
> --
> Got feedback? go/pabloem-feedback
> <https://goto.google.com/pabloem-feedback>
>
>
> --
> Got feedback? go/pabloem-feedback
> <https://goto.google.com/pabloem-feedback>
>
> --
> Got feedback? go/pabloem-feedback
> <https://goto.google.com/pabloem-feedback>
>
>
> --
> Got feedback? go/pabloem-feedback
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Posted by Ismaël Mejía <ie...@gmail.com>.
There is another issue highlighted by Scott Wegner in a non-related to
the vote PR discussion.
https://github.com/apache/beam/pull/5540
It seems that in the migration to gradle we changed the
findbugs-annotations library from com.github.stephenc.findbugs to
com.google.code.findbugs:findbugs-annotations and this one is LGPL
licensed so we should address that too for the next RC.
On Tue, Jun 12, 2018 at 9:33 PM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>
> Hi all,
>
> due to issues found by Udi and Pablo, I cancel RC1 to prepare a new release.
>
> Thanks Udi and Pablo for catching up.
>
> I will prepare a RC2 after cherry pick/fixed the found issues (tomorrow
> morning my time).
>
> Regards
> JB
>
>
> On 12/06/2018 21:04, Pablo Estrada wrote:
> > Hello all,
> > I've found another release blocker: The current findbugs dependency in
> > build_rules.gradle has a GPL license. Here's the PR to depend on the
> > correct findbugs: https://github.com/apache/beam/pull/5609
> >
> > Best
> > -P.
> >
> > On Tue, Jun 12, 2018 at 11:03 AM Udi Meiri <ehudm@google.com
> > <ma...@google.com>> wrote:
> >
> > Cherrypick created: https://github.com/apache/beam/pull/5607
> > Tests still running.
> >
> > On Tue, Jun 12, 2018 at 9:59 AM Udi Meiri <ehudm@google.com
> > <ma...@google.com>> wrote:
> >
> > -1: Would like to cherry pick a fix
> > for https://issues.apache.org/jira/browse/BEAM-4536
> >
> > On Tue, Jun 12, 2018 at 9:48 AM Jean-Baptiste Onofré
> > <jb@nanthrax.net <ma...@nanthrax.net>> wrote:
> >
> > Any update about your vote and fix ?
> >
> > Thanks !
> > Regards
> > JB
> >
> > On 12/06/2018 04:02, Udi Meiri wrote:
> > > Another bug: reading from PubSub with_attributes=True is
> > broken on
> > > Python with Dataflow.
> > > https://issues.apache.org/jira/browse/BEAM-4536
> > >
> > > JB, I'm making a PR that removes this keyword and I'd like
> > to propose it
> > > as a cherrypick to 2.5.0.
> > > (feature should be fixed in the next release)
> > >
> > > On Mon, Jun 11, 2018 at 6:19 PM Chamikara Jayalath
> > <chamikara@google.com <ma...@google.com>
> > > <mailto:chamikara@google.com
> > <ma...@google.com>>> wrote:
> > >
> > > FYI: looks like Python tests are failing for Windows. JIRA
> > > is https://issues.apache.org/jira/browse/BEAM-4535.
> > >
> > > I don't think this is a release blocker but this
> > should probably go
> > > in release notes (for any user that tries to run tests
> > on Python
> > > source build). And we should try to incorporate a fix
> > if we happen
> > > to cut another release candidate for some reason.
> > >
> > > Thanks,
> > > Cham
> > >
> > > On Mon, Jun 11, 2018 at 5:46 PM Pablo Estrada
> > <pabloem@google.com <ma...@google.com>
> > > <mailto:pabloem@google.com
> > <ma...@google.com>>> wrote:
> > >
> > > Thanks everyone who has pitched in to validate the
> > release!
> > >
> > > Boyuan Zhang and I have also run a few pipelines,
> > and verified
> > > that they work properly (see release validation
> > spreadsheet[1]).
> > >
> > > We have also found that the Game Stats pipeline is
> > failing in
> > > Python Streaming Dataflow. I have filed
> > BEAM-4534[2]. This is
> > > not a blocker, since Python streaming is not yet
> > fully supported.
> > >
> > > It seems that the uploaded artifacts look good.
> > >
> > > We have noticed that the Python artifacts are
> > still missing
> > > Python wheel files (compare [3] and [4]). JB,
> > could you please
> > > add the wheel files? Boyuan and I can try to help
> > you prepare
> > > them / upload them if necessary. Please let us know.
> > >
> > > Thanks again!
> > > -P.
> > >
> > >
> > [1] https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=152451807
> > > [2] https://issues.apache.org/jira/browse/BEAM-4534
> > > [3] https://dist.apache.org/repos/dist/dev/beam/2.4.0/
> > > [4] https://dist.apache.org/repos/dist/dev/beam/2.5.0/
> > >
> > > On Mon, Jun 11, 2018 at 12:37 PM Alan Myrvold
> > > <amyrvold@google.com <ma...@google.com>
> > <mailto:amyrvold@google.com <ma...@google.com>>>
> > wrote:
> > >
> > > +1 (non-binding)
> > >
> > > tested some of the quickstarts
> > >
> > > On Sun, Jun 10, 2018 at 1:39 AM Tim
> > > <timrobertson100@gmail.com
> > <ma...@gmail.com>
> > > <mailto:timrobertson100@gmail.com
> > <ma...@gmail.com>>> wrote:
> > >
> > > Tested by our team:
> > > - mvn inclusion
> > > - Avro, ES, Hadoop IF IO
> > > - Pipelines run on Spark (Cloudera 5.12.0
> > YARN cluster)
> > > - Reviewed release notes
> > >
> > > +1
> > >
> > > Thanks also to everyone who helped get
> > over the gradle
> > > hurdle and in particular to JB.
> > >
> > > Tim
> > >
> > > > On 9 Jun 2018, at 05:56, Jean-Baptiste
> > Onofré
> > > <jb@nanthrax.net <ma...@nanthrax.net>
> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> > > >
> > > > No problem Pablo.
> > > >
> > > > The vote period is a minimum, it can be
> > extended as
> > > requested or if we
> > > > don't have the minimum of 3 binding votes.
> > > >
> > > > Regards
> > > > JB
> > > >
> > > >> On 09/06/2018 01:54, Pablo Estrada wrote:
> > > >> Hello all,
> > > >> I'd like to request an extension of the
> > voting period
> > > until Monday
> > > >> evening (US time, so later in other
> > geographical
> > > regions). This is
> > > >> because we were only now able to
> > publish Dataflow
> > > Workers, and have not
> > > >> had the chance to run release
> > validation tests on
> > > them. The extension
> > > >> will allow us to validate and vote by
> > Monday.
> > > >> Is this acceptable to the community?
> > > >>
> > > >> Best
> > > >> -P.
> > > >>
> > > >> On Fri, Jun 8, 2018 at 6:20 AM Alexey
> > Romanenko
> > > >> <aromanenko.dev@gmail.com
> > <ma...@gmail.com>
> > > <mailto:aromanenko.dev@gmail.com
> > <ma...@gmail.com>>
> > > <mailto:aromanenko.dev@gmail.com
> > <ma...@gmail.com>
> > > <mailto:aromanenko.dev@gmail.com
> > <ma...@gmail.com>>>> wrote:
> > > >>
> > > >> Thank you JB for your work!
> > > >>
> > > >> I tested running simple streaming
> > (/KafkaIO/) and
> > > batch (/TextIO /
> > > >> HDFS/) pipelines with SparkRunner on
> > YARN cluster
> > > - it works fine.
> > > >>
> > > >> WBR,
> > > >> Alexey
> > > >>
> > > >>
> > > >>> On 8 Jun 2018, at 10:00, Etienne
> > Chauchot
> > > <echauchot@apache.org
> > <ma...@apache.org> <mailto:echauchot@apache.org
> > <ma...@apache.org>>
> > > >>> <mailto:echauchot@apache.org
> > <ma...@apache.org>
> > > <mailto:echauchot@apache.org
> > <ma...@apache.org>>>> wrote:
> > > >>>
> > > >>> I forgot to vote:
> > > >>> +1 (non binding).
> > > >>> What I tested:
> > > >>> - no functional or performance
> > regression
> > > comparing to v2.4
> > > >>> - dependencies in the poms are ok
> > > >>>
> > > >>> Etienne
> > > >>>> Le vendredi 08 juin 2018 à 08:27
> > +0200, Romain
> > > Manni-Bucau a écrit :
> > > >>>> +1 (non-binding), mainstream usage
> > is not broken
> > > by the pom
> > > >>>> changes and runtime has no known
> > regression
> > > compared to the 2.4.0
> > > >>>>
> > > >>>> (side note: kudo to JB for this
> > build tool
> > > change release, I know
> > > >>>> how it can hurt ;))
> > > >>>>
> > > >>>> Romain Manni-Bucau
> > > >>>> @rmannibucau
> > <https://twitter.com/rmannibucau>
> > > | Blog
> > > >>>>
> > <https://rmannibucau.metawerx.net/> | Old Blog
> > > >>>>
> > <http://rmannibucau.wordpress.com/> | Github
> > > >>>> <https://github.com/rmannibucau> |
> > LinkedIn
> > > >>>>
> > <https://www.linkedin.com/in/rmannibucau> | Book
> > > >>>>
> > >
> > <https://www.packtpub.com/application-development/java-ee-8-high-performance>
> > > >>>>
> > > >>>>
> > > >>>> Le jeu. 7 juin 2018 à 16:17,
> > Jean-Baptiste Onofré
> > > >>>> <jb@nanthrax.net
> > <ma...@nanthrax.net> <mailto:jb@nanthrax.net
> > <ma...@nanthrax.net>>
> > > <mailto:jb@nanthrax.net
> > <ma...@nanthrax.net> <mailto:jb@nanthrax.net
> > <ma...@nanthrax.net>>>> a écrit :
> > > >>>>> Thanks for the details Etienne !
> > > >>>>>
> > > >>>>> The good news is that the
> > artifacts seem OK and
> > > the overall Nexmark
> > > >>>>> results are consistent with the
> > 2.4.0 release ones.
> > > >>>>>
> > > >>>>> I'm starting a complete review
> > using the
> > > beam-samples as well.
> > > >>>>>
> > > >>>>> Regards
> > > >>>>> JB
> > > >>>>>
> > > >>>>>> On 07/06/2018 16:14, Etienne
> > Chauchot wrote:
> > > >>>>>> Hi,
> > > >>>>>> I've just run the nexmark queries
> > on v2.5.0-RC1 tag
> > > >>>>>> What we can notice:
> > > >>>>>> - query 3 (exercises CoGroupByKey,
> > state and
> > > timer) shows
> > > >>>>> different
> > > >>>>>> output with DR between batch and
> > streaming and
> > > with the other
> > > >>>>> runners =>
> > > >>>>>> I compared with v2.4 there were
> > still these
> > > differences but with
> > > >>>>>> different output size numbers
> > > >>>>>>
> > > >>>>>> - query 6 (exercises specialized
> > combiner) shows
> > > different output
> > > >>>>>> between the runners => the correct
> > output is 401.
> > > strange that
> > > >>>>> in batch
> > > >>>>>> mode some runners output les
> > Sellers. I compared
> > > with v2.4
> > > >>>>> same output
> > > >>>>>>
> > > >>>>>> - response time of query 7
> > (exercices Max
> > > transform, fanout
> > > >>>>> and side
> > > >>>>>> input) is very slow on DR => I
> > compared with v2.4
> > > , comparable
> > > >>>>> execution
> > > >>>>>> times
> > > >>>>>>
> > > >>>>>> I'm not comparing q10 because it is
> > a write to
> > > GCS so it is
> > > >>>>> very specific.
> > > >>>>>>
> > > >>>>>> => Basically no regression
> > comparing to v2.4
> > > >>>>>>
> > > >>>>>> For the record here is the output
> > (waiting for
> > > ongoing perfkit
> > > >>>>> integration):
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> 1. DR batch
> > > >>>>>>
> > > >>>>>> Performance:
> > > >>>>>>
> > > >>>>>>
> > > >>>>> Conf Runtime(sec) (Baseline)
> > Events(/sec)
> > > (Baseline) Results (Baseline)
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0000 5,8
> > 17283,1
> > > 100000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0001 3,2
> > 31104,2
> > > 92000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0002 1,2
> > 82918,7
> > > 351
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0003 2,2
> > 46210,7
> > > 458
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0004 1,2
> > 8503,4
> > > 40
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0005 4,0
> > 25220,7
> > > 12
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0006 0,9
> > 11148,3
> > > 401
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0007 13,2
> > 7580,9
> > > 1
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0008 1,5
> > 67340,1
> > > 6000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0009 0,7
> > 14025,2
> > > 298
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0010 12,8
> > 7793,0
> > > 1
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0011 2,4
> > 42319,1
> > > 1919
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0012 1,6
> > 61462,8
> > > 1919
> > > >>>>>>
> > > >>>>>
> > >
> > ==========================================================================================
> > > >>>>>>
> > > >>>>>> 2. DR streaming
> > > >>>>>>
> > > >>>>>> Performance:
> > > >>>>>>
> > > >>>>>>
> > > >>>>> Conf Runtime(sec) (Baseline)
> > Events(/sec)
> > > (Baseline) Results (Baseline)
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0000 6,5
> > 15285,8
> > > 100000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0001 3,7
> > 27397,3
> > > 92000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0002 1,4
> > 69108,5
> > > 351
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0003 3,2
> > 31181,8
> > > 447
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0004 1,2
> > 8361,2
> > > 40
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0005 5,3
> > 18903,6
> > > 12
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0006 0,9
> > 11111,1
> > > 401
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0007 82,5
> > 1212,2
> > > 1
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0008 2,0
> > 51072,5
> > > 6000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0009 0,8
> > 12903,2
> > > 298
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0010 49,5
> > 2021,8
> > > 1
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0011 3,9
> > 25667,4
> > > 1919
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0012 2,4
> > 41067,8
> > > 1919
> > > >>>>>>
> > > >>>>>
> > >
> > ==========================================================================================
> > > >>>>>>
> > > >>>>>> 3. Flink batch
> > > >>>>>> Performance:
> > > >>>>>>
> > > >>>>>>
> > > >>>>> Conf Runtime(sec) (Baseline)
> > Events(/sec)
> > > (Baseline) Results (Baseline)
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0000 1,0
> > 97656,3
> > > 100000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0001 0,7
> > 141643,1
> > > 92000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0002 0,4
> > 228310,5
> > > 351
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0003 1,6
> > 64020,5
> > > 580
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0004 0,7
> > 13831,3
> > > 40
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0005 1,4
> > 72939,5
> > > 12
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0006 0,5
> > 20491,8
> > > 103
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0007 1,3
> > 74239,0
> > > 1
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0008 0,8
> > 121506,7
> > > 6000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0009 0,6
> > 17953,3
> > > 298
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0010 1,3
> > 74682,6
> > > 1
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0011 1,1
> > 92936,8
> > > 1919
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0012 0,8
> > 123001,2
> > > 1919
> > > >>>>>>
> > > >>>>>
> > >
> > ==========================================================================================
> > > >>>>>>
> > > >>>>>> 4. Flink streaming
> > > >>>>>> Performance:
> > > >>>>>>
> > > >>>>>>
> > > >>>>> Conf Runtime(sec) (Baseline)
> > Events(/sec)
> > > (Baseline) Results (Baseline)
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0000 5,4
> > 18677,6
> > > 100000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0001 2,8
> > 35511,4
> > > 92000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0002 1,8
> > 54318,3
> > > 351
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0003 2,4
> > 41614,6
> > > 580
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0004 1,0
> > 10341,3
> > > 40
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0005 3,4
> > 29568,3
> > > 12
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0006 0,7
> > 13369,0
> > > 401
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0007 2,8
> > 36192,5
> > > 1
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0008 1,8
> > 54854,6
> > > 6000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0009 0,7
> > 13369,0
> > > 298
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0010 3,4
> > 29841,8
> > > 2
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0011 5,0
> > 19932,2
> > > 1919
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0012 2,6
> > 38835,0
> > > 1919
> > > >>>>>>
> > > >>>>>
> > >
> > ==========================================================================================
> > > >>>>>>
> > > >>>>>> 5. Spark batch
> > > >>>>>> Performance:
> > > >>>>>>
> > > >>>>>>
> > > >>>>> Conf Runtime(sec) (Baseline)
> > Events(/sec)
> > > (Baseline) Results (Baseline)
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0000 1,5
> > 65445,0
> > > 100000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0001 1,3
> > 79491,3
> > > 92000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0002 0,9
> > 112107,6
> > > 351
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0003 2,0
> > 48804,3
> > > 580
> > > >>
Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Posted by Eugene Kirpichov <ki...@google.com>.
FWIW I have a fix to the flaky test in
https://github.com/apache/beam/pull/5585 (open)
On Wed, Jun 13, 2018 at 5:26 PM Udi Meiri <eh...@google.com> wrote:
> +1 to ignoring flaky test.
>
> FYI there's a fourth cherrypick: https://github.com/apache/beam/pull/5624
>
> On Wed, Jun 13, 2018 at 3:45 PM Pablo Estrada <pa...@google.com> wrote:
>
>> Sent out https://github.com/apache/beam/pull/5640 to ignore the flaky
>> test. As JB is the release manager, I'l let him make the call on what to do
>> about it.
>> Best
>> -P.
>>
>> On Wed, Jun 13, 2018 at 3:34 PM Ahmet Altay <al...@google.com> wrote:
>>
>>> I would vote for second option, not a release blocker and disable the
>>> test in the release branch. My reasoning is:
>>> - ReferenceRunner is not yet the official alternative to existing direct
>>> runners.
>>> - It is bad to have flaky tests on the release branch, and we would not
>>> get good signal during validation.
>>>
>>> On Wed, Jun 13, 2018 at 3:14 PM, Pablo Estrada <pa...@google.com>
>>> wrote:
>>>
>>>> Hello all,
>>>> cherrypicks for the release branch seem to be going well, but thanks to
>>>> them we were able to surface a flaky test in the release branch. JIRA is
>>>> filed: https://issues.apache.org/jira/projects/BEAM/issues/BEAM-4558
>>>>
>>>> Given that test issue, I see the following options:
>>>> - Consider that this test is not a release blocker. Go ahead with RC2
>>>> after cherrypicks are brought in, or
>>>> - Consider that this test is not a release blocker, so we disable it
>>>> before cutting RC2.
>>>> - Consider this test a release blocker, and triage the bug for fixing.
>>>>
>>>> What do you think?
>>>>
>>>> Best
>>>> -P.
>>>>
>>>> On Wed, Jun 13, 2018 at 9:54 AM Pablo Estrada <pa...@google.com>
>>>> wrote:
>>>>
>>>>> Precommits for PR https://github.com/apache/beam/pull/5609 are now
>>>>> passing. For now I've simply set failOnWarning to false to cherrypick into
>>>>> the release, and fix in master later on.
>>>>> Best
>>>>> -P.
>>>>>
>>>>> On Wed, Jun 13, 2018 at 9:08 AM Scott Wegner <sw...@google.com>
>>>>> wrote:
>>>>>
>>>>>> From my understanding, the @SuppressFBWarnings usage is in a
>>>>>> dependency (ByteBuddy) rather than directly in our code; so we're not able
>>>>>> to modify the usage.
>>>>>>
>>>>>> Pablo, feel free to disable failOnWarning for the sdks-java-core
>>>>>> project temporarily. This isn't a major regression since we've only
>>>>>> recently made the change to enable it [1]. We can work separately on
>>>>>> figuring out how to resolve the warnings.
>>>>>>
>>>>>> [1] https://github.com/apache/beam/pull/5319
>>>>>>
>>>>>> On Tue, Jun 12, 2018 at 11:57 PM Tim Robertson <
>>>>>> timrobertson100@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Pablo,
>>>>>>>
>>>>>>> I'm afraid I couldn't find one either... there is an issue about it
>>>>>>> [1] which is old so it doesn't look likely to be resolved either.
>>>>>>>
>>>>>>> If you have time (sorry I am a bit busy) could you please verify the
>>>>>>> version does work if you install that version locally? I know the maven
>>>>>>> version of that [2] but not sure on the gradle equivalent. If we know it
>>>>>>> works, we can then find a repository that fits ok with Apache/Beam policy.
>>>>>>>
>>>>>>> Alternatively, we could consider using a fully qualified reference
>>>>>>> (i.e. @edu.umd.cs.findbugs.annotations.SuppressWarnings) to the deprecated
>>>>>>> version and leave the dependency at the 1.3.9-1. I believe our general
>>>>>>> direction is to remove findbugs when errorprone covers all aspects so I
>>>>>>> *expect* this should be considered reasonable.
>>>>>>>
>>>>>>> I hope this helps,
>>>>>>> Tim
>>>>>>>
>>>>>>> [1] https://github.com/stephenc/findbugs-annotations/issues/4
>>>>>>> [2]
>>>>>>> https://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html
>>>>>>>
>>>>>>> On Wed, Jun 13, 2018 at 8:39 AM, Pablo Estrada <pa...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Tim,
>>>>>>>> you're right. Thanks for pointing that out. There's just one
>>>>>>>> problem that I'm running into now: The 3.0.1-1 version does not seem to be
>>>>>>>> available in Maven Central[1]. Looking at the website, I am not quite sure
>>>>>>>> if there's another repository where they do stage the newer versions?[2]
>>>>>>>>
>>>>>>>> -P
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> https://repo.maven.apache.org/maven2/com/github/stephenc/findbugs/findbugs-annotations
>>>>>>>> /
>>>>>>>> [2] http://stephenc.github.io/findbugs-annotations/
>>>>>>>>
>>>>>>>> On Tue, Jun 12, 2018 at 11:10 PM Tim Robertson <
>>>>>>>> timrobertson100@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Pablo,
>>>>>>>>>
>>>>>>>>> I took only a quick look.
>>>>>>>>>
>>>>>>>>> "- The JAR from the non-LGPL findbugs does not contain the
>>>>>>>>> SuppressFBWarnings annotation"
>>>>>>>>>
>>>>>>>>> Unless I misunderstand you it looks like SuppressFBWarnings was
>>>>>>>>> added in Stephen's version in this commit [1] which was
>>>>>>>>> introduced in version 2.0.3-1 - I've checked is in the 3.0.1-1 build [2]
>>>>>>>>> I notice in your commits [1] you've been exploring version 3.0.0
>>>>>>>>> already though... what happens when you use 3.0.1-1? It sounds like the
>>>>>>>>> wrong version is coming in rather than the annotation being missing.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Tim
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>> https://github.com/stephenc/findbugs-annotations/commits/master/src/main/java/edu/umd/cs/findbugs/annotations/SuppressWarnings.java
>>>>>>>>> [2] https://github.com/stephenc/findbugs-annotations/releases
>>>>>>>>> [3]
>>>>>>>>> https://github.com/apache/beam/pull/5609/commits/32c7df706e970557f154ff6bc521b2e00f9d09ab
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jun 13, 2018 at 2:37 AM, Pablo Estrada <pabloem@google.com
>>>>>>>>> > wrote:
>>>>>>>>>
>>>>>>>>>> Hi all,
>>>>>>>>>> I'll humbly declare that after wrestling with he build to stop
>>>>>>>>>> depending on the wrong findbugs_annotations, I feel somewhat lost. The
>>>>>>>>>> issue is actually quite small:
>>>>>>>>>>
>>>>>>>>>> - The JAR from the non-LGPL findbugs does not contain the
>>>>>>>>>> SuppressFBWarnings annotation. This means that when building, ByteBuddy
>>>>>>>>>> produces a few warnings (nothing critical).
>>>>>>>>>> - The easiest way to avoid this failure is to call
>>>>>>>>>> applyJavaNature(failOnWarning: false), but this would be bad, since we want
>>>>>>>>>> to keep a high standard for tasks like ErrorProne and FindBugs itself.
>>>>>>>>>> - So I find myself lost: How do we suppress trivial warnings
>>>>>>>>>> coming from missing annotations, and honor warnings coming from other
>>>>>>>>>> plugins?
>>>>>>>>>>
>>>>>>>>>> Any help / a PR from someone more capable would be appreciated.
>>>>>>>>>> Best
>>>>>>>>>> -P.
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 12, 2018 at 3:02 PM Ismaël Mejía <ie...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Yes, ok I was not aware it was already being addressed, nice.
>>>>>>>>>>> On Tue, Jun 12, 2018 at 11:56 PM Ahmet Altay <al...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>> >
>>>>>>>>>>> > Ismaël,
>>>>>>>>>>> >
>>>>>>>>>>> > I believe Pablo's https://github.com/apache/beam/pull/5609 is
>>>>>>>>>>> fixing the issue by changing the findbugs back to
>>>>>>>>>>> "com.github.stephenc.findbugs". Is this what you are referring to?
>>>>>>>>>>> >
>>>>>>>>>>> > Ahmet
>>>>>>>>>>> >
>>>>>>>>>>> > On Tue, Jun 12, 2018 at 2:51 PM, Boyuan Zhang <
>>>>>>>>>>> boyuanz@google.com> wrote:
>>>>>>>>>>> >>
>>>>>>>>>>> >> Hey JB,
>>>>>>>>>>> >>
>>>>>>>>>>> >> I added some instructions about how to create python wheels
>>>>>>>>>>> in this PR: https://github.com/apache/beam-site/pull/467 . Hope
>>>>>>>>>>> it would be helpful.
>>>>>>>>>>> >>
>>>>>>>>>>> >> Boyuan
>>>>>>>>>>> >>
>>>>>>>>>>> >
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Got feedback? go/pabloem-feedback
>>>>>>>>>> <https://goto.google.com/pabloem-feedback>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>> Got feedback? go/pabloem-feedback
>>>>>>>> <https://goto.google.com/pabloem-feedback>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>> Got feedback? go/pabloem-feedback
>>>>> <https://goto.google.com/pabloem-feedback>
>>>>>
>>>> --
>>>> Got feedback? go/pabloem-feedback
>>>> <https://goto.google.com/pabloem-feedback>
>>>>
>>>
>>> --
>> Got feedback? go/pabloem-feedback
>> <https://goto.google.com/pabloem-feedback>
>>
>
Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Posted by Udi Meiri <eh...@google.com>.
+1 to ignoring flaky test.
FYI there's a fourth cherrypick: https://github.com/apache/beam/pull/5624
On Wed, Jun 13, 2018 at 3:45 PM Pablo Estrada <pa...@google.com> wrote:
> Sent out https://github.com/apache/beam/pull/5640 to ignore the flaky
> test. As JB is the release manager, I'l let him make the call on what to do
> about it.
> Best
> -P.
>
> On Wed, Jun 13, 2018 at 3:34 PM Ahmet Altay <al...@google.com> wrote:
>
>> I would vote for second option, not a release blocker and disable the
>> test in the release branch. My reasoning is:
>> - ReferenceRunner is not yet the official alternative to existing direct
>> runners.
>> - It is bad to have flaky tests on the release branch, and we would not
>> get good signal during validation.
>>
>> On Wed, Jun 13, 2018 at 3:14 PM, Pablo Estrada <pa...@google.com>
>> wrote:
>>
>>> Hello all,
>>> cherrypicks for the release branch seem to be going well, but thanks to
>>> them we were able to surface a flaky test in the release branch. JIRA is
>>> filed: https://issues.apache.org/jira/projects/BEAM/issues/BEAM-4558
>>>
>>> Given that test issue, I see the following options:
>>> - Consider that this test is not a release blocker. Go ahead with RC2
>>> after cherrypicks are brought in, or
>>> - Consider that this test is not a release blocker, so we disable it
>>> before cutting RC2.
>>> - Consider this test a release blocker, and triage the bug for fixing.
>>>
>>> What do you think?
>>>
>>> Best
>>> -P.
>>>
>>> On Wed, Jun 13, 2018 at 9:54 AM Pablo Estrada <pa...@google.com>
>>> wrote:
>>>
>>>> Precommits for PR https://github.com/apache/beam/pull/5609 are now
>>>> passing. For now I've simply set failOnWarning to false to cherrypick into
>>>> the release, and fix in master later on.
>>>> Best
>>>> -P.
>>>>
>>>> On Wed, Jun 13, 2018 at 9:08 AM Scott Wegner <sw...@google.com>
>>>> wrote:
>>>>
>>>>> From my understanding, the @SuppressFBWarnings usage is in a
>>>>> dependency (ByteBuddy) rather than directly in our code; so we're not able
>>>>> to modify the usage.
>>>>>
>>>>> Pablo, feel free to disable failOnWarning for the sdks-java-core
>>>>> project temporarily. This isn't a major regression since we've only
>>>>> recently made the change to enable it [1]. We can work separately on
>>>>> figuring out how to resolve the warnings.
>>>>>
>>>>> [1] https://github.com/apache/beam/pull/5319
>>>>>
>>>>> On Tue, Jun 12, 2018 at 11:57 PM Tim Robertson <
>>>>> timrobertson100@gmail.com> wrote:
>>>>>
>>>>>> Hi Pablo,
>>>>>>
>>>>>> I'm afraid I couldn't find one either... there is an issue about it
>>>>>> [1] which is old so it doesn't look likely to be resolved either.
>>>>>>
>>>>>> If you have time (sorry I am a bit busy) could you please verify the
>>>>>> version does work if you install that version locally? I know the maven
>>>>>> version of that [2] but not sure on the gradle equivalent. If we know it
>>>>>> works, we can then find a repository that fits ok with Apache/Beam policy.
>>>>>>
>>>>>> Alternatively, we could consider using a fully qualified reference
>>>>>> (i.e. @edu.umd.cs.findbugs.annotations.SuppressWarnings) to the deprecated
>>>>>> version and leave the dependency at the 1.3.9-1. I believe our general
>>>>>> direction is to remove findbugs when errorprone covers all aspects so I
>>>>>> *expect* this should be considered reasonable.
>>>>>>
>>>>>> I hope this helps,
>>>>>> Tim
>>>>>>
>>>>>> [1] https://github.com/stephenc/findbugs-annotations/issues/4
>>>>>> [2]
>>>>>> https://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html
>>>>>>
>>>>>> On Wed, Jun 13, 2018 at 8:39 AM, Pablo Estrada <pa...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Tim,
>>>>>>> you're right. Thanks for pointing that out. There's just one problem
>>>>>>> that I'm running into now: The 3.0.1-1 version does not seem to be
>>>>>>> available in Maven Central[1]. Looking at the website, I am not quite sure
>>>>>>> if there's another repository where they do stage the newer versions?[2]
>>>>>>>
>>>>>>> -P
>>>>>>>
>>>>>>> [1]
>>>>>>> https://repo.maven.apache.org/maven2/com/github/stephenc/findbugs/findbugs-annotations
>>>>>>> /
>>>>>>> [2] http://stephenc.github.io/findbugs-annotations/
>>>>>>>
>>>>>>> On Tue, Jun 12, 2018 at 11:10 PM Tim Robertson <
>>>>>>> timrobertson100@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Pablo,
>>>>>>>>
>>>>>>>> I took only a quick look.
>>>>>>>>
>>>>>>>> "- The JAR from the non-LGPL findbugs does not contain the
>>>>>>>> SuppressFBWarnings annotation"
>>>>>>>>
>>>>>>>> Unless I misunderstand you it looks like SuppressFBWarnings was
>>>>>>>> added in Stephen's version in this commit [1] which was introduced
>>>>>>>> in version 2.0.3-1 - I've checked is in the 3.0.1-1 build [2]
>>>>>>>> I notice in your commits [1] you've been exploring version 3.0.0
>>>>>>>> already though... what happens when you use 3.0.1-1? It sounds like the
>>>>>>>> wrong version is coming in rather than the annotation being missing.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Tim
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> https://github.com/stephenc/findbugs-annotations/commits/master/src/main/java/edu/umd/cs/findbugs/annotations/SuppressWarnings.java
>>>>>>>> [2] https://github.com/stephenc/findbugs-annotations/releases
>>>>>>>> [3]
>>>>>>>> https://github.com/apache/beam/pull/5609/commits/32c7df706e970557f154ff6bc521b2e00f9d09ab
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 13, 2018 at 2:37 AM, Pablo Estrada <pa...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>> I'll humbly declare that after wrestling with he build to stop
>>>>>>>>> depending on the wrong findbugs_annotations, I feel somewhat lost. The
>>>>>>>>> issue is actually quite small:
>>>>>>>>>
>>>>>>>>> - The JAR from the non-LGPL findbugs does not contain the
>>>>>>>>> SuppressFBWarnings annotation. This means that when building, ByteBuddy
>>>>>>>>> produces a few warnings (nothing critical).
>>>>>>>>> - The easiest way to avoid this failure is to call
>>>>>>>>> applyJavaNature(failOnWarning: false), but this would be bad, since we want
>>>>>>>>> to keep a high standard for tasks like ErrorProne and FindBugs itself.
>>>>>>>>> - So I find myself lost: How do we suppress trivial warnings
>>>>>>>>> coming from missing annotations, and honor warnings coming from other
>>>>>>>>> plugins?
>>>>>>>>>
>>>>>>>>> Any help / a PR from someone more capable would be appreciated.
>>>>>>>>> Best
>>>>>>>>> -P.
>>>>>>>>>
>>>>>>>>> On Tue, Jun 12, 2018 at 3:02 PM Ismaël Mejía <ie...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Yes, ok I was not aware it was already being addressed, nice.
>>>>>>>>>> On Tue, Jun 12, 2018 at 11:56 PM Ahmet Altay <al...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>> >
>>>>>>>>>> > Ismaël,
>>>>>>>>>> >
>>>>>>>>>> > I believe Pablo's https://github.com/apache/beam/pull/5609 is
>>>>>>>>>> fixing the issue by changing the findbugs back to
>>>>>>>>>> "com.github.stephenc.findbugs". Is this what you are referring to?
>>>>>>>>>> >
>>>>>>>>>> > Ahmet
>>>>>>>>>> >
>>>>>>>>>> > On Tue, Jun 12, 2018 at 2:51 PM, Boyuan Zhang <
>>>>>>>>>> boyuanz@google.com> wrote:
>>>>>>>>>> >>
>>>>>>>>>> >> Hey JB,
>>>>>>>>>> >>
>>>>>>>>>> >> I added some instructions about how to create python wheels in
>>>>>>>>>> this PR: https://github.com/apache/beam-site/pull/467 . Hope it
>>>>>>>>>> would be helpful.
>>>>>>>>>> >>
>>>>>>>>>> >> Boyuan
>>>>>>>>>> >>
>>>>>>>>>> >
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Got feedback? go/pabloem-feedback
>>>>>>>>> <https://goto.google.com/pabloem-feedback>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>> Got feedback? go/pabloem-feedback
>>>>>>> <https://goto.google.com/pabloem-feedback>
>>>>>>>
>>>>>>
>>>>>> --
>>>> Got feedback? go/pabloem-feedback
>>>> <https://goto.google.com/pabloem-feedback>
>>>>
>>> --
>>> Got feedback? go/pabloem-feedback
>>> <https://goto.google.com/pabloem-feedback>
>>>
>>
>> --
> Got feedback? go/pabloem-feedback
> <https://goto.google.com/pabloem-feedback>
>
Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Posted by Pablo Estrada <pa...@google.com>.
Sent out https://github.com/apache/beam/pull/5640 to ignore the flaky test.
As JB is the release manager, I'l let him make the call on what to do about
it.
Best
-P.
On Wed, Jun 13, 2018 at 3:34 PM Ahmet Altay <al...@google.com> wrote:
> I would vote for second option, not a release blocker and disable the test
> in the release branch. My reasoning is:
> - ReferenceRunner is not yet the official alternative to existing direct
> runners.
> - It is bad to have flaky tests on the release branch, and we would not
> get good signal during validation.
>
> On Wed, Jun 13, 2018 at 3:14 PM, Pablo Estrada <pa...@google.com> wrote:
>
>> Hello all,
>> cherrypicks for the release branch seem to be going well, but thanks to
>> them we were able to surface a flaky test in the release branch. JIRA is
>> filed: https://issues.apache.org/jira/projects/BEAM/issues/BEAM-4558
>>
>> Given that test issue, I see the following options:
>> - Consider that this test is not a release blocker. Go ahead with RC2
>> after cherrypicks are brought in, or
>> - Consider that this test is not a release blocker, so we disable it
>> before cutting RC2.
>> - Consider this test a release blocker, and triage the bug for fixing.
>>
>> What do you think?
>>
>> Best
>> -P.
>>
>> On Wed, Jun 13, 2018 at 9:54 AM Pablo Estrada <pa...@google.com> wrote:
>>
>>> Precommits for PR https://github.com/apache/beam/pull/5609 are now
>>> passing. For now I've simply set failOnWarning to false to cherrypick into
>>> the release, and fix in master later on.
>>> Best
>>> -P.
>>>
>>> On Wed, Jun 13, 2018 at 9:08 AM Scott Wegner <sw...@google.com> wrote:
>>>
>>>> From my understanding, the @SuppressFBWarnings usage is in a dependency
>>>> (ByteBuddy) rather than directly in our code; so we're not able to modify
>>>> the usage.
>>>>
>>>> Pablo, feel free to disable failOnWarning for the sdks-java-core
>>>> project temporarily. This isn't a major regression since we've only
>>>> recently made the change to enable it [1]. We can work separately on
>>>> figuring out how to resolve the warnings.
>>>>
>>>> [1] https://github.com/apache/beam/pull/5319
>>>>
>>>> On Tue, Jun 12, 2018 at 11:57 PM Tim Robertson <
>>>> timrobertson100@gmail.com> wrote:
>>>>
>>>>> Hi Pablo,
>>>>>
>>>>> I'm afraid I couldn't find one either... there is an issue about it
>>>>> [1] which is old so it doesn't look likely to be resolved either.
>>>>>
>>>>> If you have time (sorry I am a bit busy) could you please verify the
>>>>> version does work if you install that version locally? I know the maven
>>>>> version of that [2] but not sure on the gradle equivalent. If we know it
>>>>> works, we can then find a repository that fits ok with Apache/Beam policy.
>>>>>
>>>>> Alternatively, we could consider using a fully qualified reference
>>>>> (i.e. @edu.umd.cs.findbugs.annotations.SuppressWarnings) to the deprecated
>>>>> version and leave the dependency at the 1.3.9-1. I believe our general
>>>>> direction is to remove findbugs when errorprone covers all aspects so I
>>>>> *expect* this should be considered reasonable.
>>>>>
>>>>> I hope this helps,
>>>>> Tim
>>>>>
>>>>> [1] https://github.com/stephenc/findbugs-annotations/issues/4
>>>>> [2]
>>>>> https://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html
>>>>>
>>>>> On Wed, Jun 13, 2018 at 8:39 AM, Pablo Estrada <pa...@google.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Tim,
>>>>>> you're right. Thanks for pointing that out. There's just one problem
>>>>>> that I'm running into now: The 3.0.1-1 version does not seem to be
>>>>>> available in Maven Central[1]. Looking at the website, I am not quite sure
>>>>>> if there's another repository where they do stage the newer versions?[2]
>>>>>>
>>>>>> -P
>>>>>>
>>>>>> [1]
>>>>>> https://repo.maven.apache.org/maven2/com/github/stephenc/findbugs/findbugs-annotations
>>>>>> /
>>>>>> [2] http://stephenc.github.io/findbugs-annotations/
>>>>>>
>>>>>> On Tue, Jun 12, 2018 at 11:10 PM Tim Robertson <
>>>>>> timrobertson100@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Pablo,
>>>>>>>
>>>>>>> I took only a quick look.
>>>>>>>
>>>>>>> "- The JAR from the non-LGPL findbugs does not contain the
>>>>>>> SuppressFBWarnings annotation"
>>>>>>>
>>>>>>> Unless I misunderstand you it looks like SuppressFBWarnings was
>>>>>>> added in Stephen's version in this commit [1] which was introduced
>>>>>>> in version 2.0.3-1 - I've checked is in the 3.0.1-1 build [2]
>>>>>>> I notice in your commits [1] you've been exploring version 3.0.0
>>>>>>> already though... what happens when you use 3.0.1-1? It sounds like the
>>>>>>> wrong version is coming in rather than the annotation being missing.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Tim
>>>>>>>
>>>>>>> [1]
>>>>>>> https://github.com/stephenc/findbugs-annotations/commits/master/src/main/java/edu/umd/cs/findbugs/annotations/SuppressWarnings.java
>>>>>>> [2] https://github.com/stephenc/findbugs-annotations/releases
>>>>>>> [3]
>>>>>>> https://github.com/apache/beam/pull/5609/commits/32c7df706e970557f154ff6bc521b2e00f9d09ab
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 13, 2018 at 2:37 AM, Pablo Estrada <pa...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>> I'll humbly declare that after wrestling with he build to stop
>>>>>>>> depending on the wrong findbugs_annotations, I feel somewhat lost. The
>>>>>>>> issue is actually quite small:
>>>>>>>>
>>>>>>>> - The JAR from the non-LGPL findbugs does not contain the
>>>>>>>> SuppressFBWarnings annotation. This means that when building, ByteBuddy
>>>>>>>> produces a few warnings (nothing critical).
>>>>>>>> - The easiest way to avoid this failure is to call
>>>>>>>> applyJavaNature(failOnWarning: false), but this would be bad, since we want
>>>>>>>> to keep a high standard for tasks like ErrorProne and FindBugs itself.
>>>>>>>> - So I find myself lost: How do we suppress trivial warnings coming
>>>>>>>> from missing annotations, and honor warnings coming from other plugins?
>>>>>>>>
>>>>>>>> Any help / a PR from someone more capable would be appreciated.
>>>>>>>> Best
>>>>>>>> -P.
>>>>>>>>
>>>>>>>> On Tue, Jun 12, 2018 at 3:02 PM Ismaël Mejía <ie...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Yes, ok I was not aware it was already being addressed, nice.
>>>>>>>>> On Tue, Jun 12, 2018 at 11:56 PM Ahmet Altay <al...@google.com>
>>>>>>>>> wrote:
>>>>>>>>> >
>>>>>>>>> > Ismaël,
>>>>>>>>> >
>>>>>>>>> > I believe Pablo's https://github.com/apache/beam/pull/5609 is
>>>>>>>>> fixing the issue by changing the findbugs back to
>>>>>>>>> "com.github.stephenc.findbugs". Is this what you are referring to?
>>>>>>>>> >
>>>>>>>>> > Ahmet
>>>>>>>>> >
>>>>>>>>> > On Tue, Jun 12, 2018 at 2:51 PM, Boyuan Zhang <
>>>>>>>>> boyuanz@google.com> wrote:
>>>>>>>>> >>
>>>>>>>>> >> Hey JB,
>>>>>>>>> >>
>>>>>>>>> >> I added some instructions about how to create python wheels in
>>>>>>>>> this PR: https://github.com/apache/beam-site/pull/467 . Hope it
>>>>>>>>> would be helpful.
>>>>>>>>> >>
>>>>>>>>> >> Boyuan
>>>>>>>>> >>
>>>>>>>>> >
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Got feedback? go/pabloem-feedback
>>>>>>>> <https://goto.google.com/pabloem-feedback>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>> Got feedback? go/pabloem-feedback
>>>>>> <https://goto.google.com/pabloem-feedback>
>>>>>>
>>>>>
>>>>> --
>>> Got feedback? go/pabloem-feedback
>>> <https://goto.google.com/pabloem-feedback>
>>>
>> --
>> Got feedback? go/pabloem-feedback
>> <https://goto.google.com/pabloem-feedback>
>>
>
> --
Got feedback? go/pabloem-feedback
Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Posted by Ahmet Altay <al...@google.com>.
I would vote for second option, not a release blocker and disable the test
in the release branch. My reasoning is:
- ReferenceRunner is not yet the official alternative to existing direct
runners.
- It is bad to have flaky tests on the release branch, and we would not get
good signal during validation.
On Wed, Jun 13, 2018 at 3:14 PM, Pablo Estrada <pa...@google.com> wrote:
> Hello all,
> cherrypicks for the release branch seem to be going well, but thanks to
> them we were able to surface a flaky test in the release branch. JIRA is
> filed: https://issues.apache.org/jira/projects/BEAM/issues/BEAM-4558
>
> Given that test issue, I see the following options:
> - Consider that this test is not a release blocker. Go ahead with RC2
> after cherrypicks are brought in, or
> - Consider that this test is not a release blocker, so we disable it
> before cutting RC2.
> - Consider this test a release blocker, and triage the bug for fixing.
>
> What do you think?
>
> Best
> -P.
>
> On Wed, Jun 13, 2018 at 9:54 AM Pablo Estrada <pa...@google.com> wrote:
>
>> Precommits for PR https://github.com/apache/beam/pull/5609 are now
>> passing. For now I've simply set failOnWarning to false to cherrypick into
>> the release, and fix in master later on.
>> Best
>> -P.
>>
>> On Wed, Jun 13, 2018 at 9:08 AM Scott Wegner <sw...@google.com> wrote:
>>
>>> From my understanding, the @SuppressFBWarnings usage is in a dependency
>>> (ByteBuddy) rather than directly in our code; so we're not able to modify
>>> the usage.
>>>
>>> Pablo, feel free to disable failOnWarning for the sdks-java-core project
>>> temporarily. This isn't a major regression since we've only recently made
>>> the change to enable it [1]. We can work separately on figuring out how to
>>> resolve the warnings.
>>>
>>> [1] https://github.com/apache/beam/pull/5319
>>>
>>> On Tue, Jun 12, 2018 at 11:57 PM Tim Robertson <
>>> timrobertson100@gmail.com> wrote:
>>>
>>>> Hi Pablo,
>>>>
>>>> I'm afraid I couldn't find one either... there is an issue about it [1]
>>>> which is old so it doesn't look likely to be resolved either.
>>>>
>>>> If you have time (sorry I am a bit busy) could you please verify the
>>>> version does work if you install that version locally? I know the maven
>>>> version of that [2] but not sure on the gradle equivalent. If we know it
>>>> works, we can then find a repository that fits ok with Apache/Beam policy.
>>>>
>>>> Alternatively, we could consider using a fully qualified reference
>>>> (i.e. @edu.umd.cs.findbugs.annotations.SuppressWarnings) to the
>>>> deprecated version and leave the dependency at the 1.3.9-1. I believe our
>>>> general direction is to remove findbugs when errorprone covers all aspects
>>>> so I *expect* this should be considered reasonable.
>>>>
>>>> I hope this helps,
>>>> Tim
>>>>
>>>> [1] https://github.com/stephenc/findbugs-annotations/issues/4
>>>> [2] https://maven.apache.org/guides/mini/guide-3rd-party-
>>>> jars-local.html
>>>>
>>>> On Wed, Jun 13, 2018 at 8:39 AM, Pablo Estrada <pa...@google.com>
>>>> wrote:
>>>>
>>>>> Hi Tim,
>>>>> you're right. Thanks for pointing that out. There's just one problem
>>>>> that I'm running into now: The 3.0.1-1 version does not seem to be
>>>>> available in Maven Central[1]. Looking at the website, I am not quite sure
>>>>> if there's another repository where they do stage the newer versions?[2]
>>>>>
>>>>> -P
>>>>>
>>>>> [1] https://repo.maven.apache.org/maven2/com/github/
>>>>> stephenc/findbugs/findbugs-annotations/
>>>>> [2] http://stephenc.github.io/findbugs-annotations/
>>>>>
>>>>> On Tue, Jun 12, 2018 at 11:10 PM Tim Robertson <
>>>>> timrobertson100@gmail.com> wrote:
>>>>>
>>>>>> Hi Pablo,
>>>>>>
>>>>>> I took only a quick look.
>>>>>>
>>>>>> "- The JAR from the non-LGPL findbugs does not contain the
>>>>>> SuppressFBWarnings annotation"
>>>>>>
>>>>>> Unless I misunderstand you it looks like SuppressFBWarnings was added
>>>>>> in Stephen's version in this commit [1] which was introduced in
>>>>>> version 2.0.3-1 - I've checked is in the 3.0.1-1 build [2]
>>>>>> I notice in your commits [1] you've been exploring version 3.0.0
>>>>>> already though... what happens when you use 3.0.1-1? It sounds like the
>>>>>> wrong version is coming in rather than the annotation being missing.
>>>>>>
>>>>>> Thanks,
>>>>>> Tim
>>>>>>
>>>>>> [1] https://github.com/stephenc/findbugs-annotations/
>>>>>> commits/master/src/main/java/edu/umd/cs/findbugs/
>>>>>> annotations/SuppressWarnings.java
>>>>>> [2] https://github.com/stephenc/findbugs-annotations/releases
>>>>>> [3] https://github.com/apache/beam/pull/5609/commits/
>>>>>> 32c7df706e970557f154ff6bc521b2e00f9d09ab
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 13, 2018 at 2:37 AM, Pablo Estrada <pa...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>> I'll humbly declare that after wrestling with he build to stop
>>>>>>> depending on the wrong findbugs_annotations, I feel somewhat lost. The
>>>>>>> issue is actually quite small:
>>>>>>>
>>>>>>> - The JAR from the non-LGPL findbugs does not contain the
>>>>>>> SuppressFBWarnings annotation. This means that when building, ByteBuddy
>>>>>>> produces a few warnings (nothing critical).
>>>>>>> - The easiest way to avoid this failure is to call
>>>>>>> applyJavaNature(failOnWarning: false), but this would be bad, since we want
>>>>>>> to keep a high standard for tasks like ErrorProne and FindBugs itself.
>>>>>>> - So I find myself lost: How do we suppress trivial warnings coming
>>>>>>> from missing annotations, and honor warnings coming from other plugins?
>>>>>>>
>>>>>>> Any help / a PR from someone more capable would be appreciated.
>>>>>>> Best
>>>>>>> -P.
>>>>>>>
>>>>>>> On Tue, Jun 12, 2018 at 3:02 PM Ismaël Mejía <ie...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Yes, ok I was not aware it was already being addressed, nice.
>>>>>>>> On Tue, Jun 12, 2018 at 11:56 PM Ahmet Altay <al...@google.com>
>>>>>>>> wrote:
>>>>>>>> >
>>>>>>>> > Ismaël,
>>>>>>>> >
>>>>>>>> > I believe Pablo's https://github.com/apache/beam/pull/5609 is
>>>>>>>> fixing the issue by changing the findbugs back to
>>>>>>>> "com.github.stephenc.findbugs". Is this what you are referring to?
>>>>>>>> >
>>>>>>>> > Ahmet
>>>>>>>> >
>>>>>>>> > On Tue, Jun 12, 2018 at 2:51 PM, Boyuan Zhang <bo...@google.com>
>>>>>>>> wrote:
>>>>>>>> >>
>>>>>>>> >> Hey JB,
>>>>>>>> >>
>>>>>>>> >> I added some instructions about how to create python wheels in
>>>>>>>> this PR: https://github.com/apache/beam-site/pull/467 . Hope it
>>>>>>>> would be helpful.
>>>>>>>> >>
>>>>>>>> >> Boyuan
>>>>>>>> >>
>>>>>>>> >
>>>>>>>>
>>>>>>> --
>>>>>>> Got feedback? go/pabloem-feedback
>>>>>>> <https://goto.google.com/pabloem-feedback>
>>>>>>>
>>>>>>
>>>>>> --
>>>>> Got feedback? go/pabloem-feedback
>>>>> <https://goto.google.com/pabloem-feedback>
>>>>>
>>>>
>>>> --
>> Got feedback? go/pabloem-feedback
>> <https://goto.google.com/pabloem-feedback>
>>
> --
> Got feedback? go/pabloem-feedback
> <https://goto.google.com/pabloem-feedback>
>
Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Posted by Pablo Estrada <pa...@google.com>.
Hello all,
cherrypicks for the release branch seem to be going well, but thanks to
them we were able to surface a flaky test in the release branch. JIRA is
filed: https://issues.apache.org/jira/projects/BEAM/issues/BEAM-4558
Given that test issue, I see the following options:
- Consider that this test is not a release blocker. Go ahead with RC2 after
cherrypicks are brought in, or
- Consider that this test is not a release blocker, so we disable it before
cutting RC2.
- Consider this test a release blocker, and triage the bug for fixing.
What do you think?
Best
-P.
On Wed, Jun 13, 2018 at 9:54 AM Pablo Estrada <pa...@google.com> wrote:
> Precommits for PR https://github.com/apache/beam/pull/5609 are now
> passing. For now I've simply set failOnWarning to false to cherrypick into
> the release, and fix in master later on.
> Best
> -P.
>
> On Wed, Jun 13, 2018 at 9:08 AM Scott Wegner <sw...@google.com> wrote:
>
>> From my understanding, the @SuppressFBWarnings usage is in a dependency
>> (ByteBuddy) rather than directly in our code; so we're not able to modify
>> the usage.
>>
>> Pablo, feel free to disable failOnWarning for the sdks-java-core project
>> temporarily. This isn't a major regression since we've only recently made
>> the change to enable it [1]. We can work separately on figuring out how to
>> resolve the warnings.
>>
>> [1] https://github.com/apache/beam/pull/5319
>>
>> On Tue, Jun 12, 2018 at 11:57 PM Tim Robertson <ti...@gmail.com>
>> wrote:
>>
>>> Hi Pablo,
>>>
>>> I'm afraid I couldn't find one either... there is an issue about it [1]
>>> which is old so it doesn't look likely to be resolved either.
>>>
>>> If you have time (sorry I am a bit busy) could you please verify the
>>> version does work if you install that version locally? I know the maven
>>> version of that [2] but not sure on the gradle equivalent. If we know it
>>> works, we can then find a repository that fits ok with Apache/Beam policy.
>>>
>>> Alternatively, we could consider using a fully qualified reference (i.e.
>>> @edu.umd.cs.findbugs.annotations.SuppressWarnings) to the deprecated
>>> version and leave the dependency at the 1.3.9-1. I believe our general
>>> direction is to remove findbugs when errorprone covers all aspects so I
>>> *expect* this should be considered reasonable.
>>>
>>> I hope this helps,
>>> Tim
>>>
>>> [1] https://github.com/stephenc/findbugs-annotations/issues/4
>>> [2] https://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html
>>>
>>> On Wed, Jun 13, 2018 at 8:39 AM, Pablo Estrada <pa...@google.com>
>>> wrote:
>>>
>>>> Hi Tim,
>>>> you're right. Thanks for pointing that out. There's just one problem
>>>> that I'm running into now: The 3.0.1-1 version does not seem to be
>>>> available in Maven Central[1]. Looking at the website, I am not quite sure
>>>> if there's another repository where they do stage the newer versions?[2]
>>>>
>>>> -P
>>>>
>>>> [1]
>>>> https://repo.maven.apache.org/maven2/com/github/stephenc/findbugs/findbugs-annotations
>>>> /
>>>> [2] http://stephenc.github.io/findbugs-annotations/
>>>>
>>>> On Tue, Jun 12, 2018 at 11:10 PM Tim Robertson <
>>>> timrobertson100@gmail.com> wrote:
>>>>
>>>>> Hi Pablo,
>>>>>
>>>>> I took only a quick look.
>>>>>
>>>>> "- The JAR from the non-LGPL findbugs does not contain the
>>>>> SuppressFBWarnings annotation"
>>>>>
>>>>> Unless I misunderstand you it looks like SuppressFBWarnings was added
>>>>> in Stephen's version in this commit [1] which was introduced in
>>>>> version 2.0.3-1 - I've checked is in the 3.0.1-1 build [2]
>>>>> I notice in your commits [1] you've been exploring version 3.0.0
>>>>> already though... what happens when you use 3.0.1-1? It sounds like the
>>>>> wrong version is coming in rather than the annotation being missing.
>>>>>
>>>>> Thanks,
>>>>> Tim
>>>>>
>>>>> [1]
>>>>> https://github.com/stephenc/findbugs-annotations/commits/master/src/main/java/edu/umd/cs/findbugs/annotations/SuppressWarnings.java
>>>>> [2] https://github.com/stephenc/findbugs-annotations/releases
>>>>> [3]
>>>>> https://github.com/apache/beam/pull/5609/commits/32c7df706e970557f154ff6bc521b2e00f9d09ab
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 13, 2018 at 2:37 AM, Pablo Estrada <pa...@google.com>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>> I'll humbly declare that after wrestling with he build to stop
>>>>>> depending on the wrong findbugs_annotations, I feel somewhat lost. The
>>>>>> issue is actually quite small:
>>>>>>
>>>>>> - The JAR from the non-LGPL findbugs does not contain the
>>>>>> SuppressFBWarnings annotation. This means that when building, ByteBuddy
>>>>>> produces a few warnings (nothing critical).
>>>>>> - The easiest way to avoid this failure is to call
>>>>>> applyJavaNature(failOnWarning: false), but this would be bad, since we want
>>>>>> to keep a high standard for tasks like ErrorProne and FindBugs itself.
>>>>>> - So I find myself lost: How do we suppress trivial warnings coming
>>>>>> from missing annotations, and honor warnings coming from other plugins?
>>>>>>
>>>>>> Any help / a PR from someone more capable would be appreciated.
>>>>>> Best
>>>>>> -P.
>>>>>>
>>>>>> On Tue, Jun 12, 2018 at 3:02 PM Ismaël Mejía <ie...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Yes, ok I was not aware it was already being addressed, nice.
>>>>>>> On Tue, Jun 12, 2018 at 11:56 PM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > Ismaël,
>>>>>>> >
>>>>>>> > I believe Pablo's https://github.com/apache/beam/pull/5609 is
>>>>>>> fixing the issue by changing the findbugs back to
>>>>>>> "com.github.stephenc.findbugs". Is this what you are referring to?
>>>>>>> >
>>>>>>> > Ahmet
>>>>>>> >
>>>>>>> > On Tue, Jun 12, 2018 at 2:51 PM, Boyuan Zhang <bo...@google.com>
>>>>>>> wrote:
>>>>>>> >>
>>>>>>> >> Hey JB,
>>>>>>> >>
>>>>>>> >> I added some instructions about how to create python wheels in
>>>>>>> this PR: https://github.com/apache/beam-site/pull/467 . Hope it
>>>>>>> would be helpful.
>>>>>>> >>
>>>>>>> >> Boyuan
>>>>>>> >>
>>>>>>> >
>>>>>>>
>>>>>> --
>>>>>> Got feedback? go/pabloem-feedback
>>>>>> <https://goto.google.com/pabloem-feedback>
>>>>>>
>>>>>
>>>>> --
>>>> Got feedback? go/pabloem-feedback
>>>> <https://goto.google.com/pabloem-feedback>
>>>>
>>>
>>> --
> Got feedback? go/pabloem-feedback
> <https://goto.google.com/pabloem-feedback>
>
--
Got feedback? go/pabloem-feedback
Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Posted by Boyuan Zhang <bo...@google.com>.
Hey all,
Currently we have 3 PRs supposed to be cherrypicked into RC2:
Pablo: <goog_1529729135>https://github.com/apache/beam/pull/5609 (merged)
Udi: https://github.com/apache/beam/pull/5607 (open)
Charles: <goog_1529729128>https://github.com/apache/beam/pull/5636 (open)
Boyuan
On Wed, Jun 13, 2018 at 9:54 AM Pablo Estrada <pa...@google.com> wrote:
> Precommits for PR https://github.com/apache/beam/pull/5609 are now
> passing. For now I've simply set failOnWarning to false to cherrypick into
> the release, and fix in master later on.
> Best
> -P.
>
> On Wed, Jun 13, 2018 at 9:08 AM Scott Wegner <sw...@google.com> wrote:
>
>> From my understanding, the @SuppressFBWarnings usage is in a dependency
>> (ByteBuddy) rather than directly in our code; so we're not able to modify
>> the usage.
>>
>> Pablo, feel free to disable failOnWarning for the sdks-java-core project
>> temporarily. This isn't a major regression since we've only recently made
>> the change to enable it [1]. We can work separately on figuring out how to
>> resolve the warnings.
>>
>> [1] https://github.com/apache/beam/pull/5319
>>
>> On Tue, Jun 12, 2018 at 11:57 PM Tim Robertson <ti...@gmail.com>
>> wrote:
>>
>>> Hi Pablo,
>>>
>>> I'm afraid I couldn't find one either... there is an issue about it [1]
>>> which is old so it doesn't look likely to be resolved either.
>>>
>>> If you have time (sorry I am a bit busy) could you please verify the
>>> version does work if you install that version locally? I know the maven
>>> version of that [2] but not sure on the gradle equivalent. If we know it
>>> works, we can then find a repository that fits ok with Apache/Beam policy.
>>>
>>> Alternatively, we could consider using a fully qualified reference (i.e.
>>> @edu.umd.cs.findbugs.annotations.SuppressWarnings) to the deprecated
>>> version and leave the dependency at the 1.3.9-1. I believe our general
>>> direction is to remove findbugs when errorprone covers all aspects so I
>>> *expect* this should be considered reasonable.
>>>
>>> I hope this helps,
>>> Tim
>>>
>>> [1] https://github.com/stephenc/findbugs-annotations/issues/4
>>> [2] https://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html
>>>
>>> On Wed, Jun 13, 2018 at 8:39 AM, Pablo Estrada <pa...@google.com>
>>> wrote:
>>>
>>>> Hi Tim,
>>>> you're right. Thanks for pointing that out. There's just one problem
>>>> that I'm running into now: The 3.0.1-1 version does not seem to be
>>>> available in Maven Central[1]. Looking at the website, I am not quite sure
>>>> if there's another repository where they do stage the newer versions?[2]
>>>>
>>>> -P
>>>>
>>>> [1]
>>>> https://repo.maven.apache.org/maven2/com/github/stephenc/findbugs/findbugs-annotations
>>>> /
>>>> [2] http://stephenc.github.io/findbugs-annotations/
>>>>
>>>> On Tue, Jun 12, 2018 at 11:10 PM Tim Robertson <
>>>> timrobertson100@gmail.com> wrote:
>>>>
>>>>> Hi Pablo,
>>>>>
>>>>> I took only a quick look.
>>>>>
>>>>> "- The JAR from the non-LGPL findbugs does not contain the
>>>>> SuppressFBWarnings annotation"
>>>>>
>>>>> Unless I misunderstand you it looks like SuppressFBWarnings was added
>>>>> in Stephen's version in this commit [1] which was introduced in
>>>>> version 2.0.3-1 - I've checked is in the 3.0.1-1 build [2]
>>>>> I notice in your commits [1] you've been exploring version 3.0.0
>>>>> already though... what happens when you use 3.0.1-1? It sounds like the
>>>>> wrong version is coming in rather than the annotation being missing.
>>>>>
>>>>> Thanks,
>>>>> Tim
>>>>>
>>>>> [1]
>>>>> https://github.com/stephenc/findbugs-annotations/commits/master/src/main/java/edu/umd/cs/findbugs/annotations/SuppressWarnings.java
>>>>> [2] https://github.com/stephenc/findbugs-annotations/releases
>>>>> [3]
>>>>> https://github.com/apache/beam/pull/5609/commits/32c7df706e970557f154ff6bc521b2e00f9d09ab
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 13, 2018 at 2:37 AM, Pablo Estrada <pa...@google.com>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>> I'll humbly declare that after wrestling with he build to stop
>>>>>> depending on the wrong findbugs_annotations, I feel somewhat lost. The
>>>>>> issue is actually quite small:
>>>>>>
>>>>>> - The JAR from the non-LGPL findbugs does not contain the
>>>>>> SuppressFBWarnings annotation. This means that when building, ByteBuddy
>>>>>> produces a few warnings (nothing critical).
>>>>>> - The easiest way to avoid this failure is to call
>>>>>> applyJavaNature(failOnWarning: false), but this would be bad, since we want
>>>>>> to keep a high standard for tasks like ErrorProne and FindBugs itself.
>>>>>> - So I find myself lost: How do we suppress trivial warnings coming
>>>>>> from missing annotations, and honor warnings coming from other plugins?
>>>>>>
>>>>>> Any help / a PR from someone more capable would be appreciated.
>>>>>> Best
>>>>>> -P.
>>>>>>
>>>>>> On Tue, Jun 12, 2018 at 3:02 PM Ismaël Mejía <ie...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Yes, ok I was not aware it was already being addressed, nice.
>>>>>>> On Tue, Jun 12, 2018 at 11:56 PM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> > Ismaël,
>>>>>>> >
>>>>>>> > I believe Pablo's https://github.com/apache/beam/pull/5609 is
>>>>>>> fixing the issue by changing the findbugs back to
>>>>>>> "com.github.stephenc.findbugs". Is this what you are referring to?
>>>>>>> >
>>>>>>> > Ahmet
>>>>>>> >
>>>>>>> > On Tue, Jun 12, 2018 at 2:51 PM, Boyuan Zhang <bo...@google.com>
>>>>>>> wrote:
>>>>>>> >>
>>>>>>> >> Hey JB,
>>>>>>> >>
>>>>>>> >> I added some instructions about how to create python wheels in
>>>>>>> this PR: https://github.com/apache/beam-site/pull/467 . Hope it
>>>>>>> would be helpful.
>>>>>>> >>
>>>>>>> >> Boyuan
>>>>>>> >>
>>>>>>> >
>>>>>>>
>>>>>> --
>>>>>> Got feedback? go/pabloem-feedback
>>>>>> <https://goto.google.com/pabloem-feedback>
>>>>>>
>>>>>
>>>>> --
>>>> Got feedback? go/pabloem-feedback
>>>> <https://goto.google.com/pabloem-feedback>
>>>>
>>>
>>> --
> Got feedback? go/pabloem-feedback
> <https://goto.google.com/pabloem-feedback>
>
Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Posted by Pablo Estrada <pa...@google.com>.
Precommits for PR https://github.com/apache/beam/pull/5609 are now passing.
For now I've simply set failOnWarning to false to cherrypick into the
release, and fix in master later on.
Best
-P.
On Wed, Jun 13, 2018 at 9:08 AM Scott Wegner <sw...@google.com> wrote:
> From my understanding, the @SuppressFBWarnings usage is in a dependency
> (ByteBuddy) rather than directly in our code; so we're not able to modify
> the usage.
>
> Pablo, feel free to disable failOnWarning for the sdks-java-core project
> temporarily. This isn't a major regression since we've only recently made
> the change to enable it [1]. We can work separately on figuring out how to
> resolve the warnings.
>
> [1] https://github.com/apache/beam/pull/5319
>
> On Tue, Jun 12, 2018 at 11:57 PM Tim Robertson <ti...@gmail.com>
> wrote:
>
>> Hi Pablo,
>>
>> I'm afraid I couldn't find one either... there is an issue about it [1]
>> which is old so it doesn't look likely to be resolved either.
>>
>> If you have time (sorry I am a bit busy) could you please verify the
>> version does work if you install that version locally? I know the maven
>> version of that [2] but not sure on the gradle equivalent. If we know it
>> works, we can then find a repository that fits ok with Apache/Beam policy.
>>
>> Alternatively, we could consider using a fully qualified reference (i.e.
>> @edu.umd.cs.findbugs.annotations.SuppressWarnings) to the deprecated
>> version and leave the dependency at the 1.3.9-1. I believe our general
>> direction is to remove findbugs when errorprone covers all aspects so I
>> *expect* this should be considered reasonable.
>>
>> I hope this helps,
>> Tim
>>
>> [1] https://github.com/stephenc/findbugs-annotations/issues/4
>> [2] https://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html
>>
>> On Wed, Jun 13, 2018 at 8:39 AM, Pablo Estrada <pa...@google.com>
>> wrote:
>>
>>> Hi Tim,
>>> you're right. Thanks for pointing that out. There's just one problem
>>> that I'm running into now: The 3.0.1-1 version does not seem to be
>>> available in Maven Central[1]. Looking at the website, I am not quite sure
>>> if there's another repository where they do stage the newer versions?[2]
>>>
>>> -P
>>>
>>> [1]
>>> https://repo.maven.apache.org/maven2/com/github/stephenc/findbugs/findbugs-annotations
>>> /
>>> [2] http://stephenc.github.io/findbugs-annotations/
>>>
>>> On Tue, Jun 12, 2018 at 11:10 PM Tim Robertson <
>>> timrobertson100@gmail.com> wrote:
>>>
>>>> Hi Pablo,
>>>>
>>>> I took only a quick look.
>>>>
>>>> "- The JAR from the non-LGPL findbugs does not contain the
>>>> SuppressFBWarnings annotation"
>>>>
>>>> Unless I misunderstand you it looks like SuppressFBWarnings was added
>>>> in Stephen's version in this commit [1] which was introduced in
>>>> version 2.0.3-1 - I've checked is in the 3.0.1-1 build [2]
>>>> I notice in your commits [1] you've been exploring version 3.0.0
>>>> already though... what happens when you use 3.0.1-1? It sounds like the
>>>> wrong version is coming in rather than the annotation being missing.
>>>>
>>>> Thanks,
>>>> Tim
>>>>
>>>> [1]
>>>> https://github.com/stephenc/findbugs-annotations/commits/master/src/main/java/edu/umd/cs/findbugs/annotations/SuppressWarnings.java
>>>> [2] https://github.com/stephenc/findbugs-annotations/releases
>>>> [3]
>>>> https://github.com/apache/beam/pull/5609/commits/32c7df706e970557f154ff6bc521b2e00f9d09ab
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Jun 13, 2018 at 2:37 AM, Pablo Estrada <pa...@google.com>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>> I'll humbly declare that after wrestling with he build to stop
>>>>> depending on the wrong findbugs_annotations, I feel somewhat lost. The
>>>>> issue is actually quite small:
>>>>>
>>>>> - The JAR from the non-LGPL findbugs does not contain the
>>>>> SuppressFBWarnings annotation. This means that when building, ByteBuddy
>>>>> produces a few warnings (nothing critical).
>>>>> - The easiest way to avoid this failure is to call
>>>>> applyJavaNature(failOnWarning: false), but this would be bad, since we want
>>>>> to keep a high standard for tasks like ErrorProne and FindBugs itself.
>>>>> - So I find myself lost: How do we suppress trivial warnings coming
>>>>> from missing annotations, and honor warnings coming from other plugins?
>>>>>
>>>>> Any help / a PR from someone more capable would be appreciated.
>>>>> Best
>>>>> -P.
>>>>>
>>>>> On Tue, Jun 12, 2018 at 3:02 PM Ismaël Mejía <ie...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Yes, ok I was not aware it was already being addressed, nice.
>>>>>> On Tue, Jun 12, 2018 at 11:56 PM Ahmet Altay <al...@google.com>
>>>>>> wrote:
>>>>>> >
>>>>>> > Ismaël,
>>>>>> >
>>>>>> > I believe Pablo's https://github.com/apache/beam/pull/5609 is
>>>>>> fixing the issue by changing the findbugs back to
>>>>>> "com.github.stephenc.findbugs". Is this what you are referring to?
>>>>>> >
>>>>>> > Ahmet
>>>>>> >
>>>>>> > On Tue, Jun 12, 2018 at 2:51 PM, Boyuan Zhang <bo...@google.com>
>>>>>> wrote:
>>>>>> >>
>>>>>> >> Hey JB,
>>>>>> >>
>>>>>> >> I added some instructions about how to create python wheels in
>>>>>> this PR: https://github.com/apache/beam-site/pull/467 . Hope it
>>>>>> would be helpful.
>>>>>> >>
>>>>>> >> Boyuan
>>>>>> >>
>>>>>> >
>>>>>>
>>>>> --
>>>>> Got feedback? go/pabloem-feedback
>>>>> <https://goto.google.com/pabloem-feedback>
>>>>>
>>>>
>>>> --
>>> Got feedback? go/pabloem-feedback
>>> <https://goto.google.com/pabloem-feedback>
>>>
>>
>> --
Got feedback? go/pabloem-feedback
Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Posted by Scott Wegner <sw...@google.com>.
From my understanding, the @SuppressFBWarnings usage is in a dependency
(ByteBuddy) rather than directly in our code; so we're not able to modify
the usage.
Pablo, feel free to disable failOnWarning for the sdks-java-core project
temporarily. This isn't a major regression since we've only recently made
the change to enable it [1]. We can work separately on figuring out how to
resolve the warnings.
[1] https://github.com/apache/beam/pull/5319
On Tue, Jun 12, 2018 at 11:57 PM Tim Robertson <ti...@gmail.com>
wrote:
> Hi Pablo,
>
> I'm afraid I couldn't find one either... there is an issue about it [1]
> which is old so it doesn't look likely to be resolved either.
>
> If you have time (sorry I am a bit busy) could you please verify the
> version does work if you install that version locally? I know the maven
> version of that [2] but not sure on the gradle equivalent. If we know it
> works, we can then find a repository that fits ok with Apache/Beam policy.
>
> Alternatively, we could consider using a fully qualified reference (i.e.
> @edu.umd.cs.findbugs.annotations.SuppressWarnings) to the deprecated
> version and leave the dependency at the 1.3.9-1. I believe our general
> direction is to remove findbugs when errorprone covers all aspects so I
> *expect* this should be considered reasonable.
>
> I hope this helps,
> Tim
>
> [1] https://github.com/stephenc/findbugs-annotations/issues/4
> [2] https://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html
>
> On Wed, Jun 13, 2018 at 8:39 AM, Pablo Estrada <pa...@google.com> wrote:
>
>> Hi Tim,
>> you're right. Thanks for pointing that out. There's just one problem that
>> I'm running into now: The 3.0.1-1 version does not seem to be available in
>> Maven Central[1]. Looking at the website, I am not quite sure if there's
>> another repository where they do stage the newer versions?[2]
>>
>> -P
>>
>> [1]
>> https://repo.maven.apache.org/maven2/com/github/stephenc/findbugs/findbugs-annotations
>> /
>> [2] http://stephenc.github.io/findbugs-annotations/
>>
>> On Tue, Jun 12, 2018 at 11:10 PM Tim Robertson <ti...@gmail.com>
>> wrote:
>>
>>> Hi Pablo,
>>>
>>> I took only a quick look.
>>>
>>> "- The JAR from the non-LGPL findbugs does not contain the
>>> SuppressFBWarnings annotation"
>>>
>>> Unless I misunderstand you it looks like SuppressFBWarnings was added in
>>> Stephen's version in this commit [1] which was introduced in version
>>> 2.0.3-1 - I've checked is in the 3.0.1-1 build [2]
>>> I notice in your commits [1] you've been exploring version 3.0.0 already
>>> though... what happens when you use 3.0.1-1? It sounds like the wrong
>>> version is coming in rather than the annotation being missing.
>>>
>>> Thanks,
>>> Tim
>>>
>>> [1]
>>> https://github.com/stephenc/findbugs-annotations/commits/master/src/main/java/edu/umd/cs/findbugs/annotations/SuppressWarnings.java
>>> [2] https://github.com/stephenc/findbugs-annotations/releases
>>> [3]
>>> https://github.com/apache/beam/pull/5609/commits/32c7df706e970557f154ff6bc521b2e00f9d09ab
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Jun 13, 2018 at 2:37 AM, Pablo Estrada <pa...@google.com>
>>> wrote:
>>>
>>>> Hi all,
>>>> I'll humbly declare that after wrestling with he build to stop
>>>> depending on the wrong findbugs_annotations, I feel somewhat lost. The
>>>> issue is actually quite small:
>>>>
>>>> - The JAR from the non-LGPL findbugs does not contain the
>>>> SuppressFBWarnings annotation. This means that when building, ByteBuddy
>>>> produces a few warnings (nothing critical).
>>>> - The easiest way to avoid this failure is to call
>>>> applyJavaNature(failOnWarning: false), but this would be bad, since we want
>>>> to keep a high standard for tasks like ErrorProne and FindBugs itself.
>>>> - So I find myself lost: How do we suppress trivial warnings coming
>>>> from missing annotations, and honor warnings coming from other plugins?
>>>>
>>>> Any help / a PR from someone more capable would be appreciated.
>>>> Best
>>>> -P.
>>>>
>>>> On Tue, Jun 12, 2018 at 3:02 PM Ismaël Mejía <ie...@gmail.com> wrote:
>>>>
>>>>> Yes, ok I was not aware it was already being addressed, nice.
>>>>> On Tue, Jun 12, 2018 at 11:56 PM Ahmet Altay <al...@google.com> wrote:
>>>>> >
>>>>> > Ismaël,
>>>>> >
>>>>> > I believe Pablo's https://github.com/apache/beam/pull/5609 is
>>>>> fixing the issue by changing the findbugs back to
>>>>> "com.github.stephenc.findbugs". Is this what you are referring to?
>>>>> >
>>>>> > Ahmet
>>>>> >
>>>>> > On Tue, Jun 12, 2018 at 2:51 PM, Boyuan Zhang <bo...@google.com>
>>>>> wrote:
>>>>> >>
>>>>> >> Hey JB,
>>>>> >>
>>>>> >> I added some instructions about how to create python wheels in this
>>>>> PR: https://github.com/apache/beam-site/pull/467 . Hope it would be
>>>>> helpful.
>>>>> >>
>>>>> >> Boyuan
>>>>> >>
>>>>> >
>>>>>
>>>> --
>>>> Got feedback? go/pabloem-feedback
>>>> <https://goto.google.com/pabloem-feedback>
>>>>
>>>
>>> --
>> Got feedback? go/pabloem-feedback
>> <https://goto.google.com/pabloem-feedback>
>>
>
>
Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Posted by Tim Robertson <ti...@gmail.com>.
Hi Pablo,
I'm afraid I couldn't find one either... there is an issue about it [1]
which is old so it doesn't look likely to be resolved either.
If you have time (sorry I am a bit busy) could you please verify the
version does work if you install that version locally? I know the maven
version of that [2] but not sure on the gradle equivalent. If we know it
works, we can then find a repository that fits ok with Apache/Beam policy.
Alternatively, we could consider using a fully qualified reference (i.e.
@edu.umd.cs.findbugs.annotations.SuppressWarnings) to the deprecated
version and leave the dependency at the 1.3.9-1. I believe our general
direction is to remove findbugs when errorprone covers all aspects so I
*expect* this should be considered reasonable.
I hope this helps,
Tim
[1] https://github.com/stephenc/findbugs-annotations/issues/4
[2] https://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html
On Wed, Jun 13, 2018 at 8:39 AM, Pablo Estrada <pa...@google.com> wrote:
> Hi Tim,
> you're right. Thanks for pointing that out. There's just one problem that
> I'm running into now: The 3.0.1-1 version does not seem to be available in
> Maven Central[1]. Looking at the website, I am not quite sure if there's
> another repository where they do stage the newer versions?[2]
>
> -P
>
> [1] https://repo.maven.apache.org/maven2/com/github/
> stephenc/findbugs/findbugs-annotations/
> [2] http://stephenc.github.io/findbugs-annotations/
>
> On Tue, Jun 12, 2018 at 11:10 PM Tim Robertson <ti...@gmail.com>
> wrote:
>
>> Hi Pablo,
>>
>> I took only a quick look.
>>
>> "- The JAR from the non-LGPL findbugs does not contain the
>> SuppressFBWarnings annotation"
>>
>> Unless I misunderstand you it looks like SuppressFBWarnings was added in
>> Stephen's version in this commit [1] which was introduced in version
>> 2.0.3-1 - I've checked is in the 3.0.1-1 build [2]
>> I notice in your commits [1] you've been exploring version 3.0.0 already
>> though... what happens when you use 3.0.1-1? It sounds like the wrong
>> version is coming in rather than the annotation being missing.
>>
>> Thanks,
>> Tim
>>
>> [1] https://github.com/stephenc/findbugs-annotations/
>> commits/master/src/main/java/edu/umd/cs/findbugs/
>> annotations/SuppressWarnings.java
>> [2] https://github.com/stephenc/findbugs-annotations/releases
>> [3] https://github.com/apache/beam/pull/5609/commits/
>> 32c7df706e970557f154ff6bc521b2e00f9d09ab
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Jun 13, 2018 at 2:37 AM, Pablo Estrada <pa...@google.com>
>> wrote:
>>
>>> Hi all,
>>> I'll humbly declare that after wrestling with he build to stop depending
>>> on the wrong findbugs_annotations, I feel somewhat lost. The issue is
>>> actually quite small:
>>>
>>> - The JAR from the non-LGPL findbugs does not contain the
>>> SuppressFBWarnings annotation. This means that when building, ByteBuddy
>>> produces a few warnings (nothing critical).
>>> - The easiest way to avoid this failure is to call
>>> applyJavaNature(failOnWarning: false), but this would be bad, since we want
>>> to keep a high standard for tasks like ErrorProne and FindBugs itself.
>>> - So I find myself lost: How do we suppress trivial warnings coming from
>>> missing annotations, and honor warnings coming from other plugins?
>>>
>>> Any help / a PR from someone more capable would be appreciated.
>>> Best
>>> -P.
>>>
>>> On Tue, Jun 12, 2018 at 3:02 PM Ismaël Mejía <ie...@gmail.com> wrote:
>>>
>>>> Yes, ok I was not aware it was already being addressed, nice.
>>>> On Tue, Jun 12, 2018 at 11:56 PM Ahmet Altay <al...@google.com> wrote:
>>>> >
>>>> > Ismaël,
>>>> >
>>>> > I believe Pablo's https://github.com/apache/beam/pull/5609 is fixing
>>>> the issue by changing the findbugs back to "com.github.stephenc.findbugs".
>>>> Is this what you are referring to?
>>>> >
>>>> > Ahmet
>>>> >
>>>> > On Tue, Jun 12, 2018 at 2:51 PM, Boyuan Zhang <bo...@google.com>
>>>> wrote:
>>>> >>
>>>> >> Hey JB,
>>>> >>
>>>> >> I added some instructions about how to create python wheels in this
>>>> PR: https://github.com/apache/beam-site/pull/467 . Hope it would be
>>>> helpful.
>>>> >>
>>>> >> Boyuan
>>>> >>
>>>> >
>>>>
>>> --
>>> Got feedback? go/pabloem-feedback
>>> <https://goto.google.com/pabloem-feedback>
>>>
>>
>> --
> Got feedback? go/pabloem-feedback
>
Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Posted by Pablo Estrada <pa...@google.com>.
Hi Tim,
you're right. Thanks for pointing that out. There's just one problem that
I'm running into now: The 3.0.1-1 version does not seem to be available in
Maven Central[1]. Looking at the website, I am not quite sure if there's
another repository where they do stage the newer versions?[2]
-P
[1]
https://repo.maven.apache.org/maven2/com/github/stephenc/findbugs/findbugs-annotations
/
[2] http://stephenc.github.io/findbugs-annotations/
On Tue, Jun 12, 2018 at 11:10 PM Tim Robertson <ti...@gmail.com>
wrote:
> Hi Pablo,
>
> I took only a quick look.
>
> "- The JAR from the non-LGPL findbugs does not contain the
> SuppressFBWarnings annotation"
>
> Unless I misunderstand you it looks like SuppressFBWarnings was added in
> Stephen's version in this commit [1] which was introduced in version
> 2.0.3-1 - I've checked is in the 3.0.1-1 build [2]
> I notice in your commits [1] you've been exploring version 3.0.0 already
> though... what happens when you use 3.0.1-1? It sounds like the wrong
> version is coming in rather than the annotation being missing.
>
> Thanks,
> Tim
>
> [1]
> https://github.com/stephenc/findbugs-annotations/commits/master/src/main/java/edu/umd/cs/findbugs/annotations/SuppressWarnings.java
> [2] https://github.com/stephenc/findbugs-annotations/releases
> [3]
> https://github.com/apache/beam/pull/5609/commits/32c7df706e970557f154ff6bc521b2e00f9d09ab
>
>
>
>
>
>
>
> On Wed, Jun 13, 2018 at 2:37 AM, Pablo Estrada <pa...@google.com> wrote:
>
>> Hi all,
>> I'll humbly declare that after wrestling with he build to stop depending
>> on the wrong findbugs_annotations, I feel somewhat lost. The issue is
>> actually quite small:
>>
>> - The JAR from the non-LGPL findbugs does not contain the
>> SuppressFBWarnings annotation. This means that when building, ByteBuddy
>> produces a few warnings (nothing critical).
>> - The easiest way to avoid this failure is to call
>> applyJavaNature(failOnWarning: false), but this would be bad, since we want
>> to keep a high standard for tasks like ErrorProne and FindBugs itself.
>> - So I find myself lost: How do we suppress trivial warnings coming from
>> missing annotations, and honor warnings coming from other plugins?
>>
>> Any help / a PR from someone more capable would be appreciated.
>> Best
>> -P.
>>
>> On Tue, Jun 12, 2018 at 3:02 PM Ismaël Mejía <ie...@gmail.com> wrote:
>>
>>> Yes, ok I was not aware it was already being addressed, nice.
>>> On Tue, Jun 12, 2018 at 11:56 PM Ahmet Altay <al...@google.com> wrote:
>>> >
>>> > Ismaël,
>>> >
>>> > I believe Pablo's https://github.com/apache/beam/pull/5609 is fixing
>>> the issue by changing the findbugs back to "com.github.stephenc.findbugs".
>>> Is this what you are referring to?
>>> >
>>> > Ahmet
>>> >
>>> > On Tue, Jun 12, 2018 at 2:51 PM, Boyuan Zhang <bo...@google.com>
>>> wrote:
>>> >>
>>> >> Hey JB,
>>> >>
>>> >> I added some instructions about how to create python wheels in this
>>> PR: https://github.com/apache/beam-site/pull/467 . Hope it would be
>>> helpful.
>>> >>
>>> >> Boyuan
>>> >>
>>> >
>>>
>> --
>> Got feedback? go/pabloem-feedback
>> <https://goto.google.com/pabloem-feedback>
>>
>
> --
Got feedback? go/pabloem-feedback
Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Posted by Tim Robertson <ti...@gmail.com>.
Hi Pablo,
I took only a quick look.
"- The JAR from the non-LGPL findbugs does not contain the
SuppressFBWarnings annotation"
Unless I misunderstand you it looks like SuppressFBWarnings was added in
Stephen's version in this commit [1] which was introduced in version
2.0.3-1 - I've checked is in the 3.0.1-1 build [2]
I notice in your commits [1] you've been exploring version 3.0.0 already
though... what happens when you use 3.0.1-1? It sounds like the wrong
version is coming in rather than the annotation being missing.
Thanks,
Tim
[1]
https://github.com/stephenc/findbugs-annotations/commits/master/src/main/java/edu/umd/cs/findbugs/annotations/SuppressWarnings.java
[2] https://github.com/stephenc/findbugs-annotations/releases
[3]
https://github.com/apache/beam/pull/5609/commits/32c7df706e970557f154ff6bc521b2e00f9d09ab
On Wed, Jun 13, 2018 at 2:37 AM, Pablo Estrada <pa...@google.com> wrote:
> Hi all,
> I'll humbly declare that after wrestling with he build to stop depending
> on the wrong findbugs_annotations, I feel somewhat lost. The issue is
> actually quite small:
>
> - The JAR from the non-LGPL findbugs does not contain the
> SuppressFBWarnings annotation. This means that when building, ByteBuddy
> produces a few warnings (nothing critical).
> - The easiest way to avoid this failure is to call
> applyJavaNature(failOnWarning: false), but this would be bad, since we want
> to keep a high standard for tasks like ErrorProne and FindBugs itself.
> - So I find myself lost: How do we suppress trivial warnings coming from
> missing annotations, and honor warnings coming from other plugins?
>
> Any help / a PR from someone more capable would be appreciated.
> Best
> -P.
>
> On Tue, Jun 12, 2018 at 3:02 PM Ismaël Mejía <ie...@gmail.com> wrote:
>
>> Yes, ok I was not aware it was already being addressed, nice.
>> On Tue, Jun 12, 2018 at 11:56 PM Ahmet Altay <al...@google.com> wrote:
>> >
>> > Ismaël,
>> >
>> > I believe Pablo's https://github.com/apache/beam/pull/5609 is fixing
>> the issue by changing the findbugs back to "com.github.stephenc.findbugs".
>> Is this what you are referring to?
>> >
>> > Ahmet
>> >
>> > On Tue, Jun 12, 2018 at 2:51 PM, Boyuan Zhang <bo...@google.com>
>> wrote:
>> >>
>> >> Hey JB,
>> >>
>> >> I added some instructions about how to create python wheels in this
>> PR: https://github.com/apache/beam-site/pull/467 . Hope it would be
>> helpful.
>> >>
>> >> Boyuan
>> >>
>> >
>>
> --
> Got feedback? go/pabloem-feedback
>
Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Posted by Pablo Estrada <pa...@google.com>.
Hi all,
I'll humbly declare that after wrestling with he build to stop depending on
the wrong findbugs_annotations, I feel somewhat lost. The issue is actually
quite small:
- The JAR from the non-LGPL findbugs does not contain the
SuppressFBWarnings annotation. This means that when building, ByteBuddy
produces a few warnings (nothing critical).
- The easiest way to avoid this failure is to call
applyJavaNature(failOnWarning: false), but this would be bad, since we want
to keep a high standard for tasks like ErrorProne and FindBugs itself.
- So I find myself lost: How do we suppress trivial warnings coming from
missing annotations, and honor warnings coming from other plugins?
Any help / a PR from someone more capable would be appreciated.
Best
-P.
On Tue, Jun 12, 2018 at 3:02 PM Ismaël Mejía <ie...@gmail.com> wrote:
> Yes, ok I was not aware it was already being addressed, nice.
> On Tue, Jun 12, 2018 at 11:56 PM Ahmet Altay <al...@google.com> wrote:
> >
> > Ismaël,
> >
> > I believe Pablo's https://github.com/apache/beam/pull/5609 is fixing
> the issue by changing the findbugs back to "com.github.stephenc.findbugs".
> Is this what you are referring to?
> >
> > Ahmet
> >
> > On Tue, Jun 12, 2018 at 2:51 PM, Boyuan Zhang <bo...@google.com>
> wrote:
> >>
> >> Hey JB,
> >>
> >> I added some instructions about how to create python wheels in this PR:
> https://github.com/apache/beam-site/pull/467 . Hope it would be helpful.
> >>
> >> Boyuan
> >>
> >
>
--
Got feedback? go/pabloem-feedback
Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Posted by Ismaël Mejía <ie...@gmail.com>.
Yes, ok I was not aware it was already being addressed, nice.
On Tue, Jun 12, 2018 at 11:56 PM Ahmet Altay <al...@google.com> wrote:
>
> Ismaël,
>
> I believe Pablo's https://github.com/apache/beam/pull/5609 is fixing the issue by changing the findbugs back to "com.github.stephenc.findbugs". Is this what you are referring to?
>
> Ahmet
>
> On Tue, Jun 12, 2018 at 2:51 PM, Boyuan Zhang <bo...@google.com> wrote:
>>
>> Hey JB,
>>
>> I added some instructions about how to create python wheels in this PR: https://github.com/apache/beam-site/pull/467 . Hope it would be helpful.
>>
>> Boyuan
>>
>
Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Posted by Ahmet Altay <al...@google.com>.
Ismaël,
I believe Pablo's https://github.com/apache/beam/pull/5609 is fixing the
issue by changing the findbugs back to "com.github.stephenc.findbugs". Is
this what you are referring to?
Ahmet
On Tue, Jun 12, 2018 at 2:51 PM, Boyuan Zhang <bo...@google.com> wrote:
> Hey JB,
>
> I added some instructions about how to create python wheels in this PR:
> https://github.com/apache/beam-site/pull/467 . Hope it would be helpful.
>
> Boyuan
>
>
Re: [CANCEL][VOTE] Apache Beam, version 2.5.0, release candidate #1
Posted by Boyuan Zhang <bo...@google.com>.
Hey JB,
I added some instructions about how to create python wheels in this PR:
https://github.com/apache/beam-site/pull/467 . Hope it would be helpful.
Boyuan
On Tue, Jun 12, 2018 at 12:33 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Hi all,
>
> due to issues found by Udi and Pablo, I cancel RC1 to prepare a new
> release.
>
> Thanks Udi and Pablo for catching up.
>
> I will prepare a RC2 after cherry pick/fixed the found issues (tomorrow
> morning my time).
>
> Regards
> JB
>
>
> On 12/06/2018 21:04, Pablo Estrada wrote:
> > Hello all,
> > I've found another release blocker: The current findbugs dependency in
> > build_rules.gradle has a GPL license. Here's the PR to depend on the
> > correct findbugs: https://github.com/apache/beam/pull/5609
> >
> > Best
> > -P.
> >
> > On Tue, Jun 12, 2018 at 11:03 AM Udi Meiri <ehudm@google.com
> > <ma...@google.com>> wrote:
> >
> > Cherrypick created: https://github.com/apache/beam/pull/5607
> > Tests still running.
> >
> > On Tue, Jun 12, 2018 at 9:59 AM Udi Meiri <ehudm@google.com
> > <ma...@google.com>> wrote:
> >
> > -1: Would like to cherry pick a fix
> > for https://issues.apache.org/jira/browse/BEAM-4536
> >
> > On Tue, Jun 12, 2018 at 9:48 AM Jean-Baptiste Onofré
> > <jb@nanthrax.net <ma...@nanthrax.net>> wrote:
> >
> > Any update about your vote and fix ?
> >
> > Thanks !
> > Regards
> > JB
> >
> > On 12/06/2018 04:02, Udi Meiri wrote:
> > > Another bug: reading from PubSub with_attributes=True is
> > broken on
> > > Python with Dataflow.
> > > https://issues.apache.org/jira/browse/BEAM-4536
> > >
> > > JB, I'm making a PR that removes this keyword and I'd like
> > to propose it
> > > as a cherrypick to 2.5.0.
> > > (feature should be fixed in the next release)
> > >
> > > On Mon, Jun 11, 2018 at 6:19 PM Chamikara Jayalath
> > <chamikara@google.com <ma...@google.com>
> > > <mailto:chamikara@google.com
> > <ma...@google.com>>> wrote:
> > >
> > > FYI: looks like Python tests are failing for Windows.
> JIRA
> > > is https://issues.apache.org/jira/browse/BEAM-4535.
> > >
> > > I don't think this is a release blocker but this
> > should probably go
> > > in release notes (for any user that tries to run tests
> > on Python
> > > source build). And we should try to incorporate a fix
> > if we happen
> > > to cut another release candidate for some reason.
> > >
> > > Thanks,
> > > Cham
> > >
> > > On Mon, Jun 11, 2018 at 5:46 PM Pablo Estrada
> > <pabloem@google.com <ma...@google.com>
> > > <mailto:pabloem@google.com
> > <ma...@google.com>>> wrote:
> > >
> > > Thanks everyone who has pitched in to validate the
> > release!
> > >
> > > Boyuan Zhang and I have also run a few pipelines,
> > and verified
> > > that they work properly (see release validation
> > spreadsheet[1]).
> > >
> > > We have also found that the Game Stats pipeline is
> > failing in
> > > Python Streaming Dataflow. I have filed
> > BEAM-4534[2]. This is
> > > not a blocker, since Python streaming is not yet
> > fully supported.
> > >
> > > It seems that the uploaded artifacts look good.
> > >
> > > We have noticed that the Python artifacts are
> > still missing
> > > Python wheel files (compare [3] and [4]). JB,
> > could you please
> > > add the wheel files? Boyuan and I can try to help
> > you prepare
> > > them / upload them if necessary. Please let us
> know.
> > >
> > > Thanks again!
> > > -P.
> > >
> > >
> > [1]
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=152451807
> > > [2]
> https://issues.apache.org/jira/browse/BEAM-4534
> > > [3]
> https://dist.apache.org/repos/dist/dev/beam/2.4.0/
> > > [4]
> https://dist.apache.org/repos/dist/dev/beam/2.5.0/
> > >
> > > On Mon, Jun 11, 2018 at 12:37 PM Alan Myrvold
> > > <amyrvold@google.com <ma...@google.com>
> > <mailto:amyrvold@google.com <ma...@google.com>>>
> > wrote:
> > >
> > > +1 (non-binding)
> > >
> > > tested some of the quickstarts
> > >
> > > On Sun, Jun 10, 2018 at 1:39 AM Tim
> > > <timrobertson100@gmail.com
> > <ma...@gmail.com>
> > > <mailto:timrobertson100@gmail.com
> > <ma...@gmail.com>>> wrote:
> > >
> > > Tested by our team:
> > > - mvn inclusion
> > > - Avro, ES, Hadoop IF IO
> > > - Pipelines run on Spark (Cloudera 5.12.0
> > YARN cluster)
> > > - Reviewed release notes
> > >
> > > +1
> > >
> > > Thanks also to everyone who helped get
> > over the gradle
> > > hurdle and in particular to JB.
> > >
> > > Tim
> > >
> > > > On 9 Jun 2018, at 05:56, Jean-Baptiste
> > Onofré
> > > <jb@nanthrax.net <ma...@nanthrax.net>
> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> > > >
> > > > No problem Pablo.
> > > >
> > > > The vote period is a minimum, it can be
> > extended as
> > > requested or if we
> > > > don't have the minimum of 3 binding
> votes.
> > > >
> > > > Regards
> > > > JB
> > > >
> > > >> On 09/06/2018 01:54, Pablo Estrada
> wrote:
> > > >> Hello all,
> > > >> I'd like to request an extension of the
> > voting period
> > > until Monday
> > > >> evening (US time, so later in other
> > geographical
> > > regions). This is
> > > >> because we were only now able to
> > publish Dataflow
> > > Workers, and have not
> > > >> had the chance to run release
> > validation tests on
> > > them. The extension
> > > >> will allow us to validate and vote by
> > Monday.
> > > >> Is this acceptable to the community?
> > > >>
> > > >> Best
> > > >> -P.
> > > >>
> > > >> On Fri, Jun 8, 2018 at 6:20 AM Alexey
> > Romanenko
> > > >> <aromanenko.dev@gmail.com
> > <ma...@gmail.com>
> > > <mailto:aromanenko.dev@gmail.com
> > <ma...@gmail.com>>
> > > <mailto:aromanenko.dev@gmail.com
> > <ma...@gmail.com>
> > > <mailto:aromanenko.dev@gmail.com
> > <ma...@gmail.com>>>> wrote:
> > > >>
> > > >> Thank you JB for your work!
> > > >>
> > > >> I tested running simple streaming
> > (/KafkaIO/) and
> > > batch (/TextIO /
> > > >> HDFS/) pipelines with SparkRunner on
> > YARN cluster
> > > - it works fine.
> > > >>
> > > >> WBR,
> > > >> Alexey
> > > >>
> > > >>
> > > >>> On 8 Jun 2018, at 10:00, Etienne
> > Chauchot
> > > <echauchot@apache.org
> > <ma...@apache.org> <mailto:echauchot@apache.org
> > <ma...@apache.org>>
> > > >>> <mailto:echauchot@apache.org
> > <ma...@apache.org>
> > > <mailto:echauchot@apache.org
> > <ma...@apache.org>>>> wrote:
> > > >>>
> > > >>> I forgot to vote:
> > > >>> +1 (non binding).
> > > >>> What I tested:
> > > >>> - no functional or performance
> > regression
> > > comparing to v2.4
> > > >>> - dependencies in the poms are ok
> > > >>>
> > > >>> Etienne
> > > >>>> Le vendredi 08 juin 2018 à 08:27
> > +0200, Romain
> > > Manni-Bucau a écrit :
> > > >>>> +1 (non-binding), mainstream usage
> > is not broken
> > > by the pom
> > > >>>> changes and runtime has no known
> > regression
> > > compared to the 2.4.0
> > > >>>>
> > > >>>> (side note: kudo to JB for this
> > build tool
> > > change release, I know
> > > >>>> how it can hurt ;))
> > > >>>>
> > > >>>> Romain Manni-Bucau
> > > >>>> @rmannibucau
> > <https://twitter.com/rmannibucau>
> > > | Blog
> > > >>>>
> > <https://rmannibucau.metawerx.net/> | Old Blog
> > > >>>>
> > <http://rmannibucau.wordpress.com/> | Github
> > > >>>> <https://github.com/rmannibucau> |
> > LinkedIn
> > > >>>>
> > <https://www.linkedin.com/in/rmannibucau> | Book
> > > >>>>
> > >
> > <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
> > > >>>>
> > > >>>>
> > > >>>> Le jeu. 7 juin 2018 à 16:17,
> > Jean-Baptiste Onofré
> > > >>>> <jb@nanthrax.net
> > <ma...@nanthrax.net> <mailto:jb@nanthrax.net
> > <ma...@nanthrax.net>>
> > > <mailto:jb@nanthrax.net
> > <ma...@nanthrax.net> <mailto:jb@nanthrax.net
> > <ma...@nanthrax.net>>>> a écrit :
> > > >>>>> Thanks for the details Etienne !
> > > >>>>>
> > > >>>>> The good news is that the
> > artifacts seem OK and
> > > the overall Nexmark
> > > >>>>> results are consistent with the
> > 2.4.0 release ones.
> > > >>>>>
> > > >>>>> I'm starting a complete review
> > using the
> > > beam-samples as well.
> > > >>>>>
> > > >>>>> Regards
> > > >>>>> JB
> > > >>>>>
> > > >>>>>> On 07/06/2018 16:14, Etienne
> > Chauchot wrote:
> > > >>>>>> Hi,
> > > >>>>>> I've just run the nexmark queries
> > on v2.5.0-RC1 tag
> > > >>>>>> What we can notice:
> > > >>>>>> - query 3 (exercises CoGroupByKey,
> > state and
> > > timer) shows
> > > >>>>> different
> > > >>>>>> output with DR between batch and
> > streaming and
> > > with the other
> > > >>>>> runners =>
> > > >>>>>> I compared with v2.4 there were
> > still these
> > > differences but with
> > > >>>>>> different output size numbers
> > > >>>>>>
> > > >>>>>> - query 6 (exercises specialized
> > combiner) shows
> > > different output
> > > >>>>>> between the runners => the correct
> > output is 401.
> > > strange that
> > > >>>>> in batch
> > > >>>>>> mode some runners output les
> > Sellers. I compared
> > > with v2.4
> > > >>>>> same output
> > > >>>>>>
> > > >>>>>> - response time of query 7
> > (exercices Max
> > > transform, fanout
> > > >>>>> and side
> > > >>>>>> input) is very slow on DR => I
> > compared with v2.4
> > > , comparable
> > > >>>>> execution
> > > >>>>>> times
> > > >>>>>>
> > > >>>>>> I'm not comparing q10 because it is
> > a write to
> > > GCS so it is
> > > >>>>> very specific.
> > > >>>>>>
> > > >>>>>> => Basically no regression
> > comparing to v2.4
> > > >>>>>>
> > > >>>>>> For the record here is the output
> > (waiting for
> > > ongoing perfkit
> > > >>>>> integration):
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> 1. DR batch
> > > >>>>>>
> > > >>>>>> Performance:
> > > >>>>>>
> > > >>>>>>
> > > >>>>> Conf Runtime(sec) (Baseline)
> > Events(/sec)
> > > (Baseline) Results (Baseline)
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0000 5,8
> > 17283,1
> > > 100000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0001 3,2
> > 31104,2
> > > 92000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0002 1,2
> > 82918,7
> > > 351
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0003 2,2
> > 46210,7
> > > 458
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0004 1,2
> > 8503,4
> > > 40
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0005 4,0
> > 25220,7
> > > 12
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0006 0,9
> > 11148,3
> > > 401
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0007 13,2
> > 7580,9
> > > 1
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0008 1,5
> > 67340,1
> > > 6000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0009 0,7
> > 14025,2
> > > 298
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0010 12,8
> > 7793,0
> > > 1
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0011 2,4
> > 42319,1
> > > 1919
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0012 1,6
> > 61462,8
> > > 1919
> > > >>>>>>
> > > >>>>>
> > >
> >
> ==========================================================================================
> > > >>>>>>
> > > >>>>>> 2. DR streaming
> > > >>>>>>
> > > >>>>>> Performance:
> > > >>>>>>
> > > >>>>>>
> > > >>>>> Conf Runtime(sec) (Baseline)
> > Events(/sec)
> > > (Baseline) Results (Baseline)
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0000 6,5
> > 15285,8
> > > 100000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0001 3,7
> > 27397,3
> > > 92000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0002 1,4
> > 69108,5
> > > 351
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0003 3,2
> > 31181,8
> > > 447
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0004 1,2
> > 8361,2
> > > 40
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0005 5,3
> > 18903,6
> > > 12
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0006 0,9
> > 11111,1
> > > 401
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0007 82,5
> > 1212,2
> > > 1
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0008 2,0
> > 51072,5
> > > 6000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0009 0,8
> > 12903,2
> > > 298
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0010 49,5
> > 2021,8
> > > 1
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0011 3,9
> > 25667,4
> > > 1919
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0012 2,4
> > 41067,8
> > > 1919
> > > >>>>>>
> > > >>>>>
> > >
> >
> ==========================================================================================
> > > >>>>>>
> > > >>>>>> 3. Flink batch
> > > >>>>>> Performance:
> > > >>>>>>
> > > >>>>>>
> > > >>>>> Conf Runtime(sec) (Baseline)
> > Events(/sec)
> > > (Baseline) Results (Baseline)
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0000 1,0
> > 97656,3
> > > 100000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0001 0,7
> > 141643,1
> > > 92000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0002 0,4
> > 228310,5
> > > 351
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0003 1,6
> > 64020,5
> > > 580
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0004 0,7
> > 13831,3
> > > 40
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0005 1,4
> > 72939,5
> > > 12
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0006 0,5
> > 20491,8
> > > 103
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0007 1,3
> > 74239,0
> > > 1
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0008 0,8
> > 121506,7
> > > 6000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0009 0,6
> > 17953,3
> > > 298
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0010 1,3
> > 74682,6
> > > 1
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0011 1,1
> > 92936,8
> > > 1919
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0012 0,8
> > 123001,2
> > > 1919
> > > >>>>>>
> > > >>>>>
> > >
> >
> ==========================================================================================
> > > >>>>>>
> > > >>>>>> 4. Flink streaming
> > > >>>>>> Performance:
> > > >>>>>>
> > > >>>>>>
> > > >>>>> Conf Runtime(sec) (Baseline)
> > Events(/sec)
> > > (Baseline) Results (Baseline)
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0000 5,4
> > 18677,6
> > > 100000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0001 2,8
> > 35511,4
> > > 92000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0002 1,8
> > 54318,3
> > > 351
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0003 2,4
> > 41614,6
> > > 580
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0004 1,0
> > 10341,3
> > > 40
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0005 3,4
> > 29568,3
> > > 12
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0006 0,7
> > 13369,0
> > > 401
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0007 2,8
> > 36192,5
> > > 1
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0008 1,8
> > 54854,6
> > > 6000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0009 0,7
> > 13369,0
> > > 298
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0010 3,4
> > 29841,8
> > > 2
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0011 5,0
> > 19932,2
> > > 1919
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0012 2,6
> > 38835,0
> > > 1919
> > > >>>>>>
> > > >>>>>
> > >
> >
> ==========================================================================================
> > > >>>>>>
> > > >>>>>> 5. Spark batch
> > > >>>>>> Performance:
> > > >>>>>>
> > > >>>>>>
> > > >>>>> Conf Runtime(sec) (Baseline)
> > Events(/sec)
> > > (Baseline) Results (Baseline)
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0000 1,5
> > 65445,0
> > > 100000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0001 1,3
> > 79491,3
> > > 92000
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0002 0,9
> > 112107,6
> > > 351
> > > >>>>>>
> > > >>>>>>
> > > >>>>> 0003 2,0
> > 48804,3
> > > 580
> > > >>