You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Daniel Oliveira <da...@google.com.INVALID> on 2017/09/26 20:52:58 UTC
Possibility of requiring Java 8 compiler for building Java 7 sources?
So I've been working on JDK 9 support for Beam, and I have a bug in
AutoValue that can be fixed by updating our AutoValue dependency to the
latest. The problem is that AutoValue from 1.5+ seems to be banned for Beam
due to requiring Java 8 compilers. However, it should still be possible to
compile and execute Java 7 code from the Java 8 compiler by building with
the correct arguments. So the fix to this bug would essentially require
Java 8 compilers even for compiling Java 7 code.
Does anyone need to use Java 7 compilers? Because if not I would like to
continue with this fix.
Re: Possibility of requiring Java 8 compiler for building Java 7
sources?
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
+1
Regards
JB
On 09/27/2017 06:04 PM, Reuven Lax wrote:
> I would support this as well, however we probably should first poll current
> users of the Beam API to see if any of them rely on Java 7.
>
> Reuven
>
> On Wed, Sep 27, 2017 at 9:00 AM, Robert Bradshaw <
> robertwb@google.com.invalid> wrote:
>
>> I also think that it's time to seriously consider dropping support for
>> Java 7.
>>
>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
>> <da...@google.com.invalid> wrote:
>>> Yes, just as Ismaël said it's a compilation blocker right now despite
>> that
>>> (I believe) we don't use the extension that's breaking.
>>>
>>> As for other ways to solve this, if there is a way to avoid compiling the
>>> advanced features of AutoValue that might be worth a try. We could also
>> try
>>> to get a release of AutoValue with the fix that works in Java 7. However
>> I
>>> feel that slowly moving over to Java 8 is the most future-proof solution
>> if
>>> it's possible.
>>>
>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <ie...@gmail.com> wrote:
>>>
>>>> The current issue is that compilation fails on master because beam's
>>>> parent pom is configured to fail if it finds warnings):
>>>>
>>>> <compiler.error.flag>-Werror</compiler.error.flag>
>>>>
>>>> However if you remove that line from the parent pom the compilation
>> passes.
>>>>
>>>> Of course this does not mean that everything is solved for Java 9,
>>>> there are some tests that break and other issues because of other
>>>> plugins and dependencies (e.g. bytebuddy), but those are not part of
>>>> this discussion.
>>>>
>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
>>>> <ki...@google.com.invalid> wrote:
>>>>> AFAIK we don't use any advanced capabilities of AutoValue. Does that
>> mean
>>>>> this issue is moot? I didn't quite understand from your email whether
>> it
>>>> is
>>>>> a compilation blocker for Beam or not.
>>>>>
>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <ie...@gmail.com>
>> wrote:
>>>>>
>>>>>> Great that you are also working on this too Daniel and thanks for
>>>>>> bringing this subject to the mailing list, I was waiting to my
>> return
>>>>>> to office next week, but you did it first :)
>>>>>>
>>>>>> Eugene for reference (This is the issue on the migration to Java 9),
>>>>>> notice that here the goal is first that beam passes mvn clean install
>>>>>> with pure Java 9 (and also add this to jenkins), not to rewrite
>>>>>> anything to use the new stuff (e.g. modules):
>>>>>> https://issues.apache.org/jira/browse/BEAM-2530
>>>>>>
>>>>>> Eugene can you also PTAL at the AutoValue issue, more details on the
>>>>>> issue, this is a warning so I don't know if it is really critical in
>>>>>> particular because we are not using Memoization (do we?).
>>>>>> https://github.com/google/auto/issues/503
>>>>>>
>>>>>> https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d
>>>> 02247b8532
>>>>>>
>>>>>> Wouldn't the easiest way be that you guys convince the google.auto
>>>>>> guys to generate that simple fix in a Java 7 compatible way and
>>>>>> 'voila' ?
>>>>>>
>>>>>> However I agree that moving to Java 8 is an excellent idea and as
>>>>>> Eugene mentions there is less friction now since most projects are
>>>>>> moving, only pending issue are existing clusters on java 7 in the
>>>>>> hadoop world, but those are less frequent now. Anyway this discussion
>>>>>> is really important (maybe even worth a vote). Because moving to Java
>>>>>> 8 will allow us also to move some of the dependencies that we are
>>>>>> keeping in old versions and in general to move forward.
>>>>>>
>>>>>> What do the others think ?
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
>>>>>> <ki...@google.com.invalid> wrote:
>>>>>>> Very excited to hear that there's work on JDK9 support - is there a
>>>>>> public
>>>>>>> description of the plans for this work somewhere?
>>>>>>>
>>>>>>> In general, Beam could probably drop Java7 support altogether at
>> some
>>>>>> point
>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not receiving even
>>>>>> security
>>>>>>> patches) 2 years ago, and all major players in the data processing
>>>>>>> ecosystem have dropped Java7 support (Spark, Flink, Hadoop), so I
>>>> presume
>>>>>>> the demand for Java7 support in the data processing industry is
>> low.
>>>> By
>>>>>> the
>>>>>>> way: would a Java8 migration be in the scope of your work in
>> general?
>>>>>>>
>>>>>>> However, until we say that Beam requires Java8, what would be the
>>>>>>> implications of using a version of AutoValue that can only be
>> compiled
>>>>>> with
>>>>>>> Java8? Are you saying that this is simply a matter of a compiler
>> bug,
>>>> and
>>>>>>> if we use a Java8 compiler but configured to use source and target
>>>>>> versions
>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7, then the
>> resulting
>>>>>> Beam
>>>>>>> artifacts will be usable by people who don't have Java8?
>>>>>>>
>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
>>>>>>> <da...@google.com.invalid> wrote:
>>>>>>>
>>>>>>>> So I've been working on JDK 9 support for Beam, and I have a bug
>> in
>>>>>>>> AutoValue that can be fixed by updating our AutoValue dependency
>> to
>>>> the
>>>>>>>> latest. The problem is that AutoValue from 1.5+ seems to be banned
>>>> for
>>>>>> Beam
>>>>>>>> due to requiring Java 8 compilers. However, it should still be
>>>> possible
>>>>>> to
>>>>>>>> compile and execute Java 7 code from the Java 8 compiler by
>> building
>>>>>> with
>>>>>>>> the correct arguments. So the fix to this bug would essentially
>>>> require
>>>>>>>> Java 8 compilers even for compiling Java 7 code.
>>>>>>>>
>>>>>>>> Does anyone need to use Java 7 compilers? Because if not I would
>>>> like to
>>>>>>>> continue with this fix.
>>>>>>>>
>>>>>>
>>>>
>>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Reuven Lax <re...@google.com.INVALID>.
I would support this as well, however we probably should first poll current
users of the Beam API to see if any of them rely on Java 7.
Reuven
On Wed, Sep 27, 2017 at 9:00 AM, Robert Bradshaw <
robertwb@google.com.invalid> wrote:
> I also think that it's time to seriously consider dropping support for
> Java 7.
>
> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
> <da...@google.com.invalid> wrote:
> > Yes, just as Ismaël said it's a compilation blocker right now despite
> that
> > (I believe) we don't use the extension that's breaking.
> >
> > As for other ways to solve this, if there is a way to avoid compiling the
> > advanced features of AutoValue that might be worth a try. We could also
> try
> > to get a release of AutoValue with the fix that works in Java 7. However
> I
> > feel that slowly moving over to Java 8 is the most future-proof solution
> if
> > it's possible.
> >
> > On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <ie...@gmail.com> wrote:
> >
> >> The current issue is that compilation fails on master because beam's
> >> parent pom is configured to fail if it finds warnings):
> >>
> >> <compiler.error.flag>-Werror</compiler.error.flag>
> >>
> >> However if you remove that line from the parent pom the compilation
> passes.
> >>
> >> Of course this does not mean that everything is solved for Java 9,
> >> there are some tests that break and other issues because of other
> >> plugins and dependencies (e.g. bytebuddy), but those are not part of
> >> this discussion.
> >>
> >> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
> >> <ki...@google.com.invalid> wrote:
> >> > AFAIK we don't use any advanced capabilities of AutoValue. Does that
> mean
> >> > this issue is moot? I didn't quite understand from your email whether
> it
> >> is
> >> > a compilation blocker for Beam or not.
> >> >
> >> > On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <ie...@gmail.com>
> wrote:
> >> >
> >> >> Great that you are also working on this too Daniel and thanks for
> >> >> bringing this subject to the mailing list, I was waiting to my
> return
> >> >> to office next week, but you did it first :)
> >> >>
> >> >> Eugene for reference (This is the issue on the migration to Java 9),
> >> >> notice that here the goal is first that beam passes mvn clean install
> >> >> with pure Java 9 (and also add this to jenkins), not to rewrite
> >> >> anything to use the new stuff (e.g. modules):
> >> >> https://issues.apache.org/jira/browse/BEAM-2530
> >> >>
> >> >> Eugene can you also PTAL at the AutoValue issue, more details on the
> >> >> issue, this is a warning so I don't know if it is really critical in
> >> >> particular because we are not using Memoization (do we?).
> >> >> https://github.com/google/auto/issues/503
> >> >>
> >> >> https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d
> >> 02247b8532
> >> >>
> >> >> Wouldn't the easiest way be that you guys convince the google.auto
> >> >> guys to generate that simple fix in a Java 7 compatible way and
> >> >> 'voila' ?
> >> >>
> >> >> However I agree that moving to Java 8 is an excellent idea and as
> >> >> Eugene mentions there is less friction now since most projects are
> >> >> moving, only pending issue are existing clusters on java 7 in the
> >> >> hadoop world, but those are less frequent now. Anyway this discussion
> >> >> is really important (maybe even worth a vote). Because moving to Java
> >> >> 8 will allow us also to move some of the dependencies that we are
> >> >> keeping in old versions and in general to move forward.
> >> >>
> >> >> What do the others think ?
> >> >>
> >> >>
> >> >>
> >> >> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
> >> >> <ki...@google.com.invalid> wrote:
> >> >> > Very excited to hear that there's work on JDK9 support - is there a
> >> >> public
> >> >> > description of the plans for this work somewhere?
> >> >> >
> >> >> > In general, Beam could probably drop Java7 support altogether at
> some
> >> >> point
> >> >> > soon: Java7 has reached end-of-life (i.e. it's not receiving even
> >> >> security
> >> >> > patches) 2 years ago, and all major players in the data processing
> >> >> > ecosystem have dropped Java7 support (Spark, Flink, Hadoop), so I
> >> presume
> >> >> > the demand for Java7 support in the data processing industry is
> low.
> >> By
> >> >> the
> >> >> > way: would a Java8 migration be in the scope of your work in
> general?
> >> >> >
> >> >> > However, until we say that Beam requires Java8, what would be the
> >> >> > implications of using a version of AutoValue that can only be
> compiled
> >> >> with
> >> >> > Java8? Are you saying that this is simply a matter of a compiler
> bug,
> >> and
> >> >> > if we use a Java8 compiler but configured to use source and target
> >> >> versions
> >> >> > of 1.7 and using bootclasspath of rt.jar from 1.7, then the
> resulting
> >> >> Beam
> >> >> > artifacts will be usable by people who don't have Java8?
> >> >> >
> >> >> > On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
> >> >> > <da...@google.com.invalid> wrote:
> >> >> >
> >> >> >> So I've been working on JDK 9 support for Beam, and I have a bug
> in
> >> >> >> AutoValue that can be fixed by updating our AutoValue dependency
> to
> >> the
> >> >> >> latest. The problem is that AutoValue from 1.5+ seems to be banned
> >> for
> >> >> Beam
> >> >> >> due to requiring Java 8 compilers. However, it should still be
> >> possible
> >> >> to
> >> >> >> compile and execute Java 7 code from the Java 8 compiler by
> building
> >> >> with
> >> >> >> the correct arguments. So the fix to this bug would essentially
> >> require
> >> >> >> Java 8 compilers even for compiling Java 7 code.
> >> >> >>
> >> >> >> Does anyone need to use Java 7 compilers? Because if not I would
> >> like to
> >> >> >> continue with this fix.
> >> >> >>
> >> >>
> >>
>
Re: Possibility of requiring Java 8 compiler for building Java 7
sources?
Posted by Aljoscha Krettek <al...@apache.org>.
+1 for deprecating (at least)
I'm also for dropping Java 7 support because it makes the code a lot nicer. However, that doesn't really help users and they can still use Java 8 even if we don't.
> On 17. Oct 2017, at 19:17, Kenneth Knowles <kl...@google.com.INVALID> wrote:
>
> It is a good point that this is a breaking change for users of Java 7.
> Assuming we wait until 3.0.0 to drop Java 7 support, and that we wait a
> while to do a major version bump, I support deprecating support now for
> sure.
>
> If we had a very strong signal that we would not really break anyone, I
> would be open to a minor version that kept API compatibility while moving
> our build to Java 8, even though it technically violates semantic
> versioning. We don't have to be dogmatic - acceptable backwards
> incompatibility occurs all the time in the form of bugfixes (the new
> behavior is by definition different) or security-related updates.
>
> Also worth remembering that the portability framework can separate runner /
> SDK Java generation just as though they are separate languages.
>
> On Tue, Oct 17, 2017 at 9:51 AM, Reuven Lax <re...@google.com.invalid>
> wrote:
>
>> So maybe what we should do now is deprecate Java 7 support but not drop it
>> yet. I believe this is also what Spark has done.
>>
>> On Tue, Oct 17, 2017 at 9:46 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>>
>>> Agree, I would target this for Beam 3.0.0.
>>>
>>> Regards
>>> JB
>>>
>>> On 10/17/2017 06:43 PM, Reuven Lax wrote:
>>>
>>>> Should this be considered a backwards-incompatible change? If so, do we
>>>> need to wait until Beam 3.0.0 is released?
>>>>
>>>> On Tue, Oct 17, 2017 at 9:11 AM, Ismaël Mejía <ie...@gmail.com>
>> wrote:
>>>>
>>>> I am bringing the subject to the user mailing list to get some
>>>>> feedback because it makes sense anyway to discuss this there. But I
>>>>> also agree with Kenneth about the fact that runner authors must weight
>>>>> in on this subject.
>>>>>
>>>>>
>>>>> On Tue, Oct 17, 2017 at 5:24 PM, Kenneth Knowles
>> <klk@google.com.invalid
>>>>>>
>>>>> wrote:
>>>>>
>>>>>> +1 to having runner maintainers weigh in as proxies. Added a few in
>> case
>>>>>> they haven't followed this thread.
>>>>>>
>>>>>> On Mon, Oct 16, 2017 at 11:38 PM, Eugene Kirpichov <
>>>>>> kirpichov@google.com.invalid> wrote:
>>>>>>
>>>>>> Agreed that polling Dataflow users makes sense, though I think they
>> are
>>>>>>> very unlikely to have concerns, because unlike Spark/Flink users they
>>>>>>> wouldn't have a "cluster" that they need to migrate to a new JVM:
>>>>>>> they'd
>>>>>>> only need to recompile their pipelines with JDK 8.
>>>>>>>
>>>>>>> On Mon, Oct 16, 2017 at 11:21 PM Reuven Lax <relax@google.com.invalid
>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>> I think the Flink and Spark runner maintainers can weigh in here;
>>>>>>>>
>>>>>>> given
>>>>>
>>>>>> that both of those systems are moving to Java 8, I doubt they will
>>>>>>>>
>>>>>>> have
>>>>>
>>>>>> concerns. Same is true for the Dataflow runner - we should probably
>>>>>>>>
>>>>>>> poll
>>>>>
>>>>>> Dataflow users to make sure this is not a problem for any of them.
>>>>>>>>
>>>>>>>> On Mon, Oct 16, 2017 at 11:05 PM, Eugene Kirpichov <
>>>>>>>> kirpichov@google.com.invalid> wrote:
>>>>>>>>
>>>>>>>> Reuven - do you mean e.g. a poll on the Flink mailing list asking
>>>>>>>>>
>>>>>>>> whether
>>>>>>>
>>>>>>>> there are Flink users who use Beam with Java 7? Or just people who
>>>>>>>>>
>>>>>>>> use
>>>>>
>>>>>> Flink with Java 7? (the latter question I'd assume was settled by
>>>>>>>>>
>>>>>>>> the
>>>>>
>>>>>> poll
>>>>>>>>
>>>>>>>>> about making Flink itself Java8-only?)
>>>>>>>>>
>>>>>>>>> On Mon, Oct 16, 2017 at 10:32 PM Reuven Lax
>>>>>>>>>
>>>>>>>> <re...@google.com.invalid>
>>>>>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> I don't know if a vote in @user is sufficient, as it's
>>>>>>>>>>
>>>>>>>>> unfortunately
>>>>>
>>>>>> not
>>>>>>>>
>>>>>>>>> representative of all Beam users. I think the runner communities
>>>>>>>>>>
>>>>>>>>> should
>>>>>>>
>>>>>>>> be
>>>>>>>>>
>>>>>>>>>> polled as well (though I suspect the answer will be the same,
>>>>>>>>>>
>>>>>>>>> that we
>>>>>
>>>>>> can
>>>>>>>>
>>>>>>>>> go ahead and deprecate Java 7 support).
>>>>>>>>>>
>>>>>>>>>> On Mon, Oct 16, 2017 at 4:50 PM, Eugene Kirpichov <
>>>>>>>>>> kirpichov@google.com.invalid> wrote:
>>>>>>>>>>
>>>>>>>>>> Yeah, a vote on user@ sounds like a good idea. Ismaël, would
>>>>>>>>>>>
>>>>>>>>>> you
>>>>>
>>>>>> be
>>>>>>>
>>>>>>>> interested in driving this process, since you're already
>>>>>>>>>>>
>>>>>>>>>> working on
>>>>>
>>>>>> Java9
>>>>>>>>>
>>>>>>>>>> support and hence you have a good understanding of what's
>>>>>>>>>>>
>>>>>>>>>> involved
>>>>>
>>>>>> in a
>>>>>>>>
>>>>>>>>> JDK
>>>>>>>>>>
>>>>>>>>>>> version migration for a large project?
>>>>>>>>>>>
>>>>>>>>>>> As due diligence, we can look at how the other data processing
>>>>>>>>>>>
>>>>>>>>>> systems
>>>>>>>>
>>>>>>>>> handled dropping Java7.
>>>>>>>>>>>
>>>>>>>>>>> Flink:
>>>>>>>>>>> JIRA https://issues.apache.org/jira/browse/FLINK-7242
>>>>>>>>>>> Discussion
>>>>>>>>>>> http://apache-flink-user-mailing-list-archive.2336050.
>>>>>>>>>>> n4.nabble.com/POLL-Who-still-uses-Java-7-with-Flink-
>>>>>>>>>>>
>>>>>>>>>> td12216.html
>>>>>
>>>>>>
>>>>>>>>>>> They also did a Twitter poll in addition to the mailing list
>>>>>>>>>>>
>>>>>>>>>> poll,
>>>>>
>>>>>> which
>>>>>>>>>
>>>>>>>>>> seems like a good idea.
>>>>>>>>>>> Note that Flink had a number of strong reasons for dropping
>>>>>>>>>>>
>>>>>>>>>> Java7
>>>>>
>>>>>> that
>>>>>>>>
>>>>>>>>> do
>>>>>>>>>
>>>>>>>>>> not apply in Beam.
>>>>>>>>>>>
>>>>>>>>>>> Spark:
>>>>>>>>>>> JIRA https://issues.apache.org/jira/browse/SPARK-19493
>>>>>>>>>>> Discussion
>>>>>>>>>>>
>>>>>>>>>>> http://apache-spark-developers-list.1001551.n3.
>>>>>>>>>>
>>>>>>>>> nabble.com/discuss-ending-
>>>>>>>>>
>>>>>>>>>> support-for-Java-7-in-Spark-2-0-td16808.html
>>>>>>>>>>> (I
>>>>>>>>>>> couldn't find a formal poll of the user list rather than
>>>>>>>>>>>
>>>>>>>>>> developer
>>>>>
>>>>>> list)
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Hadoop:
>>>>>>>>>>> Hadoop 3.0 is Java8-only, but I couldn't quickly find a
>>>>>>>>>>>
>>>>>>>>>> discussion
>>>>>
>>>>>> of
>>>>>>>
>>>>>>>> where
>>>>>>>>>>
>>>>>>>>>>> that decision was made.
>>>>>>>>>>> https://lists.apache.org/thread.html/e5c8085ada2cca47027b63f
>>>>>>>>>>>
>>>>>>>>>> 5439839
>>>>>>>
>>>>>>>> 731a392335770386e10895be06@1444091751@%3Cmapreduce-dev.
>>>>>>>>>>> hadoop.apache.org%3E
>>>>>>>>>>> might
>>>>>>>>>>> be it.
>>>>>>>>>>>
>>>>>>>>>>> Kafka is considering it:
>>>>>>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>>>>>>>>>>> 118%3A+Drop+Support+for+Java+7+in+Kafka+0.11
>>>>>>>>>>> and
>>>>>>>>>>> quotes a number of other open-source projects that have switched
>>>>>>>>>>> http://markmail.org/message/l7s276y3xkga2eqf
>>>>>>>>>>>
>>>>>>>>>>> So basically these projects all did a mailing list poll, and one
>>>>>>>>>>>
>>>>>>>>>> did
>>>>>>>
>>>>>>>> also a
>>>>>>>>>>
>>>>>>>>>>> twitter poll.
>>>>>>>>>>>
>>>>>>>>>>> Beam has the advantage of being a relatively young project with
>>>>>>>>>>>
>>>>>>>>>> perhaps a
>>>>>>>>>
>>>>>>>>>> smaller base of users entrenched in using old versions of Java;
>>>>>>>>>>>
>>>>>>>>>> moreover,
>>>>>>>>>
>>>>>>>>>> Java version would matter only for the smaller subset of users
>>>>>>>>>>>
>>>>>>>>>> who
>>>>>
>>>>>> use
>>>>>>>>
>>>>>>>>> Beam
>>>>>>>>>>
>>>>>>>>>>> Spark/Flink/Apex/.. runners (as opposed to Cloud Dataflow),
>>>>>>>>>>>
>>>>>>>>>> which
>>>>>
>>>>>> is
>>>>>>>
>>>>>>>> likely
>>>>>>>>>>
>>>>>>>>>>> an even more "early adopter"-ish group of users, as these
>>>>>>>>>>>
>>>>>>>>>> runners
>>>>>
>>>>>> generally
>>>>>>>>>>
>>>>>>>>>>> receive less support.
>>>>>>>>>>>
>>>>>>>>>>> It may be a good idea to have at least 1 release pass between
>>>>>>>>>>>
>>>>>>>>>> announcing
>>>>>>>>>
>>>>>>>>>> the intention to drop Java8 and actually dropping it (e.g. if we
>>>>>>>>>>>
>>>>>>>>>> decided
>>>>>>>>>
>>>>>>>>>> it
>>>>>>>>>>
>>>>>>>>>>> now, then 2.4 would drop Java7). Also, we could start by
>>>>>>>>>>>
>>>>>>>>>> switching
>>>>>
>>>>>> tests
>>>>>>>>>
>>>>>>>>>> to
>>>>>>>>>>
>>>>>>>>>>> compile/run with java8 (Maven allows this). This is, I think,
>>>>>>>>>>>
>>>>>>>>>> pretty
>>>>>>>
>>>>>>>> much
>>>>>>>>>
>>>>>>>>>> safe to do immediately.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Oct 16, 2017 at 7:35 AM Ismaël Mejía <iemejia@gmail.com
>>>>>>>>>>>
>>>>>>>>>>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Any progress on this? What is the proposed way to validate if
>>>>>>>>>>>>
>>>>>>>>>>> users
>>>>>>>
>>>>>>>> are still interested on Java 7? A vote on user or something
>>>>>>>>>>>>
>>>>>>>>>>> different?
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles
>>>>>>>>>>>>
>>>>>>>>>>> <klk@google.com.invalid
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Agree with polling Beam users as well as each runner's
>>>>>>>>>>>>>
>>>>>>>>>>>> community
>>>>>>>
>>>>>>>> in
>>>>>>>>
>>>>>>>>> aggregate.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré <
>>>>>>>>>>>>>
>>>>>>>>>>>> jb@nanthrax.net
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Definitely agree
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 09/27/2017 06:00 PM, Robert Bradshaw wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I also think that it's time to seriously consider dropping
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> support
>>>>>>>>>
>>>>>>>>>> for
>>>>>>>>>>>
>>>>>>>>>>>> Java 7.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
>>>>>>>>>>>>>>> <da...@google.com.invalid> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yes, just as Ismaël said it's a compilation blocker right
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> now
>>>>>>>
>>>>>>>> despite
>>>>>>>>>>>
>>>>>>>>>>>> that
>>>>>>>>>>>>>>>> (I believe) we don't use the extension that's breaking.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> As for other ways to solve this, if there is a way to
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> avoid
>>>>>
>>>>>> compiling
>>>>>>>>>>>
>>>>>>>>>>>> the
>>>>>>>>>>>>
>>>>>>>>>>>>> advanced features of AutoValue that might be worth a
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> try. We
>>>>>
>>>>>> could
>>>>>>>>>
>>>>>>>>>> also
>>>>>>>>>>>>
>>>>>>>>>>>>> try
>>>>>>>>>>>>>>>> to get a release of AutoValue with the fix that works in
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Java
>>>>>>>
>>>>>>>> 7.
>>>>>>>>
>>>>>>>>> However
>>>>>>>>>>>>
>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>> feel that slowly moving over to Java 8 is the most
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> future-proof
>>>>>>>>
>>>>>>>>> solution
>>>>>>>>>>>>
>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>> it's possible.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> iemejia@gmail.com>
>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The current issue is that compilation fails on master
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> because
>>>>>>>
>>>>>>>> beam's
>>>>>>>>>>
>>>>>>>>>>> parent pom is configured to fail if it finds warnings):
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> <compiler.error.flag>-Werror</compiler.error.flag>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> However if you remove that line from the parent pom the
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> compilation
>>>>>>>>>>
>>>>>>>>>>> passes.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Of course this does not mean that everything is solved
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> for
>>>>>
>>>>>> Java
>>>>>>>>
>>>>>>>>> 9,
>>>>>>>>>>
>>>>>>>>>>> there are some tests that break and other issues
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> because of
>>>>>
>>>>>> other
>>>>>>>>>
>>>>>>>>>> plugins and dependencies (e.g. bytebuddy), but those are
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> not
>>>>>>>
>>>>>>>> part
>>>>>>>>>
>>>>>>>>>> of
>>>>>>>>>>>
>>>>>>>>>>>> this discussion.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
>>>>>>>>>>>>>>>>> <ki...@google.com.invalid> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> AFAIK we don't use any advanced capabilities of
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> AutoValue.
>>>>>
>>>>>> Does
>>>>>>>>>
>>>>>>>>>> that
>>>>>>>>>>>
>>>>>>>>>>>> mean
>>>>>>>>>>>>>>>>>> this issue is moot? I didn't quite understand from your
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> email
>>>>>>>>
>>>>>>>>> whether
>>>>>>>>>>>>
>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> a compilation blocker for Beam or not.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> iemejia@gmail.com
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Great that you are also working on this too Daniel and
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> thanks
>>>>>>>>
>>>>>>>>> for
>>>>>>>>>>
>>>>>>>>>>> bringing this subject to the mailing list, I was
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> waiting
>>>>>
>>>>>> to
>>>>>>>
>>>>>>>> my
>>>>>>>>>
>>>>>>>>>> return
>>>>>>>>>>>>
>>>>>>>>>>>>> to office next week, but you did it first :)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Eugene for reference (This is the issue on the
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> migration
>>>>>
>>>>>> to
>>>>>>>
>>>>>>>> Java
>>>>>>>>>>
>>>>>>>>>>> 9),
>>>>>>>>>>>>
>>>>>>>>>>>>> notice that here the goal is first that beam passes
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> mvn
>>>>>
>>>>>> clean
>>>>>>>>
>>>>>>>>> install
>>>>>>>>>>>>
>>>>>>>>>>>>> with pure Java 9 (and also add this to jenkins), not
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> to
>>>>>
>>>>>> rewrite
>>>>>>>>>
>>>>>>>>>> anything to use the new stuff (e.g. modules):
>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/BEAM-2530
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Eugene can you also PTAL at the AutoValue issue, more
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> details
>>>>>>>>
>>>>>>>>> on
>>>>>>>>>>
>>>>>>>>>>> the
>>>>>>>>>>>>
>>>>>>>>>>>>> issue, this is a warning so I don't know if it is
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> really
>>>>>
>>>>>> critical
>>>>>>>>>>
>>>>>>>>>>> in
>>>>>>>>>>>>
>>>>>>>>>>>>> particular because we are not using Memoization (do
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> we?).
>>>>>
>>>>>> https://github.com/google/auto/issues/503
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>> https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7
>>>>>>>> f0a25f5d
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 02247b8532
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Wouldn't the easiest way be that you guys convince the
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> google.auto
>>>>>>>>>>>
>>>>>>>>>>>> guys to generate that simple fix in a Java 7
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> compatible
>>>>>
>>>>>> way
>>>>>>>
>>>>>>>> and
>>>>>>>>>
>>>>>>>>>> 'voila' ?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> However I agree that moving to Java 8 is an excellent
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> idea
>>>>>>>
>>>>>>>> and
>>>>>>>>>
>>>>>>>>>> as
>>>>>>>>>>
>>>>>>>>>>> Eugene mentions there is less friction now since most
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> projects
>>>>>>>>>
>>>>>>>>>> are
>>>>>>>>>>>
>>>>>>>>>>>> moving, only pending issue are existing clusters on
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> java
>>>>>
>>>>>> 7
>>>>>>>
>>>>>>>> in
>>>>>>>>
>>>>>>>>> the
>>>>>>>>>>
>>>>>>>>>>> hadoop world, but those are less frequent now. Anyway
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> this
>>>>>>>
>>>>>>>> discussion
>>>>>>>>>>>>
>>>>>>>>>>>>> is really important (maybe even worth a vote). Because
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> moving
>>>>>>>>
>>>>>>>>> to
>>>>>>>>>>
>>>>>>>>>>> Java
>>>>>>>>>>>>
>>>>>>>>>>>>> 8 will allow us also to move some of the dependencies
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> that
>>>>>>>
>>>>>>>> we
>>>>>>>>
>>>>>>>>> are
>>>>>>>>>>
>>>>>>>>>>> keeping in old versions and in general to move
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> forward.
>>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>> What do the others think ?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
>>>>>>>>>>>>>>>>>>> <ki...@google.com.invalid> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Very excited to hear that there's work on JDK9
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> support -
>>>>>
>>>>>> is
>>>>>>>>
>>>>>>>>> there
>>>>>>>>>>>
>>>>>>>>>>>> a
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> public
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> description of the plans for this work somewhere?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> In general, Beam could probably drop Java7 support
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> altogether
>>>>>>>>>
>>>>>>>>>> at
>>>>>>>>>>
>>>>>>>>>>> some
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> point
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> receiving
>>>>>>>>
>>>>>>>>> even
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> security
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> patches) 2 years ago, and all major players in the
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> data
>>>>>
>>>>>> processing
>>>>>>>>>>>
>>>>>>>>>>>> ecosystem have dropped Java7 support (Spark, Flink,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hadoop),
>>>>>>>>
>>>>>>>>> so
>>>>>>>>>>
>>>>>>>>>>> I
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> presume
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> the demand for Java7 support in the data processing
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> industry
>>>>>>>
>>>>>>>> is
>>>>>>>>>
>>>>>>>>>> low.
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> By
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> way: would a Java8 migration be in the scope of your
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> work
>>>>>>>
>>>>>>>> in
>>>>>>>>
>>>>>>>>> general?
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> However, until we say that Beam requires Java8, what
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> would
>>>>>>>
>>>>>>>> be
>>>>>>>>>
>>>>>>>>>> the
>>>>>>>>>>>
>>>>>>>>>>>> implications of using a version of AutoValue that can
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> only
>>>>>>>
>>>>>>>> be
>>>>>>>>>
>>>>>>>>>> compiled
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Java8? Are you saying that this is simply a matter
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> of a
>>>>>
>>>>>> compiler
>>>>>>>>>>
>>>>>>>>>>> bug,
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> if we use a Java8 compiler but configured to use source
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> and
>>>>>>>
>>>>>>>> target
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> versions
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> then
>>>>>
>>>>>> the
>>>>>>>>
>>>>>>>>> resulting
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Beam
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> artifacts will be usable by people who don't have
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Java8?
>>>>>
>>>>>>
>>>>>>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
>>>>>>>>>>>>>>>>>>>> <da...@google.com.invalid> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> So I've been working on JDK 9 support for Beam, and I
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> have a
>>>>>>>>
>>>>>>>>> bug
>>>>>>>>>>
>>>>>>>>>>> in
>>>>>>>>>>>>
>>>>>>>>>>>>> AutoValue that can be fixed by updating our
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> AutoValue
>>>>>
>>>>>> dependency
>>>>>>>>>>>
>>>>>>>>>>>> to
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> latest. The problem is that AutoValue from 1.5+ seems
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> to
>>>>
>>>>
>>
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Kenneth Knowles <kl...@google.com.INVALID>.
It is a good point that this is a breaking change for users of Java 7.
Assuming we wait until 3.0.0 to drop Java 7 support, and that we wait a
while to do a major version bump, I support deprecating support now for
sure.
If we had a very strong signal that we would not really break anyone, I
would be open to a minor version that kept API compatibility while moving
our build to Java 8, even though it technically violates semantic
versioning. We don't have to be dogmatic - acceptable backwards
incompatibility occurs all the time in the form of bugfixes (the new
behavior is by definition different) or security-related updates.
Also worth remembering that the portability framework can separate runner /
SDK Java generation just as though they are separate languages.
On Tue, Oct 17, 2017 at 9:51 AM, Reuven Lax <re...@google.com.invalid>
wrote:
> So maybe what we should do now is deprecate Java 7 support but not drop it
> yet. I believe this is also what Spark has done.
>
> On Tue, Oct 17, 2017 at 9:46 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
> > Agree, I would target this for Beam 3.0.0.
> >
> > Regards
> > JB
> >
> > On 10/17/2017 06:43 PM, Reuven Lax wrote:
> >
> >> Should this be considered a backwards-incompatible change? If so, do we
> >> need to wait until Beam 3.0.0 is released?
> >>
> >> On Tue, Oct 17, 2017 at 9:11 AM, Ismaël Mejía <ie...@gmail.com>
> wrote:
> >>
> >> I am bringing the subject to the user mailing list to get some
> >>> feedback because it makes sense anyway to discuss this there. But I
> >>> also agree with Kenneth about the fact that runner authors must weight
> >>> in on this subject.
> >>>
> >>>
> >>> On Tue, Oct 17, 2017 at 5:24 PM, Kenneth Knowles
> <klk@google.com.invalid
> >>> >
> >>> wrote:
> >>>
> >>>> +1 to having runner maintainers weigh in as proxies. Added a few in
> case
> >>>> they haven't followed this thread.
> >>>>
> >>>> On Mon, Oct 16, 2017 at 11:38 PM, Eugene Kirpichov <
> >>>> kirpichov@google.com.invalid> wrote:
> >>>>
> >>>> Agreed that polling Dataflow users makes sense, though I think they
> are
> >>>>> very unlikely to have concerns, because unlike Spark/Flink users they
> >>>>> wouldn't have a "cluster" that they need to migrate to a new JVM:
> >>>>> they'd
> >>>>> only need to recompile their pipelines with JDK 8.
> >>>>>
> >>>>> On Mon, Oct 16, 2017 at 11:21 PM Reuven Lax <relax@google.com.invalid
> >
> >>>>> wrote:
> >>>>>
> >>>>> I think the Flink and Spark runner maintainers can weigh in here;
> >>>>>>
> >>>>> given
> >>>
> >>>> that both of those systems are moving to Java 8, I doubt they will
> >>>>>>
> >>>>> have
> >>>
> >>>> concerns. Same is true for the Dataflow runner - we should probably
> >>>>>>
> >>>>> poll
> >>>
> >>>> Dataflow users to make sure this is not a problem for any of them.
> >>>>>>
> >>>>>> On Mon, Oct 16, 2017 at 11:05 PM, Eugene Kirpichov <
> >>>>>> kirpichov@google.com.invalid> wrote:
> >>>>>>
> >>>>>> Reuven - do you mean e.g. a poll on the Flink mailing list asking
> >>>>>>>
> >>>>>> whether
> >>>>>
> >>>>>> there are Flink users who use Beam with Java 7? Or just people who
> >>>>>>>
> >>>>>> use
> >>>
> >>>> Flink with Java 7? (the latter question I'd assume was settled by
> >>>>>>>
> >>>>>> the
> >>>
> >>>> poll
> >>>>>>
> >>>>>>> about making Flink itself Java8-only?)
> >>>>>>>
> >>>>>>> On Mon, Oct 16, 2017 at 10:32 PM Reuven Lax
> >>>>>>>
> >>>>>> <re...@google.com.invalid>
> >>>
> >>>> wrote:
> >>>>>>>
> >>>>>>> I don't know if a vote in @user is sufficient, as it's
> >>>>>>>>
> >>>>>>> unfortunately
> >>>
> >>>> not
> >>>>>>
> >>>>>>> representative of all Beam users. I think the runner communities
> >>>>>>>>
> >>>>>>> should
> >>>>>
> >>>>>> be
> >>>>>>>
> >>>>>>>> polled as well (though I suspect the answer will be the same,
> >>>>>>>>
> >>>>>>> that we
> >>>
> >>>> can
> >>>>>>
> >>>>>>> go ahead and deprecate Java 7 support).
> >>>>>>>>
> >>>>>>>> On Mon, Oct 16, 2017 at 4:50 PM, Eugene Kirpichov <
> >>>>>>>> kirpichov@google.com.invalid> wrote:
> >>>>>>>>
> >>>>>>>> Yeah, a vote on user@ sounds like a good idea. Ismaël, would
> >>>>>>>>>
> >>>>>>>> you
> >>>
> >>>> be
> >>>>>
> >>>>>> interested in driving this process, since you're already
> >>>>>>>>>
> >>>>>>>> working on
> >>>
> >>>> Java9
> >>>>>>>
> >>>>>>>> support and hence you have a good understanding of what's
> >>>>>>>>>
> >>>>>>>> involved
> >>>
> >>>> in a
> >>>>>>
> >>>>>>> JDK
> >>>>>>>>
> >>>>>>>>> version migration for a large project?
> >>>>>>>>>
> >>>>>>>>> As due diligence, we can look at how the other data processing
> >>>>>>>>>
> >>>>>>>> systems
> >>>>>>
> >>>>>>> handled dropping Java7.
> >>>>>>>>>
> >>>>>>>>> Flink:
> >>>>>>>>> JIRA https://issues.apache.org/jira/browse/FLINK-7242
> >>>>>>>>> Discussion
> >>>>>>>>> http://apache-flink-user-mailing-list-archive.2336050.
> >>>>>>>>> n4.nabble.com/POLL-Who-still-uses-Java-7-with-Flink-
> >>>>>>>>>
> >>>>>>>> td12216.html
> >>>
> >>>>
> >>>>>>>>> They also did a Twitter poll in addition to the mailing list
> >>>>>>>>>
> >>>>>>>> poll,
> >>>
> >>>> which
> >>>>>>>
> >>>>>>>> seems like a good idea.
> >>>>>>>>> Note that Flink had a number of strong reasons for dropping
> >>>>>>>>>
> >>>>>>>> Java7
> >>>
> >>>> that
> >>>>>>
> >>>>>>> do
> >>>>>>>
> >>>>>>>> not apply in Beam.
> >>>>>>>>>
> >>>>>>>>> Spark:
> >>>>>>>>> JIRA https://issues.apache.org/jira/browse/SPARK-19493
> >>>>>>>>> Discussion
> >>>>>>>>>
> >>>>>>>>> http://apache-spark-developers-list.1001551.n3.
> >>>>>>>>
> >>>>>>> nabble.com/discuss-ending-
> >>>>>>>
> >>>>>>>> support-for-Java-7-in-Spark-2-0-td16808.html
> >>>>>>>>> (I
> >>>>>>>>> couldn't find a formal poll of the user list rather than
> >>>>>>>>>
> >>>>>>>> developer
> >>>
> >>>> list)
> >>>>>>>
> >>>>>>>>
> >>>>>>>>> Hadoop:
> >>>>>>>>> Hadoop 3.0 is Java8-only, but I couldn't quickly find a
> >>>>>>>>>
> >>>>>>>> discussion
> >>>
> >>>> of
> >>>>>
> >>>>>> where
> >>>>>>>>
> >>>>>>>>> that decision was made.
> >>>>>>>>> https://lists.apache.org/thread.html/e5c8085ada2cca47027b63f
> >>>>>>>>>
> >>>>>>>> 5439839
> >>>>>
> >>>>>> 731a392335770386e10895be06@1444091751@%3Cmapreduce-dev.
> >>>>>>>>> hadoop.apache.org%3E
> >>>>>>>>> might
> >>>>>>>>> be it.
> >>>>>>>>>
> >>>>>>>>> Kafka is considering it:
> >>>>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>>>>>>>> 118%3A+Drop+Support+for+Java+7+in+Kafka+0.11
> >>>>>>>>> and
> >>>>>>>>> quotes a number of other open-source projects that have switched
> >>>>>>>>> http://markmail.org/message/l7s276y3xkga2eqf
> >>>>>>>>>
> >>>>>>>>> So basically these projects all did a mailing list poll, and one
> >>>>>>>>>
> >>>>>>>> did
> >>>>>
> >>>>>> also a
> >>>>>>>>
> >>>>>>>>> twitter poll.
> >>>>>>>>>
> >>>>>>>>> Beam has the advantage of being a relatively young project with
> >>>>>>>>>
> >>>>>>>> perhaps a
> >>>>>>>
> >>>>>>>> smaller base of users entrenched in using old versions of Java;
> >>>>>>>>>
> >>>>>>>> moreover,
> >>>>>>>
> >>>>>>>> Java version would matter only for the smaller subset of users
> >>>>>>>>>
> >>>>>>>> who
> >>>
> >>>> use
> >>>>>>
> >>>>>>> Beam
> >>>>>>>>
> >>>>>>>>> Spark/Flink/Apex/.. runners (as opposed to Cloud Dataflow),
> >>>>>>>>>
> >>>>>>>> which
> >>>
> >>>> is
> >>>>>
> >>>>>> likely
> >>>>>>>>
> >>>>>>>>> an even more "early adopter"-ish group of users, as these
> >>>>>>>>>
> >>>>>>>> runners
> >>>
> >>>> generally
> >>>>>>>>
> >>>>>>>>> receive less support.
> >>>>>>>>>
> >>>>>>>>> It may be a good idea to have at least 1 release pass between
> >>>>>>>>>
> >>>>>>>> announcing
> >>>>>>>
> >>>>>>>> the intention to drop Java8 and actually dropping it (e.g. if we
> >>>>>>>>>
> >>>>>>>> decided
> >>>>>>>
> >>>>>>>> it
> >>>>>>>>
> >>>>>>>>> now, then 2.4 would drop Java7). Also, we could start by
> >>>>>>>>>
> >>>>>>>> switching
> >>>
> >>>> tests
> >>>>>>>
> >>>>>>>> to
> >>>>>>>>
> >>>>>>>>> compile/run with java8 (Maven allows this). This is, I think,
> >>>>>>>>>
> >>>>>>>> pretty
> >>>>>
> >>>>>> much
> >>>>>>>
> >>>>>>>> safe to do immediately.
> >>>>>>>>>
> >>>>>>>>> On Mon, Oct 16, 2017 at 7:35 AM Ismaël Mejía <iemejia@gmail.com
> >>>>>>>>>
> >>>>>>>>
> >>>> wrote:
> >>>>>>>
> >>>>>>>>
> >>>>>>>>> Any progress on this? What is the proposed way to validate if
> >>>>>>>>>>
> >>>>>>>>> users
> >>>>>
> >>>>>> are still interested on Java 7? A vote on user or something
> >>>>>>>>>>
> >>>>>>>>> different?
> >>>>>>>
> >>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles
> >>>>>>>>>>
> >>>>>>>>> <klk@google.com.invalid
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Agree with polling Beam users as well as each runner's
> >>>>>>>>>>>
> >>>>>>>>>> community
> >>>>>
> >>>>>> in
> >>>>>>
> >>>>>>> aggregate.
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré <
> >>>>>>>>>>>
> >>>>>>>>>> jb@nanthrax.net
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Definitely agree
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 09/27/2017 06:00 PM, Robert Bradshaw wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> I also think that it's time to seriously consider dropping
> >>>>>>>>>>>>>
> >>>>>>>>>>>> support
> >>>>>>>
> >>>>>>>> for
> >>>>>>>>>
> >>>>>>>>>> Java 7.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
> >>>>>>>>>>>>> <da...@google.com.invalid> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Yes, just as Ismaël said it's a compilation blocker right
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> now
> >>>>>
> >>>>>> despite
> >>>>>>>>>
> >>>>>>>>>> that
> >>>>>>>>>>>>>> (I believe) we don't use the extension that's breaking.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> As for other ways to solve this, if there is a way to
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> avoid
> >>>
> >>>> compiling
> >>>>>>>>>
> >>>>>>>>>> the
> >>>>>>>>>>
> >>>>>>>>>>> advanced features of AutoValue that might be worth a
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> try. We
> >>>
> >>>> could
> >>>>>>>
> >>>>>>>> also
> >>>>>>>>>>
> >>>>>>>>>>> try
> >>>>>>>>>>>>>> to get a release of AutoValue with the fix that works in
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> Java
> >>>>>
> >>>>>> 7.
> >>>>>>
> >>>>>>> However
> >>>>>>>>>>
> >>>>>>>>>>> I
> >>>>>>>>>>>>>> feel that slowly moving over to Java 8 is the most
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> future-proof
> >>>>>>
> >>>>>>> solution
> >>>>>>>>>>
> >>>>>>>>>>> if
> >>>>>>>>>>>>>> it's possible.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> iemejia@gmail.com>
> >>>>>>>
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>>> The current issue is that compilation fails on master
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> because
> >>>>>
> >>>>>> beam's
> >>>>>>>>
> >>>>>>>>> parent pom is configured to fail if it finds warnings):
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> <compiler.error.flag>-Werror</compiler.error.flag>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> However if you remove that line from the parent pom the
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> compilation
> >>>>>>>>
> >>>>>>>>> passes.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Of course this does not mean that everything is solved
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> for
> >>>
> >>>> Java
> >>>>>>
> >>>>>>> 9,
> >>>>>>>>
> >>>>>>>>> there are some tests that break and other issues
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> because of
> >>>
> >>>> other
> >>>>>>>
> >>>>>>>> plugins and dependencies (e.g. bytebuddy), but those are
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> not
> >>>>>
> >>>>>> part
> >>>>>>>
> >>>>>>>> of
> >>>>>>>>>
> >>>>>>>>>> this discussion.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
> >>>>>>>>>>>>>>> <ki...@google.com.invalid> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> AFAIK we don't use any advanced capabilities of
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> AutoValue.
> >>>
> >>>> Does
> >>>>>>>
> >>>>>>>> that
> >>>>>>>>>
> >>>>>>>>>> mean
> >>>>>>>>>>>>>>>> this issue is moot? I didn't quite understand from your
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> email
> >>>>>>
> >>>>>>> whether
> >>>>>>>>>>
> >>>>>>>>>>> it
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> a compilation blocker for Beam or not.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> iemejia@gmail.com
> >>>>>>>
> >>>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Great that you are also working on this too Daniel and
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> thanks
> >>>>>>
> >>>>>>> for
> >>>>>>>>
> >>>>>>>>> bringing this subject to the mailing list, I was
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> waiting
> >>>
> >>>> to
> >>>>>
> >>>>>> my
> >>>>>>>
> >>>>>>>> return
> >>>>>>>>>>
> >>>>>>>>>>> to office next week, but you did it first :)
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Eugene for reference (This is the issue on the
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> migration
> >>>
> >>>> to
> >>>>>
> >>>>>> Java
> >>>>>>>>
> >>>>>>>>> 9),
> >>>>>>>>>>
> >>>>>>>>>>> notice that here the goal is first that beam passes
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> mvn
> >>>
> >>>> clean
> >>>>>>
> >>>>>>> install
> >>>>>>>>>>
> >>>>>>>>>>> with pure Java 9 (and also add this to jenkins), not
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> to
> >>>
> >>>> rewrite
> >>>>>>>
> >>>>>>>> anything to use the new stuff (e.g. modules):
> >>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/BEAM-2530
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Eugene can you also PTAL at the AutoValue issue, more
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> details
> >>>>>>
> >>>>>>> on
> >>>>>>>>
> >>>>>>>>> the
> >>>>>>>>>>
> >>>>>>>>>>> issue, this is a warning so I don't know if it is
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> really
> >>>
> >>>> critical
> >>>>>>>>
> >>>>>>>>> in
> >>>>>>>>>>
> >>>>>>>>>>> particular because we are not using Memoization (do
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> we?).
> >>>
> >>>> https://github.com/google/auto/issues/503
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>> https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7
> >>>>>> f0a25f5d
> >>>>>>
> >>>>>>>
> >>>>>>>>>>>>>>>>> 02247b8532
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Wouldn't the easiest way be that you guys convince the
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> google.auto
> >>>>>>>>>
> >>>>>>>>>> guys to generate that simple fix in a Java 7
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> compatible
> >>>
> >>>> way
> >>>>>
> >>>>>> and
> >>>>>>>
> >>>>>>>> 'voila' ?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> However I agree that moving to Java 8 is an excellent
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> idea
> >>>>>
> >>>>>> and
> >>>>>>>
> >>>>>>>> as
> >>>>>>>>
> >>>>>>>>> Eugene mentions there is less friction now since most
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> projects
> >>>>>>>
> >>>>>>>> are
> >>>>>>>>>
> >>>>>>>>>> moving, only pending issue are existing clusters on
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> java
> >>>
> >>>> 7
> >>>>>
> >>>>>> in
> >>>>>>
> >>>>>>> the
> >>>>>>>>
> >>>>>>>>> hadoop world, but those are less frequent now. Anyway
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> this
> >>>>>
> >>>>>> discussion
> >>>>>>>>>>
> >>>>>>>>>>> is really important (maybe even worth a vote). Because
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> moving
> >>>>>>
> >>>>>>> to
> >>>>>>>>
> >>>>>>>>> Java
> >>>>>>>>>>
> >>>>>>>>>>> 8 will allow us also to move some of the dependencies
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> that
> >>>>>
> >>>>>> we
> >>>>>>
> >>>>>>> are
> >>>>>>>>
> >>>>>>>>> keeping in old versions and in general to move
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> forward.
> >>>
> >>>>
> >>>>>>>>>>>>>>>>> What do the others think ?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
> >>>>>>>>>>>>>>>>> <ki...@google.com.invalid> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Very excited to hear that there's work on JDK9
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> support -
> >>>
> >>>> is
> >>>>>>
> >>>>>>> there
> >>>>>>>>>
> >>>>>>>>>> a
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> public
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> description of the plans for this work somewhere?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> In general, Beam could probably drop Java7 support
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> altogether
> >>>>>>>
> >>>>>>>> at
> >>>>>>>>
> >>>>>>>>> some
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> point
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> receiving
> >>>>>>
> >>>>>>> even
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>>>>>>>> security
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> patches) 2 years ago, and all major players in the
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> data
> >>>
> >>>> processing
> >>>>>>>>>
> >>>>>>>>>> ecosystem have dropped Java7 support (Spark, Flink,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hadoop),
> >>>>>>
> >>>>>>> so
> >>>>>>>>
> >>>>>>>>> I
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>>>>>>>> presume
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> the demand for Java7 support in the data processing
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> industry
> >>>>>
> >>>>>> is
> >>>>>>>
> >>>>>>>> low.
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>>>>>>>> By
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> way: would a Java8 migration be in the scope of your
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> work
> >>>>>
> >>>>>> in
> >>>>>>
> >>>>>>> general?
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> However, until we say that Beam requires Java8, what
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> would
> >>>>>
> >>>>>> be
> >>>>>>>
> >>>>>>>> the
> >>>>>>>>>
> >>>>>>>>>> implications of using a version of AutoValue that can
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> only
> >>>>>
> >>>>>> be
> >>>>>>>
> >>>>>>>> compiled
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Java8? Are you saying that this is simply a matter
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> of a
> >>>
> >>>> compiler
> >>>>>>>>
> >>>>>>>>> bug,
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> if we use a Java8 compiler but configured to use source
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> and
> >>>>>
> >>>>>> target
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>>>>>>>>> versions
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> then
> >>>
> >>>> the
> >>>>>>
> >>>>>>> resulting
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Beam
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> artifacts will be usable by people who don't have
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Java8?
> >>>
> >>>>
> >>>>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
> >>>>>>>>>>>>>>>>>> <da...@google.com.invalid> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> So I've been working on JDK 9 support for Beam, and I
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> have a
> >>>>>>
> >>>>>>> bug
> >>>>>>>>
> >>>>>>>>> in
> >>>>>>>>>>
> >>>>>>>>>>> AutoValue that can be fixed by updating our
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> AutoValue
> >>>
> >>>> dependency
> >>>>>>>>>
> >>>>>>>>>> to
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> latest. The problem is that AutoValue from 1.5+ seems
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> to
> >>
> >>
>
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Reuven Lax <re...@google.com.INVALID>.
So maybe what we should do now is deprecate Java 7 support but not drop it
yet. I believe this is also what Spark has done.
On Tue, Oct 17, 2017 at 9:46 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Agree, I would target this for Beam 3.0.0.
>
> Regards
> JB
>
> On 10/17/2017 06:43 PM, Reuven Lax wrote:
>
>> Should this be considered a backwards-incompatible change? If so, do we
>> need to wait until Beam 3.0.0 is released?
>>
>> On Tue, Oct 17, 2017 at 9:11 AM, Ismaël Mejía <ie...@gmail.com> wrote:
>>
>> I am bringing the subject to the user mailing list to get some
>>> feedback because it makes sense anyway to discuss this there. But I
>>> also agree with Kenneth about the fact that runner authors must weight
>>> in on this subject.
>>>
>>>
>>> On Tue, Oct 17, 2017 at 5:24 PM, Kenneth Knowles <klk@google.com.invalid
>>> >
>>> wrote:
>>>
>>>> +1 to having runner maintainers weigh in as proxies. Added a few in case
>>>> they haven't followed this thread.
>>>>
>>>> On Mon, Oct 16, 2017 at 11:38 PM, Eugene Kirpichov <
>>>> kirpichov@google.com.invalid> wrote:
>>>>
>>>> Agreed that polling Dataflow users makes sense, though I think they are
>>>>> very unlikely to have concerns, because unlike Spark/Flink users they
>>>>> wouldn't have a "cluster" that they need to migrate to a new JVM:
>>>>> they'd
>>>>> only need to recompile their pipelines with JDK 8.
>>>>>
>>>>> On Mon, Oct 16, 2017 at 11:21 PM Reuven Lax <re...@google.com.invalid>
>>>>> wrote:
>>>>>
>>>>> I think the Flink and Spark runner maintainers can weigh in here;
>>>>>>
>>>>> given
>>>
>>>> that both of those systems are moving to Java 8, I doubt they will
>>>>>>
>>>>> have
>>>
>>>> concerns. Same is true for the Dataflow runner - we should probably
>>>>>>
>>>>> poll
>>>
>>>> Dataflow users to make sure this is not a problem for any of them.
>>>>>>
>>>>>> On Mon, Oct 16, 2017 at 11:05 PM, Eugene Kirpichov <
>>>>>> kirpichov@google.com.invalid> wrote:
>>>>>>
>>>>>> Reuven - do you mean e.g. a poll on the Flink mailing list asking
>>>>>>>
>>>>>> whether
>>>>>
>>>>>> there are Flink users who use Beam with Java 7? Or just people who
>>>>>>>
>>>>>> use
>>>
>>>> Flink with Java 7? (the latter question I'd assume was settled by
>>>>>>>
>>>>>> the
>>>
>>>> poll
>>>>>>
>>>>>>> about making Flink itself Java8-only?)
>>>>>>>
>>>>>>> On Mon, Oct 16, 2017 at 10:32 PM Reuven Lax
>>>>>>>
>>>>>> <re...@google.com.invalid>
>>>
>>>> wrote:
>>>>>>>
>>>>>>> I don't know if a vote in @user is sufficient, as it's
>>>>>>>>
>>>>>>> unfortunately
>>>
>>>> not
>>>>>>
>>>>>>> representative of all Beam users. I think the runner communities
>>>>>>>>
>>>>>>> should
>>>>>
>>>>>> be
>>>>>>>
>>>>>>>> polled as well (though I suspect the answer will be the same,
>>>>>>>>
>>>>>>> that we
>>>
>>>> can
>>>>>>
>>>>>>> go ahead and deprecate Java 7 support).
>>>>>>>>
>>>>>>>> On Mon, Oct 16, 2017 at 4:50 PM, Eugene Kirpichov <
>>>>>>>> kirpichov@google.com.invalid> wrote:
>>>>>>>>
>>>>>>>> Yeah, a vote on user@ sounds like a good idea. Ismaël, would
>>>>>>>>>
>>>>>>>> you
>>>
>>>> be
>>>>>
>>>>>> interested in driving this process, since you're already
>>>>>>>>>
>>>>>>>> working on
>>>
>>>> Java9
>>>>>>>
>>>>>>>> support and hence you have a good understanding of what's
>>>>>>>>>
>>>>>>>> involved
>>>
>>>> in a
>>>>>>
>>>>>>> JDK
>>>>>>>>
>>>>>>>>> version migration for a large project?
>>>>>>>>>
>>>>>>>>> As due diligence, we can look at how the other data processing
>>>>>>>>>
>>>>>>>> systems
>>>>>>
>>>>>>> handled dropping Java7.
>>>>>>>>>
>>>>>>>>> Flink:
>>>>>>>>> JIRA https://issues.apache.org/jira/browse/FLINK-7242
>>>>>>>>> Discussion
>>>>>>>>> http://apache-flink-user-mailing-list-archive.2336050.
>>>>>>>>> n4.nabble.com/POLL-Who-still-uses-Java-7-with-Flink-
>>>>>>>>>
>>>>>>>> td12216.html
>>>
>>>>
>>>>>>>>> They also did a Twitter poll in addition to the mailing list
>>>>>>>>>
>>>>>>>> poll,
>>>
>>>> which
>>>>>>>
>>>>>>>> seems like a good idea.
>>>>>>>>> Note that Flink had a number of strong reasons for dropping
>>>>>>>>>
>>>>>>>> Java7
>>>
>>>> that
>>>>>>
>>>>>>> do
>>>>>>>
>>>>>>>> not apply in Beam.
>>>>>>>>>
>>>>>>>>> Spark:
>>>>>>>>> JIRA https://issues.apache.org/jira/browse/SPARK-19493
>>>>>>>>> Discussion
>>>>>>>>>
>>>>>>>>> http://apache-spark-developers-list.1001551.n3.
>>>>>>>>
>>>>>>> nabble.com/discuss-ending-
>>>>>>>
>>>>>>>> support-for-Java-7-in-Spark-2-0-td16808.html
>>>>>>>>> (I
>>>>>>>>> couldn't find a formal poll of the user list rather than
>>>>>>>>>
>>>>>>>> developer
>>>
>>>> list)
>>>>>>>
>>>>>>>>
>>>>>>>>> Hadoop:
>>>>>>>>> Hadoop 3.0 is Java8-only, but I couldn't quickly find a
>>>>>>>>>
>>>>>>>> discussion
>>>
>>>> of
>>>>>
>>>>>> where
>>>>>>>>
>>>>>>>>> that decision was made.
>>>>>>>>> https://lists.apache.org/thread.html/e5c8085ada2cca47027b63f
>>>>>>>>>
>>>>>>>> 5439839
>>>>>
>>>>>> 731a392335770386e10895be06@1444091751@%3Cmapreduce-dev.
>>>>>>>>> hadoop.apache.org%3E
>>>>>>>>> might
>>>>>>>>> be it.
>>>>>>>>>
>>>>>>>>> Kafka is considering it:
>>>>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>>>>>>>>> 118%3A+Drop+Support+for+Java+7+in+Kafka+0.11
>>>>>>>>> and
>>>>>>>>> quotes a number of other open-source projects that have switched
>>>>>>>>> http://markmail.org/message/l7s276y3xkga2eqf
>>>>>>>>>
>>>>>>>>> So basically these projects all did a mailing list poll, and one
>>>>>>>>>
>>>>>>>> did
>>>>>
>>>>>> also a
>>>>>>>>
>>>>>>>>> twitter poll.
>>>>>>>>>
>>>>>>>>> Beam has the advantage of being a relatively young project with
>>>>>>>>>
>>>>>>>> perhaps a
>>>>>>>
>>>>>>>> smaller base of users entrenched in using old versions of Java;
>>>>>>>>>
>>>>>>>> moreover,
>>>>>>>
>>>>>>>> Java version would matter only for the smaller subset of users
>>>>>>>>>
>>>>>>>> who
>>>
>>>> use
>>>>>>
>>>>>>> Beam
>>>>>>>>
>>>>>>>>> Spark/Flink/Apex/.. runners (as opposed to Cloud Dataflow),
>>>>>>>>>
>>>>>>>> which
>>>
>>>> is
>>>>>
>>>>>> likely
>>>>>>>>
>>>>>>>>> an even more "early adopter"-ish group of users, as these
>>>>>>>>>
>>>>>>>> runners
>>>
>>>> generally
>>>>>>>>
>>>>>>>>> receive less support.
>>>>>>>>>
>>>>>>>>> It may be a good idea to have at least 1 release pass between
>>>>>>>>>
>>>>>>>> announcing
>>>>>>>
>>>>>>>> the intention to drop Java8 and actually dropping it (e.g. if we
>>>>>>>>>
>>>>>>>> decided
>>>>>>>
>>>>>>>> it
>>>>>>>>
>>>>>>>>> now, then 2.4 would drop Java7). Also, we could start by
>>>>>>>>>
>>>>>>>> switching
>>>
>>>> tests
>>>>>>>
>>>>>>>> to
>>>>>>>>
>>>>>>>>> compile/run with java8 (Maven allows this). This is, I think,
>>>>>>>>>
>>>>>>>> pretty
>>>>>
>>>>>> much
>>>>>>>
>>>>>>>> safe to do immediately.
>>>>>>>>>
>>>>>>>>> On Mon, Oct 16, 2017 at 7:35 AM Ismaël Mejía <iemejia@gmail.com
>>>>>>>>>
>>>>>>>>
>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>> Any progress on this? What is the proposed way to validate if
>>>>>>>>>>
>>>>>>>>> users
>>>>>
>>>>>> are still interested on Java 7? A vote on user or something
>>>>>>>>>>
>>>>>>>>> different?
>>>>>>>
>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles
>>>>>>>>>>
>>>>>>>>> <klk@google.com.invalid
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Agree with polling Beam users as well as each runner's
>>>>>>>>>>>
>>>>>>>>>> community
>>>>>
>>>>>> in
>>>>>>
>>>>>>> aggregate.
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré <
>>>>>>>>>>>
>>>>>>>>>> jb@nanthrax.net
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Definitely agree
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 09/27/2017 06:00 PM, Robert Bradshaw wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I also think that it's time to seriously consider dropping
>>>>>>>>>>>>>
>>>>>>>>>>>> support
>>>>>>>
>>>>>>>> for
>>>>>>>>>
>>>>>>>>>> Java 7.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
>>>>>>>>>>>>> <da...@google.com.invalid> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes, just as Ismaël said it's a compilation blocker right
>>>>>>>>>>>>>>
>>>>>>>>>>>>> now
>>>>>
>>>>>> despite
>>>>>>>>>
>>>>>>>>>> that
>>>>>>>>>>>>>> (I believe) we don't use the extension that's breaking.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As for other ways to solve this, if there is a way to
>>>>>>>>>>>>>>
>>>>>>>>>>>>> avoid
>>>
>>>> compiling
>>>>>>>>>
>>>>>>>>>> the
>>>>>>>>>>
>>>>>>>>>>> advanced features of AutoValue that might be worth a
>>>>>>>>>>>>>>
>>>>>>>>>>>>> try. We
>>>
>>>> could
>>>>>>>
>>>>>>>> also
>>>>>>>>>>
>>>>>>>>>>> try
>>>>>>>>>>>>>> to get a release of AutoValue with the fix that works in
>>>>>>>>>>>>>>
>>>>>>>>>>>>> Java
>>>>>
>>>>>> 7.
>>>>>>
>>>>>>> However
>>>>>>>>>>
>>>>>>>>>>> I
>>>>>>>>>>>>>> feel that slowly moving over to Java 8 is the most
>>>>>>>>>>>>>>
>>>>>>>>>>>>> future-proof
>>>>>>
>>>>>>> solution
>>>>>>>>>>
>>>>>>>>>>> if
>>>>>>>>>>>>>> it's possible.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <
>>>>>>>>>>>>>>
>>>>>>>>>>>>> iemejia@gmail.com>
>>>>>>>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>>> The current issue is that compilation fails on master
>>>>>>>>>>>>>>
>>>>>>>>>>>>> because
>>>>>
>>>>>> beam's
>>>>>>>>
>>>>>>>>> parent pom is configured to fail if it finds warnings):
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> <compiler.error.flag>-Werror</compiler.error.flag>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> However if you remove that line from the parent pom the
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> compilation
>>>>>>>>
>>>>>>>>> passes.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Of course this does not mean that everything is solved
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> for
>>>
>>>> Java
>>>>>>
>>>>>>> 9,
>>>>>>>>
>>>>>>>>> there are some tests that break and other issues
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> because of
>>>
>>>> other
>>>>>>>
>>>>>>>> plugins and dependencies (e.g. bytebuddy), but those are
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> not
>>>>>
>>>>>> part
>>>>>>>
>>>>>>>> of
>>>>>>>>>
>>>>>>>>>> this discussion.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
>>>>>>>>>>>>>>> <ki...@google.com.invalid> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> AFAIK we don't use any advanced capabilities of
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> AutoValue.
>>>
>>>> Does
>>>>>>>
>>>>>>>> that
>>>>>>>>>
>>>>>>>>>> mean
>>>>>>>>>>>>>>>> this issue is moot? I didn't quite understand from your
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> email
>>>>>>
>>>>>>> whether
>>>>>>>>>>
>>>>>>>>>>> it
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> a compilation blocker for Beam or not.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> iemejia@gmail.com
>>>>>>>
>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Great that you are also working on this too Daniel and
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> thanks
>>>>>>
>>>>>>> for
>>>>>>>>
>>>>>>>>> bringing this subject to the mailing list, I was
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> waiting
>>>
>>>> to
>>>>>
>>>>>> my
>>>>>>>
>>>>>>>> return
>>>>>>>>>>
>>>>>>>>>>> to office next week, but you did it first :)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Eugene for reference (This is the issue on the
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> migration
>>>
>>>> to
>>>>>
>>>>>> Java
>>>>>>>>
>>>>>>>>> 9),
>>>>>>>>>>
>>>>>>>>>>> notice that here the goal is first that beam passes
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> mvn
>>>
>>>> clean
>>>>>>
>>>>>>> install
>>>>>>>>>>
>>>>>>>>>>> with pure Java 9 (and also add this to jenkins), not
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> to
>>>
>>>> rewrite
>>>>>>>
>>>>>>>> anything to use the new stuff (e.g. modules):
>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/BEAM-2530
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Eugene can you also PTAL at the AutoValue issue, more
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> details
>>>>>>
>>>>>>> on
>>>>>>>>
>>>>>>>>> the
>>>>>>>>>>
>>>>>>>>>>> issue, this is a warning so I don't know if it is
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> really
>>>
>>>> critical
>>>>>>>>
>>>>>>>>> in
>>>>>>>>>>
>>>>>>>>>>> particular because we are not using Memoization (do
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> we?).
>>>
>>>> https://github.com/google/auto/issues/503
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>> https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7
>>>>>> f0a25f5d
>>>>>>
>>>>>>>
>>>>>>>>>>>>>>>>> 02247b8532
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Wouldn't the easiest way be that you guys convince the
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> google.auto
>>>>>>>>>
>>>>>>>>>> guys to generate that simple fix in a Java 7
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> compatible
>>>
>>>> way
>>>>>
>>>>>> and
>>>>>>>
>>>>>>>> 'voila' ?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> However I agree that moving to Java 8 is an excellent
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> idea
>>>>>
>>>>>> and
>>>>>>>
>>>>>>>> as
>>>>>>>>
>>>>>>>>> Eugene mentions there is less friction now since most
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> projects
>>>>>>>
>>>>>>>> are
>>>>>>>>>
>>>>>>>>>> moving, only pending issue are existing clusters on
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> java
>>>
>>>> 7
>>>>>
>>>>>> in
>>>>>>
>>>>>>> the
>>>>>>>>
>>>>>>>>> hadoop world, but those are less frequent now. Anyway
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> this
>>>>>
>>>>>> discussion
>>>>>>>>>>
>>>>>>>>>>> is really important (maybe even worth a vote). Because
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> moving
>>>>>>
>>>>>>> to
>>>>>>>>
>>>>>>>>> Java
>>>>>>>>>>
>>>>>>>>>>> 8 will allow us also to move some of the dependencies
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> that
>>>>>
>>>>>> we
>>>>>>
>>>>>>> are
>>>>>>>>
>>>>>>>>> keeping in old versions and in general to move
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> forward.
>>>
>>>>
>>>>>>>>>>>>>>>>> What do the others think ?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
>>>>>>>>>>>>>>>>> <ki...@google.com.invalid> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Very excited to hear that there's work on JDK9
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> support -
>>>
>>>> is
>>>>>>
>>>>>>> there
>>>>>>>>>
>>>>>>>>>> a
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> public
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> description of the plans for this work somewhere?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> In general, Beam could probably drop Java7 support
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> altogether
>>>>>>>
>>>>>>>> at
>>>>>>>>
>>>>>>>>> some
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> point
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> receiving
>>>>>>
>>>>>>> even
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>>>>>>>> security
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> patches) 2 years ago, and all major players in the
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> data
>>>
>>>> processing
>>>>>>>>>
>>>>>>>>>> ecosystem have dropped Java7 support (Spark, Flink,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hadoop),
>>>>>>
>>>>>>> so
>>>>>>>>
>>>>>>>>> I
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>>>>>>>> presume
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> the demand for Java7 support in the data processing
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> industry
>>>>>
>>>>>> is
>>>>>>>
>>>>>>>> low.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>>>>>>>> By
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> way: would a Java8 migration be in the scope of your
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> work
>>>>>
>>>>>> in
>>>>>>
>>>>>>> general?
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> However, until we say that Beam requires Java8, what
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> would
>>>>>
>>>>>> be
>>>>>>>
>>>>>>>> the
>>>>>>>>>
>>>>>>>>>> implications of using a version of AutoValue that can
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> only
>>>>>
>>>>>> be
>>>>>>>
>>>>>>>> compiled
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Java8? Are you saying that this is simply a matter
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> of a
>>>
>>>> compiler
>>>>>>>>
>>>>>>>>> bug,
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> if we use a Java8 compiler but configured to use source
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> and
>>>>>
>>>>>> target
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>>>>>>>> versions
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> then
>>>
>>>> the
>>>>>>
>>>>>>> resulting
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Beam
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> artifacts will be usable by people who don't have
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Java8?
>>>
>>>>
>>>>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
>>>>>>>>>>>>>>>>>> <da...@google.com.invalid> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> So I've been working on JDK 9 support for Beam, and I
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> have a
>>>>>>
>>>>>>> bug
>>>>>>>>
>>>>>>>>> in
>>>>>>>>>>
>>>>>>>>>>> AutoValue that can be fixed by updating our
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> AutoValue
>>>
>>>> dependency
>>>>>>>>>
>>>>>>>>>> to
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> latest. The problem is that AutoValue from 1.5+ seems
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> to
>>
>>
Re: Possibility of requiring Java 8 compiler for building Java 7
sources?
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Agree, I would target this for Beam 3.0.0.
Regards
JB
On 10/17/2017 06:43 PM, Reuven Lax wrote:
> Should this be considered a backwards-incompatible change? If so, do we
> need to wait until Beam 3.0.0 is released?
>
> On Tue, Oct 17, 2017 at 9:11 AM, Ismaël Mejía <ie...@gmail.com> wrote:
>
>> I am bringing the subject to the user mailing list to get some
>> feedback because it makes sense anyway to discuss this there. But I
>> also agree with Kenneth about the fact that runner authors must weight
>> in on this subject.
>>
>>
>> On Tue, Oct 17, 2017 at 5:24 PM, Kenneth Knowles <kl...@google.com.invalid>
>> wrote:
>>> +1 to having runner maintainers weigh in as proxies. Added a few in case
>>> they haven't followed this thread.
>>>
>>> On Mon, Oct 16, 2017 at 11:38 PM, Eugene Kirpichov <
>>> kirpichov@google.com.invalid> wrote:
>>>
>>>> Agreed that polling Dataflow users makes sense, though I think they are
>>>> very unlikely to have concerns, because unlike Spark/Flink users they
>>>> wouldn't have a "cluster" that they need to migrate to a new JVM: they'd
>>>> only need to recompile their pipelines with JDK 8.
>>>>
>>>> On Mon, Oct 16, 2017 at 11:21 PM Reuven Lax <re...@google.com.invalid>
>>>> wrote:
>>>>
>>>>> I think the Flink and Spark runner maintainers can weigh in here;
>> given
>>>>> that both of those systems are moving to Java 8, I doubt they will
>> have
>>>>> concerns. Same is true for the Dataflow runner - we should probably
>> poll
>>>>> Dataflow users to make sure this is not a problem for any of them.
>>>>>
>>>>> On Mon, Oct 16, 2017 at 11:05 PM, Eugene Kirpichov <
>>>>> kirpichov@google.com.invalid> wrote:
>>>>>
>>>>>> Reuven - do you mean e.g. a poll on the Flink mailing list asking
>>>> whether
>>>>>> there are Flink users who use Beam with Java 7? Or just people who
>> use
>>>>>> Flink with Java 7? (the latter question I'd assume was settled by
>> the
>>>>> poll
>>>>>> about making Flink itself Java8-only?)
>>>>>>
>>>>>> On Mon, Oct 16, 2017 at 10:32 PM Reuven Lax
>> <re...@google.com.invalid>
>>>>>> wrote:
>>>>>>
>>>>>>> I don't know if a vote in @user is sufficient, as it's
>> unfortunately
>>>>> not
>>>>>>> representative of all Beam users. I think the runner communities
>>>> should
>>>>>> be
>>>>>>> polled as well (though I suspect the answer will be the same,
>> that we
>>>>> can
>>>>>>> go ahead and deprecate Java 7 support).
>>>>>>>
>>>>>>> On Mon, Oct 16, 2017 at 4:50 PM, Eugene Kirpichov <
>>>>>>> kirpichov@google.com.invalid> wrote:
>>>>>>>
>>>>>>>> Yeah, a vote on user@ sounds like a good idea. Ismaël, would
>> you
>>>> be
>>>>>>>> interested in driving this process, since you're already
>> working on
>>>>>> Java9
>>>>>>>> support and hence you have a good understanding of what's
>> involved
>>>>> in a
>>>>>>> JDK
>>>>>>>> version migration for a large project?
>>>>>>>>
>>>>>>>> As due diligence, we can look at how the other data processing
>>>>> systems
>>>>>>>> handled dropping Java7.
>>>>>>>>
>>>>>>>> Flink:
>>>>>>>> JIRA https://issues.apache.org/jira/browse/FLINK-7242
>>>>>>>> Discussion
>>>>>>>> http://apache-flink-user-mailing-list-archive.2336050.
>>>>>>>> n4.nabble.com/POLL-Who-still-uses-Java-7-with-Flink-
>> td12216.html
>>>>>>>>
>>>>>>>> They also did a Twitter poll in addition to the mailing list
>> poll,
>>>>>> which
>>>>>>>> seems like a good idea.
>>>>>>>> Note that Flink had a number of strong reasons for dropping
>> Java7
>>>>> that
>>>>>> do
>>>>>>>> not apply in Beam.
>>>>>>>>
>>>>>>>> Spark:
>>>>>>>> JIRA https://issues.apache.org/jira/browse/SPARK-19493
>>>>>>>> Discussion
>>>>>>>>
>>>>>>> http://apache-spark-developers-list.1001551.n3.
>>>>>> nabble.com/discuss-ending-
>>>>>>>> support-for-Java-7-in-Spark-2-0-td16808.html
>>>>>>>> (I
>>>>>>>> couldn't find a formal poll of the user list rather than
>> developer
>>>>>> list)
>>>>>>>>
>>>>>>>> Hadoop:
>>>>>>>> Hadoop 3.0 is Java8-only, but I couldn't quickly find a
>> discussion
>>>> of
>>>>>>> where
>>>>>>>> that decision was made.
>>>>>>>> https://lists.apache.org/thread.html/e5c8085ada2cca47027b63f
>>>> 5439839
>>>>>>>> 731a392335770386e10895be06@1444091751@%3Cmapreduce-dev.
>>>>>>>> hadoop.apache.org%3E
>>>>>>>> might
>>>>>>>> be it.
>>>>>>>>
>>>>>>>> Kafka is considering it:
>>>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>>>>>>>> 118%3A+Drop+Support+for+Java+7+in+Kafka+0.11
>>>>>>>> and
>>>>>>>> quotes a number of other open-source projects that have switched
>>>>>>>> http://markmail.org/message/l7s276y3xkga2eqf
>>>>>>>>
>>>>>>>> So basically these projects all did a mailing list poll, and one
>>>> did
>>>>>>> also a
>>>>>>>> twitter poll.
>>>>>>>>
>>>>>>>> Beam has the advantage of being a relatively young project with
>>>>>> perhaps a
>>>>>>>> smaller base of users entrenched in using old versions of Java;
>>>>>> moreover,
>>>>>>>> Java version would matter only for the smaller subset of users
>> who
>>>>> use
>>>>>>> Beam
>>>>>>>> Spark/Flink/Apex/.. runners (as opposed to Cloud Dataflow),
>> which
>>>> is
>>>>>>> likely
>>>>>>>> an even more "early adopter"-ish group of users, as these
>> runners
>>>>>>> generally
>>>>>>>> receive less support.
>>>>>>>>
>>>>>>>> It may be a good idea to have at least 1 release pass between
>>>>>> announcing
>>>>>>>> the intention to drop Java8 and actually dropping it (e.g. if we
>>>>>> decided
>>>>>>> it
>>>>>>>> now, then 2.4 would drop Java7). Also, we could start by
>> switching
>>>>>> tests
>>>>>>> to
>>>>>>>> compile/run with java8 (Maven allows this). This is, I think,
>>>> pretty
>>>>>> much
>>>>>>>> safe to do immediately.
>>>>>>>>
>>>>>>>> On Mon, Oct 16, 2017 at 7:35 AM Ismaël Mejía <iemejia@gmail.com
>>>
>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Any progress on this? What is the proposed way to validate if
>>>> users
>>>>>>>>> are still interested on Java 7? A vote on user or something
>>>>>> different?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles
>>>>>>> <klk@google.com.invalid
>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>> Agree with polling Beam users as well as each runner's
>>>> community
>>>>> in
>>>>>>>>>> aggregate.
>>>>>>>>>>
>>>>>>>>>> On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré <
>>>>>>> jb@nanthrax.net
>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Definitely agree
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 09/27/2017 06:00 PM, Robert Bradshaw wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I also think that it's time to seriously consider dropping
>>>>>> support
>>>>>>>> for
>>>>>>>>>>>> Java 7.
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
>>>>>>>>>>>> <da...@google.com.invalid> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Yes, just as Ismaël said it's a compilation blocker right
>>>> now
>>>>>>>> despite
>>>>>>>>>>>>> that
>>>>>>>>>>>>> (I believe) we don't use the extension that's breaking.
>>>>>>>>>>>>>
>>>>>>>>>>>>> As for other ways to solve this, if there is a way to
>> avoid
>>>>>>>> compiling
>>>>>>>>> the
>>>>>>>>>>>>> advanced features of AutoValue that might be worth a
>> try. We
>>>>>> could
>>>>>>>>> also
>>>>>>>>>>>>> try
>>>>>>>>>>>>> to get a release of AutoValue with the fix that works in
>>>> Java
>>>>> 7.
>>>>>>>>> However
>>>>>>>>>>>>> I
>>>>>>>>>>>>> feel that slowly moving over to Java 8 is the most
>>>>> future-proof
>>>>>>>>> solution
>>>>>>>>>>>>> if
>>>>>>>>>>>>> it's possible.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <
>>>>>> iemejia@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> The current issue is that compilation fails on master
>>>> because
>>>>>>> beam's
>>>>>>>>>>>>>> parent pom is configured to fail if it finds warnings):
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <compiler.error.flag>-Werror</compiler.error.flag>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> However if you remove that line from the parent pom the
>>>>>>> compilation
>>>>>>>>>>>>>> passes.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Of course this does not mean that everything is solved
>> for
>>>>> Java
>>>>>>> 9,
>>>>>>>>>>>>>> there are some tests that break and other issues
>> because of
>>>>>> other
>>>>>>>>>>>>>> plugins and dependencies (e.g. bytebuddy), but those are
>>>> not
>>>>>> part
>>>>>>>> of
>>>>>>>>>>>>>> this discussion.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
>>>>>>>>>>>>>> <ki...@google.com.invalid> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> AFAIK we don't use any advanced capabilities of
>> AutoValue.
>>>>>> Does
>>>>>>>> that
>>>>>>>>>>>>>>> mean
>>>>>>>>>>>>>>> this issue is moot? I didn't quite understand from your
>>>>> email
>>>>>>>>> whether
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> a compilation blocker for Beam or not.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <
>>>>>> iemejia@gmail.com
>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Great that you are also working on this too Daniel and
>>>>> thanks
>>>>>>> for
>>>>>>>>>>>>>>>> bringing this subject to the mailing list, I was
>> waiting
>>>> to
>>>>>> my
>>>>>>>>> return
>>>>>>>>>>>>>>>> to office next week, but you did it first :)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Eugene for reference (This is the issue on the
>> migration
>>>> to
>>>>>>> Java
>>>>>>>>> 9),
>>>>>>>>>>>>>>>> notice that here the goal is first that beam passes
>> mvn
>>>>> clean
>>>>>>>>> install
>>>>>>>>>>>>>>>> with pure Java 9 (and also add this to jenkins), not
>> to
>>>>>> rewrite
>>>>>>>>>>>>>>>> anything to use the new stuff (e.g. modules):
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/BEAM-2530
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Eugene can you also PTAL at the AutoValue issue, more
>>>>> details
>>>>>>> on
>>>>>>>>> the
>>>>>>>>>>>>>>>> issue, this is a warning so I don't know if it is
>> really
>>>>>>> critical
>>>>>>>>> in
>>>>>>>>>>>>>>>> particular because we are not using Memoization (do
>> we?).
>>>>>>>>>>>>>>>> https://github.com/google/auto/issues/503
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>
>>>>> https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 02247b8532
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Wouldn't the easiest way be that you guys convince the
>>>>>>>> google.auto
>>>>>>>>>>>>>>>> guys to generate that simple fix in a Java 7
>> compatible
>>>> way
>>>>>> and
>>>>>>>>>>>>>>>> 'voila' ?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> However I agree that moving to Java 8 is an excellent
>>>> idea
>>>>>> and
>>>>>>> as
>>>>>>>>>>>>>>>> Eugene mentions there is less friction now since most
>>>>>> projects
>>>>>>>> are
>>>>>>>>>>>>>>>> moving, only pending issue are existing clusters on
>> java
>>>> 7
>>>>> in
>>>>>>> the
>>>>>>>>>>>>>>>> hadoop world, but those are less frequent now. Anyway
>>>> this
>>>>>>>>> discussion
>>>>>>>>>>>>>>>> is really important (maybe even worth a vote). Because
>>>>> moving
>>>>>>> to
>>>>>>>>> Java
>>>>>>>>>>>>>>>> 8 will allow us also to move some of the dependencies
>>>> that
>>>>> we
>>>>>>> are
>>>>>>>>>>>>>>>> keeping in old versions and in general to move
>> forward.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> What do the others think ?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
>>>>>>>>>>>>>>>> <ki...@google.com.invalid> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Very excited to hear that there's work on JDK9
>> support -
>>>>> is
>>>>>>>> there
>>>>>>>>> a
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> public
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> description of the plans for this work somewhere?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> In general, Beam could probably drop Java7 support
>>>>>> altogether
>>>>>>> at
>>>>>>>>> some
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> point
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not
>>>>> receiving
>>>>>>>> even
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> security
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> patches) 2 years ago, and all major players in the
>> data
>>>>>>>> processing
>>>>>>>>>>>>>>>>> ecosystem have dropped Java7 support (Spark, Flink,
>>>>> Hadoop),
>>>>>>> so
>>>>>>>> I
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> presume
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> the demand for Java7 support in the data processing
>>>> industry
>>>>>> is
>>>>>>>> low.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> By
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> way: would a Java8 migration be in the scope of your
>>>> work
>>>>> in
>>>>>>>>> general?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> However, until we say that Beam requires Java8, what
>>>> would
>>>>>> be
>>>>>>>> the
>>>>>>>>>>>>>>>>> implications of using a version of AutoValue that can
>>>> only
>>>>>> be
>>>>>>>>>>>>>>>>> compiled
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Java8? Are you saying that this is simply a matter
>> of a
>>>>>>> compiler
>>>>>>>>> bug,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> if we use a Java8 compiler but configured to use source
>>>> and
>>>>>>> target
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> versions
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7,
>> then
>>>>> the
>>>>>>>>> resulting
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Beam
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> artifacts will be usable by people who don't have
>> Java8?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
>>>>>>>>>>>>>>>>> <da...@google.com.invalid> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> So I've been working on JDK 9 support for Beam, and I
>>>>> have a
>>>>>>> bug
>>>>>>>>> in
>>>>>>>>>>>>>>>>>> AutoValue that can be fixed by updating our
>> AutoValue
>>>>>>>> dependency
>>>>>>>>> to
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> latest. The problem is that AutoValue from 1.5+ seems
>> to
>>>> be
>>>>>>> banned
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Beam
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> due to requiring Java 8 compilers. However, it should
>>>>> still
>>>>>> be
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> possible
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> compile and execute Java 7 code from the Java 8
>> compiler
>>>>> by
>>>>>>>>> building
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> the correct arguments. So the fix to this bug would
>>>>>>> essentially
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> require
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Java 8 compilers even for compiling Java 7 code.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Does anyone need to use Java 7 compilers? Because if
>>>> not
>>>>> I
>>>>>>>> would
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> like to
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> continue with this fix.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Jean-Baptiste Onofré
>>>>>>>>>>> jbonofre@apache.org
>>>>>>>>>>> http://blog.nanthrax.net
>>>>>>>>>>> Talend - http://www.talend.com
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Reuven Lax <re...@google.com.INVALID>.
Should this be considered a backwards-incompatible change? If so, do we
need to wait until Beam 3.0.0 is released?
On Tue, Oct 17, 2017 at 9:11 AM, Ismaël Mejía <ie...@gmail.com> wrote:
> I am bringing the subject to the user mailing list to get some
> feedback because it makes sense anyway to discuss this there. But I
> also agree with Kenneth about the fact that runner authors must weight
> in on this subject.
>
>
> On Tue, Oct 17, 2017 at 5:24 PM, Kenneth Knowles <kl...@google.com.invalid>
> wrote:
> > +1 to having runner maintainers weigh in as proxies. Added a few in case
> > they haven't followed this thread.
> >
> > On Mon, Oct 16, 2017 at 11:38 PM, Eugene Kirpichov <
> > kirpichov@google.com.invalid> wrote:
> >
> >> Agreed that polling Dataflow users makes sense, though I think they are
> >> very unlikely to have concerns, because unlike Spark/Flink users they
> >> wouldn't have a "cluster" that they need to migrate to a new JVM: they'd
> >> only need to recompile their pipelines with JDK 8.
> >>
> >> On Mon, Oct 16, 2017 at 11:21 PM Reuven Lax <re...@google.com.invalid>
> >> wrote:
> >>
> >> > I think the Flink and Spark runner maintainers can weigh in here;
> given
> >> > that both of those systems are moving to Java 8, I doubt they will
> have
> >> > concerns. Same is true for the Dataflow runner - we should probably
> poll
> >> > Dataflow users to make sure this is not a problem for any of them.
> >> >
> >> > On Mon, Oct 16, 2017 at 11:05 PM, Eugene Kirpichov <
> >> > kirpichov@google.com.invalid> wrote:
> >> >
> >> > > Reuven - do you mean e.g. a poll on the Flink mailing list asking
> >> whether
> >> > > there are Flink users who use Beam with Java 7? Or just people who
> use
> >> > > Flink with Java 7? (the latter question I'd assume was settled by
> the
> >> > poll
> >> > > about making Flink itself Java8-only?)
> >> > >
> >> > > On Mon, Oct 16, 2017 at 10:32 PM Reuven Lax
> <re...@google.com.invalid>
> >> > > wrote:
> >> > >
> >> > > > I don't know if a vote in @user is sufficient, as it's
> unfortunately
> >> > not
> >> > > > representative of all Beam users. I think the runner communities
> >> should
> >> > > be
> >> > > > polled as well (though I suspect the answer will be the same,
> that we
> >> > can
> >> > > > go ahead and deprecate Java 7 support).
> >> > > >
> >> > > > On Mon, Oct 16, 2017 at 4:50 PM, Eugene Kirpichov <
> >> > > > kirpichov@google.com.invalid> wrote:
> >> > > >
> >> > > > > Yeah, a vote on user@ sounds like a good idea. Ismaël, would
> you
> >> be
> >> > > > > interested in driving this process, since you're already
> working on
> >> > > Java9
> >> > > > > support and hence you have a good understanding of what's
> involved
> >> > in a
> >> > > > JDK
> >> > > > > version migration for a large project?
> >> > > > >
> >> > > > > As due diligence, we can look at how the other data processing
> >> > systems
> >> > > > > handled dropping Java7.
> >> > > > >
> >> > > > > Flink:
> >> > > > > JIRA https://issues.apache.org/jira/browse/FLINK-7242
> >> > > > > Discussion
> >> > > > > http://apache-flink-user-mailing-list-archive.2336050.
> >> > > > > n4.nabble.com/POLL-Who-still-uses-Java-7-with-Flink-
> td12216.html
> >> > > > >
> >> > > > > They also did a Twitter poll in addition to the mailing list
> poll,
> >> > > which
> >> > > > > seems like a good idea.
> >> > > > > Note that Flink had a number of strong reasons for dropping
> Java7
> >> > that
> >> > > do
> >> > > > > not apply in Beam.
> >> > > > >
> >> > > > > Spark:
> >> > > > > JIRA https://issues.apache.org/jira/browse/SPARK-19493
> >> > > > > Discussion
> >> > > > >
> >> > > > http://apache-spark-developers-list.1001551.n3.
> >> > > nabble.com/discuss-ending-
> >> > > > > support-for-Java-7-in-Spark-2-0-td16808.html
> >> > > > > (I
> >> > > > > couldn't find a formal poll of the user list rather than
> developer
> >> > > list)
> >> > > > >
> >> > > > > Hadoop:
> >> > > > > Hadoop 3.0 is Java8-only, but I couldn't quickly find a
> discussion
> >> of
> >> > > > where
> >> > > > > that decision was made.
> >> > > > > https://lists.apache.org/thread.html/e5c8085ada2cca47027b63f
> >> 5439839
> >> > > > > 731a392335770386e10895be06@1444091751@%3Cmapreduce-dev.
> >> > > > > hadoop.apache.org%3E
> >> > > > > might
> >> > > > > be it.
> >> > > > >
> >> > > > > Kafka is considering it:
> >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >> > > > > 118%3A+Drop+Support+for+Java+7+in+Kafka+0.11
> >> > > > > and
> >> > > > > quotes a number of other open-source projects that have switched
> >> > > > > http://markmail.org/message/l7s276y3xkga2eqf
> >> > > > >
> >> > > > > So basically these projects all did a mailing list poll, and one
> >> did
> >> > > > also a
> >> > > > > twitter poll.
> >> > > > >
> >> > > > > Beam has the advantage of being a relatively young project with
> >> > > perhaps a
> >> > > > > smaller base of users entrenched in using old versions of Java;
> >> > > moreover,
> >> > > > > Java version would matter only for the smaller subset of users
> who
> >> > use
> >> > > > Beam
> >> > > > > Spark/Flink/Apex/.. runners (as opposed to Cloud Dataflow),
> which
> >> is
> >> > > > likely
> >> > > > > an even more "early adopter"-ish group of users, as these
> runners
> >> > > > generally
> >> > > > > receive less support.
> >> > > > >
> >> > > > > It may be a good idea to have at least 1 release pass between
> >> > > announcing
> >> > > > > the intention to drop Java8 and actually dropping it (e.g. if we
> >> > > decided
> >> > > > it
> >> > > > > now, then 2.4 would drop Java7). Also, we could start by
> switching
> >> > > tests
> >> > > > to
> >> > > > > compile/run with java8 (Maven allows this). This is, I think,
> >> pretty
> >> > > much
> >> > > > > safe to do immediately.
> >> > > > >
> >> > > > > On Mon, Oct 16, 2017 at 7:35 AM Ismaël Mejía <iemejia@gmail.com
> >
> >> > > wrote:
> >> > > > >
> >> > > > > > Any progress on this? What is the proposed way to validate if
> >> users
> >> > > > > > are still interested on Java 7? A vote on user or something
> >> > > different?
> >> > > > > >
> >> > > > > >
> >> > > > > > On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles
> >> > > > <klk@google.com.invalid
> >> > > > > >
> >> > > > > > wrote:
> >> > > > > > > Agree with polling Beam users as well as each runner's
> >> community
> >> > in
> >> > > > > > > aggregate.
> >> > > > > > >
> >> > > > > > > On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré <
> >> > > > jb@nanthrax.net
> >> > > > > >
> >> > > > > > > wrote:
> >> > > > > > >
> >> > > > > > >> Definitely agree
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >> On 09/27/2017 06:00 PM, Robert Bradshaw wrote:
> >> > > > > > >>
> >> > > > > > >>> I also think that it's time to seriously consider dropping
> >> > > support
> >> > > > > for
> >> > > > > > >>> Java 7.
> >> > > > > > >>>
> >> > > > > > >>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
> >> > > > > > >>> <da...@google.com.invalid> wrote:
> >> > > > > > >>>
> >> > > > > > >>>> Yes, just as Ismaël said it's a compilation blocker right
> >> now
> >> > > > > despite
> >> > > > > > >>>> that
> >> > > > > > >>>> (I believe) we don't use the extension that's breaking.
> >> > > > > > >>>>
> >> > > > > > >>>> As for other ways to solve this, if there is a way to
> avoid
> >> > > > > compiling
> >> > > > > > the
> >> > > > > > >>>> advanced features of AutoValue that might be worth a
> try. We
> >> > > could
> >> > > > > > also
> >> > > > > > >>>> try
> >> > > > > > >>>> to get a release of AutoValue with the fix that works in
> >> Java
> >> > 7.
> >> > > > > > However
> >> > > > > > >>>> I
> >> > > > > > >>>> feel that slowly moving over to Java 8 is the most
> >> > future-proof
> >> > > > > > solution
> >> > > > > > >>>> if
> >> > > > > > >>>> it's possible.
> >> > > > > > >>>>
> >> > > > > > >>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <
> >> > > iemejia@gmail.com>
> >> > > > > > wrote:
> >> > > > > > >>>>
> >> > > > > > >>>> The current issue is that compilation fails on master
> >> because
> >> > > > beam's
> >> > > > > > >>>>> parent pom is configured to fail if it finds warnings):
> >> > > > > > >>>>>
> >> > > > > > >>>>> <compiler.error.flag>-Werror</compiler.error.flag>
> >> > > > > > >>>>>
> >> > > > > > >>>>> However if you remove that line from the parent pom the
> >> > > > compilation
> >> > > > > > >>>>> passes.
> >> > > > > > >>>>>
> >> > > > > > >>>>> Of course this does not mean that everything is solved
> for
> >> > Java
> >> > > > 9,
> >> > > > > > >>>>> there are some tests that break and other issues
> because of
> >> > > other
> >> > > > > > >>>>> plugins and dependencies (e.g. bytebuddy), but those are
> >> not
> >> > > part
> >> > > > > of
> >> > > > > > >>>>> this discussion.
> >> > > > > > >>>>>
> >> > > > > > >>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
> >> > > > > > >>>>> <ki...@google.com.invalid> wrote:
> >> > > > > > >>>>>
> >> > > > > > >>>>>> AFAIK we don't use any advanced capabilities of
> AutoValue.
> >> > > Does
> >> > > > > that
> >> > > > > > >>>>>> mean
> >> > > > > > >>>>>> this issue is moot? I didn't quite understand from your
> >> > email
> >> > > > > > whether
> >> > > > > > >>>>>> it
> >> > > > > > >>>>>>
> >> > > > > > >>>>> is
> >> > > > > > >>>>>
> >> > > > > > >>>>>> a compilation blocker for Beam or not.
> >> > > > > > >>>>>>
> >> > > > > > >>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <
> >> > > iemejia@gmail.com
> >> > > > >
> >> > > > > > >>>>>> wrote:
> >> > > > > > >>>>>>
> >> > > > > > >>>>>> Great that you are also working on this too Daniel and
> >> > thanks
> >> > > > for
> >> > > > > > >>>>>>> bringing this subject to the mailing list, I was
> waiting
> >> to
> >> > > my
> >> > > > > > return
> >> > > > > > >>>>>>> to office next week, but you did it first :)
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>> Eugene for reference (This is the issue on the
> migration
> >> to
> >> > > > Java
> >> > > > > > 9),
> >> > > > > > >>>>>>> notice that here the goal is first that beam passes
> mvn
> >> > clean
> >> > > > > > install
> >> > > > > > >>>>>>> with pure Java 9 (and also add this to jenkins), not
> to
> >> > > rewrite
> >> > > > > > >>>>>>> anything to use the new stuff (e.g. modules):
> >> > > > > > >>>>>>> https://issues.apache.org/jira/browse/BEAM-2530
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>> Eugene can you also PTAL at the AutoValue issue, more
> >> > details
> >> > > > on
> >> > > > > > the
> >> > > > > > >>>>>>> issue, this is a warning so I don't know if it is
> really
> >> > > > critical
> >> > > > > > in
> >> > > > > > >>>>>>> particular because we are not using Memoization (do
> we?).
> >> > > > > > >>>>>>> https://github.com/google/auto/issues/503
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>>
> >> > > > > >
> >> > https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>> 02247b8532
> >> > > > > > >>>>>
> >> > > > > > >>>>>>
> >> > > > > > >>>>>>> Wouldn't the easiest way be that you guys convince the
> >> > > > > google.auto
> >> > > > > > >>>>>>> guys to generate that simple fix in a Java 7
> compatible
> >> way
> >> > > and
> >> > > > > > >>>>>>> 'voila' ?
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>> However I agree that moving to Java 8 is an excellent
> >> idea
> >> > > and
> >> > > > as
> >> > > > > > >>>>>>> Eugene mentions there is less friction now since most
> >> > > projects
> >> > > > > are
> >> > > > > > >>>>>>> moving, only pending issue are existing clusters on
> java
> >> 7
> >> > in
> >> > > > the
> >> > > > > > >>>>>>> hadoop world, but those are less frequent now. Anyway
> >> this
> >> > > > > > discussion
> >> > > > > > >>>>>>> is really important (maybe even worth a vote). Because
> >> > moving
> >> > > > to
> >> > > > > > Java
> >> > > > > > >>>>>>> 8 will allow us also to move some of the dependencies
> >> that
> >> > we
> >> > > > are
> >> > > > > > >>>>>>> keeping in old versions and in general to move
> forward.
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>> What do the others think ?
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
> >> > > > > > >>>>>>> <ki...@google.com.invalid> wrote:
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>>> Very excited to hear that there's work on JDK9
> support -
> >> > is
> >> > > > > there
> >> > > > > > a
> >> > > > > > >>>>>>>>
> >> > > > > > >>>>>>> public
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>>> description of the plans for this work somewhere?
> >> > > > > > >>>>>>>>
> >> > > > > > >>>>>>>> In general, Beam could probably drop Java7 support
> >> > > altogether
> >> > > > at
> >> > > > > > some
> >> > > > > > >>>>>>>>
> >> > > > > > >>>>>>> point
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not
> >> > receiving
> >> > > > > even
> >> > > > > > >>>>>>>>
> >> > > > > > >>>>>>> security
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>>> patches) 2 years ago, and all major players in the
> data
> >> > > > > processing
> >> > > > > > >>>>>>>> ecosystem have dropped Java7 support (Spark, Flink,
> >> > Hadoop),
> >> > > > so
> >> > > > > I
> >> > > > > > >>>>>>>>
> >> > > > > > >>>>>>> presume
> >> > > > > > >>>>>
> >> > > > > > >>>>>> the demand for Java7 support in the data processing
> >> industry
> >> > > is
> >> > > > > low.
> >> > > > > > >>>>>>>>
> >> > > > > > >>>>>>> By
> >> > > > > > >>>>>
> >> > > > > > >>>>>> the
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>>> way: would a Java8 migration be in the scope of your
> >> work
> >> > in
> >> > > > > > general?
> >> > > > > > >>>>>>>>
> >> > > > > > >>>>>>>> However, until we say that Beam requires Java8, what
> >> would
> >> > > be
> >> > > > > the
> >> > > > > > >>>>>>>> implications of using a version of AutoValue that can
> >> only
> >> > > be
> >> > > > > > >>>>>>>> compiled
> >> > > > > > >>>>>>>>
> >> > > > > > >>>>>>> with
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>>> Java8? Are you saying that this is simply a matter
> of a
> >> > > > compiler
> >> > > > > > bug,
> >> > > > > > >>>>>>>>
> >> > > > > > >>>>>>> and
> >> > > > > > >>>>>
> >> > > > > > >>>>>> if we use a Java8 compiler but configured to use source
> >> and
> >> > > > target
> >> > > > > > >>>>>>>>
> >> > > > > > >>>>>>> versions
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7,
> then
> >> > the
> >> > > > > > resulting
> >> > > > > > >>>>>>>>
> >> > > > > > >>>>>>> Beam
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>>> artifacts will be usable by people who don't have
> Java8?
> >> > > > > > >>>>>>>>
> >> > > > > > >>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
> >> > > > > > >>>>>>>> <da...@google.com.invalid> wrote:
> >> > > > > > >>>>>>>>
> >> > > > > > >>>>>>>> So I've been working on JDK 9 support for Beam, and I
> >> > have a
> >> > > > bug
> >> > > > > > in
> >> > > > > > >>>>>>>>> AutoValue that can be fixed by updating our
> AutoValue
> >> > > > > dependency
> >> > > > > > to
> >> > > > > > >>>>>>>>>
> >> > > > > > >>>>>>>> the
> >> > > > > > >>>>>
> >> > > > > > >>>>>> latest. The problem is that AutoValue from 1.5+ seems
> to
> >> be
> >> > > > banned
> >> > > > > > >>>>>>>>>
> >> > > > > > >>>>>>>> for
> >> > > > > > >>>>>
> >> > > > > > >>>>>> Beam
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>>> due to requiring Java 8 compilers. However, it should
> >> > still
> >> > > be
> >> > > > > > >>>>>>>>>
> >> > > > > > >>>>>>>> possible
> >> > > > > > >>>>>
> >> > > > > > >>>>>> to
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>>> compile and execute Java 7 code from the Java 8
> compiler
> >> > by
> >> > > > > > building
> >> > > > > > >>>>>>>>>
> >> > > > > > >>>>>>>> with
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>>>> the correct arguments. So the fix to this bug would
> >> > > > essentially
> >> > > > > > >>>>>>>>>
> >> > > > > > >>>>>>>> require
> >> > > > > > >>>>>
> >> > > > > > >>>>>> Java 8 compilers even for compiling Java 7 code.
> >> > > > > > >>>>>>>>>
> >> > > > > > >>>>>>>>> Does anyone need to use Java 7 compilers? Because if
> >> not
> >> > I
> >> > > > > would
> >> > > > > > >>>>>>>>>
> >> > > > > > >>>>>>>> like to
> >> > > > > > >>>>>
> >> > > > > > >>>>>> continue with this fix.
> >> > > > > > >>>>>>>>>
> >> > > > > > >>>>>>>>>
> >> > > > > > >>>>>>>
> >> > > > > > >>>>>
> >> > > > > > >> --
> >> > > > > > >> Jean-Baptiste Onofré
> >> > > > > > >> jbonofre@apache.org
> >> > > > > > >> http://blog.nanthrax.net
> >> > > > > > >> Talend - http://www.talend.com
> >> > > > > > >>
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Ismaël Mejía <ie...@gmail.com>.
I am bringing the subject to the user mailing list to get some
feedback because it makes sense anyway to discuss this there. But I
also agree with Kenneth about the fact that runner authors must weight
in on this subject.
On Tue, Oct 17, 2017 at 5:24 PM, Kenneth Knowles <kl...@google.com.invalid> wrote:
> +1 to having runner maintainers weigh in as proxies. Added a few in case
> they haven't followed this thread.
>
> On Mon, Oct 16, 2017 at 11:38 PM, Eugene Kirpichov <
> kirpichov@google.com.invalid> wrote:
>
>> Agreed that polling Dataflow users makes sense, though I think they are
>> very unlikely to have concerns, because unlike Spark/Flink users they
>> wouldn't have a "cluster" that they need to migrate to a new JVM: they'd
>> only need to recompile their pipelines with JDK 8.
>>
>> On Mon, Oct 16, 2017 at 11:21 PM Reuven Lax <re...@google.com.invalid>
>> wrote:
>>
>> > I think the Flink and Spark runner maintainers can weigh in here; given
>> > that both of those systems are moving to Java 8, I doubt they will have
>> > concerns. Same is true for the Dataflow runner - we should probably poll
>> > Dataflow users to make sure this is not a problem for any of them.
>> >
>> > On Mon, Oct 16, 2017 at 11:05 PM, Eugene Kirpichov <
>> > kirpichov@google.com.invalid> wrote:
>> >
>> > > Reuven - do you mean e.g. a poll on the Flink mailing list asking
>> whether
>> > > there are Flink users who use Beam with Java 7? Or just people who use
>> > > Flink with Java 7? (the latter question I'd assume was settled by the
>> > poll
>> > > about making Flink itself Java8-only?)
>> > >
>> > > On Mon, Oct 16, 2017 at 10:32 PM Reuven Lax <re...@google.com.invalid>
>> > > wrote:
>> > >
>> > > > I don't know if a vote in @user is sufficient, as it's unfortunately
>> > not
>> > > > representative of all Beam users. I think the runner communities
>> should
>> > > be
>> > > > polled as well (though I suspect the answer will be the same, that we
>> > can
>> > > > go ahead and deprecate Java 7 support).
>> > > >
>> > > > On Mon, Oct 16, 2017 at 4:50 PM, Eugene Kirpichov <
>> > > > kirpichov@google.com.invalid> wrote:
>> > > >
>> > > > > Yeah, a vote on user@ sounds like a good idea. Ismaël, would you
>> be
>> > > > > interested in driving this process, since you're already working on
>> > > Java9
>> > > > > support and hence you have a good understanding of what's involved
>> > in a
>> > > > JDK
>> > > > > version migration for a large project?
>> > > > >
>> > > > > As due diligence, we can look at how the other data processing
>> > systems
>> > > > > handled dropping Java7.
>> > > > >
>> > > > > Flink:
>> > > > > JIRA https://issues.apache.org/jira/browse/FLINK-7242
>> > > > > Discussion
>> > > > > http://apache-flink-user-mailing-list-archive.2336050.
>> > > > > n4.nabble.com/POLL-Who-still-uses-Java-7-with-Flink-td12216.html
>> > > > >
>> > > > > They also did a Twitter poll in addition to the mailing list poll,
>> > > which
>> > > > > seems like a good idea.
>> > > > > Note that Flink had a number of strong reasons for dropping Java7
>> > that
>> > > do
>> > > > > not apply in Beam.
>> > > > >
>> > > > > Spark:
>> > > > > JIRA https://issues.apache.org/jira/browse/SPARK-19493
>> > > > > Discussion
>> > > > >
>> > > > http://apache-spark-developers-list.1001551.n3.
>> > > nabble.com/discuss-ending-
>> > > > > support-for-Java-7-in-Spark-2-0-td16808.html
>> > > > > (I
>> > > > > couldn't find a formal poll of the user list rather than developer
>> > > list)
>> > > > >
>> > > > > Hadoop:
>> > > > > Hadoop 3.0 is Java8-only, but I couldn't quickly find a discussion
>> of
>> > > > where
>> > > > > that decision was made.
>> > > > > https://lists.apache.org/thread.html/e5c8085ada2cca47027b63f
>> 5439839
>> > > > > 731a392335770386e10895be06@1444091751@%3Cmapreduce-dev.
>> > > > > hadoop.apache.org%3E
>> > > > > might
>> > > > > be it.
>> > > > >
>> > > > > Kafka is considering it:
>> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> > > > > 118%3A+Drop+Support+for+Java+7+in+Kafka+0.11
>> > > > > and
>> > > > > quotes a number of other open-source projects that have switched
>> > > > > http://markmail.org/message/l7s276y3xkga2eqf
>> > > > >
>> > > > > So basically these projects all did a mailing list poll, and one
>> did
>> > > > also a
>> > > > > twitter poll.
>> > > > >
>> > > > > Beam has the advantage of being a relatively young project with
>> > > perhaps a
>> > > > > smaller base of users entrenched in using old versions of Java;
>> > > moreover,
>> > > > > Java version would matter only for the smaller subset of users who
>> > use
>> > > > Beam
>> > > > > Spark/Flink/Apex/.. runners (as opposed to Cloud Dataflow), which
>> is
>> > > > likely
>> > > > > an even more "early adopter"-ish group of users, as these runners
>> > > > generally
>> > > > > receive less support.
>> > > > >
>> > > > > It may be a good idea to have at least 1 release pass between
>> > > announcing
>> > > > > the intention to drop Java8 and actually dropping it (e.g. if we
>> > > decided
>> > > > it
>> > > > > now, then 2.4 would drop Java7). Also, we could start by switching
>> > > tests
>> > > > to
>> > > > > compile/run with java8 (Maven allows this). This is, I think,
>> pretty
>> > > much
>> > > > > safe to do immediately.
>> > > > >
>> > > > > On Mon, Oct 16, 2017 at 7:35 AM Ismaël Mejía <ie...@gmail.com>
>> > > wrote:
>> > > > >
>> > > > > > Any progress on this? What is the proposed way to validate if
>> users
>> > > > > > are still interested on Java 7? A vote on user or something
>> > > different?
>> > > > > >
>> > > > > >
>> > > > > > On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles
>> > > > <klk@google.com.invalid
>> > > > > >
>> > > > > > wrote:
>> > > > > > > Agree with polling Beam users as well as each runner's
>> community
>> > in
>> > > > > > > aggregate.
>> > > > > > >
>> > > > > > > On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré <
>> > > > jb@nanthrax.net
>> > > > > >
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > >> Definitely agree
>> > > > > > >>
>> > > > > > >>
>> > > > > > >> On 09/27/2017 06:00 PM, Robert Bradshaw wrote:
>> > > > > > >>
>> > > > > > >>> I also think that it's time to seriously consider dropping
>> > > support
>> > > > > for
>> > > > > > >>> Java 7.
>> > > > > > >>>
>> > > > > > >>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
>> > > > > > >>> <da...@google.com.invalid> wrote:
>> > > > > > >>>
>> > > > > > >>>> Yes, just as Ismaël said it's a compilation blocker right
>> now
>> > > > > despite
>> > > > > > >>>> that
>> > > > > > >>>> (I believe) we don't use the extension that's breaking.
>> > > > > > >>>>
>> > > > > > >>>> As for other ways to solve this, if there is a way to avoid
>> > > > > compiling
>> > > > > > the
>> > > > > > >>>> advanced features of AutoValue that might be worth a try. We
>> > > could
>> > > > > > also
>> > > > > > >>>> try
>> > > > > > >>>> to get a release of AutoValue with the fix that works in
>> Java
>> > 7.
>> > > > > > However
>> > > > > > >>>> I
>> > > > > > >>>> feel that slowly moving over to Java 8 is the most
>> > future-proof
>> > > > > > solution
>> > > > > > >>>> if
>> > > > > > >>>> it's possible.
>> > > > > > >>>>
>> > > > > > >>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <
>> > > iemejia@gmail.com>
>> > > > > > wrote:
>> > > > > > >>>>
>> > > > > > >>>> The current issue is that compilation fails on master
>> because
>> > > > beam's
>> > > > > > >>>>> parent pom is configured to fail if it finds warnings):
>> > > > > > >>>>>
>> > > > > > >>>>> <compiler.error.flag>-Werror</compiler.error.flag>
>> > > > > > >>>>>
>> > > > > > >>>>> However if you remove that line from the parent pom the
>> > > > compilation
>> > > > > > >>>>> passes.
>> > > > > > >>>>>
>> > > > > > >>>>> Of course this does not mean that everything is solved for
>> > Java
>> > > > 9,
>> > > > > > >>>>> there are some tests that break and other issues because of
>> > > other
>> > > > > > >>>>> plugins and dependencies (e.g. bytebuddy), but those are
>> not
>> > > part
>> > > > > of
>> > > > > > >>>>> this discussion.
>> > > > > > >>>>>
>> > > > > > >>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
>> > > > > > >>>>> <ki...@google.com.invalid> wrote:
>> > > > > > >>>>>
>> > > > > > >>>>>> AFAIK we don't use any advanced capabilities of AutoValue.
>> > > Does
>> > > > > that
>> > > > > > >>>>>> mean
>> > > > > > >>>>>> this issue is moot? I didn't quite understand from your
>> > email
>> > > > > > whether
>> > > > > > >>>>>> it
>> > > > > > >>>>>>
>> > > > > > >>>>> is
>> > > > > > >>>>>
>> > > > > > >>>>>> a compilation blocker for Beam or not.
>> > > > > > >>>>>>
>> > > > > > >>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <
>> > > iemejia@gmail.com
>> > > > >
>> > > > > > >>>>>> wrote:
>> > > > > > >>>>>>
>> > > > > > >>>>>> Great that you are also working on this too Daniel and
>> > thanks
>> > > > for
>> > > > > > >>>>>>> bringing this subject to the mailing list, I was waiting
>> to
>> > > my
>> > > > > > return
>> > > > > > >>>>>>> to office next week, but you did it first :)
>> > > > > > >>>>>>>
>> > > > > > >>>>>>> Eugene for reference (This is the issue on the migration
>> to
>> > > > Java
>> > > > > > 9),
>> > > > > > >>>>>>> notice that here the goal is first that beam passes mvn
>> > clean
>> > > > > > install
>> > > > > > >>>>>>> with pure Java 9 (and also add this to jenkins), not to
>> > > rewrite
>> > > > > > >>>>>>> anything to use the new stuff (e.g. modules):
>> > > > > > >>>>>>> https://issues.apache.org/jira/browse/BEAM-2530
>> > > > > > >>>>>>>
>> > > > > > >>>>>>> Eugene can you also PTAL at the AutoValue issue, more
>> > details
>> > > > on
>> > > > > > the
>> > > > > > >>>>>>> issue, this is a warning so I don't know if it is really
>> > > > critical
>> > > > > > in
>> > > > > > >>>>>>> particular because we are not using Memoization (do we?).
>> > > > > > >>>>>>> https://github.com/google/auto/issues/503
>> > > > > > >>>>>>>
>> > > > > > >>>>>>>
>> > > > > >
>> > https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d
>> > > > > > >>>>>>>
>> > > > > > >>>>>> 02247b8532
>> > > > > > >>>>>
>> > > > > > >>>>>>
>> > > > > > >>>>>>> Wouldn't the easiest way be that you guys convince the
>> > > > > google.auto
>> > > > > > >>>>>>> guys to generate that simple fix in a Java 7 compatible
>> way
>> > > and
>> > > > > > >>>>>>> 'voila' ?
>> > > > > > >>>>>>>
>> > > > > > >>>>>>> However I agree that moving to Java 8 is an excellent
>> idea
>> > > and
>> > > > as
>> > > > > > >>>>>>> Eugene mentions there is less friction now since most
>> > > projects
>> > > > > are
>> > > > > > >>>>>>> moving, only pending issue are existing clusters on java
>> 7
>> > in
>> > > > the
>> > > > > > >>>>>>> hadoop world, but those are less frequent now. Anyway
>> this
>> > > > > > discussion
>> > > > > > >>>>>>> is really important (maybe even worth a vote). Because
>> > moving
>> > > > to
>> > > > > > Java
>> > > > > > >>>>>>> 8 will allow us also to move some of the dependencies
>> that
>> > we
>> > > > are
>> > > > > > >>>>>>> keeping in old versions and in general to move forward.
>> > > > > > >>>>>>>
>> > > > > > >>>>>>> What do the others think ?
>> > > > > > >>>>>>>
>> > > > > > >>>>>>>
>> > > > > > >>>>>>>
>> > > > > > >>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
>> > > > > > >>>>>>> <ki...@google.com.invalid> wrote:
>> > > > > > >>>>>>>
>> > > > > > >>>>>>>> Very excited to hear that there's work on JDK9 support -
>> > is
>> > > > > there
>> > > > > > a
>> > > > > > >>>>>>>>
>> > > > > > >>>>>>> public
>> > > > > > >>>>>>>
>> > > > > > >>>>>>>> description of the plans for this work somewhere?
>> > > > > > >>>>>>>>
>> > > > > > >>>>>>>> In general, Beam could probably drop Java7 support
>> > > altogether
>> > > > at
>> > > > > > some
>> > > > > > >>>>>>>>
>> > > > > > >>>>>>> point
>> > > > > > >>>>>>>
>> > > > > > >>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not
>> > receiving
>> > > > > even
>> > > > > > >>>>>>>>
>> > > > > > >>>>>>> security
>> > > > > > >>>>>>>
>> > > > > > >>>>>>>> patches) 2 years ago, and all major players in the data
>> > > > > processing
>> > > > > > >>>>>>>> ecosystem have dropped Java7 support (Spark, Flink,
>> > Hadoop),
>> > > > so
>> > > > > I
>> > > > > > >>>>>>>>
>> > > > > > >>>>>>> presume
>> > > > > > >>>>>
>> > > > > > >>>>>> the demand for Java7 support in the data processing
>> industry
>> > > is
>> > > > > low.
>> > > > > > >>>>>>>>
>> > > > > > >>>>>>> By
>> > > > > > >>>>>
>> > > > > > >>>>>> the
>> > > > > > >>>>>>>
>> > > > > > >>>>>>>> way: would a Java8 migration be in the scope of your
>> work
>> > in
>> > > > > > general?
>> > > > > > >>>>>>>>
>> > > > > > >>>>>>>> However, until we say that Beam requires Java8, what
>> would
>> > > be
>> > > > > the
>> > > > > > >>>>>>>> implications of using a version of AutoValue that can
>> only
>> > > be
>> > > > > > >>>>>>>> compiled
>> > > > > > >>>>>>>>
>> > > > > > >>>>>>> with
>> > > > > > >>>>>>>
>> > > > > > >>>>>>>> Java8? Are you saying that this is simply a matter of a
>> > > > compiler
>> > > > > > bug,
>> > > > > > >>>>>>>>
>> > > > > > >>>>>>> and
>> > > > > > >>>>>
>> > > > > > >>>>>> if we use a Java8 compiler but configured to use source
>> and
>> > > > target
>> > > > > > >>>>>>>>
>> > > > > > >>>>>>> versions
>> > > > > > >>>>>>>
>> > > > > > >>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7, then
>> > the
>> > > > > > resulting
>> > > > > > >>>>>>>>
>> > > > > > >>>>>>> Beam
>> > > > > > >>>>>>>
>> > > > > > >>>>>>>> artifacts will be usable by people who don't have Java8?
>> > > > > > >>>>>>>>
>> > > > > > >>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
>> > > > > > >>>>>>>> <da...@google.com.invalid> wrote:
>> > > > > > >>>>>>>>
>> > > > > > >>>>>>>> So I've been working on JDK 9 support for Beam, and I
>> > have a
>> > > > bug
>> > > > > > in
>> > > > > > >>>>>>>>> AutoValue that can be fixed by updating our AutoValue
>> > > > > dependency
>> > > > > > to
>> > > > > > >>>>>>>>>
>> > > > > > >>>>>>>> the
>> > > > > > >>>>>
>> > > > > > >>>>>> latest. The problem is that AutoValue from 1.5+ seems to
>> be
>> > > > banned
>> > > > > > >>>>>>>>>
>> > > > > > >>>>>>>> for
>> > > > > > >>>>>
>> > > > > > >>>>>> Beam
>> > > > > > >>>>>>>
>> > > > > > >>>>>>>> due to requiring Java 8 compilers. However, it should
>> > still
>> > > be
>> > > > > > >>>>>>>>>
>> > > > > > >>>>>>>> possible
>> > > > > > >>>>>
>> > > > > > >>>>>> to
>> > > > > > >>>>>>>
>> > > > > > >>>>>>>> compile and execute Java 7 code from the Java 8 compiler
>> > by
>> > > > > > building
>> > > > > > >>>>>>>>>
>> > > > > > >>>>>>>> with
>> > > > > > >>>>>>>
>> > > > > > >>>>>>>> the correct arguments. So the fix to this bug would
>> > > > essentially
>> > > > > > >>>>>>>>>
>> > > > > > >>>>>>>> require
>> > > > > > >>>>>
>> > > > > > >>>>>> Java 8 compilers even for compiling Java 7 code.
>> > > > > > >>>>>>>>>
>> > > > > > >>>>>>>>> Does anyone need to use Java 7 compilers? Because if
>> not
>> > I
>> > > > > would
>> > > > > > >>>>>>>>>
>> > > > > > >>>>>>>> like to
>> > > > > > >>>>>
>> > > > > > >>>>>> continue with this fix.
>> > > > > > >>>>>>>>>
>> > > > > > >>>>>>>>>
>> > > > > > >>>>>>>
>> > > > > > >>>>>
>> > > > > > >> --
>> > > > > > >> Jean-Baptiste Onofré
>> > > > > > >> jbonofre@apache.org
>> > > > > > >> http://blog.nanthrax.net
>> > > > > > >> Talend - http://www.talend.com
>> > > > > > >>
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Kenneth Knowles <kl...@google.com.INVALID>.
+1 to having runner maintainers weigh in as proxies. Added a few in case
they haven't followed this thread.
On Mon, Oct 16, 2017 at 11:38 PM, Eugene Kirpichov <
kirpichov@google.com.invalid> wrote:
> Agreed that polling Dataflow users makes sense, though I think they are
> very unlikely to have concerns, because unlike Spark/Flink users they
> wouldn't have a "cluster" that they need to migrate to a new JVM: they'd
> only need to recompile their pipelines with JDK 8.
>
> On Mon, Oct 16, 2017 at 11:21 PM Reuven Lax <re...@google.com.invalid>
> wrote:
>
> > I think the Flink and Spark runner maintainers can weigh in here; given
> > that both of those systems are moving to Java 8, I doubt they will have
> > concerns. Same is true for the Dataflow runner - we should probably poll
> > Dataflow users to make sure this is not a problem for any of them.
> >
> > On Mon, Oct 16, 2017 at 11:05 PM, Eugene Kirpichov <
> > kirpichov@google.com.invalid> wrote:
> >
> > > Reuven - do you mean e.g. a poll on the Flink mailing list asking
> whether
> > > there are Flink users who use Beam with Java 7? Or just people who use
> > > Flink with Java 7? (the latter question I'd assume was settled by the
> > poll
> > > about making Flink itself Java8-only?)
> > >
> > > On Mon, Oct 16, 2017 at 10:32 PM Reuven Lax <re...@google.com.invalid>
> > > wrote:
> > >
> > > > I don't know if a vote in @user is sufficient, as it's unfortunately
> > not
> > > > representative of all Beam users. I think the runner communities
> should
> > > be
> > > > polled as well (though I suspect the answer will be the same, that we
> > can
> > > > go ahead and deprecate Java 7 support).
> > > >
> > > > On Mon, Oct 16, 2017 at 4:50 PM, Eugene Kirpichov <
> > > > kirpichov@google.com.invalid> wrote:
> > > >
> > > > > Yeah, a vote on user@ sounds like a good idea. Ismaël, would you
> be
> > > > > interested in driving this process, since you're already working on
> > > Java9
> > > > > support and hence you have a good understanding of what's involved
> > in a
> > > > JDK
> > > > > version migration for a large project?
> > > > >
> > > > > As due diligence, we can look at how the other data processing
> > systems
> > > > > handled dropping Java7.
> > > > >
> > > > > Flink:
> > > > > JIRA https://issues.apache.org/jira/browse/FLINK-7242
> > > > > Discussion
> > > > > http://apache-flink-user-mailing-list-archive.2336050.
> > > > > n4.nabble.com/POLL-Who-still-uses-Java-7-with-Flink-td12216.html
> > > > >
> > > > > They also did a Twitter poll in addition to the mailing list poll,
> > > which
> > > > > seems like a good idea.
> > > > > Note that Flink had a number of strong reasons for dropping Java7
> > that
> > > do
> > > > > not apply in Beam.
> > > > >
> > > > > Spark:
> > > > > JIRA https://issues.apache.org/jira/browse/SPARK-19493
> > > > > Discussion
> > > > >
> > > > http://apache-spark-developers-list.1001551.n3.
> > > nabble.com/discuss-ending-
> > > > > support-for-Java-7-in-Spark-2-0-td16808.html
> > > > > (I
> > > > > couldn't find a formal poll of the user list rather than developer
> > > list)
> > > > >
> > > > > Hadoop:
> > > > > Hadoop 3.0 is Java8-only, but I couldn't quickly find a discussion
> of
> > > > where
> > > > > that decision was made.
> > > > > https://lists.apache.org/thread.html/e5c8085ada2cca47027b63f
> 5439839
> > > > > 731a392335770386e10895be06@1444091751@%3Cmapreduce-dev.
> > > > > hadoop.apache.org%3E
> > > > > might
> > > > > be it.
> > > > >
> > > > > Kafka is considering it:
> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > 118%3A+Drop+Support+for+Java+7+in+Kafka+0.11
> > > > > and
> > > > > quotes a number of other open-source projects that have switched
> > > > > http://markmail.org/message/l7s276y3xkga2eqf
> > > > >
> > > > > So basically these projects all did a mailing list poll, and one
> did
> > > > also a
> > > > > twitter poll.
> > > > >
> > > > > Beam has the advantage of being a relatively young project with
> > > perhaps a
> > > > > smaller base of users entrenched in using old versions of Java;
> > > moreover,
> > > > > Java version would matter only for the smaller subset of users who
> > use
> > > > Beam
> > > > > Spark/Flink/Apex/.. runners (as opposed to Cloud Dataflow), which
> is
> > > > likely
> > > > > an even more "early adopter"-ish group of users, as these runners
> > > > generally
> > > > > receive less support.
> > > > >
> > > > > It may be a good idea to have at least 1 release pass between
> > > announcing
> > > > > the intention to drop Java8 and actually dropping it (e.g. if we
> > > decided
> > > > it
> > > > > now, then 2.4 would drop Java7). Also, we could start by switching
> > > tests
> > > > to
> > > > > compile/run with java8 (Maven allows this). This is, I think,
> pretty
> > > much
> > > > > safe to do immediately.
> > > > >
> > > > > On Mon, Oct 16, 2017 at 7:35 AM Ismaël Mejía <ie...@gmail.com>
> > > wrote:
> > > > >
> > > > > > Any progress on this? What is the proposed way to validate if
> users
> > > > > > are still interested on Java 7? A vote on user or something
> > > different?
> > > > > >
> > > > > >
> > > > > > On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles
> > > > <klk@google.com.invalid
> > > > > >
> > > > > > wrote:
> > > > > > > Agree with polling Beam users as well as each runner's
> community
> > in
> > > > > > > aggregate.
> > > > > > >
> > > > > > > On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré <
> > > > jb@nanthrax.net
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Definitely agree
> > > > > > >>
> > > > > > >>
> > > > > > >> On 09/27/2017 06:00 PM, Robert Bradshaw wrote:
> > > > > > >>
> > > > > > >>> I also think that it's time to seriously consider dropping
> > > support
> > > > > for
> > > > > > >>> Java 7.
> > > > > > >>>
> > > > > > >>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
> > > > > > >>> <da...@google.com.invalid> wrote:
> > > > > > >>>
> > > > > > >>>> Yes, just as Ismaël said it's a compilation blocker right
> now
> > > > > despite
> > > > > > >>>> that
> > > > > > >>>> (I believe) we don't use the extension that's breaking.
> > > > > > >>>>
> > > > > > >>>> As for other ways to solve this, if there is a way to avoid
> > > > > compiling
> > > > > > the
> > > > > > >>>> advanced features of AutoValue that might be worth a try. We
> > > could
> > > > > > also
> > > > > > >>>> try
> > > > > > >>>> to get a release of AutoValue with the fix that works in
> Java
> > 7.
> > > > > > However
> > > > > > >>>> I
> > > > > > >>>> feel that slowly moving over to Java 8 is the most
> > future-proof
> > > > > > solution
> > > > > > >>>> if
> > > > > > >>>> it's possible.
> > > > > > >>>>
> > > > > > >>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <
> > > iemejia@gmail.com>
> > > > > > wrote:
> > > > > > >>>>
> > > > > > >>>> The current issue is that compilation fails on master
> because
> > > > beam's
> > > > > > >>>>> parent pom is configured to fail if it finds warnings):
> > > > > > >>>>>
> > > > > > >>>>> <compiler.error.flag>-Werror</compiler.error.flag>
> > > > > > >>>>>
> > > > > > >>>>> However if you remove that line from the parent pom the
> > > > compilation
> > > > > > >>>>> passes.
> > > > > > >>>>>
> > > > > > >>>>> Of course this does not mean that everything is solved for
> > Java
> > > > 9,
> > > > > > >>>>> there are some tests that break and other issues because of
> > > other
> > > > > > >>>>> plugins and dependencies (e.g. bytebuddy), but those are
> not
> > > part
> > > > > of
> > > > > > >>>>> this discussion.
> > > > > > >>>>>
> > > > > > >>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
> > > > > > >>>>> <ki...@google.com.invalid> wrote:
> > > > > > >>>>>
> > > > > > >>>>>> AFAIK we don't use any advanced capabilities of AutoValue.
> > > Does
> > > > > that
> > > > > > >>>>>> mean
> > > > > > >>>>>> this issue is moot? I didn't quite understand from your
> > email
> > > > > > whether
> > > > > > >>>>>> it
> > > > > > >>>>>>
> > > > > > >>>>> is
> > > > > > >>>>>
> > > > > > >>>>>> a compilation blocker for Beam or not.
> > > > > > >>>>>>
> > > > > > >>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <
> > > iemejia@gmail.com
> > > > >
> > > > > > >>>>>> wrote:
> > > > > > >>>>>>
> > > > > > >>>>>> Great that you are also working on this too Daniel and
> > thanks
> > > > for
> > > > > > >>>>>>> bringing this subject to the mailing list, I was waiting
> to
> > > my
> > > > > > return
> > > > > > >>>>>>> to office next week, but you did it first :)
> > > > > > >>>>>>>
> > > > > > >>>>>>> Eugene for reference (This is the issue on the migration
> to
> > > > Java
> > > > > > 9),
> > > > > > >>>>>>> notice that here the goal is first that beam passes mvn
> > clean
> > > > > > install
> > > > > > >>>>>>> with pure Java 9 (and also add this to jenkins), not to
> > > rewrite
> > > > > > >>>>>>> anything to use the new stuff (e.g. modules):
> > > > > > >>>>>>> https://issues.apache.org/jira/browse/BEAM-2530
> > > > > > >>>>>>>
> > > > > > >>>>>>> Eugene can you also PTAL at the AutoValue issue, more
> > details
> > > > on
> > > > > > the
> > > > > > >>>>>>> issue, this is a warning so I don't know if it is really
> > > > critical
> > > > > > in
> > > > > > >>>>>>> particular because we are not using Memoization (do we?).
> > > > > > >>>>>>> https://github.com/google/auto/issues/503
> > > > > > >>>>>>>
> > > > > > >>>>>>>
> > > > > >
> > https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d
> > > > > > >>>>>>>
> > > > > > >>>>>> 02247b8532
> > > > > > >>>>>
> > > > > > >>>>>>
> > > > > > >>>>>>> Wouldn't the easiest way be that you guys convince the
> > > > > google.auto
> > > > > > >>>>>>> guys to generate that simple fix in a Java 7 compatible
> way
> > > and
> > > > > > >>>>>>> 'voila' ?
> > > > > > >>>>>>>
> > > > > > >>>>>>> However I agree that moving to Java 8 is an excellent
> idea
> > > and
> > > > as
> > > > > > >>>>>>> Eugene mentions there is less friction now since most
> > > projects
> > > > > are
> > > > > > >>>>>>> moving, only pending issue are existing clusters on java
> 7
> > in
> > > > the
> > > > > > >>>>>>> hadoop world, but those are less frequent now. Anyway
> this
> > > > > > discussion
> > > > > > >>>>>>> is really important (maybe even worth a vote). Because
> > moving
> > > > to
> > > > > > Java
> > > > > > >>>>>>> 8 will allow us also to move some of the dependencies
> that
> > we
> > > > are
> > > > > > >>>>>>> keeping in old versions and in general to move forward.
> > > > > > >>>>>>>
> > > > > > >>>>>>> What do the others think ?
> > > > > > >>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
> > > > > > >>>>>>> <ki...@google.com.invalid> wrote:
> > > > > > >>>>>>>
> > > > > > >>>>>>>> Very excited to hear that there's work on JDK9 support -
> > is
> > > > > there
> > > > > > a
> > > > > > >>>>>>>>
> > > > > > >>>>>>> public
> > > > > > >>>>>>>
> > > > > > >>>>>>>> description of the plans for this work somewhere?
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> In general, Beam could probably drop Java7 support
> > > altogether
> > > > at
> > > > > > some
> > > > > > >>>>>>>>
> > > > > > >>>>>>> point
> > > > > > >>>>>>>
> > > > > > >>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not
> > receiving
> > > > > even
> > > > > > >>>>>>>>
> > > > > > >>>>>>> security
> > > > > > >>>>>>>
> > > > > > >>>>>>>> patches) 2 years ago, and all major players in the data
> > > > > processing
> > > > > > >>>>>>>> ecosystem have dropped Java7 support (Spark, Flink,
> > Hadoop),
> > > > so
> > > > > I
> > > > > > >>>>>>>>
> > > > > > >>>>>>> presume
> > > > > > >>>>>
> > > > > > >>>>>> the demand for Java7 support in the data processing
> industry
> > > is
> > > > > low.
> > > > > > >>>>>>>>
> > > > > > >>>>>>> By
> > > > > > >>>>>
> > > > > > >>>>>> the
> > > > > > >>>>>>>
> > > > > > >>>>>>>> way: would a Java8 migration be in the scope of your
> work
> > in
> > > > > > general?
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> However, until we say that Beam requires Java8, what
> would
> > > be
> > > > > the
> > > > > > >>>>>>>> implications of using a version of AutoValue that can
> only
> > > be
> > > > > > >>>>>>>> compiled
> > > > > > >>>>>>>>
> > > > > > >>>>>>> with
> > > > > > >>>>>>>
> > > > > > >>>>>>>> Java8? Are you saying that this is simply a matter of a
> > > > compiler
> > > > > > bug,
> > > > > > >>>>>>>>
> > > > > > >>>>>>> and
> > > > > > >>>>>
> > > > > > >>>>>> if we use a Java8 compiler but configured to use source
> and
> > > > target
> > > > > > >>>>>>>>
> > > > > > >>>>>>> versions
> > > > > > >>>>>>>
> > > > > > >>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7, then
> > the
> > > > > > resulting
> > > > > > >>>>>>>>
> > > > > > >>>>>>> Beam
> > > > > > >>>>>>>
> > > > > > >>>>>>>> artifacts will be usable by people who don't have Java8?
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
> > > > > > >>>>>>>> <da...@google.com.invalid> wrote:
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> So I've been working on JDK 9 support for Beam, and I
> > have a
> > > > bug
> > > > > > in
> > > > > > >>>>>>>>> AutoValue that can be fixed by updating our AutoValue
> > > > > dependency
> > > > > > to
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>> the
> > > > > > >>>>>
> > > > > > >>>>>> latest. The problem is that AutoValue from 1.5+ seems to
> be
> > > > banned
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>> for
> > > > > > >>>>>
> > > > > > >>>>>> Beam
> > > > > > >>>>>>>
> > > > > > >>>>>>>> due to requiring Java 8 compilers. However, it should
> > still
> > > be
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>> possible
> > > > > > >>>>>
> > > > > > >>>>>> to
> > > > > > >>>>>>>
> > > > > > >>>>>>>> compile and execute Java 7 code from the Java 8 compiler
> > by
> > > > > > building
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>> with
> > > > > > >>>>>>>
> > > > > > >>>>>>>> the correct arguments. So the fix to this bug would
> > > > essentially
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>> require
> > > > > > >>>>>
> > > > > > >>>>>> Java 8 compilers even for compiling Java 7 code.
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>> Does anyone need to use Java 7 compilers? Because if
> not
> > I
> > > > > would
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>> like to
> > > > > > >>>>>
> > > > > > >>>>>> continue with this fix.
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>
> > > > > > >> --
> > > > > > >> Jean-Baptiste Onofré
> > > > > > >> jbonofre@apache.org
> > > > > > >> http://blog.nanthrax.net
> > > > > > >> Talend - http://www.talend.com
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Eugene Kirpichov <ki...@google.com.INVALID>.
Agreed that polling Dataflow users makes sense, though I think they are
very unlikely to have concerns, because unlike Spark/Flink users they
wouldn't have a "cluster" that they need to migrate to a new JVM: they'd
only need to recompile their pipelines with JDK 8.
On Mon, Oct 16, 2017 at 11:21 PM Reuven Lax <re...@google.com.invalid>
wrote:
> I think the Flink and Spark runner maintainers can weigh in here; given
> that both of those systems are moving to Java 8, I doubt they will have
> concerns. Same is true for the Dataflow runner - we should probably poll
> Dataflow users to make sure this is not a problem for any of them.
>
> On Mon, Oct 16, 2017 at 11:05 PM, Eugene Kirpichov <
> kirpichov@google.com.invalid> wrote:
>
> > Reuven - do you mean e.g. a poll on the Flink mailing list asking whether
> > there are Flink users who use Beam with Java 7? Or just people who use
> > Flink with Java 7? (the latter question I'd assume was settled by the
> poll
> > about making Flink itself Java8-only?)
> >
> > On Mon, Oct 16, 2017 at 10:32 PM Reuven Lax <re...@google.com.invalid>
> > wrote:
> >
> > > I don't know if a vote in @user is sufficient, as it's unfortunately
> not
> > > representative of all Beam users. I think the runner communities should
> > be
> > > polled as well (though I suspect the answer will be the same, that we
> can
> > > go ahead and deprecate Java 7 support).
> > >
> > > On Mon, Oct 16, 2017 at 4:50 PM, Eugene Kirpichov <
> > > kirpichov@google.com.invalid> wrote:
> > >
> > > > Yeah, a vote on user@ sounds like a good idea. Ismaël, would you be
> > > > interested in driving this process, since you're already working on
> > Java9
> > > > support and hence you have a good understanding of what's involved
> in a
> > > JDK
> > > > version migration for a large project?
> > > >
> > > > As due diligence, we can look at how the other data processing
> systems
> > > > handled dropping Java7.
> > > >
> > > > Flink:
> > > > JIRA https://issues.apache.org/jira/browse/FLINK-7242
> > > > Discussion
> > > > http://apache-flink-user-mailing-list-archive.2336050.
> > > > n4.nabble.com/POLL-Who-still-uses-Java-7-with-Flink-td12216.html
> > > >
> > > > They also did a Twitter poll in addition to the mailing list poll,
> > which
> > > > seems like a good idea.
> > > > Note that Flink had a number of strong reasons for dropping Java7
> that
> > do
> > > > not apply in Beam.
> > > >
> > > > Spark:
> > > > JIRA https://issues.apache.org/jira/browse/SPARK-19493
> > > > Discussion
> > > >
> > > http://apache-spark-developers-list.1001551.n3.
> > nabble.com/discuss-ending-
> > > > support-for-Java-7-in-Spark-2-0-td16808.html
> > > > (I
> > > > couldn't find a formal poll of the user list rather than developer
> > list)
> > > >
> > > > Hadoop:
> > > > Hadoop 3.0 is Java8-only, but I couldn't quickly find a discussion of
> > > where
> > > > that decision was made.
> > > > https://lists.apache.org/thread.html/e5c8085ada2cca47027b63f5439839
> > > > 731a392335770386e10895be06@1444091751@%3Cmapreduce-dev.
> > > > hadoop.apache.org%3E
> > > > might
> > > > be it.
> > > >
> > > > Kafka is considering it:
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 118%3A+Drop+Support+for+Java+7+in+Kafka+0.11
> > > > and
> > > > quotes a number of other open-source projects that have switched
> > > > http://markmail.org/message/l7s276y3xkga2eqf
> > > >
> > > > So basically these projects all did a mailing list poll, and one did
> > > also a
> > > > twitter poll.
> > > >
> > > > Beam has the advantage of being a relatively young project with
> > perhaps a
> > > > smaller base of users entrenched in using old versions of Java;
> > moreover,
> > > > Java version would matter only for the smaller subset of users who
> use
> > > Beam
> > > > Spark/Flink/Apex/.. runners (as opposed to Cloud Dataflow), which is
> > > likely
> > > > an even more "early adopter"-ish group of users, as these runners
> > > generally
> > > > receive less support.
> > > >
> > > > It may be a good idea to have at least 1 release pass between
> > announcing
> > > > the intention to drop Java8 and actually dropping it (e.g. if we
> > decided
> > > it
> > > > now, then 2.4 would drop Java7). Also, we could start by switching
> > tests
> > > to
> > > > compile/run with java8 (Maven allows this). This is, I think, pretty
> > much
> > > > safe to do immediately.
> > > >
> > > > On Mon, Oct 16, 2017 at 7:35 AM Ismaël Mejía <ie...@gmail.com>
> > wrote:
> > > >
> > > > > Any progress on this? What is the proposed way to validate if users
> > > > > are still interested on Java 7? A vote on user or something
> > different?
> > > > >
> > > > >
> > > > > On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles
> > > <klk@google.com.invalid
> > > > >
> > > > > wrote:
> > > > > > Agree with polling Beam users as well as each runner's community
> in
> > > > > > aggregate.
> > > > > >
> > > > > > On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré <
> > > jb@nanthrax.net
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > >> Definitely agree
> > > > > >>
> > > > > >>
> > > > > >> On 09/27/2017 06:00 PM, Robert Bradshaw wrote:
> > > > > >>
> > > > > >>> I also think that it's time to seriously consider dropping
> > support
> > > > for
> > > > > >>> Java 7.
> > > > > >>>
> > > > > >>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
> > > > > >>> <da...@google.com.invalid> wrote:
> > > > > >>>
> > > > > >>>> Yes, just as Ismaël said it's a compilation blocker right now
> > > > despite
> > > > > >>>> that
> > > > > >>>> (I believe) we don't use the extension that's breaking.
> > > > > >>>>
> > > > > >>>> As for other ways to solve this, if there is a way to avoid
> > > > compiling
> > > > > the
> > > > > >>>> advanced features of AutoValue that might be worth a try. We
> > could
> > > > > also
> > > > > >>>> try
> > > > > >>>> to get a release of AutoValue with the fix that works in Java
> 7.
> > > > > However
> > > > > >>>> I
> > > > > >>>> feel that slowly moving over to Java 8 is the most
> future-proof
> > > > > solution
> > > > > >>>> if
> > > > > >>>> it's possible.
> > > > > >>>>
> > > > > >>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <
> > iemejia@gmail.com>
> > > > > wrote:
> > > > > >>>>
> > > > > >>>> The current issue is that compilation fails on master because
> > > beam's
> > > > > >>>>> parent pom is configured to fail if it finds warnings):
> > > > > >>>>>
> > > > > >>>>> <compiler.error.flag>-Werror</compiler.error.flag>
> > > > > >>>>>
> > > > > >>>>> However if you remove that line from the parent pom the
> > > compilation
> > > > > >>>>> passes.
> > > > > >>>>>
> > > > > >>>>> Of course this does not mean that everything is solved for
> Java
> > > 9,
> > > > > >>>>> there are some tests that break and other issues because of
> > other
> > > > > >>>>> plugins and dependencies (e.g. bytebuddy), but those are not
> > part
> > > > of
> > > > > >>>>> this discussion.
> > > > > >>>>>
> > > > > >>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
> > > > > >>>>> <ki...@google.com.invalid> wrote:
> > > > > >>>>>
> > > > > >>>>>> AFAIK we don't use any advanced capabilities of AutoValue.
> > Does
> > > > that
> > > > > >>>>>> mean
> > > > > >>>>>> this issue is moot? I didn't quite understand from your
> email
> > > > > whether
> > > > > >>>>>> it
> > > > > >>>>>>
> > > > > >>>>> is
> > > > > >>>>>
> > > > > >>>>>> a compilation blocker for Beam or not.
> > > > > >>>>>>
> > > > > >>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <
> > iemejia@gmail.com
> > > >
> > > > > >>>>>> wrote:
> > > > > >>>>>>
> > > > > >>>>>> Great that you are also working on this too Daniel and
> thanks
> > > for
> > > > > >>>>>>> bringing this subject to the mailing list, I was waiting to
> > my
> > > > > return
> > > > > >>>>>>> to office next week, but you did it first :)
> > > > > >>>>>>>
> > > > > >>>>>>> Eugene for reference (This is the issue on the migration to
> > > Java
> > > > > 9),
> > > > > >>>>>>> notice that here the goal is first that beam passes mvn
> clean
> > > > > install
> > > > > >>>>>>> with pure Java 9 (and also add this to jenkins), not to
> > rewrite
> > > > > >>>>>>> anything to use the new stuff (e.g. modules):
> > > > > >>>>>>> https://issues.apache.org/jira/browse/BEAM-2530
> > > > > >>>>>>>
> > > > > >>>>>>> Eugene can you also PTAL at the AutoValue issue, more
> details
> > > on
> > > > > the
> > > > > >>>>>>> issue, this is a warning so I don't know if it is really
> > > critical
> > > > > in
> > > > > >>>>>>> particular because we are not using Memoization (do we?).
> > > > > >>>>>>> https://github.com/google/auto/issues/503
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > >
> https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d
> > > > > >>>>>>>
> > > > > >>>>>> 02247b8532
> > > > > >>>>>
> > > > > >>>>>>
> > > > > >>>>>>> Wouldn't the easiest way be that you guys convince the
> > > > google.auto
> > > > > >>>>>>> guys to generate that simple fix in a Java 7 compatible way
> > and
> > > > > >>>>>>> 'voila' ?
> > > > > >>>>>>>
> > > > > >>>>>>> However I agree that moving to Java 8 is an excellent idea
> > and
> > > as
> > > > > >>>>>>> Eugene mentions there is less friction now since most
> > projects
> > > > are
> > > > > >>>>>>> moving, only pending issue are existing clusters on java 7
> in
> > > the
> > > > > >>>>>>> hadoop world, but those are less frequent now. Anyway this
> > > > > discussion
> > > > > >>>>>>> is really important (maybe even worth a vote). Because
> moving
> > > to
> > > > > Java
> > > > > >>>>>>> 8 will allow us also to move some of the dependencies that
> we
> > > are
> > > > > >>>>>>> keeping in old versions and in general to move forward.
> > > > > >>>>>>>
> > > > > >>>>>>> What do the others think ?
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
> > > > > >>>>>>> <ki...@google.com.invalid> wrote:
> > > > > >>>>>>>
> > > > > >>>>>>>> Very excited to hear that there's work on JDK9 support -
> is
> > > > there
> > > > > a
> > > > > >>>>>>>>
> > > > > >>>>>>> public
> > > > > >>>>>>>
> > > > > >>>>>>>> description of the plans for this work somewhere?
> > > > > >>>>>>>>
> > > > > >>>>>>>> In general, Beam could probably drop Java7 support
> > altogether
> > > at
> > > > > some
> > > > > >>>>>>>>
> > > > > >>>>>>> point
> > > > > >>>>>>>
> > > > > >>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not
> receiving
> > > > even
> > > > > >>>>>>>>
> > > > > >>>>>>> security
> > > > > >>>>>>>
> > > > > >>>>>>>> patches) 2 years ago, and all major players in the data
> > > > processing
> > > > > >>>>>>>> ecosystem have dropped Java7 support (Spark, Flink,
> Hadoop),
> > > so
> > > > I
> > > > > >>>>>>>>
> > > > > >>>>>>> presume
> > > > > >>>>>
> > > > > >>>>>> the demand for Java7 support in the data processing industry
> > is
> > > > low.
> > > > > >>>>>>>>
> > > > > >>>>>>> By
> > > > > >>>>>
> > > > > >>>>>> the
> > > > > >>>>>>>
> > > > > >>>>>>>> way: would a Java8 migration be in the scope of your work
> in
> > > > > general?
> > > > > >>>>>>>>
> > > > > >>>>>>>> However, until we say that Beam requires Java8, what would
> > be
> > > > the
> > > > > >>>>>>>> implications of using a version of AutoValue that can only
> > be
> > > > > >>>>>>>> compiled
> > > > > >>>>>>>>
> > > > > >>>>>>> with
> > > > > >>>>>>>
> > > > > >>>>>>>> Java8? Are you saying that this is simply a matter of a
> > > compiler
> > > > > bug,
> > > > > >>>>>>>>
> > > > > >>>>>>> and
> > > > > >>>>>
> > > > > >>>>>> if we use a Java8 compiler but configured to use source and
> > > target
> > > > > >>>>>>>>
> > > > > >>>>>>> versions
> > > > > >>>>>>>
> > > > > >>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7, then
> the
> > > > > resulting
> > > > > >>>>>>>>
> > > > > >>>>>>> Beam
> > > > > >>>>>>>
> > > > > >>>>>>>> artifacts will be usable by people who don't have Java8?
> > > > > >>>>>>>>
> > > > > >>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
> > > > > >>>>>>>> <da...@google.com.invalid> wrote:
> > > > > >>>>>>>>
> > > > > >>>>>>>> So I've been working on JDK 9 support for Beam, and I
> have a
> > > bug
> > > > > in
> > > > > >>>>>>>>> AutoValue that can be fixed by updating our AutoValue
> > > > dependency
> > > > > to
> > > > > >>>>>>>>>
> > > > > >>>>>>>> the
> > > > > >>>>>
> > > > > >>>>>> latest. The problem is that AutoValue from 1.5+ seems to be
> > > banned
> > > > > >>>>>>>>>
> > > > > >>>>>>>> for
> > > > > >>>>>
> > > > > >>>>>> Beam
> > > > > >>>>>>>
> > > > > >>>>>>>> due to requiring Java 8 compilers. However, it should
> still
> > be
> > > > > >>>>>>>>>
> > > > > >>>>>>>> possible
> > > > > >>>>>
> > > > > >>>>>> to
> > > > > >>>>>>>
> > > > > >>>>>>>> compile and execute Java 7 code from the Java 8 compiler
> by
> > > > > building
> > > > > >>>>>>>>>
> > > > > >>>>>>>> with
> > > > > >>>>>>>
> > > > > >>>>>>>> the correct arguments. So the fix to this bug would
> > > essentially
> > > > > >>>>>>>>>
> > > > > >>>>>>>> require
> > > > > >>>>>
> > > > > >>>>>> Java 8 compilers even for compiling Java 7 code.
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> Does anyone need to use Java 7 compilers? Because if not
> I
> > > > would
> > > > > >>>>>>>>>
> > > > > >>>>>>>> like to
> > > > > >>>>>
> > > > > >>>>>> continue with this fix.
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>
> > > > > >> --
> > > > > >> Jean-Baptiste Onofré
> > > > > >> jbonofre@apache.org
> > > > > >> http://blog.nanthrax.net
> > > > > >> Talend - http://www.talend.com
> > > > > >>
> > > > >
> > > >
> > >
> >
>
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Reuven Lax <re...@google.com.INVALID>.
I think the Flink and Spark runner maintainers can weigh in here; given
that both of those systems are moving to Java 8, I doubt they will have
concerns. Same is true for the Dataflow runner - we should probably poll
Dataflow users to make sure this is not a problem for any of them.
On Mon, Oct 16, 2017 at 11:05 PM, Eugene Kirpichov <
kirpichov@google.com.invalid> wrote:
> Reuven - do you mean e.g. a poll on the Flink mailing list asking whether
> there are Flink users who use Beam with Java 7? Or just people who use
> Flink with Java 7? (the latter question I'd assume was settled by the poll
> about making Flink itself Java8-only?)
>
> On Mon, Oct 16, 2017 at 10:32 PM Reuven Lax <re...@google.com.invalid>
> wrote:
>
> > I don't know if a vote in @user is sufficient, as it's unfortunately not
> > representative of all Beam users. I think the runner communities should
> be
> > polled as well (though I suspect the answer will be the same, that we can
> > go ahead and deprecate Java 7 support).
> >
> > On Mon, Oct 16, 2017 at 4:50 PM, Eugene Kirpichov <
> > kirpichov@google.com.invalid> wrote:
> >
> > > Yeah, a vote on user@ sounds like a good idea. Ismaël, would you be
> > > interested in driving this process, since you're already working on
> Java9
> > > support and hence you have a good understanding of what's involved in a
> > JDK
> > > version migration for a large project?
> > >
> > > As due diligence, we can look at how the other data processing systems
> > > handled dropping Java7.
> > >
> > > Flink:
> > > JIRA https://issues.apache.org/jira/browse/FLINK-7242
> > > Discussion
> > > http://apache-flink-user-mailing-list-archive.2336050.
> > > n4.nabble.com/POLL-Who-still-uses-Java-7-with-Flink-td12216.html
> > >
> > > They also did a Twitter poll in addition to the mailing list poll,
> which
> > > seems like a good idea.
> > > Note that Flink had a number of strong reasons for dropping Java7 that
> do
> > > not apply in Beam.
> > >
> > > Spark:
> > > JIRA https://issues.apache.org/jira/browse/SPARK-19493
> > > Discussion
> > >
> > http://apache-spark-developers-list.1001551.n3.
> nabble.com/discuss-ending-
> > > support-for-Java-7-in-Spark-2-0-td16808.html
> > > (I
> > > couldn't find a formal poll of the user list rather than developer
> list)
> > >
> > > Hadoop:
> > > Hadoop 3.0 is Java8-only, but I couldn't quickly find a discussion of
> > where
> > > that decision was made.
> > > https://lists.apache.org/thread.html/e5c8085ada2cca47027b63f5439839
> > > 731a392335770386e10895be06@1444091751@%3Cmapreduce-dev.
> > > hadoop.apache.org%3E
> > > might
> > > be it.
> > >
> > > Kafka is considering it:
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 118%3A+Drop+Support+for+Java+7+in+Kafka+0.11
> > > and
> > > quotes a number of other open-source projects that have switched
> > > http://markmail.org/message/l7s276y3xkga2eqf
> > >
> > > So basically these projects all did a mailing list poll, and one did
> > also a
> > > twitter poll.
> > >
> > > Beam has the advantage of being a relatively young project with
> perhaps a
> > > smaller base of users entrenched in using old versions of Java;
> moreover,
> > > Java version would matter only for the smaller subset of users who use
> > Beam
> > > Spark/Flink/Apex/.. runners (as opposed to Cloud Dataflow), which is
> > likely
> > > an even more "early adopter"-ish group of users, as these runners
> > generally
> > > receive less support.
> > >
> > > It may be a good idea to have at least 1 release pass between
> announcing
> > > the intention to drop Java8 and actually dropping it (e.g. if we
> decided
> > it
> > > now, then 2.4 would drop Java7). Also, we could start by switching
> tests
> > to
> > > compile/run with java8 (Maven allows this). This is, I think, pretty
> much
> > > safe to do immediately.
> > >
> > > On Mon, Oct 16, 2017 at 7:35 AM Ismaël Mejía <ie...@gmail.com>
> wrote:
> > >
> > > > Any progress on this? What is the proposed way to validate if users
> > > > are still interested on Java 7? A vote on user or something
> different?
> > > >
> > > >
> > > > On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles
> > <klk@google.com.invalid
> > > >
> > > > wrote:
> > > > > Agree with polling Beam users as well as each runner's community in
> > > > > aggregate.
> > > > >
> > > > > On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré <
> > jb@nanthrax.net
> > > >
> > > > > wrote:
> > > > >
> > > > >> Definitely agree
> > > > >>
> > > > >>
> > > > >> On 09/27/2017 06:00 PM, Robert Bradshaw wrote:
> > > > >>
> > > > >>> I also think that it's time to seriously consider dropping
> support
> > > for
> > > > >>> Java 7.
> > > > >>>
> > > > >>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
> > > > >>> <da...@google.com.invalid> wrote:
> > > > >>>
> > > > >>>> Yes, just as Ismaël said it's a compilation blocker right now
> > > despite
> > > > >>>> that
> > > > >>>> (I believe) we don't use the extension that's breaking.
> > > > >>>>
> > > > >>>> As for other ways to solve this, if there is a way to avoid
> > > compiling
> > > > the
> > > > >>>> advanced features of AutoValue that might be worth a try. We
> could
> > > > also
> > > > >>>> try
> > > > >>>> to get a release of AutoValue with the fix that works in Java 7.
> > > > However
> > > > >>>> I
> > > > >>>> feel that slowly moving over to Java 8 is the most future-proof
> > > > solution
> > > > >>>> if
> > > > >>>> it's possible.
> > > > >>>>
> > > > >>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <
> iemejia@gmail.com>
> > > > wrote:
> > > > >>>>
> > > > >>>> The current issue is that compilation fails on master because
> > beam's
> > > > >>>>> parent pom is configured to fail if it finds warnings):
> > > > >>>>>
> > > > >>>>> <compiler.error.flag>-Werror</compiler.error.flag>
> > > > >>>>>
> > > > >>>>> However if you remove that line from the parent pom the
> > compilation
> > > > >>>>> passes.
> > > > >>>>>
> > > > >>>>> Of course this does not mean that everything is solved for Java
> > 9,
> > > > >>>>> there are some tests that break and other issues because of
> other
> > > > >>>>> plugins and dependencies (e.g. bytebuddy), but those are not
> part
> > > of
> > > > >>>>> this discussion.
> > > > >>>>>
> > > > >>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
> > > > >>>>> <ki...@google.com.invalid> wrote:
> > > > >>>>>
> > > > >>>>>> AFAIK we don't use any advanced capabilities of AutoValue.
> Does
> > > that
> > > > >>>>>> mean
> > > > >>>>>> this issue is moot? I didn't quite understand from your email
> > > > whether
> > > > >>>>>> it
> > > > >>>>>>
> > > > >>>>> is
> > > > >>>>>
> > > > >>>>>> a compilation blocker for Beam or not.
> > > > >>>>>>
> > > > >>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <
> iemejia@gmail.com
> > >
> > > > >>>>>> wrote:
> > > > >>>>>>
> > > > >>>>>> Great that you are also working on this too Daniel and thanks
> > for
> > > > >>>>>>> bringing this subject to the mailing list, I was waiting to
> my
> > > > return
> > > > >>>>>>> to office next week, but you did it first :)
> > > > >>>>>>>
> > > > >>>>>>> Eugene for reference (This is the issue on the migration to
> > Java
> > > > 9),
> > > > >>>>>>> notice that here the goal is first that beam passes mvn clean
> > > > install
> > > > >>>>>>> with pure Java 9 (and also add this to jenkins), not to
> rewrite
> > > > >>>>>>> anything to use the new stuff (e.g. modules):
> > > > >>>>>>> https://issues.apache.org/jira/browse/BEAM-2530
> > > > >>>>>>>
> > > > >>>>>>> Eugene can you also PTAL at the AutoValue issue, more details
> > on
> > > > the
> > > > >>>>>>> issue, this is a warning so I don't know if it is really
> > critical
> > > > in
> > > > >>>>>>> particular because we are not using Memoization (do we?).
> > > > >>>>>>> https://github.com/google/auto/issues/503
> > > > >>>>>>>
> > > > >>>>>>>
> > > > https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d
> > > > >>>>>>>
> > > > >>>>>> 02247b8532
> > > > >>>>>
> > > > >>>>>>
> > > > >>>>>>> Wouldn't the easiest way be that you guys convince the
> > > google.auto
> > > > >>>>>>> guys to generate that simple fix in a Java 7 compatible way
> and
> > > > >>>>>>> 'voila' ?
> > > > >>>>>>>
> > > > >>>>>>> However I agree that moving to Java 8 is an excellent idea
> and
> > as
> > > > >>>>>>> Eugene mentions there is less friction now since most
> projects
> > > are
> > > > >>>>>>> moving, only pending issue are existing clusters on java 7 in
> > the
> > > > >>>>>>> hadoop world, but those are less frequent now. Anyway this
> > > > discussion
> > > > >>>>>>> is really important (maybe even worth a vote). Because moving
> > to
> > > > Java
> > > > >>>>>>> 8 will allow us also to move some of the dependencies that we
> > are
> > > > >>>>>>> keeping in old versions and in general to move forward.
> > > > >>>>>>>
> > > > >>>>>>> What do the others think ?
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
> > > > >>>>>>> <ki...@google.com.invalid> wrote:
> > > > >>>>>>>
> > > > >>>>>>>> Very excited to hear that there's work on JDK9 support - is
> > > there
> > > > a
> > > > >>>>>>>>
> > > > >>>>>>> public
> > > > >>>>>>>
> > > > >>>>>>>> description of the plans for this work somewhere?
> > > > >>>>>>>>
> > > > >>>>>>>> In general, Beam could probably drop Java7 support
> altogether
> > at
> > > > some
> > > > >>>>>>>>
> > > > >>>>>>> point
> > > > >>>>>>>
> > > > >>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not receiving
> > > even
> > > > >>>>>>>>
> > > > >>>>>>> security
> > > > >>>>>>>
> > > > >>>>>>>> patches) 2 years ago, and all major players in the data
> > > processing
> > > > >>>>>>>> ecosystem have dropped Java7 support (Spark, Flink, Hadoop),
> > so
> > > I
> > > > >>>>>>>>
> > > > >>>>>>> presume
> > > > >>>>>
> > > > >>>>>> the demand for Java7 support in the data processing industry
> is
> > > low.
> > > > >>>>>>>>
> > > > >>>>>>> By
> > > > >>>>>
> > > > >>>>>> the
> > > > >>>>>>>
> > > > >>>>>>>> way: would a Java8 migration be in the scope of your work in
> > > > general?
> > > > >>>>>>>>
> > > > >>>>>>>> However, until we say that Beam requires Java8, what would
> be
> > > the
> > > > >>>>>>>> implications of using a version of AutoValue that can only
> be
> > > > >>>>>>>> compiled
> > > > >>>>>>>>
> > > > >>>>>>> with
> > > > >>>>>>>
> > > > >>>>>>>> Java8? Are you saying that this is simply a matter of a
> > compiler
> > > > bug,
> > > > >>>>>>>>
> > > > >>>>>>> and
> > > > >>>>>
> > > > >>>>>> if we use a Java8 compiler but configured to use source and
> > target
> > > > >>>>>>>>
> > > > >>>>>>> versions
> > > > >>>>>>>
> > > > >>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7, then the
> > > > resulting
> > > > >>>>>>>>
> > > > >>>>>>> Beam
> > > > >>>>>>>
> > > > >>>>>>>> artifacts will be usable by people who don't have Java8?
> > > > >>>>>>>>
> > > > >>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
> > > > >>>>>>>> <da...@google.com.invalid> wrote:
> > > > >>>>>>>>
> > > > >>>>>>>> So I've been working on JDK 9 support for Beam, and I have a
> > bug
> > > > in
> > > > >>>>>>>>> AutoValue that can be fixed by updating our AutoValue
> > > dependency
> > > > to
> > > > >>>>>>>>>
> > > > >>>>>>>> the
> > > > >>>>>
> > > > >>>>>> latest. The problem is that AutoValue from 1.5+ seems to be
> > banned
> > > > >>>>>>>>>
> > > > >>>>>>>> for
> > > > >>>>>
> > > > >>>>>> Beam
> > > > >>>>>>>
> > > > >>>>>>>> due to requiring Java 8 compilers. However, it should still
> be
> > > > >>>>>>>>>
> > > > >>>>>>>> possible
> > > > >>>>>
> > > > >>>>>> to
> > > > >>>>>>>
> > > > >>>>>>>> compile and execute Java 7 code from the Java 8 compiler by
> > > > building
> > > > >>>>>>>>>
> > > > >>>>>>>> with
> > > > >>>>>>>
> > > > >>>>>>>> the correct arguments. So the fix to this bug would
> > essentially
> > > > >>>>>>>>>
> > > > >>>>>>>> require
> > > > >>>>>
> > > > >>>>>> Java 8 compilers even for compiling Java 7 code.
> > > > >>>>>>>>>
> > > > >>>>>>>>> Does anyone need to use Java 7 compilers? Because if not I
> > > would
> > > > >>>>>>>>>
> > > > >>>>>>>> like to
> > > > >>>>>
> > > > >>>>>> continue with this fix.
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>
> > > > >>>>>
> > > > >> --
> > > > >> Jean-Baptiste Onofré
> > > > >> jbonofre@apache.org
> > > > >> http://blog.nanthrax.net
> > > > >> Talend - http://www.talend.com
> > > > >>
> > > >
> > >
> >
>
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Eugene Kirpichov <ki...@google.com.INVALID>.
Reuven - do you mean e.g. a poll on the Flink mailing list asking whether
there are Flink users who use Beam with Java 7? Or just people who use
Flink with Java 7? (the latter question I'd assume was settled by the poll
about making Flink itself Java8-only?)
On Mon, Oct 16, 2017 at 10:32 PM Reuven Lax <re...@google.com.invalid>
wrote:
> I don't know if a vote in @user is sufficient, as it's unfortunately not
> representative of all Beam users. I think the runner communities should be
> polled as well (though I suspect the answer will be the same, that we can
> go ahead and deprecate Java 7 support).
>
> On Mon, Oct 16, 2017 at 4:50 PM, Eugene Kirpichov <
> kirpichov@google.com.invalid> wrote:
>
> > Yeah, a vote on user@ sounds like a good idea. Ismaël, would you be
> > interested in driving this process, since you're already working on Java9
> > support and hence you have a good understanding of what's involved in a
> JDK
> > version migration for a large project?
> >
> > As due diligence, we can look at how the other data processing systems
> > handled dropping Java7.
> >
> > Flink:
> > JIRA https://issues.apache.org/jira/browse/FLINK-7242
> > Discussion
> > http://apache-flink-user-mailing-list-archive.2336050.
> > n4.nabble.com/POLL-Who-still-uses-Java-7-with-Flink-td12216.html
> >
> > They also did a Twitter poll in addition to the mailing list poll, which
> > seems like a good idea.
> > Note that Flink had a number of strong reasons for dropping Java7 that do
> > not apply in Beam.
> >
> > Spark:
> > JIRA https://issues.apache.org/jira/browse/SPARK-19493
> > Discussion
> >
> http://apache-spark-developers-list.1001551.n3.nabble.com/discuss-ending-
> > support-for-Java-7-in-Spark-2-0-td16808.html
> > (I
> > couldn't find a formal poll of the user list rather than developer list)
> >
> > Hadoop:
> > Hadoop 3.0 is Java8-only, but I couldn't quickly find a discussion of
> where
> > that decision was made.
> > https://lists.apache.org/thread.html/e5c8085ada2cca47027b63f5439839
> > 731a392335770386e10895be06@1444091751@%3Cmapreduce-dev.
> > hadoop.apache.org%3E
> > might
> > be it.
> >
> > Kafka is considering it:
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 118%3A+Drop+Support+for+Java+7+in+Kafka+0.11
> > and
> > quotes a number of other open-source projects that have switched
> > http://markmail.org/message/l7s276y3xkga2eqf
> >
> > So basically these projects all did a mailing list poll, and one did
> also a
> > twitter poll.
> >
> > Beam has the advantage of being a relatively young project with perhaps a
> > smaller base of users entrenched in using old versions of Java; moreover,
> > Java version would matter only for the smaller subset of users who use
> Beam
> > Spark/Flink/Apex/.. runners (as opposed to Cloud Dataflow), which is
> likely
> > an even more "early adopter"-ish group of users, as these runners
> generally
> > receive less support.
> >
> > It may be a good idea to have at least 1 release pass between announcing
> > the intention to drop Java8 and actually dropping it (e.g. if we decided
> it
> > now, then 2.4 would drop Java7). Also, we could start by switching tests
> to
> > compile/run with java8 (Maven allows this). This is, I think, pretty much
> > safe to do immediately.
> >
> > On Mon, Oct 16, 2017 at 7:35 AM Ismaël Mejía <ie...@gmail.com> wrote:
> >
> > > Any progress on this? What is the proposed way to validate if users
> > > are still interested on Java 7? A vote on user or something different?
> > >
> > >
> > > On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles
> <klk@google.com.invalid
> > >
> > > wrote:
> > > > Agree with polling Beam users as well as each runner's community in
> > > > aggregate.
> > > >
> > > > On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré <
> jb@nanthrax.net
> > >
> > > > wrote:
> > > >
> > > >> Definitely agree
> > > >>
> > > >>
> > > >> On 09/27/2017 06:00 PM, Robert Bradshaw wrote:
> > > >>
> > > >>> I also think that it's time to seriously consider dropping support
> > for
> > > >>> Java 7.
> > > >>>
> > > >>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
> > > >>> <da...@google.com.invalid> wrote:
> > > >>>
> > > >>>> Yes, just as Ismaël said it's a compilation blocker right now
> > despite
> > > >>>> that
> > > >>>> (I believe) we don't use the extension that's breaking.
> > > >>>>
> > > >>>> As for other ways to solve this, if there is a way to avoid
> > compiling
> > > the
> > > >>>> advanced features of AutoValue that might be worth a try. We could
> > > also
> > > >>>> try
> > > >>>> to get a release of AutoValue with the fix that works in Java 7.
> > > However
> > > >>>> I
> > > >>>> feel that slowly moving over to Java 8 is the most future-proof
> > > solution
> > > >>>> if
> > > >>>> it's possible.
> > > >>>>
> > > >>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <ie...@gmail.com>
> > > wrote:
> > > >>>>
> > > >>>> The current issue is that compilation fails on master because
> beam's
> > > >>>>> parent pom is configured to fail if it finds warnings):
> > > >>>>>
> > > >>>>> <compiler.error.flag>-Werror</compiler.error.flag>
> > > >>>>>
> > > >>>>> However if you remove that line from the parent pom the
> compilation
> > > >>>>> passes.
> > > >>>>>
> > > >>>>> Of course this does not mean that everything is solved for Java
> 9,
> > > >>>>> there are some tests that break and other issues because of other
> > > >>>>> plugins and dependencies (e.g. bytebuddy), but those are not part
> > of
> > > >>>>> this discussion.
> > > >>>>>
> > > >>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
> > > >>>>> <ki...@google.com.invalid> wrote:
> > > >>>>>
> > > >>>>>> AFAIK we don't use any advanced capabilities of AutoValue. Does
> > that
> > > >>>>>> mean
> > > >>>>>> this issue is moot? I didn't quite understand from your email
> > > whether
> > > >>>>>> it
> > > >>>>>>
> > > >>>>> is
> > > >>>>>
> > > >>>>>> a compilation blocker for Beam or not.
> > > >>>>>>
> > > >>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <iemejia@gmail.com
> >
> > > >>>>>> wrote:
> > > >>>>>>
> > > >>>>>> Great that you are also working on this too Daniel and thanks
> for
> > > >>>>>>> bringing this subject to the mailing list, I was waiting to my
> > > return
> > > >>>>>>> to office next week, but you did it first :)
> > > >>>>>>>
> > > >>>>>>> Eugene for reference (This is the issue on the migration to
> Java
> > > 9),
> > > >>>>>>> notice that here the goal is first that beam passes mvn clean
> > > install
> > > >>>>>>> with pure Java 9 (and also add this to jenkins), not to rewrite
> > > >>>>>>> anything to use the new stuff (e.g. modules):
> > > >>>>>>> https://issues.apache.org/jira/browse/BEAM-2530
> > > >>>>>>>
> > > >>>>>>> Eugene can you also PTAL at the AutoValue issue, more details
> on
> > > the
> > > >>>>>>> issue, this is a warning so I don't know if it is really
> critical
> > > in
> > > >>>>>>> particular because we are not using Memoization (do we?).
> > > >>>>>>> https://github.com/google/auto/issues/503
> > > >>>>>>>
> > > >>>>>>>
> > > https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d
> > > >>>>>>>
> > > >>>>>> 02247b8532
> > > >>>>>
> > > >>>>>>
> > > >>>>>>> Wouldn't the easiest way be that you guys convince the
> > google.auto
> > > >>>>>>> guys to generate that simple fix in a Java 7 compatible way and
> > > >>>>>>> 'voila' ?
> > > >>>>>>>
> > > >>>>>>> However I agree that moving to Java 8 is an excellent idea and
> as
> > > >>>>>>> Eugene mentions there is less friction now since most projects
> > are
> > > >>>>>>> moving, only pending issue are existing clusters on java 7 in
> the
> > > >>>>>>> hadoop world, but those are less frequent now. Anyway this
> > > discussion
> > > >>>>>>> is really important (maybe even worth a vote). Because moving
> to
> > > Java
> > > >>>>>>> 8 will allow us also to move some of the dependencies that we
> are
> > > >>>>>>> keeping in old versions and in general to move forward.
> > > >>>>>>>
> > > >>>>>>> What do the others think ?
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
> > > >>>>>>> <ki...@google.com.invalid> wrote:
> > > >>>>>>>
> > > >>>>>>>> Very excited to hear that there's work on JDK9 support - is
> > there
> > > a
> > > >>>>>>>>
> > > >>>>>>> public
> > > >>>>>>>
> > > >>>>>>>> description of the plans for this work somewhere?
> > > >>>>>>>>
> > > >>>>>>>> In general, Beam could probably drop Java7 support altogether
> at
> > > some
> > > >>>>>>>>
> > > >>>>>>> point
> > > >>>>>>>
> > > >>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not receiving
> > even
> > > >>>>>>>>
> > > >>>>>>> security
> > > >>>>>>>
> > > >>>>>>>> patches) 2 years ago, and all major players in the data
> > processing
> > > >>>>>>>> ecosystem have dropped Java7 support (Spark, Flink, Hadoop),
> so
> > I
> > > >>>>>>>>
> > > >>>>>>> presume
> > > >>>>>
> > > >>>>>> the demand for Java7 support in the data processing industry is
> > low.
> > > >>>>>>>>
> > > >>>>>>> By
> > > >>>>>
> > > >>>>>> the
> > > >>>>>>>
> > > >>>>>>>> way: would a Java8 migration be in the scope of your work in
> > > general?
> > > >>>>>>>>
> > > >>>>>>>> However, until we say that Beam requires Java8, what would be
> > the
> > > >>>>>>>> implications of using a version of AutoValue that can only be
> > > >>>>>>>> compiled
> > > >>>>>>>>
> > > >>>>>>> with
> > > >>>>>>>
> > > >>>>>>>> Java8? Are you saying that this is simply a matter of a
> compiler
> > > bug,
> > > >>>>>>>>
> > > >>>>>>> and
> > > >>>>>
> > > >>>>>> if we use a Java8 compiler but configured to use source and
> target
> > > >>>>>>>>
> > > >>>>>>> versions
> > > >>>>>>>
> > > >>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7, then the
> > > resulting
> > > >>>>>>>>
> > > >>>>>>> Beam
> > > >>>>>>>
> > > >>>>>>>> artifacts will be usable by people who don't have Java8?
> > > >>>>>>>>
> > > >>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
> > > >>>>>>>> <da...@google.com.invalid> wrote:
> > > >>>>>>>>
> > > >>>>>>>> So I've been working on JDK 9 support for Beam, and I have a
> bug
> > > in
> > > >>>>>>>>> AutoValue that can be fixed by updating our AutoValue
> > dependency
> > > to
> > > >>>>>>>>>
> > > >>>>>>>> the
> > > >>>>>
> > > >>>>>> latest. The problem is that AutoValue from 1.5+ seems to be
> banned
> > > >>>>>>>>>
> > > >>>>>>>> for
> > > >>>>>
> > > >>>>>> Beam
> > > >>>>>>>
> > > >>>>>>>> due to requiring Java 8 compilers. However, it should still be
> > > >>>>>>>>>
> > > >>>>>>>> possible
> > > >>>>>
> > > >>>>>> to
> > > >>>>>>>
> > > >>>>>>>> compile and execute Java 7 code from the Java 8 compiler by
> > > building
> > > >>>>>>>>>
> > > >>>>>>>> with
> > > >>>>>>>
> > > >>>>>>>> the correct arguments. So the fix to this bug would
> essentially
> > > >>>>>>>>>
> > > >>>>>>>> require
> > > >>>>>
> > > >>>>>> Java 8 compilers even for compiling Java 7 code.
> > > >>>>>>>>>
> > > >>>>>>>>> Does anyone need to use Java 7 compilers? Because if not I
> > would
> > > >>>>>>>>>
> > > >>>>>>>> like to
> > > >>>>>
> > > >>>>>> continue with this fix.
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>
> > > >>>>>
> > > >> --
> > > >> Jean-Baptiste Onofré
> > > >> jbonofre@apache.org
> > > >> http://blog.nanthrax.net
> > > >> Talend - http://www.talend.com
> > > >>
> > >
> >
>
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Reuven Lax <re...@google.com.INVALID>.
I don't know if a vote in @user is sufficient, as it's unfortunately not
representative of all Beam users. I think the runner communities should be
polled as well (though I suspect the answer will be the same, that we can
go ahead and deprecate Java 7 support).
On Mon, Oct 16, 2017 at 4:50 PM, Eugene Kirpichov <
kirpichov@google.com.invalid> wrote:
> Yeah, a vote on user@ sounds like a good idea. Ismaël, would you be
> interested in driving this process, since you're already working on Java9
> support and hence you have a good understanding of what's involved in a JDK
> version migration for a large project?
>
> As due diligence, we can look at how the other data processing systems
> handled dropping Java7.
>
> Flink:
> JIRA https://issues.apache.org/jira/browse/FLINK-7242
> Discussion
> http://apache-flink-user-mailing-list-archive.2336050.
> n4.nabble.com/POLL-Who-still-uses-Java-7-with-Flink-td12216.html
>
> They also did a Twitter poll in addition to the mailing list poll, which
> seems like a good idea.
> Note that Flink had a number of strong reasons for dropping Java7 that do
> not apply in Beam.
>
> Spark:
> JIRA https://issues.apache.org/jira/browse/SPARK-19493
> Discussion
> http://apache-spark-developers-list.1001551.n3.nabble.com/discuss-ending-
> support-for-Java-7-in-Spark-2-0-td16808.html
> (I
> couldn't find a formal poll of the user list rather than developer list)
>
> Hadoop:
> Hadoop 3.0 is Java8-only, but I couldn't quickly find a discussion of where
> that decision was made.
> https://lists.apache.org/thread.html/e5c8085ada2cca47027b63f5439839
> 731a392335770386e10895be06@1444091751@%3Cmapreduce-dev.
> hadoop.apache.org%3E
> might
> be it.
>
> Kafka is considering it:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 118%3A+Drop+Support+for+Java+7+in+Kafka+0.11
> and
> quotes a number of other open-source projects that have switched
> http://markmail.org/message/l7s276y3xkga2eqf
>
> So basically these projects all did a mailing list poll, and one did also a
> twitter poll.
>
> Beam has the advantage of being a relatively young project with perhaps a
> smaller base of users entrenched in using old versions of Java; moreover,
> Java version would matter only for the smaller subset of users who use Beam
> Spark/Flink/Apex/.. runners (as opposed to Cloud Dataflow), which is likely
> an even more "early adopter"-ish group of users, as these runners generally
> receive less support.
>
> It may be a good idea to have at least 1 release pass between announcing
> the intention to drop Java8 and actually dropping it (e.g. if we decided it
> now, then 2.4 would drop Java7). Also, we could start by switching tests to
> compile/run with java8 (Maven allows this). This is, I think, pretty much
> safe to do immediately.
>
> On Mon, Oct 16, 2017 at 7:35 AM Ismaël Mejía <ie...@gmail.com> wrote:
>
> > Any progress on this? What is the proposed way to validate if users
> > are still interested on Java 7? A vote on user or something different?
> >
> >
> > On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles <klk@google.com.invalid
> >
> > wrote:
> > > Agree with polling Beam users as well as each runner's community in
> > > aggregate.
> > >
> > > On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré <jb@nanthrax.net
> >
> > > wrote:
> > >
> > >> Definitely agree
> > >>
> > >>
> > >> On 09/27/2017 06:00 PM, Robert Bradshaw wrote:
> > >>
> > >>> I also think that it's time to seriously consider dropping support
> for
> > >>> Java 7.
> > >>>
> > >>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
> > >>> <da...@google.com.invalid> wrote:
> > >>>
> > >>>> Yes, just as Ismaël said it's a compilation blocker right now
> despite
> > >>>> that
> > >>>> (I believe) we don't use the extension that's breaking.
> > >>>>
> > >>>> As for other ways to solve this, if there is a way to avoid
> compiling
> > the
> > >>>> advanced features of AutoValue that might be worth a try. We could
> > also
> > >>>> try
> > >>>> to get a release of AutoValue with the fix that works in Java 7.
> > However
> > >>>> I
> > >>>> feel that slowly moving over to Java 8 is the most future-proof
> > solution
> > >>>> if
> > >>>> it's possible.
> > >>>>
> > >>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <ie...@gmail.com>
> > wrote:
> > >>>>
> > >>>> The current issue is that compilation fails on master because beam's
> > >>>>> parent pom is configured to fail if it finds warnings):
> > >>>>>
> > >>>>> <compiler.error.flag>-Werror</compiler.error.flag>
> > >>>>>
> > >>>>> However if you remove that line from the parent pom the compilation
> > >>>>> passes.
> > >>>>>
> > >>>>> Of course this does not mean that everything is solved for Java 9,
> > >>>>> there are some tests that break and other issues because of other
> > >>>>> plugins and dependencies (e.g. bytebuddy), but those are not part
> of
> > >>>>> this discussion.
> > >>>>>
> > >>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
> > >>>>> <ki...@google.com.invalid> wrote:
> > >>>>>
> > >>>>>> AFAIK we don't use any advanced capabilities of AutoValue. Does
> that
> > >>>>>> mean
> > >>>>>> this issue is moot? I didn't quite understand from your email
> > whether
> > >>>>>> it
> > >>>>>>
> > >>>>> is
> > >>>>>
> > >>>>>> a compilation blocker for Beam or not.
> > >>>>>>
> > >>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <ie...@gmail.com>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>> Great that you are also working on this too Daniel and thanks for
> > >>>>>>> bringing this subject to the mailing list, I was waiting to my
> > return
> > >>>>>>> to office next week, but you did it first :)
> > >>>>>>>
> > >>>>>>> Eugene for reference (This is the issue on the migration to Java
> > 9),
> > >>>>>>> notice that here the goal is first that beam passes mvn clean
> > install
> > >>>>>>> with pure Java 9 (and also add this to jenkins), not to rewrite
> > >>>>>>> anything to use the new stuff (e.g. modules):
> > >>>>>>> https://issues.apache.org/jira/browse/BEAM-2530
> > >>>>>>>
> > >>>>>>> Eugene can you also PTAL at the AutoValue issue, more details on
> > the
> > >>>>>>> issue, this is a warning so I don't know if it is really critical
> > in
> > >>>>>>> particular because we are not using Memoization (do we?).
> > >>>>>>> https://github.com/google/auto/issues/503
> > >>>>>>>
> > >>>>>>>
> > https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d
> > >>>>>>>
> > >>>>>> 02247b8532
> > >>>>>
> > >>>>>>
> > >>>>>>> Wouldn't the easiest way be that you guys convince the
> google.auto
> > >>>>>>> guys to generate that simple fix in a Java 7 compatible way and
> > >>>>>>> 'voila' ?
> > >>>>>>>
> > >>>>>>> However I agree that moving to Java 8 is an excellent idea and as
> > >>>>>>> Eugene mentions there is less friction now since most projects
> are
> > >>>>>>> moving, only pending issue are existing clusters on java 7 in the
> > >>>>>>> hadoop world, but those are less frequent now. Anyway this
> > discussion
> > >>>>>>> is really important (maybe even worth a vote). Because moving to
> > Java
> > >>>>>>> 8 will allow us also to move some of the dependencies that we are
> > >>>>>>> keeping in old versions and in general to move forward.
> > >>>>>>>
> > >>>>>>> What do the others think ?
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
> > >>>>>>> <ki...@google.com.invalid> wrote:
> > >>>>>>>
> > >>>>>>>> Very excited to hear that there's work on JDK9 support - is
> there
> > a
> > >>>>>>>>
> > >>>>>>> public
> > >>>>>>>
> > >>>>>>>> description of the plans for this work somewhere?
> > >>>>>>>>
> > >>>>>>>> In general, Beam could probably drop Java7 support altogether at
> > some
> > >>>>>>>>
> > >>>>>>> point
> > >>>>>>>
> > >>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not receiving
> even
> > >>>>>>>>
> > >>>>>>> security
> > >>>>>>>
> > >>>>>>>> patches) 2 years ago, and all major players in the data
> processing
> > >>>>>>>> ecosystem have dropped Java7 support (Spark, Flink, Hadoop), so
> I
> > >>>>>>>>
> > >>>>>>> presume
> > >>>>>
> > >>>>>> the demand for Java7 support in the data processing industry is
> low.
> > >>>>>>>>
> > >>>>>>> By
> > >>>>>
> > >>>>>> the
> > >>>>>>>
> > >>>>>>>> way: would a Java8 migration be in the scope of your work in
> > general?
> > >>>>>>>>
> > >>>>>>>> However, until we say that Beam requires Java8, what would be
> the
> > >>>>>>>> implications of using a version of AutoValue that can only be
> > >>>>>>>> compiled
> > >>>>>>>>
> > >>>>>>> with
> > >>>>>>>
> > >>>>>>>> Java8? Are you saying that this is simply a matter of a compiler
> > bug,
> > >>>>>>>>
> > >>>>>>> and
> > >>>>>
> > >>>>>> if we use a Java8 compiler but configured to use source and target
> > >>>>>>>>
> > >>>>>>> versions
> > >>>>>>>
> > >>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7, then the
> > resulting
> > >>>>>>>>
> > >>>>>>> Beam
> > >>>>>>>
> > >>>>>>>> artifacts will be usable by people who don't have Java8?
> > >>>>>>>>
> > >>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
> > >>>>>>>> <da...@google.com.invalid> wrote:
> > >>>>>>>>
> > >>>>>>>> So I've been working on JDK 9 support for Beam, and I have a bug
> > in
> > >>>>>>>>> AutoValue that can be fixed by updating our AutoValue
> dependency
> > to
> > >>>>>>>>>
> > >>>>>>>> the
> > >>>>>
> > >>>>>> latest. The problem is that AutoValue from 1.5+ seems to be banned
> > >>>>>>>>>
> > >>>>>>>> for
> > >>>>>
> > >>>>>> Beam
> > >>>>>>>
> > >>>>>>>> due to requiring Java 8 compilers. However, it should still be
> > >>>>>>>>>
> > >>>>>>>> possible
> > >>>>>
> > >>>>>> to
> > >>>>>>>
> > >>>>>>>> compile and execute Java 7 code from the Java 8 compiler by
> > building
> > >>>>>>>>>
> > >>>>>>>> with
> > >>>>>>>
> > >>>>>>>> the correct arguments. So the fix to this bug would essentially
> > >>>>>>>>>
> > >>>>>>>> require
> > >>>>>
> > >>>>>> Java 8 compilers even for compiling Java 7 code.
> > >>>>>>>>>
> > >>>>>>>>> Does anyone need to use Java 7 compilers? Because if not I
> would
> > >>>>>>>>>
> > >>>>>>>> like to
> > >>>>>
> > >>>>>> continue with this fix.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>
> > >>>>>
> > >> --
> > >> Jean-Baptiste Onofré
> > >> jbonofre@apache.org
> > >> http://blog.nanthrax.net
> > >> Talend - http://www.talend.com
> > >>
> >
>
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Eugene Kirpichov <ki...@google.com.INVALID>.
Yeah, a vote on user@ sounds like a good idea. Ismaël, would you be
interested in driving this process, since you're already working on Java9
support and hence you have a good understanding of what's involved in a JDK
version migration for a large project?
As due diligence, we can look at how the other data processing systems
handled dropping Java7.
Flink:
JIRA https://issues.apache.org/jira/browse/FLINK-7242
Discussion
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/POLL-Who-still-uses-Java-7-with-Flink-td12216.html
They also did a Twitter poll in addition to the mailing list poll, which
seems like a good idea.
Note that Flink had a number of strong reasons for dropping Java7 that do
not apply in Beam.
Spark:
JIRA https://issues.apache.org/jira/browse/SPARK-19493
Discussion
http://apache-spark-developers-list.1001551.n3.nabble.com/discuss-ending-support-for-Java-7-in-Spark-2-0-td16808.html
(I
couldn't find a formal poll of the user list rather than developer list)
Hadoop:
Hadoop 3.0 is Java8-only, but I couldn't quickly find a discussion of where
that decision was made.
https://lists.apache.org/thread.html/e5c8085ada2cca47027b63f5439839731a392335770386e10895be06@1444091751@%3Cmapreduce-dev.hadoop.apache.org%3E
might
be it.
Kafka is considering it:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-118%3A+Drop+Support+for+Java+7+in+Kafka+0.11
and
quotes a number of other open-source projects that have switched
http://markmail.org/message/l7s276y3xkga2eqf
So basically these projects all did a mailing list poll, and one did also a
twitter poll.
Beam has the advantage of being a relatively young project with perhaps a
smaller base of users entrenched in using old versions of Java; moreover,
Java version would matter only for the smaller subset of users who use Beam
Spark/Flink/Apex/.. runners (as opposed to Cloud Dataflow), which is likely
an even more "early adopter"-ish group of users, as these runners generally
receive less support.
It may be a good idea to have at least 1 release pass between announcing
the intention to drop Java8 and actually dropping it (e.g. if we decided it
now, then 2.4 would drop Java7). Also, we could start by switching tests to
compile/run with java8 (Maven allows this). This is, I think, pretty much
safe to do immediately.
On Mon, Oct 16, 2017 at 7:35 AM Ismaël Mejía <ie...@gmail.com> wrote:
> Any progress on this? What is the proposed way to validate if users
> are still interested on Java 7? A vote on user or something different?
>
>
> On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles <kl...@google.com.invalid>
> wrote:
> > Agree with polling Beam users as well as each runner's community in
> > aggregate.
> >
> > On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> > wrote:
> >
> >> Definitely agree
> >>
> >>
> >> On 09/27/2017 06:00 PM, Robert Bradshaw wrote:
> >>
> >>> I also think that it's time to seriously consider dropping support for
> >>> Java 7.
> >>>
> >>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
> >>> <da...@google.com.invalid> wrote:
> >>>
> >>>> Yes, just as Ismaël said it's a compilation blocker right now despite
> >>>> that
> >>>> (I believe) we don't use the extension that's breaking.
> >>>>
> >>>> As for other ways to solve this, if there is a way to avoid compiling
> the
> >>>> advanced features of AutoValue that might be worth a try. We could
> also
> >>>> try
> >>>> to get a release of AutoValue with the fix that works in Java 7.
> However
> >>>> I
> >>>> feel that slowly moving over to Java 8 is the most future-proof
> solution
> >>>> if
> >>>> it's possible.
> >>>>
> >>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <ie...@gmail.com>
> wrote:
> >>>>
> >>>> The current issue is that compilation fails on master because beam's
> >>>>> parent pom is configured to fail if it finds warnings):
> >>>>>
> >>>>> <compiler.error.flag>-Werror</compiler.error.flag>
> >>>>>
> >>>>> However if you remove that line from the parent pom the compilation
> >>>>> passes.
> >>>>>
> >>>>> Of course this does not mean that everything is solved for Java 9,
> >>>>> there are some tests that break and other issues because of other
> >>>>> plugins and dependencies (e.g. bytebuddy), but those are not part of
> >>>>> this discussion.
> >>>>>
> >>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
> >>>>> <ki...@google.com.invalid> wrote:
> >>>>>
> >>>>>> AFAIK we don't use any advanced capabilities of AutoValue. Does that
> >>>>>> mean
> >>>>>> this issue is moot? I didn't quite understand from your email
> whether
> >>>>>> it
> >>>>>>
> >>>>> is
> >>>>>
> >>>>>> a compilation blocker for Beam or not.
> >>>>>>
> >>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <ie...@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>> Great that you are also working on this too Daniel and thanks for
> >>>>>>> bringing this subject to the mailing list, I was waiting to my
> return
> >>>>>>> to office next week, but you did it first :)
> >>>>>>>
> >>>>>>> Eugene for reference (This is the issue on the migration to Java
> 9),
> >>>>>>> notice that here the goal is first that beam passes mvn clean
> install
> >>>>>>> with pure Java 9 (and also add this to jenkins), not to rewrite
> >>>>>>> anything to use the new stuff (e.g. modules):
> >>>>>>> https://issues.apache.org/jira/browse/BEAM-2530
> >>>>>>>
> >>>>>>> Eugene can you also PTAL at the AutoValue issue, more details on
> the
> >>>>>>> issue, this is a warning so I don't know if it is really critical
> in
> >>>>>>> particular because we are not using Memoization (do we?).
> >>>>>>> https://github.com/google/auto/issues/503
> >>>>>>>
> >>>>>>>
> https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d
> >>>>>>>
> >>>>>> 02247b8532
> >>>>>
> >>>>>>
> >>>>>>> Wouldn't the easiest way be that you guys convince the google.auto
> >>>>>>> guys to generate that simple fix in a Java 7 compatible way and
> >>>>>>> 'voila' ?
> >>>>>>>
> >>>>>>> However I agree that moving to Java 8 is an excellent idea and as
> >>>>>>> Eugene mentions there is less friction now since most projects are
> >>>>>>> moving, only pending issue are existing clusters on java 7 in the
> >>>>>>> hadoop world, but those are less frequent now. Anyway this
> discussion
> >>>>>>> is really important (maybe even worth a vote). Because moving to
> Java
> >>>>>>> 8 will allow us also to move some of the dependencies that we are
> >>>>>>> keeping in old versions and in general to move forward.
> >>>>>>>
> >>>>>>> What do the others think ?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
> >>>>>>> <ki...@google.com.invalid> wrote:
> >>>>>>>
> >>>>>>>> Very excited to hear that there's work on JDK9 support - is there
> a
> >>>>>>>>
> >>>>>>> public
> >>>>>>>
> >>>>>>>> description of the plans for this work somewhere?
> >>>>>>>>
> >>>>>>>> In general, Beam could probably drop Java7 support altogether at
> some
> >>>>>>>>
> >>>>>>> point
> >>>>>>>
> >>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not receiving even
> >>>>>>>>
> >>>>>>> security
> >>>>>>>
> >>>>>>>> patches) 2 years ago, and all major players in the data processing
> >>>>>>>> ecosystem have dropped Java7 support (Spark, Flink, Hadoop), so I
> >>>>>>>>
> >>>>>>> presume
> >>>>>
> >>>>>> the demand for Java7 support in the data processing industry is low.
> >>>>>>>>
> >>>>>>> By
> >>>>>
> >>>>>> the
> >>>>>>>
> >>>>>>>> way: would a Java8 migration be in the scope of your work in
> general?
> >>>>>>>>
> >>>>>>>> However, until we say that Beam requires Java8, what would be the
> >>>>>>>> implications of using a version of AutoValue that can only be
> >>>>>>>> compiled
> >>>>>>>>
> >>>>>>> with
> >>>>>>>
> >>>>>>>> Java8? Are you saying that this is simply a matter of a compiler
> bug,
> >>>>>>>>
> >>>>>>> and
> >>>>>
> >>>>>> if we use a Java8 compiler but configured to use source and target
> >>>>>>>>
> >>>>>>> versions
> >>>>>>>
> >>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7, then the
> resulting
> >>>>>>>>
> >>>>>>> Beam
> >>>>>>>
> >>>>>>>> artifacts will be usable by people who don't have Java8?
> >>>>>>>>
> >>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
> >>>>>>>> <da...@google.com.invalid> wrote:
> >>>>>>>>
> >>>>>>>> So I've been working on JDK 9 support for Beam, and I have a bug
> in
> >>>>>>>>> AutoValue that can be fixed by updating our AutoValue dependency
> to
> >>>>>>>>>
> >>>>>>>> the
> >>>>>
> >>>>>> latest. The problem is that AutoValue from 1.5+ seems to be banned
> >>>>>>>>>
> >>>>>>>> for
> >>>>>
> >>>>>> Beam
> >>>>>>>
> >>>>>>>> due to requiring Java 8 compilers. However, it should still be
> >>>>>>>>>
> >>>>>>>> possible
> >>>>>
> >>>>>> to
> >>>>>>>
> >>>>>>>> compile and execute Java 7 code from the Java 8 compiler by
> building
> >>>>>>>>>
> >>>>>>>> with
> >>>>>>>
> >>>>>>>> the correct arguments. So the fix to this bug would essentially
> >>>>>>>>>
> >>>>>>>> require
> >>>>>
> >>>>>> Java 8 compilers even for compiling Java 7 code.
> >>>>>>>>>
> >>>>>>>>> Does anyone need to use Java 7 compilers? Because if not I would
> >>>>>>>>>
> >>>>>>>> like to
> >>>>>
> >>>>>> continue with this fix.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >> --
> >> Jean-Baptiste Onofré
> >> jbonofre@apache.org
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
> >>
>
Re: Possibility of requiring Java 8 compiler for building Java 7
sources?
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
+1 to vote on @user.
Not sure anyway it requires a formal vote. As Java 7 is deprecated, it should be
an implicit "decision".
Regards
JB
On 10/16/2017 04:35 PM, Ismaël Mejía wrote:
> Any progress on this? What is the proposed way to validate if users
> are still interested on Java 7? A vote on user or something different?
>
>
> On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles <kl...@google.com.invalid> wrote:
>> Agree with polling Beam users as well as each runner's community in
>> aggregate.
>>
>> On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>>
>>> Definitely agree
>>>
>>>
>>> On 09/27/2017 06:00 PM, Robert Bradshaw wrote:
>>>
>>>> I also think that it's time to seriously consider dropping support for
>>>> Java 7.
>>>>
>>>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
>>>> <da...@google.com.invalid> wrote:
>>>>
>>>>> Yes, just as Ismaël said it's a compilation blocker right now despite
>>>>> that
>>>>> (I believe) we don't use the extension that's breaking.
>>>>>
>>>>> As for other ways to solve this, if there is a way to avoid compiling the
>>>>> advanced features of AutoValue that might be worth a try. We could also
>>>>> try
>>>>> to get a release of AutoValue with the fix that works in Java 7. However
>>>>> I
>>>>> feel that slowly moving over to Java 8 is the most future-proof solution
>>>>> if
>>>>> it's possible.
>>>>>
>>>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <ie...@gmail.com> wrote:
>>>>>
>>>>> The current issue is that compilation fails on master because beam's
>>>>>> parent pom is configured to fail if it finds warnings):
>>>>>>
>>>>>> <compiler.error.flag>-Werror</compiler.error.flag>
>>>>>>
>>>>>> However if you remove that line from the parent pom the compilation
>>>>>> passes.
>>>>>>
>>>>>> Of course this does not mean that everything is solved for Java 9,
>>>>>> there are some tests that break and other issues because of other
>>>>>> plugins and dependencies (e.g. bytebuddy), but those are not part of
>>>>>> this discussion.
>>>>>>
>>>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
>>>>>> <ki...@google.com.invalid> wrote:
>>>>>>
>>>>>>> AFAIK we don't use any advanced capabilities of AutoValue. Does that
>>>>>>> mean
>>>>>>> this issue is moot? I didn't quite understand from your email whether
>>>>>>> it
>>>>>>>
>>>>>> is
>>>>>>
>>>>>>> a compilation blocker for Beam or not.
>>>>>>>
>>>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <ie...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Great that you are also working on this too Daniel and thanks for
>>>>>>>> bringing this subject to the mailing list, I was waiting to my return
>>>>>>>> to office next week, but you did it first :)
>>>>>>>>
>>>>>>>> Eugene for reference (This is the issue on the migration to Java 9),
>>>>>>>> notice that here the goal is first that beam passes mvn clean install
>>>>>>>> with pure Java 9 (and also add this to jenkins), not to rewrite
>>>>>>>> anything to use the new stuff (e.g. modules):
>>>>>>>> https://issues.apache.org/jira/browse/BEAM-2530
>>>>>>>>
>>>>>>>> Eugene can you also PTAL at the AutoValue issue, more details on the
>>>>>>>> issue, this is a warning so I don't know if it is really critical in
>>>>>>>> particular because we are not using Memoization (do we?).
>>>>>>>> https://github.com/google/auto/issues/503
>>>>>>>>
>>>>>>>> https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d
>>>>>>>>
>>>>>>> 02247b8532
>>>>>>
>>>>>>>
>>>>>>>> Wouldn't the easiest way be that you guys convince the google.auto
>>>>>>>> guys to generate that simple fix in a Java 7 compatible way and
>>>>>>>> 'voila' ?
>>>>>>>>
>>>>>>>> However I agree that moving to Java 8 is an excellent idea and as
>>>>>>>> Eugene mentions there is less friction now since most projects are
>>>>>>>> moving, only pending issue are existing clusters on java 7 in the
>>>>>>>> hadoop world, but those are less frequent now. Anyway this discussion
>>>>>>>> is really important (maybe even worth a vote). Because moving to Java
>>>>>>>> 8 will allow us also to move some of the dependencies that we are
>>>>>>>> keeping in old versions and in general to move forward.
>>>>>>>>
>>>>>>>> What do the others think ?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
>>>>>>>> <ki...@google.com.invalid> wrote:
>>>>>>>>
>>>>>>>>> Very excited to hear that there's work on JDK9 support - is there a
>>>>>>>>>
>>>>>>>> public
>>>>>>>>
>>>>>>>>> description of the plans for this work somewhere?
>>>>>>>>>
>>>>>>>>> In general, Beam could probably drop Java7 support altogether at some
>>>>>>>>>
>>>>>>>> point
>>>>>>>>
>>>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not receiving even
>>>>>>>>>
>>>>>>>> security
>>>>>>>>
>>>>>>>>> patches) 2 years ago, and all major players in the data processing
>>>>>>>>> ecosystem have dropped Java7 support (Spark, Flink, Hadoop), so I
>>>>>>>>>
>>>>>>>> presume
>>>>>>
>>>>>>> the demand for Java7 support in the data processing industry is low.
>>>>>>>>>
>>>>>>>> By
>>>>>>
>>>>>>> the
>>>>>>>>
>>>>>>>>> way: would a Java8 migration be in the scope of your work in general?
>>>>>>>>>
>>>>>>>>> However, until we say that Beam requires Java8, what would be the
>>>>>>>>> implications of using a version of AutoValue that can only be
>>>>>>>>> compiled
>>>>>>>>>
>>>>>>>> with
>>>>>>>>
>>>>>>>>> Java8? Are you saying that this is simply a matter of a compiler bug,
>>>>>>>>>
>>>>>>>> and
>>>>>>
>>>>>>> if we use a Java8 compiler but configured to use source and target
>>>>>>>>>
>>>>>>>> versions
>>>>>>>>
>>>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7, then the resulting
>>>>>>>>>
>>>>>>>> Beam
>>>>>>>>
>>>>>>>>> artifacts will be usable by people who don't have Java8?
>>>>>>>>>
>>>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
>>>>>>>>> <da...@google.com.invalid> wrote:
>>>>>>>>>
>>>>>>>>> So I've been working on JDK 9 support for Beam, and I have a bug in
>>>>>>>>>> AutoValue that can be fixed by updating our AutoValue dependency to
>>>>>>>>>>
>>>>>>>>> the
>>>>>>
>>>>>>> latest. The problem is that AutoValue from 1.5+ seems to be banned
>>>>>>>>>>
>>>>>>>>> for
>>>>>>
>>>>>>> Beam
>>>>>>>>
>>>>>>>>> due to requiring Java 8 compilers. However, it should still be
>>>>>>>>>>
>>>>>>>>> possible
>>>>>>
>>>>>>> to
>>>>>>>>
>>>>>>>>> compile and execute Java 7 code from the Java 8 compiler by building
>>>>>>>>>>
>>>>>>>>> with
>>>>>>>>
>>>>>>>>> the correct arguments. So the fix to this bug would essentially
>>>>>>>>>>
>>>>>>>>> require
>>>>>>
>>>>>>> Java 8 compilers even for compiling Java 7 code.
>>>>>>>>>>
>>>>>>>>>> Does anyone need to use Java 7 compilers? Because if not I would
>>>>>>>>>>
>>>>>>>>> like to
>>>>>>
>>>>>>> continue with this fix.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>> --
>>> Jean-Baptiste Onofré
>>> jbonofre@apache.org
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Ismaël Mejía <ie...@gmail.com>.
Any progress on this? What is the proposed way to validate if users
are still interested on Java 7? A vote on user or something different?
On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles <kl...@google.com.invalid> wrote:
> Agree with polling Beam users as well as each runner's community in
> aggregate.
>
> On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Definitely agree
>>
>>
>> On 09/27/2017 06:00 PM, Robert Bradshaw wrote:
>>
>>> I also think that it's time to seriously consider dropping support for
>>> Java 7.
>>>
>>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
>>> <da...@google.com.invalid> wrote:
>>>
>>>> Yes, just as Ismaël said it's a compilation blocker right now despite
>>>> that
>>>> (I believe) we don't use the extension that's breaking.
>>>>
>>>> As for other ways to solve this, if there is a way to avoid compiling the
>>>> advanced features of AutoValue that might be worth a try. We could also
>>>> try
>>>> to get a release of AutoValue with the fix that works in Java 7. However
>>>> I
>>>> feel that slowly moving over to Java 8 is the most future-proof solution
>>>> if
>>>> it's possible.
>>>>
>>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <ie...@gmail.com> wrote:
>>>>
>>>> The current issue is that compilation fails on master because beam's
>>>>> parent pom is configured to fail if it finds warnings):
>>>>>
>>>>> <compiler.error.flag>-Werror</compiler.error.flag>
>>>>>
>>>>> However if you remove that line from the parent pom the compilation
>>>>> passes.
>>>>>
>>>>> Of course this does not mean that everything is solved for Java 9,
>>>>> there are some tests that break and other issues because of other
>>>>> plugins and dependencies (e.g. bytebuddy), but those are not part of
>>>>> this discussion.
>>>>>
>>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
>>>>> <ki...@google.com.invalid> wrote:
>>>>>
>>>>>> AFAIK we don't use any advanced capabilities of AutoValue. Does that
>>>>>> mean
>>>>>> this issue is moot? I didn't quite understand from your email whether
>>>>>> it
>>>>>>
>>>>> is
>>>>>
>>>>>> a compilation blocker for Beam or not.
>>>>>>
>>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <ie...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Great that you are also working on this too Daniel and thanks for
>>>>>>> bringing this subject to the mailing list, I was waiting to my return
>>>>>>> to office next week, but you did it first :)
>>>>>>>
>>>>>>> Eugene for reference (This is the issue on the migration to Java 9),
>>>>>>> notice that here the goal is first that beam passes mvn clean install
>>>>>>> with pure Java 9 (and also add this to jenkins), not to rewrite
>>>>>>> anything to use the new stuff (e.g. modules):
>>>>>>> https://issues.apache.org/jira/browse/BEAM-2530
>>>>>>>
>>>>>>> Eugene can you also PTAL at the AutoValue issue, more details on the
>>>>>>> issue, this is a warning so I don't know if it is really critical in
>>>>>>> particular because we are not using Memoization (do we?).
>>>>>>> https://github.com/google/auto/issues/503
>>>>>>>
>>>>>>> https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d
>>>>>>>
>>>>>> 02247b8532
>>>>>
>>>>>>
>>>>>>> Wouldn't the easiest way be that you guys convince the google.auto
>>>>>>> guys to generate that simple fix in a Java 7 compatible way and
>>>>>>> 'voila' ?
>>>>>>>
>>>>>>> However I agree that moving to Java 8 is an excellent idea and as
>>>>>>> Eugene mentions there is less friction now since most projects are
>>>>>>> moving, only pending issue are existing clusters on java 7 in the
>>>>>>> hadoop world, but those are less frequent now. Anyway this discussion
>>>>>>> is really important (maybe even worth a vote). Because moving to Java
>>>>>>> 8 will allow us also to move some of the dependencies that we are
>>>>>>> keeping in old versions and in general to move forward.
>>>>>>>
>>>>>>> What do the others think ?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
>>>>>>> <ki...@google.com.invalid> wrote:
>>>>>>>
>>>>>>>> Very excited to hear that there's work on JDK9 support - is there a
>>>>>>>>
>>>>>>> public
>>>>>>>
>>>>>>>> description of the plans for this work somewhere?
>>>>>>>>
>>>>>>>> In general, Beam could probably drop Java7 support altogether at some
>>>>>>>>
>>>>>>> point
>>>>>>>
>>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not receiving even
>>>>>>>>
>>>>>>> security
>>>>>>>
>>>>>>>> patches) 2 years ago, and all major players in the data processing
>>>>>>>> ecosystem have dropped Java7 support (Spark, Flink, Hadoop), so I
>>>>>>>>
>>>>>>> presume
>>>>>
>>>>>> the demand for Java7 support in the data processing industry is low.
>>>>>>>>
>>>>>>> By
>>>>>
>>>>>> the
>>>>>>>
>>>>>>>> way: would a Java8 migration be in the scope of your work in general?
>>>>>>>>
>>>>>>>> However, until we say that Beam requires Java8, what would be the
>>>>>>>> implications of using a version of AutoValue that can only be
>>>>>>>> compiled
>>>>>>>>
>>>>>>> with
>>>>>>>
>>>>>>>> Java8? Are you saying that this is simply a matter of a compiler bug,
>>>>>>>>
>>>>>>> and
>>>>>
>>>>>> if we use a Java8 compiler but configured to use source and target
>>>>>>>>
>>>>>>> versions
>>>>>>>
>>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7, then the resulting
>>>>>>>>
>>>>>>> Beam
>>>>>>>
>>>>>>>> artifacts will be usable by people who don't have Java8?
>>>>>>>>
>>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
>>>>>>>> <da...@google.com.invalid> wrote:
>>>>>>>>
>>>>>>>> So I've been working on JDK 9 support for Beam, and I have a bug in
>>>>>>>>> AutoValue that can be fixed by updating our AutoValue dependency to
>>>>>>>>>
>>>>>>>> the
>>>>>
>>>>>> latest. The problem is that AutoValue from 1.5+ seems to be banned
>>>>>>>>>
>>>>>>>> for
>>>>>
>>>>>> Beam
>>>>>>>
>>>>>>>> due to requiring Java 8 compilers. However, it should still be
>>>>>>>>>
>>>>>>>> possible
>>>>>
>>>>>> to
>>>>>>>
>>>>>>>> compile and execute Java 7 code from the Java 8 compiler by building
>>>>>>>>>
>>>>>>>> with
>>>>>>>
>>>>>>>> the correct arguments. So the fix to this bug would essentially
>>>>>>>>>
>>>>>>>> require
>>>>>
>>>>>> Java 8 compilers even for compiling Java 7 code.
>>>>>>>>>
>>>>>>>>> Does anyone need to use Java 7 compilers? Because if not I would
>>>>>>>>>
>>>>>>>> like to
>>>>>
>>>>>> continue with this fix.
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Kenneth Knowles <kl...@google.com.INVALID>.
Agree with polling Beam users as well as each runner's community in
aggregate.
On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Definitely agree
>
>
> On 09/27/2017 06:00 PM, Robert Bradshaw wrote:
>
>> I also think that it's time to seriously consider dropping support for
>> Java 7.
>>
>> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
>> <da...@google.com.invalid> wrote:
>>
>>> Yes, just as Ismaël said it's a compilation blocker right now despite
>>> that
>>> (I believe) we don't use the extension that's breaking.
>>>
>>> As for other ways to solve this, if there is a way to avoid compiling the
>>> advanced features of AutoValue that might be worth a try. We could also
>>> try
>>> to get a release of AutoValue with the fix that works in Java 7. However
>>> I
>>> feel that slowly moving over to Java 8 is the most future-proof solution
>>> if
>>> it's possible.
>>>
>>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <ie...@gmail.com> wrote:
>>>
>>> The current issue is that compilation fails on master because beam's
>>>> parent pom is configured to fail if it finds warnings):
>>>>
>>>> <compiler.error.flag>-Werror</compiler.error.flag>
>>>>
>>>> However if you remove that line from the parent pom the compilation
>>>> passes.
>>>>
>>>> Of course this does not mean that everything is solved for Java 9,
>>>> there are some tests that break and other issues because of other
>>>> plugins and dependencies (e.g. bytebuddy), but those are not part of
>>>> this discussion.
>>>>
>>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
>>>> <ki...@google.com.invalid> wrote:
>>>>
>>>>> AFAIK we don't use any advanced capabilities of AutoValue. Does that
>>>>> mean
>>>>> this issue is moot? I didn't quite understand from your email whether
>>>>> it
>>>>>
>>>> is
>>>>
>>>>> a compilation blocker for Beam or not.
>>>>>
>>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <ie...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Great that you are also working on this too Daniel and thanks for
>>>>>> bringing this subject to the mailing list, I was waiting to my return
>>>>>> to office next week, but you did it first :)
>>>>>>
>>>>>> Eugene for reference (This is the issue on the migration to Java 9),
>>>>>> notice that here the goal is first that beam passes mvn clean install
>>>>>> with pure Java 9 (and also add this to jenkins), not to rewrite
>>>>>> anything to use the new stuff (e.g. modules):
>>>>>> https://issues.apache.org/jira/browse/BEAM-2530
>>>>>>
>>>>>> Eugene can you also PTAL at the AutoValue issue, more details on the
>>>>>> issue, this is a warning so I don't know if it is really critical in
>>>>>> particular because we are not using Memoization (do we?).
>>>>>> https://github.com/google/auto/issues/503
>>>>>>
>>>>>> https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d
>>>>>>
>>>>> 02247b8532
>>>>
>>>>>
>>>>>> Wouldn't the easiest way be that you guys convince the google.auto
>>>>>> guys to generate that simple fix in a Java 7 compatible way and
>>>>>> 'voila' ?
>>>>>>
>>>>>> However I agree that moving to Java 8 is an excellent idea and as
>>>>>> Eugene mentions there is less friction now since most projects are
>>>>>> moving, only pending issue are existing clusters on java 7 in the
>>>>>> hadoop world, but those are less frequent now. Anyway this discussion
>>>>>> is really important (maybe even worth a vote). Because moving to Java
>>>>>> 8 will allow us also to move some of the dependencies that we are
>>>>>> keeping in old versions and in general to move forward.
>>>>>>
>>>>>> What do the others think ?
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
>>>>>> <ki...@google.com.invalid> wrote:
>>>>>>
>>>>>>> Very excited to hear that there's work on JDK9 support - is there a
>>>>>>>
>>>>>> public
>>>>>>
>>>>>>> description of the plans for this work somewhere?
>>>>>>>
>>>>>>> In general, Beam could probably drop Java7 support altogether at some
>>>>>>>
>>>>>> point
>>>>>>
>>>>>>> soon: Java7 has reached end-of-life (i.e. it's not receiving even
>>>>>>>
>>>>>> security
>>>>>>
>>>>>>> patches) 2 years ago, and all major players in the data processing
>>>>>>> ecosystem have dropped Java7 support (Spark, Flink, Hadoop), so I
>>>>>>>
>>>>>> presume
>>>>
>>>>> the demand for Java7 support in the data processing industry is low.
>>>>>>>
>>>>>> By
>>>>
>>>>> the
>>>>>>
>>>>>>> way: would a Java8 migration be in the scope of your work in general?
>>>>>>>
>>>>>>> However, until we say that Beam requires Java8, what would be the
>>>>>>> implications of using a version of AutoValue that can only be
>>>>>>> compiled
>>>>>>>
>>>>>> with
>>>>>>
>>>>>>> Java8? Are you saying that this is simply a matter of a compiler bug,
>>>>>>>
>>>>>> and
>>>>
>>>>> if we use a Java8 compiler but configured to use source and target
>>>>>>>
>>>>>> versions
>>>>>>
>>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7, then the resulting
>>>>>>>
>>>>>> Beam
>>>>>>
>>>>>>> artifacts will be usable by people who don't have Java8?
>>>>>>>
>>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
>>>>>>> <da...@google.com.invalid> wrote:
>>>>>>>
>>>>>>> So I've been working on JDK 9 support for Beam, and I have a bug in
>>>>>>>> AutoValue that can be fixed by updating our AutoValue dependency to
>>>>>>>>
>>>>>>> the
>>>>
>>>>> latest. The problem is that AutoValue from 1.5+ seems to be banned
>>>>>>>>
>>>>>>> for
>>>>
>>>>> Beam
>>>>>>
>>>>>>> due to requiring Java 8 compilers. However, it should still be
>>>>>>>>
>>>>>>> possible
>>>>
>>>>> to
>>>>>>
>>>>>>> compile and execute Java 7 code from the Java 8 compiler by building
>>>>>>>>
>>>>>>> with
>>>>>>
>>>>>>> the correct arguments. So the fix to this bug would essentially
>>>>>>>>
>>>>>>> require
>>>>
>>>>> Java 8 compilers even for compiling Java 7 code.
>>>>>>>>
>>>>>>>> Does anyone need to use Java 7 compilers? Because if not I would
>>>>>>>>
>>>>>>> like to
>>>>
>>>>> continue with this fix.
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Possibility of requiring Java 8 compiler for building Java 7
sources?
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Definitely agree
On 09/27/2017 06:00 PM, Robert Bradshaw wrote:
> I also think that it's time to seriously consider dropping support for Java 7.
>
> On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
> <da...@google.com.invalid> wrote:
>> Yes, just as Ismaël said it's a compilation blocker right now despite that
>> (I believe) we don't use the extension that's breaking.
>>
>> As for other ways to solve this, if there is a way to avoid compiling the
>> advanced features of AutoValue that might be worth a try. We could also try
>> to get a release of AutoValue with the fix that works in Java 7. However I
>> feel that slowly moving over to Java 8 is the most future-proof solution if
>> it's possible.
>>
>> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <ie...@gmail.com> wrote:
>>
>>> The current issue is that compilation fails on master because beam's
>>> parent pom is configured to fail if it finds warnings):
>>>
>>> <compiler.error.flag>-Werror</compiler.error.flag>
>>>
>>> However if you remove that line from the parent pom the compilation passes.
>>>
>>> Of course this does not mean that everything is solved for Java 9,
>>> there are some tests that break and other issues because of other
>>> plugins and dependencies (e.g. bytebuddy), but those are not part of
>>> this discussion.
>>>
>>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
>>> <ki...@google.com.invalid> wrote:
>>>> AFAIK we don't use any advanced capabilities of AutoValue. Does that mean
>>>> this issue is moot? I didn't quite understand from your email whether it
>>> is
>>>> a compilation blocker for Beam or not.
>>>>
>>>> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <ie...@gmail.com> wrote:
>>>>
>>>>> Great that you are also working on this too Daniel and thanks for
>>>>> bringing this subject to the mailing list, I was waiting to my return
>>>>> to office next week, but you did it first :)
>>>>>
>>>>> Eugene for reference (This is the issue on the migration to Java 9),
>>>>> notice that here the goal is first that beam passes mvn clean install
>>>>> with pure Java 9 (and also add this to jenkins), not to rewrite
>>>>> anything to use the new stuff (e.g. modules):
>>>>> https://issues.apache.org/jira/browse/BEAM-2530
>>>>>
>>>>> Eugene can you also PTAL at the AutoValue issue, more details on the
>>>>> issue, this is a warning so I don't know if it is really critical in
>>>>> particular because we are not using Memoization (do we?).
>>>>> https://github.com/google/auto/issues/503
>>>>>
>>>>> https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d
>>> 02247b8532
>>>>>
>>>>> Wouldn't the easiest way be that you guys convince the google.auto
>>>>> guys to generate that simple fix in a Java 7 compatible way and
>>>>> 'voila' ?
>>>>>
>>>>> However I agree that moving to Java 8 is an excellent idea and as
>>>>> Eugene mentions there is less friction now since most projects are
>>>>> moving, only pending issue are existing clusters on java 7 in the
>>>>> hadoop world, but those are less frequent now. Anyway this discussion
>>>>> is really important (maybe even worth a vote). Because moving to Java
>>>>> 8 will allow us also to move some of the dependencies that we are
>>>>> keeping in old versions and in general to move forward.
>>>>>
>>>>> What do the others think ?
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
>>>>> <ki...@google.com.invalid> wrote:
>>>>>> Very excited to hear that there's work on JDK9 support - is there a
>>>>> public
>>>>>> description of the plans for this work somewhere?
>>>>>>
>>>>>> In general, Beam could probably drop Java7 support altogether at some
>>>>> point
>>>>>> soon: Java7 has reached end-of-life (i.e. it's not receiving even
>>>>> security
>>>>>> patches) 2 years ago, and all major players in the data processing
>>>>>> ecosystem have dropped Java7 support (Spark, Flink, Hadoop), so I
>>> presume
>>>>>> the demand for Java7 support in the data processing industry is low.
>>> By
>>>>> the
>>>>>> way: would a Java8 migration be in the scope of your work in general?
>>>>>>
>>>>>> However, until we say that Beam requires Java8, what would be the
>>>>>> implications of using a version of AutoValue that can only be compiled
>>>>> with
>>>>>> Java8? Are you saying that this is simply a matter of a compiler bug,
>>> and
>>>>>> if we use a Java8 compiler but configured to use source and target
>>>>> versions
>>>>>> of 1.7 and using bootclasspath of rt.jar from 1.7, then the resulting
>>>>> Beam
>>>>>> artifacts will be usable by people who don't have Java8?
>>>>>>
>>>>>> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
>>>>>> <da...@google.com.invalid> wrote:
>>>>>>
>>>>>>> So I've been working on JDK 9 support for Beam, and I have a bug in
>>>>>>> AutoValue that can be fixed by updating our AutoValue dependency to
>>> the
>>>>>>> latest. The problem is that AutoValue from 1.5+ seems to be banned
>>> for
>>>>> Beam
>>>>>>> due to requiring Java 8 compilers. However, it should still be
>>> possible
>>>>> to
>>>>>>> compile and execute Java 7 code from the Java 8 compiler by building
>>>>> with
>>>>>>> the correct arguments. So the fix to this bug would essentially
>>> require
>>>>>>> Java 8 compilers even for compiling Java 7 code.
>>>>>>>
>>>>>>> Does anyone need to use Java 7 compilers? Because if not I would
>>> like to
>>>>>>> continue with this fix.
>>>>>>>
>>>>>
>>>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Robert Bradshaw <ro...@google.com.INVALID>.
I also think that it's time to seriously consider dropping support for Java 7.
On Tue, Sep 26, 2017 at 9:14 PM, Daniel Oliveira
<da...@google.com.invalid> wrote:
> Yes, just as Ismaël said it's a compilation blocker right now despite that
> (I believe) we don't use the extension that's breaking.
>
> As for other ways to solve this, if there is a way to avoid compiling the
> advanced features of AutoValue that might be worth a try. We could also try
> to get a release of AutoValue with the fix that works in Java 7. However I
> feel that slowly moving over to Java 8 is the most future-proof solution if
> it's possible.
>
> On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <ie...@gmail.com> wrote:
>
>> The current issue is that compilation fails on master because beam's
>> parent pom is configured to fail if it finds warnings):
>>
>> <compiler.error.flag>-Werror</compiler.error.flag>
>>
>> However if you remove that line from the parent pom the compilation passes.
>>
>> Of course this does not mean that everything is solved for Java 9,
>> there are some tests that break and other issues because of other
>> plugins and dependencies (e.g. bytebuddy), but those are not part of
>> this discussion.
>>
>> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
>> <ki...@google.com.invalid> wrote:
>> > AFAIK we don't use any advanced capabilities of AutoValue. Does that mean
>> > this issue is moot? I didn't quite understand from your email whether it
>> is
>> > a compilation blocker for Beam or not.
>> >
>> > On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <ie...@gmail.com> wrote:
>> >
>> >> Great that you are also working on this too Daniel and thanks for
>> >> bringing this subject to the mailing list, I was waiting to my return
>> >> to office next week, but you did it first :)
>> >>
>> >> Eugene for reference (This is the issue on the migration to Java 9),
>> >> notice that here the goal is first that beam passes mvn clean install
>> >> with pure Java 9 (and also add this to jenkins), not to rewrite
>> >> anything to use the new stuff (e.g. modules):
>> >> https://issues.apache.org/jira/browse/BEAM-2530
>> >>
>> >> Eugene can you also PTAL at the AutoValue issue, more details on the
>> >> issue, this is a warning so I don't know if it is really critical in
>> >> particular because we are not using Memoization (do we?).
>> >> https://github.com/google/auto/issues/503
>> >>
>> >> https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d
>> 02247b8532
>> >>
>> >> Wouldn't the easiest way be that you guys convince the google.auto
>> >> guys to generate that simple fix in a Java 7 compatible way and
>> >> 'voila' ?
>> >>
>> >> However I agree that moving to Java 8 is an excellent idea and as
>> >> Eugene mentions there is less friction now since most projects are
>> >> moving, only pending issue are existing clusters on java 7 in the
>> >> hadoop world, but those are less frequent now. Anyway this discussion
>> >> is really important (maybe even worth a vote). Because moving to Java
>> >> 8 will allow us also to move some of the dependencies that we are
>> >> keeping in old versions and in general to move forward.
>> >>
>> >> What do the others think ?
>> >>
>> >>
>> >>
>> >> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
>> >> <ki...@google.com.invalid> wrote:
>> >> > Very excited to hear that there's work on JDK9 support - is there a
>> >> public
>> >> > description of the plans for this work somewhere?
>> >> >
>> >> > In general, Beam could probably drop Java7 support altogether at some
>> >> point
>> >> > soon: Java7 has reached end-of-life (i.e. it's not receiving even
>> >> security
>> >> > patches) 2 years ago, and all major players in the data processing
>> >> > ecosystem have dropped Java7 support (Spark, Flink, Hadoop), so I
>> presume
>> >> > the demand for Java7 support in the data processing industry is low.
>> By
>> >> the
>> >> > way: would a Java8 migration be in the scope of your work in general?
>> >> >
>> >> > However, until we say that Beam requires Java8, what would be the
>> >> > implications of using a version of AutoValue that can only be compiled
>> >> with
>> >> > Java8? Are you saying that this is simply a matter of a compiler bug,
>> and
>> >> > if we use a Java8 compiler but configured to use source and target
>> >> versions
>> >> > of 1.7 and using bootclasspath of rt.jar from 1.7, then the resulting
>> >> Beam
>> >> > artifacts will be usable by people who don't have Java8?
>> >> >
>> >> > On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
>> >> > <da...@google.com.invalid> wrote:
>> >> >
>> >> >> So I've been working on JDK 9 support for Beam, and I have a bug in
>> >> >> AutoValue that can be fixed by updating our AutoValue dependency to
>> the
>> >> >> latest. The problem is that AutoValue from 1.5+ seems to be banned
>> for
>> >> Beam
>> >> >> due to requiring Java 8 compilers. However, it should still be
>> possible
>> >> to
>> >> >> compile and execute Java 7 code from the Java 8 compiler by building
>> >> with
>> >> >> the correct arguments. So the fix to this bug would essentially
>> require
>> >> >> Java 8 compilers even for compiling Java 7 code.
>> >> >>
>> >> >> Does anyone need to use Java 7 compilers? Because if not I would
>> like to
>> >> >> continue with this fix.
>> >> >>
>> >>
>>
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Daniel Oliveira <da...@google.com.INVALID>.
Yes, just as Ismaël said it's a compilation blocker right now despite that
(I believe) we don't use the extension that's breaking.
As for other ways to solve this, if there is a way to avoid compiling the
advanced features of AutoValue that might be worth a try. We could also try
to get a release of AutoValue with the fix that works in Java 7. However I
feel that slowly moving over to Java 8 is the most future-proof solution if
it's possible.
On Tue, Sep 26, 2017 at 2:47 PM, Ismaël Mejía <ie...@gmail.com> wrote:
> The current issue is that compilation fails on master because beam's
> parent pom is configured to fail if it finds warnings):
>
> <compiler.error.flag>-Werror</compiler.error.flag>
>
> However if you remove that line from the parent pom the compilation passes.
>
> Of course this does not mean that everything is solved for Java 9,
> there are some tests that break and other issues because of other
> plugins and dependencies (e.g. bytebuddy), but those are not part of
> this discussion.
>
> On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
> <ki...@google.com.invalid> wrote:
> > AFAIK we don't use any advanced capabilities of AutoValue. Does that mean
> > this issue is moot? I didn't quite understand from your email whether it
> is
> > a compilation blocker for Beam or not.
> >
> > On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <ie...@gmail.com> wrote:
> >
> >> Great that you are also working on this too Daniel and thanks for
> >> bringing this subject to the mailing list, I was waiting to my return
> >> to office next week, but you did it first :)
> >>
> >> Eugene for reference (This is the issue on the migration to Java 9),
> >> notice that here the goal is first that beam passes mvn clean install
> >> with pure Java 9 (and also add this to jenkins), not to rewrite
> >> anything to use the new stuff (e.g. modules):
> >> https://issues.apache.org/jira/browse/BEAM-2530
> >>
> >> Eugene can you also PTAL at the AutoValue issue, more details on the
> >> issue, this is a warning so I don't know if it is really critical in
> >> particular because we are not using Memoization (do we?).
> >> https://github.com/google/auto/issues/503
> >>
> >> https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d
> 02247b8532
> >>
> >> Wouldn't the easiest way be that you guys convince the google.auto
> >> guys to generate that simple fix in a Java 7 compatible way and
> >> 'voila' ?
> >>
> >> However I agree that moving to Java 8 is an excellent idea and as
> >> Eugene mentions there is less friction now since most projects are
> >> moving, only pending issue are existing clusters on java 7 in the
> >> hadoop world, but those are less frequent now. Anyway this discussion
> >> is really important (maybe even worth a vote). Because moving to Java
> >> 8 will allow us also to move some of the dependencies that we are
> >> keeping in old versions and in general to move forward.
> >>
> >> What do the others think ?
> >>
> >>
> >>
> >> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
> >> <ki...@google.com.invalid> wrote:
> >> > Very excited to hear that there's work on JDK9 support - is there a
> >> public
> >> > description of the plans for this work somewhere?
> >> >
> >> > In general, Beam could probably drop Java7 support altogether at some
> >> point
> >> > soon: Java7 has reached end-of-life (i.e. it's not receiving even
> >> security
> >> > patches) 2 years ago, and all major players in the data processing
> >> > ecosystem have dropped Java7 support (Spark, Flink, Hadoop), so I
> presume
> >> > the demand for Java7 support in the data processing industry is low.
> By
> >> the
> >> > way: would a Java8 migration be in the scope of your work in general?
> >> >
> >> > However, until we say that Beam requires Java8, what would be the
> >> > implications of using a version of AutoValue that can only be compiled
> >> with
> >> > Java8? Are you saying that this is simply a matter of a compiler bug,
> and
> >> > if we use a Java8 compiler but configured to use source and target
> >> versions
> >> > of 1.7 and using bootclasspath of rt.jar from 1.7, then the resulting
> >> Beam
> >> > artifacts will be usable by people who don't have Java8?
> >> >
> >> > On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
> >> > <da...@google.com.invalid> wrote:
> >> >
> >> >> So I've been working on JDK 9 support for Beam, and I have a bug in
> >> >> AutoValue that can be fixed by updating our AutoValue dependency to
> the
> >> >> latest. The problem is that AutoValue from 1.5+ seems to be banned
> for
> >> Beam
> >> >> due to requiring Java 8 compilers. However, it should still be
> possible
> >> to
> >> >> compile and execute Java 7 code from the Java 8 compiler by building
> >> with
> >> >> the correct arguments. So the fix to this bug would essentially
> require
> >> >> Java 8 compilers even for compiling Java 7 code.
> >> >>
> >> >> Does anyone need to use Java 7 compilers? Because if not I would
> like to
> >> >> continue with this fix.
> >> >>
> >>
>
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Ismaël Mejía <ie...@gmail.com>.
The current issue is that compilation fails on master because beam's
parent pom is configured to fail if it finds warnings):
<compiler.error.flag>-Werror</compiler.error.flag>
However if you remove that line from the parent pom the compilation passes.
Of course this does not mean that everything is solved for Java 9,
there are some tests that break and other issues because of other
plugins and dependencies (e.g. bytebuddy), but those are not part of
this discussion.
On Tue, Sep 26, 2017 at 11:38 PM, Eugene Kirpichov
<ki...@google.com.invalid> wrote:
> AFAIK we don't use any advanced capabilities of AutoValue. Does that mean
> this issue is moot? I didn't quite understand from your email whether it is
> a compilation blocker for Beam or not.
>
> On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <ie...@gmail.com> wrote:
>
>> Great that you are also working on this too Daniel and thanks for
>> bringing this subject to the mailing list, I was waiting to my return
>> to office next week, but you did it first :)
>>
>> Eugene for reference (This is the issue on the migration to Java 9),
>> notice that here the goal is first that beam passes mvn clean install
>> with pure Java 9 (and also add this to jenkins), not to rewrite
>> anything to use the new stuff (e.g. modules):
>> https://issues.apache.org/jira/browse/BEAM-2530
>>
>> Eugene can you also PTAL at the AutoValue issue, more details on the
>> issue, this is a warning so I don't know if it is really critical in
>> particular because we are not using Memoization (do we?).
>> https://github.com/google/auto/issues/503
>>
>> https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d02247b8532
>>
>> Wouldn't the easiest way be that you guys convince the google.auto
>> guys to generate that simple fix in a Java 7 compatible way and
>> 'voila' ?
>>
>> However I agree that moving to Java 8 is an excellent idea and as
>> Eugene mentions there is less friction now since most projects are
>> moving, only pending issue are existing clusters on java 7 in the
>> hadoop world, but those are less frequent now. Anyway this discussion
>> is really important (maybe even worth a vote). Because moving to Java
>> 8 will allow us also to move some of the dependencies that we are
>> keeping in old versions and in general to move forward.
>>
>> What do the others think ?
>>
>>
>>
>> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
>> <ki...@google.com.invalid> wrote:
>> > Very excited to hear that there's work on JDK9 support - is there a
>> public
>> > description of the plans for this work somewhere?
>> >
>> > In general, Beam could probably drop Java7 support altogether at some
>> point
>> > soon: Java7 has reached end-of-life (i.e. it's not receiving even
>> security
>> > patches) 2 years ago, and all major players in the data processing
>> > ecosystem have dropped Java7 support (Spark, Flink, Hadoop), so I presume
>> > the demand for Java7 support in the data processing industry is low. By
>> the
>> > way: would a Java8 migration be in the scope of your work in general?
>> >
>> > However, until we say that Beam requires Java8, what would be the
>> > implications of using a version of AutoValue that can only be compiled
>> with
>> > Java8? Are you saying that this is simply a matter of a compiler bug, and
>> > if we use a Java8 compiler but configured to use source and target
>> versions
>> > of 1.7 and using bootclasspath of rt.jar from 1.7, then the resulting
>> Beam
>> > artifacts will be usable by people who don't have Java8?
>> >
>> > On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
>> > <da...@google.com.invalid> wrote:
>> >
>> >> So I've been working on JDK 9 support for Beam, and I have a bug in
>> >> AutoValue that can be fixed by updating our AutoValue dependency to the
>> >> latest. The problem is that AutoValue from 1.5+ seems to be banned for
>> Beam
>> >> due to requiring Java 8 compilers. However, it should still be possible
>> to
>> >> compile and execute Java 7 code from the Java 8 compiler by building
>> with
>> >> the correct arguments. So the fix to this bug would essentially require
>> >> Java 8 compilers even for compiling Java 7 code.
>> >>
>> >> Does anyone need to use Java 7 compilers? Because if not I would like to
>> >> continue with this fix.
>> >>
>>
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Eugene Kirpichov <ki...@google.com.INVALID>.
AFAIK we don't use any advanced capabilities of AutoValue. Does that mean
this issue is moot? I didn't quite understand from your email whether it is
a compilation blocker for Beam or not.
On Tue, Sep 26, 2017 at 2:32 PM Ismaël Mejía <ie...@gmail.com> wrote:
> Great that you are also working on this too Daniel and thanks for
> bringing this subject to the mailing list, I was waiting to my return
> to office next week, but you did it first :)
>
> Eugene for reference (This is the issue on the migration to Java 9),
> notice that here the goal is first that beam passes mvn clean install
> with pure Java 9 (and also add this to jenkins), not to rewrite
> anything to use the new stuff (e.g. modules):
> https://issues.apache.org/jira/browse/BEAM-2530
>
> Eugene can you also PTAL at the AutoValue issue, more details on the
> issue, this is a warning so I don't know if it is really critical in
> particular because we are not using Memoization (do we?).
> https://github.com/google/auto/issues/503
>
> https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d02247b8532
>
> Wouldn't the easiest way be that you guys convince the google.auto
> guys to generate that simple fix in a Java 7 compatible way and
> 'voila' ?
>
> However I agree that moving to Java 8 is an excellent idea and as
> Eugene mentions there is less friction now since most projects are
> moving, only pending issue are existing clusters on java 7 in the
> hadoop world, but those are less frequent now. Anyway this discussion
> is really important (maybe even worth a vote). Because moving to Java
> 8 will allow us also to move some of the dependencies that we are
> keeping in old versions and in general to move forward.
>
> What do the others think ?
>
>
>
> On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
> <ki...@google.com.invalid> wrote:
> > Very excited to hear that there's work on JDK9 support - is there a
> public
> > description of the plans for this work somewhere?
> >
> > In general, Beam could probably drop Java7 support altogether at some
> point
> > soon: Java7 has reached end-of-life (i.e. it's not receiving even
> security
> > patches) 2 years ago, and all major players in the data processing
> > ecosystem have dropped Java7 support (Spark, Flink, Hadoop), so I presume
> > the demand for Java7 support in the data processing industry is low. By
> the
> > way: would a Java8 migration be in the scope of your work in general?
> >
> > However, until we say that Beam requires Java8, what would be the
> > implications of using a version of AutoValue that can only be compiled
> with
> > Java8? Are you saying that this is simply a matter of a compiler bug, and
> > if we use a Java8 compiler but configured to use source and target
> versions
> > of 1.7 and using bootclasspath of rt.jar from 1.7, then the resulting
> Beam
> > artifacts will be usable by people who don't have Java8?
> >
> > On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
> > <da...@google.com.invalid> wrote:
> >
> >> So I've been working on JDK 9 support for Beam, and I have a bug in
> >> AutoValue that can be fixed by updating our AutoValue dependency to the
> >> latest. The problem is that AutoValue from 1.5+ seems to be banned for
> Beam
> >> due to requiring Java 8 compilers. However, it should still be possible
> to
> >> compile and execute Java 7 code from the Java 8 compiler by building
> with
> >> the correct arguments. So the fix to this bug would essentially require
> >> Java 8 compilers even for compiling Java 7 code.
> >>
> >> Does anyone need to use Java 7 compilers? Because if not I would like to
> >> continue with this fix.
> >>
>
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Ismaël Mejía <ie...@gmail.com>.
Great that you are also working on this too Daniel and thanks for
bringing this subject to the mailing list, I was waiting to my return
to office next week, but you did it first :)
Eugene for reference (This is the issue on the migration to Java 9),
notice that here the goal is first that beam passes mvn clean install
with pure Java 9 (and also add this to jenkins), not to rewrite
anything to use the new stuff (e.g. modules):
https://issues.apache.org/jira/browse/BEAM-2530
Eugene can you also PTAL at the AutoValue issue, more details on the
issue, this is a warning so I don't know if it is really critical in
particular because we are not using Memoization (do we?).
https://github.com/google/auto/issues/503
https://github.com/google/auto/commit/71514081f2ca6fb4ead2b7f0a25f5d02247b8532
Wouldn't the easiest way be that you guys convince the google.auto
guys to generate that simple fix in a Java 7 compatible way and
'voila' ?
However I agree that moving to Java 8 is an excellent idea and as
Eugene mentions there is less friction now since most projects are
moving, only pending issue are existing clusters on java 7 in the
hadoop world, but those are less frequent now. Anyway this discussion
is really important (maybe even worth a vote). Because moving to Java
8 will allow us also to move some of the dependencies that we are
keeping in old versions and in general to move forward.
What do the others think ?
On Tue, Sep 26, 2017 at 11:09 PM, Eugene Kirpichov
<ki...@google.com.invalid> wrote:
> Very excited to hear that there's work on JDK9 support - is there a public
> description of the plans for this work somewhere?
>
> In general, Beam could probably drop Java7 support altogether at some point
> soon: Java7 has reached end-of-life (i.e. it's not receiving even security
> patches) 2 years ago, and all major players in the data processing
> ecosystem have dropped Java7 support (Spark, Flink, Hadoop), so I presume
> the demand for Java7 support in the data processing industry is low. By the
> way: would a Java8 migration be in the scope of your work in general?
>
> However, until we say that Beam requires Java8, what would be the
> implications of using a version of AutoValue that can only be compiled with
> Java8? Are you saying that this is simply a matter of a compiler bug, and
> if we use a Java8 compiler but configured to use source and target versions
> of 1.7 and using bootclasspath of rt.jar from 1.7, then the resulting Beam
> artifacts will be usable by people who don't have Java8?
>
> On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
> <da...@google.com.invalid> wrote:
>
>> So I've been working on JDK 9 support for Beam, and I have a bug in
>> AutoValue that can be fixed by updating our AutoValue dependency to the
>> latest. The problem is that AutoValue from 1.5+ seems to be banned for Beam
>> due to requiring Java 8 compilers. However, it should still be possible to
>> compile and execute Java 7 code from the Java 8 compiler by building with
>> the correct arguments. So the fix to this bug would essentially require
>> Java 8 compilers even for compiling Java 7 code.
>>
>> Does anyone need to use Java 7 compilers? Because if not I would like to
>> continue with this fix.
>>
Re: Possibility of requiring Java 8 compiler for building Java 7 sources?
Posted by Eugene Kirpichov <ki...@google.com.INVALID>.
Very excited to hear that there's work on JDK9 support - is there a public
description of the plans for this work somewhere?
In general, Beam could probably drop Java7 support altogether at some point
soon: Java7 has reached end-of-life (i.e. it's not receiving even security
patches) 2 years ago, and all major players in the data processing
ecosystem have dropped Java7 support (Spark, Flink, Hadoop), so I presume
the demand for Java7 support in the data processing industry is low. By the
way: would a Java8 migration be in the scope of your work in general?
However, until we say that Beam requires Java8, what would be the
implications of using a version of AutoValue that can only be compiled with
Java8? Are you saying that this is simply a matter of a compiler bug, and
if we use a Java8 compiler but configured to use source and target versions
of 1.7 and using bootclasspath of rt.jar from 1.7, then the resulting Beam
artifacts will be usable by people who don't have Java8?
On Tue, Sep 26, 2017 at 1:53 PM Daniel Oliveira
<da...@google.com.invalid> wrote:
> So I've been working on JDK 9 support for Beam, and I have a bug in
> AutoValue that can be fixed by updating our AutoValue dependency to the
> latest. The problem is that AutoValue from 1.5+ seems to be banned for Beam
> due to requiring Java 8 compilers. However, it should still be possible to
> compile and execute Java 7 code from the Java 8 compiler by building with
> the correct arguments. So the fix to this bug would essentially require
> Java 8 compilers even for compiling Java 7 code.
>
> Does anyone need to use Java 7 compilers? Because if not I would like to
> continue with this fix.
>