You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Robert Kanter <rk...@cloudera.com> on 2015/04/22 00:14:50 UTC
Set minimum version of Hadoop 3 to JDK 8
Hi all,
Moving forward on some of the discussions on Hadoop 3, I've created
HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
wanted to let everyone know in case there's some reason we shouldn't go
ahead with this.
thanks
- Robert
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Robert Kanter <rk...@cloudera.com>.
You're absolutely right Steve. We should get the jobs under control.
At the very least, before we move to JDK 8 in Jenkins, we should get the
Java8 nightly builds working. I didn't realize we had these, and I had
thought all compiling and most test issues with JDK8 were already fixed
based on what HADOOP-11090 looks like; but I guess there's more.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-common-trunk-Java8/
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Hdfs-trunk-Java8/
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Mapreduce-trunk-Java8/
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Yarn-trunk-Java8/
I'll take a deeper look at these and file JIRAs as necessary later today.
On Wed, Apr 22, 2015 at 3:13 AM, Steve Loughran <st...@hortonworks.com>
wrote:
>
> > On 21 Apr 2015, at 23:31, Andrew Wang <an...@cloudera.com> wrote:
> >
> > Hey Robert,
> >
> > As a first step, could we try switching all our precommit and nightly
> > builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
> > to do in any case since it'll still target 1.7.
>
> +1
>
> we have jenkin builds running on Java 8 -its just they have enough
> longstanding and intermittent test failures that they get ignored
>
> I've been trying to keep hadoop-common somewhat under control, but
>
> -YARN is failing due to to hard coded ports in tests - this needs to be
> fixed urgently: YARN-3528
> -HDFS build process itself is failing with some config problem checkstyle
>
>
> >
> > I'll note that HADOOP-10530 details the pain Steve went through switching
> > us to JDK7. Might be some lessons learned about how to do this transition
> > more smoothly.
> >
>
>
> The key thing is to have Jenkins happy before trying to change anything.
> And make sure that "happy" means that jenkins really is building and
> testing against the version of Java that you think it is. I spent about 15
> minutes/hour of my weekend kicking off builds and tuning them. -I
> volunteered Colin Patrick McCabe to to that bit of the switch, though of
> course he doesn't have to accept that opportunuty (
> https://issues.apache.org/jira/browse/HADOOP-10530?focusedCommentId=14239250&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14239250
>
> Someone needs to take ownership of the "Jenkins builds are broken" problem
> -ideally someone in each subproject. That's a prerequisite to any major
> changes in the build process, but should also be best practice anyway. Ask
> your QE teams "would you like the ASF projects to care whether their
> jenkins test runs succeed" and see what they say.
>
> as a reminder, all the jenkins builds are here
> https://builds.apache.org/view/H-L/view/Hadoop/
>
> please look at them and try to fix them —without even waiting for a switch
> to Java 8
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Robert Kanter <rk...@cloudera.com>.
You're absolutely right Steve. We should get the jobs under control.
At the very least, before we move to JDK 8 in Jenkins, we should get the
Java8 nightly builds working. I didn't realize we had these, and I had
thought all compiling and most test issues with JDK8 were already fixed
based on what HADOOP-11090 looks like; but I guess there's more.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-common-trunk-Java8/
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Hdfs-trunk-Java8/
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Mapreduce-trunk-Java8/
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Yarn-trunk-Java8/
I'll take a deeper look at these and file JIRAs as necessary later today.
On Wed, Apr 22, 2015 at 3:13 AM, Steve Loughran <st...@hortonworks.com>
wrote:
>
> > On 21 Apr 2015, at 23:31, Andrew Wang <an...@cloudera.com> wrote:
> >
> > Hey Robert,
> >
> > As a first step, could we try switching all our precommit and nightly
> > builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
> > to do in any case since it'll still target 1.7.
>
> +1
>
> we have jenkin builds running on Java 8 -its just they have enough
> longstanding and intermittent test failures that they get ignored
>
> I've been trying to keep hadoop-common somewhat under control, but
>
> -YARN is failing due to to hard coded ports in tests - this needs to be
> fixed urgently: YARN-3528
> -HDFS build process itself is failing with some config problem checkstyle
>
>
> >
> > I'll note that HADOOP-10530 details the pain Steve went through switching
> > us to JDK7. Might be some lessons learned about how to do this transition
> > more smoothly.
> >
>
>
> The key thing is to have Jenkins happy before trying to change anything.
> And make sure that "happy" means that jenkins really is building and
> testing against the version of Java that you think it is. I spent about 15
> minutes/hour of my weekend kicking off builds and tuning them. -I
> volunteered Colin Patrick McCabe to to that bit of the switch, though of
> course he doesn't have to accept that opportunuty (
> https://issues.apache.org/jira/browse/HADOOP-10530?focusedCommentId=14239250&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14239250
>
> Someone needs to take ownership of the "Jenkins builds are broken" problem
> -ideally someone in each subproject. That's a prerequisite to any major
> changes in the build process, but should also be best practice anyway. Ask
> your QE teams "would you like the ASF projects to care whether their
> jenkins test runs succeed" and see what they say.
>
> as a reminder, all the jenkins builds are here
> https://builds.apache.org/view/H-L/view/Hadoop/
>
> please look at them and try to fix them —without even waiting for a switch
> to Java 8
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Robert Kanter <rk...@cloudera.com>.
You're absolutely right Steve. We should get the jobs under control.
At the very least, before we move to JDK 8 in Jenkins, we should get the
Java8 nightly builds working. I didn't realize we had these, and I had
thought all compiling and most test issues with JDK8 were already fixed
based on what HADOOP-11090 looks like; but I guess there's more.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-common-trunk-Java8/
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Hdfs-trunk-Java8/
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Mapreduce-trunk-Java8/
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Yarn-trunk-Java8/
I'll take a deeper look at these and file JIRAs as necessary later today.
On Wed, Apr 22, 2015 at 3:13 AM, Steve Loughran <st...@hortonworks.com>
wrote:
>
> > On 21 Apr 2015, at 23:31, Andrew Wang <an...@cloudera.com> wrote:
> >
> > Hey Robert,
> >
> > As a first step, could we try switching all our precommit and nightly
> > builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
> > to do in any case since it'll still target 1.7.
>
> +1
>
> we have jenkin builds running on Java 8 -its just they have enough
> longstanding and intermittent test failures that they get ignored
>
> I've been trying to keep hadoop-common somewhat under control, but
>
> -YARN is failing due to to hard coded ports in tests - this needs to be
> fixed urgently: YARN-3528
> -HDFS build process itself is failing with some config problem checkstyle
>
>
> >
> > I'll note that HADOOP-10530 details the pain Steve went through switching
> > us to JDK7. Might be some lessons learned about how to do this transition
> > more smoothly.
> >
>
>
> The key thing is to have Jenkins happy before trying to change anything.
> And make sure that "happy" means that jenkins really is building and
> testing against the version of Java that you think it is. I spent about 15
> minutes/hour of my weekend kicking off builds and tuning them. -I
> volunteered Colin Patrick McCabe to to that bit of the switch, though of
> course he doesn't have to accept that opportunuty (
> https://issues.apache.org/jira/browse/HADOOP-10530?focusedCommentId=14239250&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14239250
>
> Someone needs to take ownership of the "Jenkins builds are broken" problem
> -ideally someone in each subproject. That's a prerequisite to any major
> changes in the build process, but should also be best practice anyway. Ask
> your QE teams "would you like the ASF projects to care whether their
> jenkins test runs succeed" and see what they say.
>
> as a reminder, all the jenkins builds are here
> https://builds.apache.org/view/H-L/view/Hadoop/
>
> please look at them and try to fix them —without even waiting for a switch
> to Java 8
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Robert Kanter <rk...@cloudera.com>.
You're absolutely right Steve. We should get the jobs under control.
At the very least, before we move to JDK 8 in Jenkins, we should get the
Java8 nightly builds working. I didn't realize we had these, and I had
thought all compiling and most test issues with JDK8 were already fixed
based on what HADOOP-11090 looks like; but I guess there's more.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-common-trunk-Java8/
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Hdfs-trunk-Java8/
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Mapreduce-trunk-Java8/
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Yarn-trunk-Java8/
I'll take a deeper look at these and file JIRAs as necessary later today.
On Wed, Apr 22, 2015 at 3:13 AM, Steve Loughran <st...@hortonworks.com>
wrote:
>
> > On 21 Apr 2015, at 23:31, Andrew Wang <an...@cloudera.com> wrote:
> >
> > Hey Robert,
> >
> > As a first step, could we try switching all our precommit and nightly
> > builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
> > to do in any case since it'll still target 1.7.
>
> +1
>
> we have jenkin builds running on Java 8 -its just they have enough
> longstanding and intermittent test failures that they get ignored
>
> I've been trying to keep hadoop-common somewhat under control, but
>
> -YARN is failing due to to hard coded ports in tests - this needs to be
> fixed urgently: YARN-3528
> -HDFS build process itself is failing with some config problem checkstyle
>
>
> >
> > I'll note that HADOOP-10530 details the pain Steve went through switching
> > us to JDK7. Might be some lessons learned about how to do this transition
> > more smoothly.
> >
>
>
> The key thing is to have Jenkins happy before trying to change anything.
> And make sure that "happy" means that jenkins really is building and
> testing against the version of Java that you think it is. I spent about 15
> minutes/hour of my weekend kicking off builds and tuning them. -I
> volunteered Colin Patrick McCabe to to that bit of the switch, though of
> course he doesn't have to accept that opportunuty (
> https://issues.apache.org/jira/browse/HADOOP-10530?focusedCommentId=14239250&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14239250
>
> Someone needs to take ownership of the "Jenkins builds are broken" problem
> -ideally someone in each subproject. That's a prerequisite to any major
> changes in the build process, but should also be best practice anyway. Ask
> your QE teams "would you like the ASF projects to care whether their
> jenkins test runs succeed" and see what they say.
>
> as a reminder, all the jenkins builds are here
> https://builds.apache.org/view/H-L/view/Hadoop/
>
> please look at them and try to fix them —without even waiting for a switch
> to Java 8
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Steve Loughran <st...@hortonworks.com>.
> On 21 Apr 2015, at 23:31, Andrew Wang <an...@cloudera.com> wrote:
>
> Hey Robert,
>
> As a first step, could we try switching all our precommit and nightly
> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
> to do in any case since it'll still target 1.7.
+1
we have jenkin builds running on Java 8 -its just they have enough longstanding and intermittent test failures that they get ignored
I've been trying to keep hadoop-common somewhat under control, but
-YARN is failing due to to hard coded ports in tests - this needs to be fixed urgently: YARN-3528
-HDFS build process itself is failing with some config problem checkstyle
>
> I'll note that HADOOP-10530 details the pain Steve went through switching
> us to JDK7. Might be some lessons learned about how to do this transition
> more smoothly.
>
The key thing is to have Jenkins happy before trying to change anything. And make sure that "happy" means that jenkins really is building and testing against the version of Java that you think it is. I spent about 15 minutes/hour of my weekend kicking off builds and tuning them. -I volunteered Colin Patrick McCabe to to that bit of the switch, though of course he doesn't have to accept that opportunuty ( https://issues.apache.org/jira/browse/HADOOP-10530?focusedCommentId=14239250&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14239250
Someone needs to take ownership of the "Jenkins builds are broken" problem -ideally someone in each subproject. That's a prerequisite to any major changes in the build process, but should also be best practice anyway. Ask your QE teams "would you like the ASF projects to care whether their jenkins test runs succeed" and see what they say.
as a reminder, all the jenkins builds are here
https://builds.apache.org/view/H-L/view/Hadoop/
please look at them and try to fix them —without even waiting for a switch to Java 8
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Steve Loughran <st...@hortonworks.com>.
> On 21 Apr 2015, at 23:31, Andrew Wang <an...@cloudera.com> wrote:
>
> Hey Robert,
>
> As a first step, could we try switching all our precommit and nightly
> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
> to do in any case since it'll still target 1.7.
+1
we have jenkin builds running on Java 8 -its just they have enough longstanding and intermittent test failures that they get ignored
I've been trying to keep hadoop-common somewhat under control, but
-YARN is failing due to to hard coded ports in tests - this needs to be fixed urgently: YARN-3528
-HDFS build process itself is failing with some config problem checkstyle
>
> I'll note that HADOOP-10530 details the pain Steve went through switching
> us to JDK7. Might be some lessons learned about how to do this transition
> more smoothly.
>
The key thing is to have Jenkins happy before trying to change anything. And make sure that "happy" means that jenkins really is building and testing against the version of Java that you think it is. I spent about 15 minutes/hour of my weekend kicking off builds and tuning them. -I volunteered Colin Patrick McCabe to to that bit of the switch, though of course he doesn't have to accept that opportunuty ( https://issues.apache.org/jira/browse/HADOOP-10530?focusedCommentId=14239250&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14239250
Someone needs to take ownership of the "Jenkins builds are broken" problem -ideally someone in each subproject. That's a prerequisite to any major changes in the build process, but should also be best practice anyway. Ask your QE teams "would you like the ASF projects to care whether their jenkins test runs succeed" and see what they say.
as a reminder, all the jenkins builds are here
https://builds.apache.org/view/H-L/view/Hadoop/
please look at them and try to fix them —without even waiting for a switch to Java 8
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Steve Loughran <st...@hortonworks.com>.
> On 21 Apr 2015, at 23:31, Andrew Wang <an...@cloudera.com> wrote:
>
> Hey Robert,
>
> As a first step, could we try switching all our precommit and nightly
> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
> to do in any case since it'll still target 1.7.
+1
we have jenkin builds running on Java 8 -its just they have enough longstanding and intermittent test failures that they get ignored
I've been trying to keep hadoop-common somewhat under control, but
-YARN is failing due to to hard coded ports in tests - this needs to be fixed urgently: YARN-3528
-HDFS build process itself is failing with some config problem checkstyle
>
> I'll note that HADOOP-10530 details the pain Steve went through switching
> us to JDK7. Might be some lessons learned about how to do this transition
> more smoothly.
>
The key thing is to have Jenkins happy before trying to change anything. And make sure that "happy" means that jenkins really is building and testing against the version of Java that you think it is. I spent about 15 minutes/hour of my weekend kicking off builds and tuning them. -I volunteered Colin Patrick McCabe to to that bit of the switch, though of course he doesn't have to accept that opportunuty ( https://issues.apache.org/jira/browse/HADOOP-10530?focusedCommentId=14239250&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14239250
Someone needs to take ownership of the "Jenkins builds are broken" problem -ideally someone in each subproject. That's a prerequisite to any major changes in the build process, but should also be best practice anyway. Ask your QE teams "would you like the ASF projects to care whether their jenkins test runs succeed" and see what they say.
as a reminder, all the jenkins builds are here
https://builds.apache.org/view/H-L/view/Hadoop/
please look at them and try to fix them —without even waiting for a switch to Java 8
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Chris Nauroth <cn...@hortonworks.com>.
Suppose we configure maven-compiler-plugin with <source> set to 1.7 but
<target> set to 1.8 in trunk. I believe this would have the effect of
generating JDK 8 bytecode, but enforcing that our code sticks to JDK 7
compatibility at compile time. Does that still satisfy requirements for
HADOOP-11858?
I'd prefer to avoid executing duplicate builds for different JDK versions.
Pre-commit already takes a long time, and I suspect doubling the amount
of builds will make us starved for executors in Jenkins.
Chris Nauroth
Hortonworks
http://hortonworks.com/
On 4/21/15, 8:38 PM, "Sean Busbey" <bu...@cloudera.com> wrote:
>A few options:
>
>* Only change the builds for master to use jdk8
>* build with both jdk7 and jdk8 by copying jobs
>* build with both jdk7 and jdk8 using a jenkins matrix build
>
>Robert, if you'd like help with any of these please send me a ping
>off-list.
>
>On Tue, Apr 21, 2015 at 8:19 PM, Vinod Kumar Vavilapalli <
>vinodkv@hortonworks.com> wrote:
>
>> We don't want JDK 8 only code going into branch-2 line. Moving Jenkins
>>to
>> 1.8 right-away will shield such code, how do we address that?
>>
>> Thanks,
>> +Vinod
>>
>> On Apr 21, 2015, at 5:54 PM, Robert Kanter <rk...@cloudera.com> wrote:
>>
>> > Sure, I'll try to change the Jenkins builds to 1.8 first.
>> >
>> > On Tue, Apr 21, 2015 at 3:31 PM, Andrew Wang
>><an...@cloudera.com>
>> > wrote:
>> >
>> >> Hey Robert,
>> >>
>> >> As a first step, could we try switching all our precommit and nightly
>> >> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and
>> safe
>> >> to do in any case since it'll still target 1.7.
>> >>
>> >> I'll note that HADOOP-10530 details the pain Steve went through
>> switching
>> >> us to JDK7. Might be some lessons learned about how to do this
>> transition
>> >> more smoothly.
>> >>
>> >> Thanks,
>> >> Andrew
>> >>
>> >> On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com>
>> >> wrote:
>> >>
>> >>> + yarn-dev, hdfs-dev, mapred-dev
>> >>>
>> >>> On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter
>><rk...@cloudera.com>
>> >>> wrote:
>> >>>
>> >>>> Hi all,
>> >>>>
>> >>>> Moving forward on some of the discussions on Hadoop 3, I've created
>> >>>> HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I
>>just
>> >>>> wanted to let everyone know in case there's some reason we
>>shouldn't
>> go
>> >>>> ahead with this.
>> >>>>
>> >>>> thanks
>> >>>> - Robert
>> >>>>
>> >>>
>> >>
>>
>>
>
>
>--
>Sean
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Chris Nauroth <cn...@hortonworks.com>.
Suppose we configure maven-compiler-plugin with <source> set to 1.7 but
<target> set to 1.8 in trunk. I believe this would have the effect of
generating JDK 8 bytecode, but enforcing that our code sticks to JDK 7
compatibility at compile time. Does that still satisfy requirements for
HADOOP-11858?
I'd prefer to avoid executing duplicate builds for different JDK versions.
Pre-commit already takes a long time, and I suspect doubling the amount
of builds will make us starved for executors in Jenkins.
Chris Nauroth
Hortonworks
http://hortonworks.com/
On 4/21/15, 8:38 PM, "Sean Busbey" <bu...@cloudera.com> wrote:
>A few options:
>
>* Only change the builds for master to use jdk8
>* build with both jdk7 and jdk8 by copying jobs
>* build with both jdk7 and jdk8 using a jenkins matrix build
>
>Robert, if you'd like help with any of these please send me a ping
>off-list.
>
>On Tue, Apr 21, 2015 at 8:19 PM, Vinod Kumar Vavilapalli <
>vinodkv@hortonworks.com> wrote:
>
>> We don't want JDK 8 only code going into branch-2 line. Moving Jenkins
>>to
>> 1.8 right-away will shield such code, how do we address that?
>>
>> Thanks,
>> +Vinod
>>
>> On Apr 21, 2015, at 5:54 PM, Robert Kanter <rk...@cloudera.com> wrote:
>>
>> > Sure, I'll try to change the Jenkins builds to 1.8 first.
>> >
>> > On Tue, Apr 21, 2015 at 3:31 PM, Andrew Wang
>><an...@cloudera.com>
>> > wrote:
>> >
>> >> Hey Robert,
>> >>
>> >> As a first step, could we try switching all our precommit and nightly
>> >> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and
>> safe
>> >> to do in any case since it'll still target 1.7.
>> >>
>> >> I'll note that HADOOP-10530 details the pain Steve went through
>> switching
>> >> us to JDK7. Might be some lessons learned about how to do this
>> transition
>> >> more smoothly.
>> >>
>> >> Thanks,
>> >> Andrew
>> >>
>> >> On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com>
>> >> wrote:
>> >>
>> >>> + yarn-dev, hdfs-dev, mapred-dev
>> >>>
>> >>> On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter
>><rk...@cloudera.com>
>> >>> wrote:
>> >>>
>> >>>> Hi all,
>> >>>>
>> >>>> Moving forward on some of the discussions on Hadoop 3, I've created
>> >>>> HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I
>>just
>> >>>> wanted to let everyone know in case there's some reason we
>>shouldn't
>> go
>> >>>> ahead with this.
>> >>>>
>> >>>> thanks
>> >>>> - Robert
>> >>>>
>> >>>
>> >>
>>
>>
>
>
>--
>Sean
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Chris Nauroth <cn...@hortonworks.com>.
Suppose we configure maven-compiler-plugin with <source> set to 1.7 but
<target> set to 1.8 in trunk. I believe this would have the effect of
generating JDK 8 bytecode, but enforcing that our code sticks to JDK 7
compatibility at compile time. Does that still satisfy requirements for
HADOOP-11858?
I'd prefer to avoid executing duplicate builds for different JDK versions.
Pre-commit already takes a long time, and I suspect doubling the amount
of builds will make us starved for executors in Jenkins.
Chris Nauroth
Hortonworks
http://hortonworks.com/
On 4/21/15, 8:38 PM, "Sean Busbey" <bu...@cloudera.com> wrote:
>A few options:
>
>* Only change the builds for master to use jdk8
>* build with both jdk7 and jdk8 by copying jobs
>* build with both jdk7 and jdk8 using a jenkins matrix build
>
>Robert, if you'd like help with any of these please send me a ping
>off-list.
>
>On Tue, Apr 21, 2015 at 8:19 PM, Vinod Kumar Vavilapalli <
>vinodkv@hortonworks.com> wrote:
>
>> We don't want JDK 8 only code going into branch-2 line. Moving Jenkins
>>to
>> 1.8 right-away will shield such code, how do we address that?
>>
>> Thanks,
>> +Vinod
>>
>> On Apr 21, 2015, at 5:54 PM, Robert Kanter <rk...@cloudera.com> wrote:
>>
>> > Sure, I'll try to change the Jenkins builds to 1.8 first.
>> >
>> > On Tue, Apr 21, 2015 at 3:31 PM, Andrew Wang
>><an...@cloudera.com>
>> > wrote:
>> >
>> >> Hey Robert,
>> >>
>> >> As a first step, could we try switching all our precommit and nightly
>> >> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and
>> safe
>> >> to do in any case since it'll still target 1.7.
>> >>
>> >> I'll note that HADOOP-10530 details the pain Steve went through
>> switching
>> >> us to JDK7. Might be some lessons learned about how to do this
>> transition
>> >> more smoothly.
>> >>
>> >> Thanks,
>> >> Andrew
>> >>
>> >> On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com>
>> >> wrote:
>> >>
>> >>> + yarn-dev, hdfs-dev, mapred-dev
>> >>>
>> >>> On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter
>><rk...@cloudera.com>
>> >>> wrote:
>> >>>
>> >>>> Hi all,
>> >>>>
>> >>>> Moving forward on some of the discussions on Hadoop 3, I've created
>> >>>> HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I
>>just
>> >>>> wanted to let everyone know in case there's some reason we
>>shouldn't
>> go
>> >>>> ahead with this.
>> >>>>
>> >>>> thanks
>> >>>> - Robert
>> >>>>
>> >>>
>> >>
>>
>>
>
>
>--
>Sean
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Chris Nauroth <cn...@hortonworks.com>.
Suppose we configure maven-compiler-plugin with <source> set to 1.7 but
<target> set to 1.8 in trunk. I believe this would have the effect of
generating JDK 8 bytecode, but enforcing that our code sticks to JDK 7
compatibility at compile time. Does that still satisfy requirements for
HADOOP-11858?
I'd prefer to avoid executing duplicate builds for different JDK versions.
Pre-commit already takes a long time, and I suspect doubling the amount
of builds will make us starved for executors in Jenkins.
Chris Nauroth
Hortonworks
http://hortonworks.com/
On 4/21/15, 8:38 PM, "Sean Busbey" <bu...@cloudera.com> wrote:
>A few options:
>
>* Only change the builds for master to use jdk8
>* build with both jdk7 and jdk8 by copying jobs
>* build with both jdk7 and jdk8 using a jenkins matrix build
>
>Robert, if you'd like help with any of these please send me a ping
>off-list.
>
>On Tue, Apr 21, 2015 at 8:19 PM, Vinod Kumar Vavilapalli <
>vinodkv@hortonworks.com> wrote:
>
>> We don't want JDK 8 only code going into branch-2 line. Moving Jenkins
>>to
>> 1.8 right-away will shield such code, how do we address that?
>>
>> Thanks,
>> +Vinod
>>
>> On Apr 21, 2015, at 5:54 PM, Robert Kanter <rk...@cloudera.com> wrote:
>>
>> > Sure, I'll try to change the Jenkins builds to 1.8 first.
>> >
>> > On Tue, Apr 21, 2015 at 3:31 PM, Andrew Wang
>><an...@cloudera.com>
>> > wrote:
>> >
>> >> Hey Robert,
>> >>
>> >> As a first step, could we try switching all our precommit and nightly
>> >> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and
>> safe
>> >> to do in any case since it'll still target 1.7.
>> >>
>> >> I'll note that HADOOP-10530 details the pain Steve went through
>> switching
>> >> us to JDK7. Might be some lessons learned about how to do this
>> transition
>> >> more smoothly.
>> >>
>> >> Thanks,
>> >> Andrew
>> >>
>> >> On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com>
>> >> wrote:
>> >>
>> >>> + yarn-dev, hdfs-dev, mapred-dev
>> >>>
>> >>> On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter
>><rk...@cloudera.com>
>> >>> wrote:
>> >>>
>> >>>> Hi all,
>> >>>>
>> >>>> Moving forward on some of the discussions on Hadoop 3, I've created
>> >>>> HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I
>>just
>> >>>> wanted to let everyone know in case there's some reason we
>>shouldn't
>> go
>> >>>> ahead with this.
>> >>>>
>> >>>> thanks
>> >>>> - Robert
>> >>>>
>> >>>
>> >>
>>
>>
>
>
>--
>Sean
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Sean Busbey <bu...@cloudera.com>.
A few options:
* Only change the builds for master to use jdk8
* build with both jdk7 and jdk8 by copying jobs
* build with both jdk7 and jdk8 using a jenkins matrix build
Robert, if you'd like help with any of these please send me a ping off-list.
On Tue, Apr 21, 2015 at 8:19 PM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:
> We don't want JDK 8 only code going into branch-2 line. Moving Jenkins to
> 1.8 right-away will shield such code, how do we address that?
>
> Thanks,
> +Vinod
>
> On Apr 21, 2015, at 5:54 PM, Robert Kanter <rk...@cloudera.com> wrote:
>
> > Sure, I'll try to change the Jenkins builds to 1.8 first.
> >
> > On Tue, Apr 21, 2015 at 3:31 PM, Andrew Wang <an...@cloudera.com>
> > wrote:
> >
> >> Hey Robert,
> >>
> >> As a first step, could we try switching all our precommit and nightly
> >> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and
> safe
> >> to do in any case since it'll still target 1.7.
> >>
> >> I'll note that HADOOP-10530 details the pain Steve went through
> switching
> >> us to JDK7. Might be some lessons learned about how to do this
> transition
> >> more smoothly.
> >>
> >> Thanks,
> >> Andrew
> >>
> >> On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com>
> >> wrote:
> >>
> >>> + yarn-dev, hdfs-dev, mapred-dev
> >>>
> >>> On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com>
> >>> wrote:
> >>>
> >>>> Hi all,
> >>>>
> >>>> Moving forward on some of the discussions on Hadoop 3, I've created
> >>>> HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
> >>>> wanted to let everyone know in case there's some reason we shouldn't
> go
> >>>> ahead with this.
> >>>>
> >>>> thanks
> >>>> - Robert
> >>>>
> >>>
> >>
>
>
--
Sean
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Sean Busbey <bu...@cloudera.com>.
A few options:
* Only change the builds for master to use jdk8
* build with both jdk7 and jdk8 by copying jobs
* build with both jdk7 and jdk8 using a jenkins matrix build
Robert, if you'd like help with any of these please send me a ping off-list.
On Tue, Apr 21, 2015 at 8:19 PM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:
> We don't want JDK 8 only code going into branch-2 line. Moving Jenkins to
> 1.8 right-away will shield such code, how do we address that?
>
> Thanks,
> +Vinod
>
> On Apr 21, 2015, at 5:54 PM, Robert Kanter <rk...@cloudera.com> wrote:
>
> > Sure, I'll try to change the Jenkins builds to 1.8 first.
> >
> > On Tue, Apr 21, 2015 at 3:31 PM, Andrew Wang <an...@cloudera.com>
> > wrote:
> >
> >> Hey Robert,
> >>
> >> As a first step, could we try switching all our precommit and nightly
> >> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and
> safe
> >> to do in any case since it'll still target 1.7.
> >>
> >> I'll note that HADOOP-10530 details the pain Steve went through
> switching
> >> us to JDK7. Might be some lessons learned about how to do this
> transition
> >> more smoothly.
> >>
> >> Thanks,
> >> Andrew
> >>
> >> On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com>
> >> wrote:
> >>
> >>> + yarn-dev, hdfs-dev, mapred-dev
> >>>
> >>> On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com>
> >>> wrote:
> >>>
> >>>> Hi all,
> >>>>
> >>>> Moving forward on some of the discussions on Hadoop 3, I've created
> >>>> HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
> >>>> wanted to let everyone know in case there's some reason we shouldn't
> go
> >>>> ahead with this.
> >>>>
> >>>> thanks
> >>>> - Robert
> >>>>
> >>>
> >>
>
>
--
Sean
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Sean Busbey <bu...@cloudera.com>.
A few options:
* Only change the builds for master to use jdk8
* build with both jdk7 and jdk8 by copying jobs
* build with both jdk7 and jdk8 using a jenkins matrix build
Robert, if you'd like help with any of these please send me a ping off-list.
On Tue, Apr 21, 2015 at 8:19 PM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:
> We don't want JDK 8 only code going into branch-2 line. Moving Jenkins to
> 1.8 right-away will shield such code, how do we address that?
>
> Thanks,
> +Vinod
>
> On Apr 21, 2015, at 5:54 PM, Robert Kanter <rk...@cloudera.com> wrote:
>
> > Sure, I'll try to change the Jenkins builds to 1.8 first.
> >
> > On Tue, Apr 21, 2015 at 3:31 PM, Andrew Wang <an...@cloudera.com>
> > wrote:
> >
> >> Hey Robert,
> >>
> >> As a first step, could we try switching all our precommit and nightly
> >> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and
> safe
> >> to do in any case since it'll still target 1.7.
> >>
> >> I'll note that HADOOP-10530 details the pain Steve went through
> switching
> >> us to JDK7. Might be some lessons learned about how to do this
> transition
> >> more smoothly.
> >>
> >> Thanks,
> >> Andrew
> >>
> >> On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com>
> >> wrote:
> >>
> >>> + yarn-dev, hdfs-dev, mapred-dev
> >>>
> >>> On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com>
> >>> wrote:
> >>>
> >>>> Hi all,
> >>>>
> >>>> Moving forward on some of the discussions on Hadoop 3, I've created
> >>>> HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
> >>>> wanted to let everyone know in case there's some reason we shouldn't
> go
> >>>> ahead with this.
> >>>>
> >>>> thanks
> >>>> - Robert
> >>>>
> >>>
> >>
>
>
--
Sean
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Sean Busbey <bu...@cloudera.com>.
A few options:
* Only change the builds for master to use jdk8
* build with both jdk7 and jdk8 by copying jobs
* build with both jdk7 and jdk8 using a jenkins matrix build
Robert, if you'd like help with any of these please send me a ping off-list.
On Tue, Apr 21, 2015 at 8:19 PM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:
> We don't want JDK 8 only code going into branch-2 line. Moving Jenkins to
> 1.8 right-away will shield such code, how do we address that?
>
> Thanks,
> +Vinod
>
> On Apr 21, 2015, at 5:54 PM, Robert Kanter <rk...@cloudera.com> wrote:
>
> > Sure, I'll try to change the Jenkins builds to 1.8 first.
> >
> > On Tue, Apr 21, 2015 at 3:31 PM, Andrew Wang <an...@cloudera.com>
> > wrote:
> >
> >> Hey Robert,
> >>
> >> As a first step, could we try switching all our precommit and nightly
> >> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and
> safe
> >> to do in any case since it'll still target 1.7.
> >>
> >> I'll note that HADOOP-10530 details the pain Steve went through
> switching
> >> us to JDK7. Might be some lessons learned about how to do this
> transition
> >> more smoothly.
> >>
> >> Thanks,
> >> Andrew
> >>
> >> On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com>
> >> wrote:
> >>
> >>> + yarn-dev, hdfs-dev, mapred-dev
> >>>
> >>> On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com>
> >>> wrote:
> >>>
> >>>> Hi all,
> >>>>
> >>>> Moving forward on some of the discussions on Hadoop 3, I've created
> >>>> HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
> >>>> wanted to let everyone know in case there's some reason we shouldn't
> go
> >>>> ahead with this.
> >>>>
> >>>> thanks
> >>>> - Robert
> >>>>
> >>>
> >>
>
>
--
Sean
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
We don't want JDK 8 only code going into branch-2 line. Moving Jenkins to 1.8 right-away will shield such code, how do we address that?
Thanks,
+Vinod
On Apr 21, 2015, at 5:54 PM, Robert Kanter <rk...@cloudera.com> wrote:
> Sure, I'll try to change the Jenkins builds to 1.8 first.
>
> On Tue, Apr 21, 2015 at 3:31 PM, Andrew Wang <an...@cloudera.com>
> wrote:
>
>> Hey Robert,
>>
>> As a first step, could we try switching all our precommit and nightly
>> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
>> to do in any case since it'll still target 1.7.
>>
>> I'll note that HADOOP-10530 details the pain Steve went through switching
>> us to JDK7. Might be some lessons learned about how to do this transition
>> more smoothly.
>>
>> Thanks,
>> Andrew
>>
>> On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com>
>> wrote:
>>
>>> + yarn-dev, hdfs-dev, mapred-dev
>>>
>>> On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Moving forward on some of the discussions on Hadoop 3, I've created
>>>> HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
>>>> wanted to let everyone know in case there's some reason we shouldn't go
>>>> ahead with this.
>>>>
>>>> thanks
>>>> - Robert
>>>>
>>>
>>
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
We don't want JDK 8 only code going into branch-2 line. Moving Jenkins to 1.8 right-away will shield such code, how do we address that?
Thanks,
+Vinod
On Apr 21, 2015, at 5:54 PM, Robert Kanter <rk...@cloudera.com> wrote:
> Sure, I'll try to change the Jenkins builds to 1.8 first.
>
> On Tue, Apr 21, 2015 at 3:31 PM, Andrew Wang <an...@cloudera.com>
> wrote:
>
>> Hey Robert,
>>
>> As a first step, could we try switching all our precommit and nightly
>> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
>> to do in any case since it'll still target 1.7.
>>
>> I'll note that HADOOP-10530 details the pain Steve went through switching
>> us to JDK7. Might be some lessons learned about how to do this transition
>> more smoothly.
>>
>> Thanks,
>> Andrew
>>
>> On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com>
>> wrote:
>>
>>> + yarn-dev, hdfs-dev, mapred-dev
>>>
>>> On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Moving forward on some of the discussions on Hadoop 3, I've created
>>>> HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
>>>> wanted to let everyone know in case there's some reason we shouldn't go
>>>> ahead with this.
>>>>
>>>> thanks
>>>> - Robert
>>>>
>>>
>>
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
We don't want JDK 8 only code going into branch-2 line. Moving Jenkins to 1.8 right-away will shield such code, how do we address that?
Thanks,
+Vinod
On Apr 21, 2015, at 5:54 PM, Robert Kanter <rk...@cloudera.com> wrote:
> Sure, I'll try to change the Jenkins builds to 1.8 first.
>
> On Tue, Apr 21, 2015 at 3:31 PM, Andrew Wang <an...@cloudera.com>
> wrote:
>
>> Hey Robert,
>>
>> As a first step, could we try switching all our precommit and nightly
>> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
>> to do in any case since it'll still target 1.7.
>>
>> I'll note that HADOOP-10530 details the pain Steve went through switching
>> us to JDK7. Might be some lessons learned about how to do this transition
>> more smoothly.
>>
>> Thanks,
>> Andrew
>>
>> On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com>
>> wrote:
>>
>>> + yarn-dev, hdfs-dev, mapred-dev
>>>
>>> On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Moving forward on some of the discussions on Hadoop 3, I've created
>>>> HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
>>>> wanted to let everyone know in case there's some reason we shouldn't go
>>>> ahead with this.
>>>>
>>>> thanks
>>>> - Robert
>>>>
>>>
>>
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
We don't want JDK 8 only code going into branch-2 line. Moving Jenkins to 1.8 right-away will shield such code, how do we address that?
Thanks,
+Vinod
On Apr 21, 2015, at 5:54 PM, Robert Kanter <rk...@cloudera.com> wrote:
> Sure, I'll try to change the Jenkins builds to 1.8 first.
>
> On Tue, Apr 21, 2015 at 3:31 PM, Andrew Wang <an...@cloudera.com>
> wrote:
>
>> Hey Robert,
>>
>> As a first step, could we try switching all our precommit and nightly
>> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
>> to do in any case since it'll still target 1.7.
>>
>> I'll note that HADOOP-10530 details the pain Steve went through switching
>> us to JDK7. Might be some lessons learned about how to do this transition
>> more smoothly.
>>
>> Thanks,
>> Andrew
>>
>> On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com>
>> wrote:
>>
>>> + yarn-dev, hdfs-dev, mapred-dev
>>>
>>> On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Moving forward on some of the discussions on Hadoop 3, I've created
>>>> HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
>>>> wanted to let everyone know in case there's some reason we shouldn't go
>>>> ahead with this.
>>>>
>>>> thanks
>>>> - Robert
>>>>
>>>
>>
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Robert Kanter <rk...@cloudera.com>.
Sure, I'll try to change the Jenkins builds to 1.8 first.
On Tue, Apr 21, 2015 at 3:31 PM, Andrew Wang <an...@cloudera.com>
wrote:
> Hey Robert,
>
> As a first step, could we try switching all our precommit and nightly
> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
> to do in any case since it'll still target 1.7.
>
> I'll note that HADOOP-10530 details the pain Steve went through switching
> us to JDK7. Might be some lessons learned about how to do this transition
> more smoothly.
>
> Thanks,
> Andrew
>
> On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com>
> wrote:
>
> > + yarn-dev, hdfs-dev, mapred-dev
> >
> > On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > Moving forward on some of the discussions on Hadoop 3, I've created
> > > HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
> > > wanted to let everyone know in case there's some reason we shouldn't go
> > > ahead with this.
> > >
> > > thanks
> > > - Robert
> > >
> >
>
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Robert Kanter <rk...@cloudera.com>.
Sure, I'll try to change the Jenkins builds to 1.8 first.
On Tue, Apr 21, 2015 at 3:31 PM, Andrew Wang <an...@cloudera.com>
wrote:
> Hey Robert,
>
> As a first step, could we try switching all our precommit and nightly
> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
> to do in any case since it'll still target 1.7.
>
> I'll note that HADOOP-10530 details the pain Steve went through switching
> us to JDK7. Might be some lessons learned about how to do this transition
> more smoothly.
>
> Thanks,
> Andrew
>
> On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com>
> wrote:
>
> > + yarn-dev, hdfs-dev, mapred-dev
> >
> > On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > Moving forward on some of the discussions on Hadoop 3, I've created
> > > HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
> > > wanted to let everyone know in case there's some reason we shouldn't go
> > > ahead with this.
> > >
> > > thanks
> > > - Robert
> > >
> >
>
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Robert Kanter <rk...@cloudera.com>.
Sure, I'll try to change the Jenkins builds to 1.8 first.
On Tue, Apr 21, 2015 at 3:31 PM, Andrew Wang <an...@cloudera.com>
wrote:
> Hey Robert,
>
> As a first step, could we try switching all our precommit and nightly
> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
> to do in any case since it'll still target 1.7.
>
> I'll note that HADOOP-10530 details the pain Steve went through switching
> us to JDK7. Might be some lessons learned about how to do this transition
> more smoothly.
>
> Thanks,
> Andrew
>
> On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com>
> wrote:
>
> > + yarn-dev, hdfs-dev, mapred-dev
> >
> > On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > Moving forward on some of the discussions on Hadoop 3, I've created
> > > HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
> > > wanted to let everyone know in case there's some reason we shouldn't go
> > > ahead with this.
> > >
> > > thanks
> > > - Robert
> > >
> >
>
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Robert Kanter <rk...@cloudera.com>.
Sure, I'll try to change the Jenkins builds to 1.8 first.
On Tue, Apr 21, 2015 at 3:31 PM, Andrew Wang <an...@cloudera.com>
wrote:
> Hey Robert,
>
> As a first step, could we try switching all our precommit and nightly
> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
> to do in any case since it'll still target 1.7.
>
> I'll note that HADOOP-10530 details the pain Steve went through switching
> us to JDK7. Might be some lessons learned about how to do this transition
> more smoothly.
>
> Thanks,
> Andrew
>
> On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com>
> wrote:
>
> > + yarn-dev, hdfs-dev, mapred-dev
> >
> > On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > Moving forward on some of the discussions on Hadoop 3, I've created
> > > HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
> > > wanted to let everyone know in case there's some reason we shouldn't go
> > > ahead with this.
> > >
> > > thanks
> > > - Robert
> > >
> >
>
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Steve Loughran <st...@hortonworks.com>.
> On 21 Apr 2015, at 23:31, Andrew Wang <an...@cloudera.com> wrote:
>
> Hey Robert,
>
> As a first step, could we try switching all our precommit and nightly
> builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
> to do in any case since it'll still target 1.7.
+1
we have jenkin builds running on Java 8 -its just they have enough longstanding and intermittent test failures that they get ignored
I've been trying to keep hadoop-common somewhat under control, but
-YARN is failing due to to hard coded ports in tests - this needs to be fixed urgently: YARN-3528
-HDFS build process itself is failing with some config problem checkstyle
>
> I'll note that HADOOP-10530 details the pain Steve went through switching
> us to JDK7. Might be some lessons learned about how to do this transition
> more smoothly.
>
The key thing is to have Jenkins happy before trying to change anything. And make sure that "happy" means that jenkins really is building and testing against the version of Java that you think it is. I spent about 15 minutes/hour of my weekend kicking off builds and tuning them. -I volunteered Colin Patrick McCabe to to that bit of the switch, though of course he doesn't have to accept that opportunuty ( https://issues.apache.org/jira/browse/HADOOP-10530?focusedCommentId=14239250&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14239250
Someone needs to take ownership of the "Jenkins builds are broken" problem -ideally someone in each subproject. That's a prerequisite to any major changes in the build process, but should also be best practice anyway. Ask your QE teams "would you like the ASF projects to care whether their jenkins test runs succeed" and see what they say.
as a reminder, all the jenkins builds are here
https://builds.apache.org/view/H-L/view/Hadoop/
please look at them and try to fix them —without even waiting for a switch to Java 8
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Andrew Wang <an...@cloudera.com>.
Hey Robert,
As a first step, could we try switching all our precommit and nightly
builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
to do in any case since it'll still target 1.7.
I'll note that HADOOP-10530 details the pain Steve went through switching
us to JDK7. Might be some lessons learned about how to do this transition
more smoothly.
Thanks,
Andrew
On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com> wrote:
> + yarn-dev, hdfs-dev, mapred-dev
>
> On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com>
> wrote:
>
> > Hi all,
> >
> > Moving forward on some of the discussions on Hadoop 3, I've created
> > HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
> > wanted to let everyone know in case there's some reason we shouldn't go
> > ahead with this.
> >
> > thanks
> > - Robert
> >
>
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Andrew Wang <an...@cloudera.com>.
Hey Robert,
As a first step, could we try switching all our precommit and nightly
builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
to do in any case since it'll still target 1.7.
I'll note that HADOOP-10530 details the pain Steve went through switching
us to JDK7. Might be some lessons learned about how to do this transition
more smoothly.
Thanks,
Andrew
On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com> wrote:
> + yarn-dev, hdfs-dev, mapred-dev
>
> On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com>
> wrote:
>
> > Hi all,
> >
> > Moving forward on some of the discussions on Hadoop 3, I've created
> > HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
> > wanted to let everyone know in case there's some reason we shouldn't go
> > ahead with this.
> >
> > thanks
> > - Robert
> >
>
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Andrew Wang <an...@cloudera.com>.
Hey Robert,
As a first step, could we try switching all our precommit and nightly
builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
to do in any case since it'll still target 1.7.
I'll note that HADOOP-10530 details the pain Steve went through switching
us to JDK7. Might be some lessons learned about how to do this transition
more smoothly.
Thanks,
Andrew
On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com> wrote:
> + yarn-dev, hdfs-dev, mapred-dev
>
> On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com>
> wrote:
>
> > Hi all,
> >
> > Moving forward on some of the discussions on Hadoop 3, I've created
> > HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
> > wanted to let everyone know in case there's some reason we shouldn't go
> > ahead with this.
> >
> > thanks
> > - Robert
> >
>
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Andrew Wang <an...@cloudera.com>.
Hey Robert,
As a first step, could we try switching all our precommit and nightly
builds over to use 1.8? This is a prerequisite for HADOOP-11858, and safe
to do in any case since it'll still target 1.7.
I'll note that HADOOP-10530 details the pain Steve went through switching
us to JDK7. Might be some lessons learned about how to do this transition
more smoothly.
Thanks,
Andrew
On Tue, Apr 21, 2015 at 3:15 PM, Robert Kanter <rk...@cloudera.com> wrote:
> + yarn-dev, hdfs-dev, mapred-dev
>
> On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com>
> wrote:
>
> > Hi all,
> >
> > Moving forward on some of the discussions on Hadoop 3, I've created
> > HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
> > wanted to let everyone know in case there's some reason we shouldn't go
> > ahead with this.
> >
> > thanks
> > - Robert
> >
>
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Robert Kanter <rk...@cloudera.com>.
+ yarn-dev, hdfs-dev, mapred-dev
On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com> wrote:
> Hi all,
>
> Moving forward on some of the discussions on Hadoop 3, I've created
> HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
> wanted to let everyone know in case there's some reason we shouldn't go
> ahead with this.
>
> thanks
> - Robert
>
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Actually I didn't see any conclusion on this, on the previous third. There were proposals and counter points, but we never made a decision. We should get consensus first.
Thanks
+Vinod
On Apr 21, 2015, at 3:14 PM, Robert Kanter <rk...@cloudera.com> wrote:
> Hi all,
>
> Moving forward on some of the discussions on Hadoop 3, I've created
> HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
> wanted to let everyone know in case there's some reason we shouldn't go
> ahead with this.
>
> thanks
> - Robert
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Robert Kanter <rk...@cloudera.com>.
+ yarn-dev, hdfs-dev, mapred-dev
On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com> wrote:
> Hi all,
>
> Moving forward on some of the discussions on Hadoop 3, I've created
> HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
> wanted to let everyone know in case there's some reason we shouldn't go
> ahead with this.
>
> thanks
> - Robert
>
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Robert Kanter <rk...@cloudera.com>.
+ yarn-dev, hdfs-dev, mapred-dev
On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com> wrote:
> Hi all,
>
> Moving forward on some of the discussions on Hadoop 3, I've created
> HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
> wanted to let everyone know in case there's some reason we shouldn't go
> ahead with this.
>
> thanks
> - Robert
>
Re: Set minimum version of Hadoop 3 to JDK 8
Posted by Robert Kanter <rk...@cloudera.com>.
+ yarn-dev, hdfs-dev, mapred-dev
On Tue, Apr 21, 2015 at 3:14 PM, Robert Kanter <rk...@cloudera.com> wrote:
> Hi all,
>
> Moving forward on some of the discussions on Hadoop 3, I've created
> HADOOP-11858 to set the minimum version of Hadoop 3 to JDK 8. I just
> wanted to let everyone know in case there's some reason we shouldn't go
> ahead with this.
>
> thanks
> - Robert
>