You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Robert Rati <rr...@redhat.com> on 2013/11/01 15:57:56 UTC
Re: Hadoop in Fedora updated to 2.2.0
Putting the java 6 vs java 7 issue aside, what about the other patches
to update dependencies? Can those be looked at an planned for inclusion
into a releation?
Rob
On 10/31/2013 05:51 PM, Andrew Wang wrote:
> I'm in agreement with Steve on this one. We're aware that Java 6 is EOL,
> but we can't drop support for the lifetime of the 2.x line since it's a
> (very) incompatible change. AFAIK a 3.x release fixing this isn't on any of
> our horizons yet.
>
> Best,
> Andrew
>
>
> On Thu, Oct 31, 2013 at 6:15 AM, Robert Rati <rr...@redhat.com> wrote:
>
>> https://issues.apache.org/****jira/browse/HADOOP-9594<https://issues.apache.org/**jira/browse/HADOOP-9594>
>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9594<https://issues.apache.org/jira/browse/HADOOP-9594>
>>>>>
>>>> https://issues.apache.org/****jira/browse/MAPREDUCE-5431<https://issues.apache.org/**jira/browse/MAPREDUCE-5431>
>>>> <htt**ps://issues.apache.org/jira/**browse/MAPREDUCE-5431<https://issues.apache.org/jira/browse/MAPREDUCE-5431>
>>>>>
>>>> https://issues.apache.org/****jira/browse/HADOOP-9611<https://issues.apache.org/**jira/browse/HADOOP-9611>
>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9611<https://issues.apache.org/jira/browse/HADOOP-9611>
>>>>>
>>>> https://issues.apache.org/****jira/browse/HADOOP-9613<https://issues.apache.org/**jira/browse/HADOOP-9613>
>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9613<https://issues.apache.org/jira/browse/HADOOP-9613>
>>>>>
>>>> https://issues.apache.org/****jira/browse/HADOOP-9623<https://issues.apache.org/**jira/browse/HADOOP-9623>
>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9623<https://issues.apache.org/jira/browse/HADOOP-9623>
>>>>>
>>>> https://issues.apache.org/****jira/browse/HDFS-5411<https://issues.apache.org/**jira/browse/HDFS-5411>
>>>> <https://**issues.apache.org/jira/browse/**HDFS-5411<https://issues.apache.org/jira/browse/HDFS-5411>
>>>>>
>>>> https://issues.apache.org/****jira/browse/HADOOP-10067<https://issues.apache.org/**jira/browse/HADOOP-10067>
>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10067<https://issues.apache.org/jira/browse/HADOOP-10067>
>>>>>
>>>> https://issues.apache.org/****jira/browse/HDFS-5075<https://issues.apache.org/**jira/browse/HDFS-5075>
>>>> <https://**issues.apache.org/jira/browse/**HDFS-5075<https://issues.apache.org/jira/browse/HDFS-5075>
>>>>>
>>>> https://issues.apache.org/****jira/browse/HADOOP-10068<https://issues.apache.org/**jira/browse/HADOOP-10068>
>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10068<https://issues.apache.org/jira/browse/HADOOP-10068>
>>>>>
>>>> https://issues.apache.org/****jira/browse/HADOOP-10075<https://issues.apache.org/**jira/browse/HADOOP-10075>
>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10075<https://issues.apache.org/jira/browse/HADOOP-10075>
>>>>>
>>>> https://issues.apache.org/****jira/browse/HADOOP-10076<https://issues.apache.org/**jira/browse/HADOOP-10076>
>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10076<https://issues.apache.org/jira/browse/HADOOP-10076>
>>>>>
>>>> https://issues.apache.org/****jira/browse/HADOOP-9849<https://issues.apache.org/**jira/browse/HADOOP-9849>
>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9849<https://issues.apache.org/jira/browse/HADOOP-9849>
>>>>>
>>>>
>>>>
>>>> most (all?) of these are pom changes
>>>
>>
>> A good number are basically pom changes to update to newer versions of
>> dependencies. A few, such as commons-math3, required code changes as well
>> because of a namespace change. Some are minor code changes to enhance
>> compatibility with newer dependencies. Even tomcat is mostly changes in
>> pom files.
>>
>>
>> Most of the changes are minor. There are 2 big updates though: Jetty 9
>>>> (which requires java 7) and tomcat 7. These are also the most difficult
>>>> patches to rebase when hadoop produces a new release.
>>>>
>>>>
>>>> that's not going to go in the 2.x branch. Java 6 is still a common
>>> platform
>>> that people are using, because historically java7 (or any leading edge
>>> java
>>> version) is buggy.
>>>
>>> that said, our QA team did test hadoop 2 & HDP-2 at scale on java7 and
>>> openjdk 7, so it all works -it's just the commit "java7 only" is a big
>>> decision that
>>>
>>
>> I realize moving to java 7 is a big decision and wasn't trying to imply
>> this should happen without discussion and planning, just that it would be
>> nice to have the discussion and see where things land. It can also help
>> minimize work. There is an open bz for updating jetty to jetty 8 (the last
>> version that would work on java 6), but if there are plans to move to
>> java7, maybe it makes sense to just to jetty 9 and not test a new version
>> of jetty twice.
>>
>> With Hadoop in Fedora running on these newer deps there is a test bed to
>> play with to give some level of confidence before taking the plunge on any
>> major change.
>>
>> Rob
>>
>
Re: Hadoop in Fedora updated to 2.2.0
Posted by Robert Rati <rr...@redhat.com>.
Bah, typos. Can those other patches be looked at and planned for
inclusion into a release?
Rob
On 11/01/2013 10:57 AM, Robert Rati wrote:
> Putting the java 6 vs java 7 issue aside, what about the other patches
> to update dependencies? Can those be looked at an planned for inclusion
> into a releation?
>
> Rob
>
> On 10/31/2013 05:51 PM, Andrew Wang wrote:
>> I'm in agreement with Steve on this one. We're aware that Java 6 is EOL,
>> but we can't drop support for the lifetime of the 2.x line since it's a
>> (very) incompatible change. AFAIK a 3.x release fixing this isn't on
>> any of
>> our horizons yet.
>>
>> Best,
>> Andrew
>>
>>
>> On Thu, Oct 31, 2013 at 6:15 AM, Robert Rati <rr...@redhat.com> wrote:
>>
>>> https://issues.apache.org/****jira/browse/HADOOP-9594<https://issues.apache.org/**jira/browse/HADOOP-9594>
>>>
>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9594<https://issues.apache.org/jira/browse/HADOOP-9594>
>>>>>
>>>>>>
>>>>> https://issues.apache.org/****jira/browse/MAPREDUCE-5431<https://issues.apache.org/**jira/browse/MAPREDUCE-5431>
>>>>>
>>>>> <htt**ps://issues.apache.org/jira/**browse/MAPREDUCE-5431<https://issues.apache.org/jira/browse/MAPREDUCE-5431>
>>>>>
>>>>>>
>>>>> https://issues.apache.org/****jira/browse/HADOOP-9611<https://issues.apache.org/**jira/browse/HADOOP-9611>
>>>>>
>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9611<https://issues.apache.org/jira/browse/HADOOP-9611>
>>>>>
>>>>>>
>>>>> https://issues.apache.org/****jira/browse/HADOOP-9613<https://issues.apache.org/**jira/browse/HADOOP-9613>
>>>>>
>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9613<https://issues.apache.org/jira/browse/HADOOP-9613>
>>>>>
>>>>>>
>>>>> https://issues.apache.org/****jira/browse/HADOOP-9623<https://issues.apache.org/**jira/browse/HADOOP-9623>
>>>>>
>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9623<https://issues.apache.org/jira/browse/HADOOP-9623>
>>>>>
>>>>>>
>>>>> https://issues.apache.org/****jira/browse/HDFS-5411<https://issues.apache.org/**jira/browse/HDFS-5411>
>>>>>
>>>>> <https://**issues.apache.org/jira/browse/**HDFS-5411<https://issues.apache.org/jira/browse/HDFS-5411>
>>>>>
>>>>>>
>>>>> https://issues.apache.org/****jira/browse/HADOOP-10067<https://issues.apache.org/**jira/browse/HADOOP-10067>
>>>>>
>>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10067<https://issues.apache.org/jira/browse/HADOOP-10067>
>>>>>
>>>>>>
>>>>> https://issues.apache.org/****jira/browse/HDFS-5075<https://issues.apache.org/**jira/browse/HDFS-5075>
>>>>>
>>>>> <https://**issues.apache.org/jira/browse/**HDFS-5075<https://issues.apache.org/jira/browse/HDFS-5075>
>>>>>
>>>>>>
>>>>> https://issues.apache.org/****jira/browse/HADOOP-10068<https://issues.apache.org/**jira/browse/HADOOP-10068>
>>>>>
>>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10068<https://issues.apache.org/jira/browse/HADOOP-10068>
>>>>>
>>>>>>
>>>>> https://issues.apache.org/****jira/browse/HADOOP-10075<https://issues.apache.org/**jira/browse/HADOOP-10075>
>>>>>
>>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10075<https://issues.apache.org/jira/browse/HADOOP-10075>
>>>>>
>>>>>>
>>>>> https://issues.apache.org/****jira/browse/HADOOP-10076<https://issues.apache.org/**jira/browse/HADOOP-10076>
>>>>>
>>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10076<https://issues.apache.org/jira/browse/HADOOP-10076>
>>>>>
>>>>>>
>>>>> https://issues.apache.org/****jira/browse/HADOOP-9849<https://issues.apache.org/**jira/browse/HADOOP-9849>
>>>>>
>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9849<https://issues.apache.org/jira/browse/HADOOP-9849>
>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> most (all?) of these are pom changes
>>>>
>>>
>>> A good number are basically pom changes to update to newer versions of
>>> dependencies. A few, such as commons-math3, required code changes as
>>> well
>>> because of a namespace change. Some are minor code changes to enhance
>>> compatibility with newer dependencies. Even tomcat is mostly changes in
>>> pom files.
>>>
>>>
>>> Most of the changes are minor. There are 2 big updates though:
>>> Jetty 9
>>>>> (which requires java 7) and tomcat 7. These are also the most
>>>>> difficult
>>>>> patches to rebase when hadoop produces a new release.
>>>>>
>>>>>
>>>>> that's not going to go in the 2.x branch. Java 6 is still a common
>>>> platform
>>>> that people are using, because historically java7 (or any leading edge
>>>> java
>>>> version) is buggy.
>>>>
>>>> that said, our QA team did test hadoop 2 & HDP-2 at scale on java7 and
>>>> openjdk 7, so it all works -it's just the commit "java7 only" is a big
>>>> decision that
>>>>
>>>
>>> I realize moving to java 7 is a big decision and wasn't trying to imply
>>> this should happen without discussion and planning, just that it
>>> would be
>>> nice to have the discussion and see where things land. It can also help
>>> minimize work. There is an open bz for updating jetty to jetty 8
>>> (the last
>>> version that would work on java 6), but if there are plans to move to
>>> java7, maybe it makes sense to just to jetty 9 and not test a new
>>> version
>>> of jetty twice.
>>>
>>> With Hadoop in Fedora running on these newer deps there is a test bed to
>>> play with to give some level of confidence before taking the plunge
>>> on any
>>> major change.
>>>
>>> Rob
>>>
>>
Re: Hadoop in Fedora updated to 2.2.0
Posted by Steve Loughran <st...@hortonworks.com>.
On 21 November 2013 14:59, Robert Rati <rr...@redhat.com> wrote:
> Just to clarify, the tomcat/jasper updates and the jersey updates should
> be able to go in without any jetty changes.
OK
> There is also a separate BZ for updating jetty to jetty 8, which is the
> last jetty version that will run on java 6, if there is a desire to update
> jetty without requiring java 7.
>
> If jetty 9 is being looked at for inclusion it will affect the jasper bits
> in the poms. Jetty 9 has its own jsp compiler and would need to replace
> jasper, but that's largely just pom changes iirc. Jetty 9 does revamp some
> apis and should definitely be looked at by people more knowledgeable with
> how jetty is used, especially as it relates to secure mode.
>
>
Jetty gets used in the shuffle phase, and its where there's already code to
detect and react to it playing up -which is why its an area people are
scared of.
https://issues.apache.org/jira/browse/HADOOP-1158
now, if you were to work with the MR team to replace that with some
netty-based protocol that issue would go away (or at least move)
> Rob
>
>
> On 11/13/2013 03:31 PM, Steve Loughran wrote:
>
>> I've just been through some of these as part of my background project,
>> "fix
>> up the POMs" https://issues.apache.org/jira/browse/HADOOP-9991.
>>
>>
>> 1. I've applied the simple low/risk ones.
>> 2. I've not done the bookkeeper one, as people working with that code
>>
>> need to play with it first.
>> 3. I've not touched anything related to {jersey, tomcat, jetty}
>>
>>
>> This is more than just a java6/7 issue, is is that Jetty has been very
>> brittle in the past, and there's code in hadoop to detect when it's not
>> actually servicing requests properly. Moving up Jetty/web server versions
>> is something that needs to be done carefull and with consensus -and once
>> you leave Jetty alone, I don't know where the jersey and tomcat changes
>> go.
>>
>> There is always the option of s/jetty/r/grizzly/
>>
>> -steve
>>
>>
>>
>>
>> On 1 November 2013 14:57, Robert Rati <rr...@redhat.com> wrote:
>>
>> Putting the java 6 vs java 7 issue aside, what about the other patches to
>>> update dependencies? Can those be looked at an planned for inclusion
>>> into
>>> a releation?
>>>
>>> Rob
>>>
>>>
>>> On 10/31/2013 05:51 PM, Andrew Wang wrote:
>>>
>>> I'm in agreement with Steve on this one. We're aware that Java 6 is EOL,
>>>> but we can't drop support for the lifetime of the 2.x line since it's a
>>>> (very) incompatible change. AFAIK a 3.x release fixing this isn't on any
>>>> of
>>>> our horizons yet.
>>>>
>>>> Best,
>>>> Andrew
>>>>
>>>>
>>>> On Thu, Oct 31, 2013 at 6:15 AM, Robert Rati <rr...@redhat.com> wrote:
>>>>
>>>> https://issues.apache.org/****jira/browse/HADOOP-9594<https:
>>>>
>>>>> //issues.apache.org/**jira/browse/HADOOP-9594>
>>>>>
>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9594<http
>>>>>>
>>>>>>> s://issues.apache.org/jira/browse/HADOOP-9594>
>>>>>>>
>>>>>>>
>>>>>>>> https://issues.apache.org/****jira/browse/MAPREDUCE-5431<htt
>>>>>>>>
>>>>>>> ps://issues.apache.org/**jira/browse/MAPREDUCE-5431>
>>>>>>> <htt**ps://issues.apache.org/jira/**browse/MAPREDUCE-5431<h
>>>>>>> ttps://issues.apache.org/jira/browse/MAPREDUCE-5431>
>>>>>>>
>>>>>>>
>>>>>>>> https://issues.apache.org/****jira/browse/HADOOP-9611<https:
>>>>>>>>
>>>>>>> //issues.apache.org/**jira/browse/HADOOP-9611>
>>>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9611<http
>>>>>>> s://issues.apache.org/jira/browse/HADOOP-9611>
>>>>>>>
>>>>>>>
>>>>>>>> https://issues.apache.org/****jira/browse/HADOOP-9613<https:
>>>>>>>>
>>>>>>> //issues.apache.org/**jira/browse/HADOOP-9613>
>>>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9613<http
>>>>>>> s://issues.apache.org/jira/browse/HADOOP-9613>
>>>>>>>
>>>>>>>
>>>>>>>> https://issues.apache.org/****jira/browse/HADOOP-9623<https:
>>>>>>>>
>>>>>>> //issues.apache.org/**jira/browse/HADOOP-9623>
>>>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9623<http
>>>>>>> s://issues.apache.org/jira/browse/HADOOP-9623>
>>>>>>>
>>>>>>>
>>>>>>>> https://issues.apache.org/****jira/browse/HDFS-5411<https://
>>>>>>>>
>>>>>>> issues.apache.org/**jira/browse/HDFS-5411>
>>>>>>> <https://**issues.apache.org/jira/browse/**HDFS-5411<https:
>>>>>>> //issues.apache.org/jira/browse/HDFS-5411>
>>>>>>>
>>>>>>>
>>>>>>>> https://issues.apache.org/****jira/browse/HADOOP-10067<https
>>>>>>>>
>>>>>>> ://issues.apache.org/**jira/browse/HADOOP-10067>
>>>>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10067<htt
>>>>>>> ps://issues.apache.org/jira/browse/HADOOP-10067>
>>>>>>>
>>>>>>>
>>>>>>>> https://issues.apache.org/****jira/browse/HDFS-5075<https://
>>>>>>>>
>>>>>>> issues.apache.org/**jira/browse/HDFS-5075>
>>>>>>> <https://**issues.apache.org/jira/browse/**HDFS-5075<https:
>>>>>>> //issues.apache.org/jira/browse/HDFS-5075>
>>>>>>>
>>>>>>>
>>>>>>>> https://issues.apache.org/****jira/browse/HADOOP-10068<https
>>>>>>>>
>>>>>>> ://issues.apache.org/**jira/browse/HADOOP-10068>
>>>>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10068<htt
>>>>>>> ps://issues.apache.org/jira/browse/HADOOP-10068>
>>>>>>>
>>>>>>>
>>>>>>>> https://issues.apache.org/****jira/browse/HADOOP-10075<https
>>>>>>>>
>>>>>>> ://issues.apache.org/**jira/browse/HADOOP-10075>
>>>>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10075<htt
>>>>>>> ps://issues.apache.org/jira/browse/HADOOP-10075>
>>>>>>>
>>>>>>>
>>>>>>>> https://issues.apache.org/****jira/browse/HADOOP-10076<https
>>>>>>>>
>>>>>>> ://issues.apache.org/**jira/browse/HADOOP-10076>
>>>>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10076<htt
>>>>>>> ps://issues.apache.org/jira/browse/HADOOP-10076>
>>>>>>>
>>>>>>>
>>>>>>>> https://issues.apache.org/****jira/browse/HADOOP-9849<https:
>>>>>>>>
>>>>>>> //issues.apache.org/**jira/browse/HADOOP-9849>
>>>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9849<http
>>>>>>> s://issues.apache.org/jira/browse/HADOOP-9849>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> most (all?) of these are pom changes
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> A good number are basically pom changes to update to newer versions
>>>>> of
>>>>> dependencies. A few, such as commons-math3, required code changes as
>>>>> well
>>>>> because of a namespace change. Some are minor code changes to enhance
>>>>> compatibility with newer dependencies. Even tomcat is mostly changes
>>>>> in
>>>>> pom files.
>>>>>
>>>>>
>>>>> Most of the changes are minor. There are 2 big updates though:
>>>>> Jetty 9
>>>>>
>>>>> (which requires java 7) and tomcat 7. These are also the most
>>>>>> difficult
>>>>>>
>>>>>>> patches to rebase when hadoop produces a new release.
>>>>>>>
>>>>>>>
>>>>>>> that's not going to go in the 2.x branch. Java 6 is still a common
>>>>>>>
>>>>>>> platform
>>>>>> that people are using, because historically java7 (or any leading edge
>>>>>> java
>>>>>> version) is buggy.
>>>>>>
>>>>>> that said, our QA team did test hadoop 2 & HDP-2 at scale on java7 and
>>>>>> openjdk 7, so it all works -it's just the commit "java7 only" is a big
>>>>>> decision that
>>>>>>
>>>>>>
>>>>>> I realize moving to java 7 is a big decision and wasn't trying to
>>>>> imply
>>>>> this should happen without discussion and planning, just that it would
>>>>> be
>>>>> nice to have the discussion and see where things land. It can also
>>>>> help
>>>>> minimize work. There is an open bz for updating jetty to jetty 8 (the
>>>>> last
>>>>> version that would work on java 6), but if there are plans to move to
>>>>> java7, maybe it makes sense to just to jetty 9 and not test a new
>>>>> version
>>>>> of jetty twice.
>>>>>
>>>>> With Hadoop in Fedora running on these newer deps there is a test bed
>>>>> to
>>>>> play with to give some level of confidence before taking the plunge on
>>>>> any
>>>>> major change.
>>>>>
>>>>> Rob
>>>>>
>>>>>
>>>>>
>>>>
>>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Hadoop in Fedora updated to 2.2.0
Posted by Robert Rati <rr...@redhat.com>.
Just to clarify, the tomcat/jasper updates and the jersey updates should
be able to go in without any jetty changes. There is also a separate BZ
for updating jetty to jetty 8, which is the last jetty version that will
run on java 6, if there is a desire to update jetty without requiring
java 7.
If jetty 9 is being looked at for inclusion it will affect the jasper
bits in the poms. Jetty 9 has its own jsp compiler and would need to
replace jasper, but that's largely just pom changes iirc. Jetty 9 does
revamp some apis and should definitely be looked at by people more
knowledgeable with how jetty is used, especially as it relates to secure
mode.
Rob
On 11/13/2013 03:31 PM, Steve Loughran wrote:
> I've just been through some of these as part of my background project, "fix
> up the POMs" https://issues.apache.org/jira/browse/HADOOP-9991.
>
>
> 1. I've applied the simple low/risk ones.
> 2. I've not done the bookkeeper one, as people working with that code
> need to play with it first.
> 3. I've not touched anything related to {jersey, tomcat, jetty}
>
> This is more than just a java6/7 issue, is is that Jetty has been very
> brittle in the past, and there's code in hadoop to detect when it's not
> actually servicing requests properly. Moving up Jetty/web server versions
> is something that needs to be done carefull and with consensus -and once
> you leave Jetty alone, I don't know where the jersey and tomcat changes go.
>
> There is always the option of s/jetty/r/grizzly/
>
> -steve
>
>
>
>
> On 1 November 2013 14:57, Robert Rati <rr...@redhat.com> wrote:
>
>> Putting the java 6 vs java 7 issue aside, what about the other patches to
>> update dependencies? Can those be looked at an planned for inclusion into
>> a releation?
>>
>> Rob
>>
>>
>> On 10/31/2013 05:51 PM, Andrew Wang wrote:
>>
>>> I'm in agreement with Steve on this one. We're aware that Java 6 is EOL,
>>> but we can't drop support for the lifetime of the 2.x line since it's a
>>> (very) incompatible change. AFAIK a 3.x release fixing this isn't on any
>>> of
>>> our horizons yet.
>>>
>>> Best,
>>> Andrew
>>>
>>>
>>> On Thu, Oct 31, 2013 at 6:15 AM, Robert Rati <rr...@redhat.com> wrote:
>>>
>>> https://issues.apache.org/****jira/browse/HADOOP-9594<https:
>>>> //issues.apache.org/**jira/browse/HADOOP-9594>
>>>>
>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9594<http
>>>>>> s://issues.apache.org/jira/browse/HADOOP-9594>
>>>>>>
>>>>>>>
>>>>>>> https://issues.apache.org/****jira/browse/MAPREDUCE-5431<htt
>>>>>> ps://issues.apache.org/**jira/browse/MAPREDUCE-5431>
>>>>>> <htt**ps://issues.apache.org/jira/**browse/MAPREDUCE-5431<h
>>>>>> ttps://issues.apache.org/jira/browse/MAPREDUCE-5431>
>>>>>>
>>>>>>>
>>>>>>> https://issues.apache.org/****jira/browse/HADOOP-9611<https:
>>>>>> //issues.apache.org/**jira/browse/HADOOP-9611>
>>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9611<http
>>>>>> s://issues.apache.org/jira/browse/HADOOP-9611>
>>>>>>
>>>>>>>
>>>>>>> https://issues.apache.org/****jira/browse/HADOOP-9613<https:
>>>>>> //issues.apache.org/**jira/browse/HADOOP-9613>
>>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9613<http
>>>>>> s://issues.apache.org/jira/browse/HADOOP-9613>
>>>>>>
>>>>>>>
>>>>>>> https://issues.apache.org/****jira/browse/HADOOP-9623<https:
>>>>>> //issues.apache.org/**jira/browse/HADOOP-9623>
>>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9623<http
>>>>>> s://issues.apache.org/jira/browse/HADOOP-9623>
>>>>>>
>>>>>>>
>>>>>>> https://issues.apache.org/****jira/browse/HDFS-5411<https://
>>>>>> issues.apache.org/**jira/browse/HDFS-5411>
>>>>>> <https://**issues.apache.org/jira/browse/**HDFS-5411<https:
>>>>>> //issues.apache.org/jira/browse/HDFS-5411>
>>>>>>
>>>>>>>
>>>>>>> https://issues.apache.org/****jira/browse/HADOOP-10067<https
>>>>>> ://issues.apache.org/**jira/browse/HADOOP-10067>
>>>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10067<htt
>>>>>> ps://issues.apache.org/jira/browse/HADOOP-10067>
>>>>>>
>>>>>>>
>>>>>>> https://issues.apache.org/****jira/browse/HDFS-5075<https://
>>>>>> issues.apache.org/**jira/browse/HDFS-5075>
>>>>>> <https://**issues.apache.org/jira/browse/**HDFS-5075<https:
>>>>>> //issues.apache.org/jira/browse/HDFS-5075>
>>>>>>
>>>>>>>
>>>>>>> https://issues.apache.org/****jira/browse/HADOOP-10068<https
>>>>>> ://issues.apache.org/**jira/browse/HADOOP-10068>
>>>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10068<htt
>>>>>> ps://issues.apache.org/jira/browse/HADOOP-10068>
>>>>>>
>>>>>>>
>>>>>>> https://issues.apache.org/****jira/browse/HADOOP-10075<https
>>>>>> ://issues.apache.org/**jira/browse/HADOOP-10075>
>>>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10075<htt
>>>>>> ps://issues.apache.org/jira/browse/HADOOP-10075>
>>>>>>
>>>>>>>
>>>>>>> https://issues.apache.org/****jira/browse/HADOOP-10076<https
>>>>>> ://issues.apache.org/**jira/browse/HADOOP-10076>
>>>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10076<htt
>>>>>> ps://issues.apache.org/jira/browse/HADOOP-10076>
>>>>>>
>>>>>>>
>>>>>>> https://issues.apache.org/****jira/browse/HADOOP-9849<https:
>>>>>> //issues.apache.org/**jira/browse/HADOOP-9849>
>>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9849<http
>>>>>> s://issues.apache.org/jira/browse/HADOOP-9849>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> most (all?) of these are pom changes
>>>>>>
>>>>>
>>>>>
>>>> A good number are basically pom changes to update to newer versions of
>>>> dependencies. A few, such as commons-math3, required code changes as
>>>> well
>>>> because of a namespace change. Some are minor code changes to enhance
>>>> compatibility with newer dependencies. Even tomcat is mostly changes in
>>>> pom files.
>>>>
>>>>
>>>> Most of the changes are minor. There are 2 big updates though: Jetty 9
>>>>
>>>>> (which requires java 7) and tomcat 7. These are also the most difficult
>>>>>> patches to rebase when hadoop produces a new release.
>>>>>>
>>>>>>
>>>>>> that's not going to go in the 2.x branch. Java 6 is still a common
>>>>>>
>>>>> platform
>>>>> that people are using, because historically java7 (or any leading edge
>>>>> java
>>>>> version) is buggy.
>>>>>
>>>>> that said, our QA team did test hadoop 2 & HDP-2 at scale on java7 and
>>>>> openjdk 7, so it all works -it's just the commit "java7 only" is a big
>>>>> decision that
>>>>>
>>>>>
>>>> I realize moving to java 7 is a big decision and wasn't trying to imply
>>>> this should happen without discussion and planning, just that it would be
>>>> nice to have the discussion and see where things land. It can also help
>>>> minimize work. There is an open bz for updating jetty to jetty 8 (the
>>>> last
>>>> version that would work on java 6), but if there are plans to move to
>>>> java7, maybe it makes sense to just to jetty 9 and not test a new version
>>>> of jetty twice.
>>>>
>>>> With Hadoop in Fedora running on these newer deps there is a test bed to
>>>> play with to give some level of confidence before taking the plunge on
>>>> any
>>>> major change.
>>>>
>>>> Rob
>>>>
>>>>
>>>
>
Re: Hadoop in Fedora updated to 2.2.0
Posted by Steve Loughran <st...@hortonworks.com>.
I've just been through some of these as part of my background project, "fix
up the POMs" https://issues.apache.org/jira/browse/HADOOP-9991.
1. I've applied the simple low/risk ones.
2. I've not done the bookkeeper one, as people working with that code
need to play with it first.
3. I've not touched anything related to {jersey, tomcat, jetty}
This is more than just a java6/7 issue, is is that Jetty has been very
brittle in the past, and there's code in hadoop to detect when it's not
actually servicing requests properly. Moving up Jetty/web server versions
is something that needs to be done carefull and with consensus -and once
you leave Jetty alone, I don't know where the jersey and tomcat changes go.
There is always the option of s/jetty/r/grizzly/
-steve
On 1 November 2013 14:57, Robert Rati <rr...@redhat.com> wrote:
> Putting the java 6 vs java 7 issue aside, what about the other patches to
> update dependencies? Can those be looked at an planned for inclusion into
> a releation?
>
> Rob
>
>
> On 10/31/2013 05:51 PM, Andrew Wang wrote:
>
>> I'm in agreement with Steve on this one. We're aware that Java 6 is EOL,
>> but we can't drop support for the lifetime of the 2.x line since it's a
>> (very) incompatible change. AFAIK a 3.x release fixing this isn't on any
>> of
>> our horizons yet.
>>
>> Best,
>> Andrew
>>
>>
>> On Thu, Oct 31, 2013 at 6:15 AM, Robert Rati <rr...@redhat.com> wrote:
>>
>> https://issues.apache.org/****jira/browse/HADOOP-9594<https:
>>> //issues.apache.org/**jira/browse/HADOOP-9594>
>>>
>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9594<http
>>>>> s://issues.apache.org/jira/browse/HADOOP-9594>
>>>>>
>>>>>>
>>>>>> https://issues.apache.org/****jira/browse/MAPREDUCE-5431<htt
>>>>> ps://issues.apache.org/**jira/browse/MAPREDUCE-5431>
>>>>> <htt**ps://issues.apache.org/jira/**browse/MAPREDUCE-5431<h
>>>>> ttps://issues.apache.org/jira/browse/MAPREDUCE-5431>
>>>>>
>>>>>>
>>>>>> https://issues.apache.org/****jira/browse/HADOOP-9611<https:
>>>>> //issues.apache.org/**jira/browse/HADOOP-9611>
>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9611<http
>>>>> s://issues.apache.org/jira/browse/HADOOP-9611>
>>>>>
>>>>>>
>>>>>> https://issues.apache.org/****jira/browse/HADOOP-9613<https:
>>>>> //issues.apache.org/**jira/browse/HADOOP-9613>
>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9613<http
>>>>> s://issues.apache.org/jira/browse/HADOOP-9613>
>>>>>
>>>>>>
>>>>>> https://issues.apache.org/****jira/browse/HADOOP-9623<https:
>>>>> //issues.apache.org/**jira/browse/HADOOP-9623>
>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9623<http
>>>>> s://issues.apache.org/jira/browse/HADOOP-9623>
>>>>>
>>>>>>
>>>>>> https://issues.apache.org/****jira/browse/HDFS-5411<https://
>>>>> issues.apache.org/**jira/browse/HDFS-5411>
>>>>> <https://**issues.apache.org/jira/browse/**HDFS-5411<https:
>>>>> //issues.apache.org/jira/browse/HDFS-5411>
>>>>>
>>>>>>
>>>>>> https://issues.apache.org/****jira/browse/HADOOP-10067<https
>>>>> ://issues.apache.org/**jira/browse/HADOOP-10067>
>>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10067<htt
>>>>> ps://issues.apache.org/jira/browse/HADOOP-10067>
>>>>>
>>>>>>
>>>>>> https://issues.apache.org/****jira/browse/HDFS-5075<https://
>>>>> issues.apache.org/**jira/browse/HDFS-5075>
>>>>> <https://**issues.apache.org/jira/browse/**HDFS-5075<https:
>>>>> //issues.apache.org/jira/browse/HDFS-5075>
>>>>>
>>>>>>
>>>>>> https://issues.apache.org/****jira/browse/HADOOP-10068<https
>>>>> ://issues.apache.org/**jira/browse/HADOOP-10068>
>>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10068<htt
>>>>> ps://issues.apache.org/jira/browse/HADOOP-10068>
>>>>>
>>>>>>
>>>>>> https://issues.apache.org/****jira/browse/HADOOP-10075<https
>>>>> ://issues.apache.org/**jira/browse/HADOOP-10075>
>>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10075<htt
>>>>> ps://issues.apache.org/jira/browse/HADOOP-10075>
>>>>>
>>>>>>
>>>>>> https://issues.apache.org/****jira/browse/HADOOP-10076<https
>>>>> ://issues.apache.org/**jira/browse/HADOOP-10076>
>>>>> <https**://issues.apache.org/jira/**browse/HADOOP-10076<htt
>>>>> ps://issues.apache.org/jira/browse/HADOOP-10076>
>>>>>
>>>>>>
>>>>>> https://issues.apache.org/****jira/browse/HADOOP-9849<https:
>>>>> //issues.apache.org/**jira/browse/HADOOP-9849>
>>>>> <https:**//issues.apache.org/jira/**browse/HADOOP-9849<http
>>>>> s://issues.apache.org/jira/browse/HADOOP-9849>
>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> most (all?) of these are pom changes
>>>>>
>>>>
>>>>
>>> A good number are basically pom changes to update to newer versions of
>>> dependencies. A few, such as commons-math3, required code changes as
>>> well
>>> because of a namespace change. Some are minor code changes to enhance
>>> compatibility with newer dependencies. Even tomcat is mostly changes in
>>> pom files.
>>>
>>>
>>> Most of the changes are minor. There are 2 big updates though: Jetty 9
>>>
>>>> (which requires java 7) and tomcat 7. These are also the most difficult
>>>>> patches to rebase when hadoop produces a new release.
>>>>>
>>>>>
>>>>> that's not going to go in the 2.x branch. Java 6 is still a common
>>>>>
>>>> platform
>>>> that people are using, because historically java7 (or any leading edge
>>>> java
>>>> version) is buggy.
>>>>
>>>> that said, our QA team did test hadoop 2 & HDP-2 at scale on java7 and
>>>> openjdk 7, so it all works -it's just the commit "java7 only" is a big
>>>> decision that
>>>>
>>>>
>>> I realize moving to java 7 is a big decision and wasn't trying to imply
>>> this should happen without discussion and planning, just that it would be
>>> nice to have the discussion and see where things land. It can also help
>>> minimize work. There is an open bz for updating jetty to jetty 8 (the
>>> last
>>> version that would work on java 6), but if there are plans to move to
>>> java7, maybe it makes sense to just to jetty 9 and not test a new version
>>> of jetty twice.
>>>
>>> With Hadoop in Fedora running on these newer deps there is a test bed to
>>> play with to give some level of confidence before taking the plunge on
>>> any
>>> major change.
>>>
>>> Rob
>>>
>>>
>>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.