You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Steve Loughran <st...@hortonworks.com> on 2015/10/25 14:34:16 UTC
Java 8 + Jersey updates
https://issues.apache.org/jira/browse/HADOOP-9613 covers the issue of updating Jersey to 3.0 to cope with the move to Java 8
1. this is trunk
2. it's "significant"
we've been frozen on an old version of Jersey with issues, known ones needing separate threads to detect jersey startup failures, and putting off the Update-Jersey issue for 2- years.
For Java 8 there's no choice but to move on.
thoughts and comments on the JIRA please. I'll try to rebuild a YARN app (slider) with the patch to see how it fares
Re: Java 8 + Jersey updates
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
On Fri, Oct 30, 2015 at 11:16 PM, Steve Loughran <st...@hortonworks.com> wrote:
>
>> On 29 Oct 2015, at 15:40, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>>
>> Steve,
>>
>>> If you exclude jax-rs 2 and try to stay @ jersey 1.9 for your client, all the http clients: KMS, webhdfs, ATS, aren't going to link.
>>
>> I thought we can use Jersey 1.19 on JDK 8 with client-side
>> compatibility, but do you mean that we cannot use Jersey 1.19 on JDK
>> 8? Please correct me if I've misunderstood.
>>
>
> jersey 1.19 works on JDK8. What I'm worried about is having two versions of jersey on the classpath -as the patch you've put up rewrites all the client-side code to move to jersey 2
>
A latest patch I updated(HADOOP-9613.007.incompatible.patch) uses
Jersey 1.19. All rewrites at client-side code I updated is to avoid
using deprecated method, ClientResponse.getClientResponseStatus(). In
fact, we can still use the method though it's marked as deprecated
since 1.18.
https://jersey.java.net/nonav/apidocs/1.19/jersey/com/sun/jersey/api/client/ClientResponse.html
https://jersey.java.net/nonav/apidocs/1.19/jersey/com/sun/jersey/api/client/ClientResponse.html#getClientResponseStatus()
Maybe the latest patch confuses you with non-essential change. I'm
sorry for that. Please correct me if I'm wrong.
Best,
- Tsuyoshi
Re: Java 8 + Jersey updates
Posted by Steve Loughran <st...@hortonworks.com>.
> On 29 Oct 2015, at 15:40, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>
> Steve,
>
>> If you exclude jax-rs 2 and try to stay @ jersey 1.9 for your client, all the http clients: KMS, webhdfs, ATS, aren't going to link.
>
> I thought we can use Jersey 1.19 on JDK 8 with client-side
> compatibility, but do you mean that we cannot use Jersey 1.19 on JDK
> 8? Please correct me if I've misunderstood.
>
> Best,
> - Tsuyoshi
>
jersey 1.19 works on JDK8. What I'm worried about is having two versions of jersey on the classpath -as the patch you've put up rewrites all the client-side code to move to jersey 2
Re: Java 8 + Jersey updates
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
Steve,
> If you exclude jax-rs 2 and try to stay @ jersey 1.9 for your client, all the http clients: KMS, webhdfs, ATS, aren't going to link.
I thought we can use Jersey 1.19 on JDK 8 with client-side
compatibility, but do you mean that we cannot use Jersey 1.19 on JDK
8? Please correct me if I've misunderstood.
Best,
- Tsuyoshi
On Thu, Oct 29, 2015 at 7:52 PM, Steve Loughran <st...@hortonworks.com> wrote:
> as an update, this looks like a "protbuf-class" incompatibility
>
> the problem here is that this updates to JAX-RS 2.0, which is incompatible at the client API
>
> https://jersey.java.net/nonav/documentation/2.0/migration.html
>
> which means everything downstream gets to rewrite their Jersey client code.
>
> This is going to add yet another barrier to adoption of Hadoop 3.x; you will not be able to seamlessly update your client apps, and you will never be able to have code which compiles against 2.x and 3.x
>
> If you exclude jax-rs 2 and try to stay @ jersey 1.9 for your client, all the http clients: KMS, webhdfs, ATS, aren't going to link.
>
> I think to pull this off we'll need to somehow wrap that client-side use of jersey with enough introspection that at least the Hadoop REST clients can use whichever jersey lib is on the classpath
>
> Any volunteers.
>
>
>> On 27 Oct 2015, at 02:38, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>>
>>> I assume you are targetting this only at trunk / 3.0 based on the "target version" and the incompatibility discussion?
>>
>> Yes, you're right.
>>
>> Best regards,
>> - Tsuyoshi
>>
>> On Tue, Oct 27, 2015 at 5:12 AM, Colin P. McCabe <cm...@apache.org> wrote:
>>> Looks like a good idea. I assume you are targetting this only at trunk /
>>> 3.0 based on the "target version" and the incompatibility discussion?
>>>
>>> best,
>>> Colin
>>>
>>> On Mon, Oct 26, 2015 at 7:07 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>>>
>>>> Hi Steve,
>>>>
>>>> Thanks for your help.
>>>>
>>>>> 2. it's "significant"
>>>>
>>>> This change includes upgrading not only Jersey, but also its
>>>> dependencies like grizzly, asm, and so on.
>>>>
>>>>> I'll try to rebuild a YARN app (slider) with the patch to see how it
>>>> fares
>>>>
>>>> It helps us a lot. I'd like to suggest that the incompatibility
>>>> clarified here be described on release note of jira(HADOOP-9613).
>>>>
>>>> - Tsuyoshi
>>>>
>>>> On Sun, Oct 25, 2015 at 10:34 PM, Steve Loughran <st...@hortonworks.com>
>>>> wrote:
>>>>>
>>>>>
>>>>> https://issues.apache.org/jira/browse/HADOOP-9613 covers the issue of
>>>> updating Jersey to 3.0 to cope with the move to Java 8
>>>>>
>>>>> 1. this is trunk
>>>>> 2. it's "significant"
>>>>>
>>>>> we've been frozen on an old version of Jersey with issues, known ones
>>>> needing separate threads to detect jersey startup failures, and putting off
>>>> the Update-Jersey issue for 2- years.
>>>>>
>>>>> For Java 8 there's no choice but to move on.
>>>>>
>>>>> thoughts and comments on the JIRA please. I'll try to rebuild a YARN app
>>>> (slider) with the patch to see how it fares
>>>>>
>>>>>
>>>>
>>
>
Re: Java 8 + Jersey updates
Posted by Steve Loughran <st...@hortonworks.com>.
as an update, this looks like a "protbuf-class" incompatibility
the problem here is that this updates to JAX-RS 2.0, which is incompatible at the client API
https://jersey.java.net/nonav/documentation/2.0/migration.html
which means everything downstream gets to rewrite their Jersey client code.
This is going to add yet another barrier to adoption of Hadoop 3.x; you will not be able to seamlessly update your client apps, and you will never be able to have code which compiles against 2.x and 3.x
If you exclude jax-rs 2 and try to stay @ jersey 1.9 for your client, all the http clients: KMS, webhdfs, ATS, aren't going to link.
I think to pull this off we'll need to somehow wrap that client-side use of jersey with enough introspection that at least the Hadoop REST clients can use whichever jersey lib is on the classpath
Any volunteers.
> On 27 Oct 2015, at 02:38, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>
>> I assume you are targetting this only at trunk / 3.0 based on the "target version" and the incompatibility discussion?
>
> Yes, you're right.
>
> Best regards,
> - Tsuyoshi
>
> On Tue, Oct 27, 2015 at 5:12 AM, Colin P. McCabe <cm...@apache.org> wrote:
>> Looks like a good idea. I assume you are targetting this only at trunk /
>> 3.0 based on the "target version" and the incompatibility discussion?
>>
>> best,
>> Colin
>>
>> On Mon, Oct 26, 2015 at 7:07 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>>
>>> Hi Steve,
>>>
>>> Thanks for your help.
>>>
>>>> 2. it's "significant"
>>>
>>> This change includes upgrading not only Jersey, but also its
>>> dependencies like grizzly, asm, and so on.
>>>
>>>> I'll try to rebuild a YARN app (slider) with the patch to see how it
>>> fares
>>>
>>> It helps us a lot. I'd like to suggest that the incompatibility
>>> clarified here be described on release note of jira(HADOOP-9613).
>>>
>>> - Tsuyoshi
>>>
>>> On Sun, Oct 25, 2015 at 10:34 PM, Steve Loughran <st...@hortonworks.com>
>>> wrote:
>>>>
>>>>
>>>> https://issues.apache.org/jira/browse/HADOOP-9613 covers the issue of
>>> updating Jersey to 3.0 to cope with the move to Java 8
>>>>
>>>> 1. this is trunk
>>>> 2. it's "significant"
>>>>
>>>> we've been frozen on an old version of Jersey with issues, known ones
>>> needing separate threads to detect jersey startup failures, and putting off
>>> the Update-Jersey issue for 2- years.
>>>>
>>>> For Java 8 there's no choice but to move on.
>>>>
>>>> thoughts and comments on the JIRA please. I'll try to rebuild a YARN app
>>> (slider) with the patch to see how it fares
>>>>
>>>>
>>>
>
Re: Java 8 + Jersey updates
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
> I assume you are targetting this only at trunk / 3.0 based on the "target version" and the incompatibility discussion?
Yes, you're right.
Best regards,
- Tsuyoshi
On Tue, Oct 27, 2015 at 5:12 AM, Colin P. McCabe <cm...@apache.org> wrote:
> Looks like a good idea. I assume you are targetting this only at trunk /
> 3.0 based on the "target version" and the incompatibility discussion?
>
> best,
> Colin
>
> On Mon, Oct 26, 2015 at 7:07 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>
>> Hi Steve,
>>
>> Thanks for your help.
>>
>> > 2. it's "significant"
>>
>> This change includes upgrading not only Jersey, but also its
>> dependencies like grizzly, asm, and so on.
>>
>> > I'll try to rebuild a YARN app (slider) with the patch to see how it
>> fares
>>
>> It helps us a lot. I'd like to suggest that the incompatibility
>> clarified here be described on release note of jira(HADOOP-9613).
>>
>> - Tsuyoshi
>>
>> On Sun, Oct 25, 2015 at 10:34 PM, Steve Loughran <st...@hortonworks.com>
>> wrote:
>> >
>> >
>> > https://issues.apache.org/jira/browse/HADOOP-9613 covers the issue of
>> updating Jersey to 3.0 to cope with the move to Java 8
>> >
>> > 1. this is trunk
>> > 2. it's "significant"
>> >
>> > we've been frozen on an old version of Jersey with issues, known ones
>> needing separate threads to detect jersey startup failures, and putting off
>> the Update-Jersey issue for 2- years.
>> >
>> > For Java 8 there's no choice but to move on.
>> >
>> > thoughts and comments on the JIRA please. I'll try to rebuild a YARN app
>> (slider) with the patch to see how it fares
>> >
>> >
>>
Re: Java 8 + Jersey updates
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
> I assume you are targetting this only at trunk / 3.0 based on the "target version" and the incompatibility discussion?
Yes, you're right.
Best regards,
- Tsuyoshi
On Tue, Oct 27, 2015 at 5:12 AM, Colin P. McCabe <cm...@apache.org> wrote:
> Looks like a good idea. I assume you are targetting this only at trunk /
> 3.0 based on the "target version" and the incompatibility discussion?
>
> best,
> Colin
>
> On Mon, Oct 26, 2015 at 7:07 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>
>> Hi Steve,
>>
>> Thanks for your help.
>>
>> > 2. it's "significant"
>>
>> This change includes upgrading not only Jersey, but also its
>> dependencies like grizzly, asm, and so on.
>>
>> > I'll try to rebuild a YARN app (slider) with the patch to see how it
>> fares
>>
>> It helps us a lot. I'd like to suggest that the incompatibility
>> clarified here be described on release note of jira(HADOOP-9613).
>>
>> - Tsuyoshi
>>
>> On Sun, Oct 25, 2015 at 10:34 PM, Steve Loughran <st...@hortonworks.com>
>> wrote:
>> >
>> >
>> > https://issues.apache.org/jira/browse/HADOOP-9613 covers the issue of
>> updating Jersey to 3.0 to cope with the move to Java 8
>> >
>> > 1. this is trunk
>> > 2. it's "significant"
>> >
>> > we've been frozen on an old version of Jersey with issues, known ones
>> needing separate threads to detect jersey startup failures, and putting off
>> the Update-Jersey issue for 2- years.
>> >
>> > For Java 8 there's no choice but to move on.
>> >
>> > thoughts and comments on the JIRA please. I'll try to rebuild a YARN app
>> (slider) with the patch to see how it fares
>> >
>> >
>>
Re: Java 8 + Jersey updates
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
> I assume you are targetting this only at trunk / 3.0 based on the "target version" and the incompatibility discussion?
Yes, you're right.
Best regards,
- Tsuyoshi
On Tue, Oct 27, 2015 at 5:12 AM, Colin P. McCabe <cm...@apache.org> wrote:
> Looks like a good idea. I assume you are targetting this only at trunk /
> 3.0 based on the "target version" and the incompatibility discussion?
>
> best,
> Colin
>
> On Mon, Oct 26, 2015 at 7:07 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>
>> Hi Steve,
>>
>> Thanks for your help.
>>
>> > 2. it's "significant"
>>
>> This change includes upgrading not only Jersey, but also its
>> dependencies like grizzly, asm, and so on.
>>
>> > I'll try to rebuild a YARN app (slider) with the patch to see how it
>> fares
>>
>> It helps us a lot. I'd like to suggest that the incompatibility
>> clarified here be described on release note of jira(HADOOP-9613).
>>
>> - Tsuyoshi
>>
>> On Sun, Oct 25, 2015 at 10:34 PM, Steve Loughran <st...@hortonworks.com>
>> wrote:
>> >
>> >
>> > https://issues.apache.org/jira/browse/HADOOP-9613 covers the issue of
>> updating Jersey to 3.0 to cope with the move to Java 8
>> >
>> > 1. this is trunk
>> > 2. it's "significant"
>> >
>> > we've been frozen on an old version of Jersey with issues, known ones
>> needing separate threads to detect jersey startup failures, and putting off
>> the Update-Jersey issue for 2- years.
>> >
>> > For Java 8 there's no choice but to move on.
>> >
>> > thoughts and comments on the JIRA please. I'll try to rebuild a YARN app
>> (slider) with the patch to see how it fares
>> >
>> >
>>
Re: Java 8 + Jersey updates
Posted by "Colin P. McCabe" <cm...@apache.org>.
Looks like a good idea. I assume you are targetting this only at trunk /
3.0 based on the "target version" and the incompatibility discussion?
best,
Colin
On Mon, Oct 26, 2015 at 7:07 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
> Hi Steve,
>
> Thanks for your help.
>
> > 2. it's "significant"
>
> This change includes upgrading not only Jersey, but also its
> dependencies like grizzly, asm, and so on.
>
> > I'll try to rebuild a YARN app (slider) with the patch to see how it
> fares
>
> It helps us a lot. I'd like to suggest that the incompatibility
> clarified here be described on release note of jira(HADOOP-9613).
>
> - Tsuyoshi
>
> On Sun, Oct 25, 2015 at 10:34 PM, Steve Loughran <st...@hortonworks.com>
> wrote:
> >
> >
> > https://issues.apache.org/jira/browse/HADOOP-9613 covers the issue of
> updating Jersey to 3.0 to cope with the move to Java 8
> >
> > 1. this is trunk
> > 2. it's "significant"
> >
> > we've been frozen on an old version of Jersey with issues, known ones
> needing separate threads to detect jersey startup failures, and putting off
> the Update-Jersey issue for 2- years.
> >
> > For Java 8 there's no choice but to move on.
> >
> > thoughts and comments on the JIRA please. I'll try to rebuild a YARN app
> (slider) with the patch to see how it fares
> >
> >
>
Re: Java 8 + Jersey updates
Posted by "Colin P. McCabe" <cm...@apache.org>.
Looks like a good idea. I assume you are targetting this only at trunk /
3.0 based on the "target version" and the incompatibility discussion?
best,
Colin
On Mon, Oct 26, 2015 at 7:07 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
> Hi Steve,
>
> Thanks for your help.
>
> > 2. it's "significant"
>
> This change includes upgrading not only Jersey, but also its
> dependencies like grizzly, asm, and so on.
>
> > I'll try to rebuild a YARN app (slider) with the patch to see how it
> fares
>
> It helps us a lot. I'd like to suggest that the incompatibility
> clarified here be described on release note of jira(HADOOP-9613).
>
> - Tsuyoshi
>
> On Sun, Oct 25, 2015 at 10:34 PM, Steve Loughran <st...@hortonworks.com>
> wrote:
> >
> >
> > https://issues.apache.org/jira/browse/HADOOP-9613 covers the issue of
> updating Jersey to 3.0 to cope with the move to Java 8
> >
> > 1. this is trunk
> > 2. it's "significant"
> >
> > we've been frozen on an old version of Jersey with issues, known ones
> needing separate threads to detect jersey startup failures, and putting off
> the Update-Jersey issue for 2- years.
> >
> > For Java 8 there's no choice but to move on.
> >
> > thoughts and comments on the JIRA please. I'll try to rebuild a YARN app
> (slider) with the patch to see how it fares
> >
> >
>
Re: Java 8 + Jersey updates
Posted by "Colin P. McCabe" <cm...@apache.org>.
Looks like a good idea. I assume you are targetting this only at trunk /
3.0 based on the "target version" and the incompatibility discussion?
best,
Colin
On Mon, Oct 26, 2015 at 7:07 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
> Hi Steve,
>
> Thanks for your help.
>
> > 2. it's "significant"
>
> This change includes upgrading not only Jersey, but also its
> dependencies like grizzly, asm, and so on.
>
> > I'll try to rebuild a YARN app (slider) with the patch to see how it
> fares
>
> It helps us a lot. I'd like to suggest that the incompatibility
> clarified here be described on release note of jira(HADOOP-9613).
>
> - Tsuyoshi
>
> On Sun, Oct 25, 2015 at 10:34 PM, Steve Loughran <st...@hortonworks.com>
> wrote:
> >
> >
> > https://issues.apache.org/jira/browse/HADOOP-9613 covers the issue of
> updating Jersey to 3.0 to cope with the move to Java 8
> >
> > 1. this is trunk
> > 2. it's "significant"
> >
> > we've been frozen on an old version of Jersey with issues, known ones
> needing separate threads to detect jersey startup failures, and putting off
> the Update-Jersey issue for 2- years.
> >
> > For Java 8 there's no choice but to move on.
> >
> > thoughts and comments on the JIRA please. I'll try to rebuild a YARN app
> (slider) with the patch to see how it fares
> >
> >
>
Re: Java 8 + Jersey updates
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
Hi Steve,
Thanks for your help.
> 2. it's "significant"
This change includes upgrading not only Jersey, but also its
dependencies like grizzly, asm, and so on.
> I'll try to rebuild a YARN app (slider) with the patch to see how it fares
It helps us a lot. I'd like to suggest that the incompatibility
clarified here be described on release note of jira(HADOOP-9613).
- Tsuyoshi
On Sun, Oct 25, 2015 at 10:34 PM, Steve Loughran <st...@hortonworks.com> wrote:
>
>
> https://issues.apache.org/jira/browse/HADOOP-9613 covers the issue of updating Jersey to 3.0 to cope with the move to Java 8
>
> 1. this is trunk
> 2. it's "significant"
>
> we've been frozen on an old version of Jersey with issues, known ones needing separate threads to detect jersey startup failures, and putting off the Update-Jersey issue for 2- years.
>
> For Java 8 there's no choice but to move on.
>
> thoughts and comments on the JIRA please. I'll try to rebuild a YARN app (slider) with the patch to see how it fares
>
>
Re: Java 8 + Jersey updates
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
Hi Steve,
Thanks for your help.
> 2. it's "significant"
This change includes upgrading not only Jersey, but also its
dependencies like grizzly, asm, and so on.
> I'll try to rebuild a YARN app (slider) with the patch to see how it fares
It helps us a lot. I'd like to suggest that the incompatibility
clarified here be described on release note of jira(HADOOP-9613).
- Tsuyoshi
On Sun, Oct 25, 2015 at 10:34 PM, Steve Loughran <st...@hortonworks.com> wrote:
>
>
> https://issues.apache.org/jira/browse/HADOOP-9613 covers the issue of updating Jersey to 3.0 to cope with the move to Java 8
>
> 1. this is trunk
> 2. it's "significant"
>
> we've been frozen on an old version of Jersey with issues, known ones needing separate threads to detect jersey startup failures, and putting off the Update-Jersey issue for 2- years.
>
> For Java 8 there's no choice but to move on.
>
> thoughts and comments on the JIRA please. I'll try to rebuild a YARN app (slider) with the patch to see how it fares
>
>
Re: Java 8 + Jersey updates
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
Hi Steve,
Thanks for your help.
> 2. it's "significant"
This change includes upgrading not only Jersey, but also its
dependencies like grizzly, asm, and so on.
> I'll try to rebuild a YARN app (slider) with the patch to see how it fares
It helps us a lot. I'd like to suggest that the incompatibility
clarified here be described on release note of jira(HADOOP-9613).
- Tsuyoshi
On Sun, Oct 25, 2015 at 10:34 PM, Steve Loughran <st...@hortonworks.com> wrote:
>
>
> https://issues.apache.org/jira/browse/HADOOP-9613 covers the issue of updating Jersey to 3.0 to cope with the move to Java 8
>
> 1. this is trunk
> 2. it's "significant"
>
> we've been frozen on an old version of Jersey with issues, known ones needing separate threads to detect jersey startup failures, and putting off the Update-Jersey issue for 2- years.
>
> For Java 8 there's no choice but to move on.
>
> thoughts and comments on the JIRA please. I'll try to rebuild a YARN app (slider) with the patch to see how it fares
>
>