You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Robert Metzger <rm...@apache.org> on 2017/03/13 13:33:16 UTC
Re: Frontend classpath issue
I think the change reduces the chances to run into classloading issues in
case there's a bug in Flink (= it is using the wrong CL)
I've filed a JIRA for the problem:
https://issues.apache.org/jira/browse/FLINK-6031
On Fri, Feb 24, 2017 at 9:29 PM, Gyula Fóra <gy...@gmail.com> wrote:
> Hi,
> I am wondering whether there is any scenario where the new way makes
> anything better under normal circumstances.
>
> I can only see how it will break things in subtle ways.
>
> If you think there is any real benefit to the current approach I dont mind
> having it as a default, otherwise I am in favor of reverting to the 1.1
> default. (My logic is that the user will only observe a difference in
> behavior when the new setup actually causes problems)
>
> Gyula
>
> On Fri, Feb 24, 2017, 17:53 Robert Metzger <rm...@apache.org> wrote:
>
>> The JIRA (https://issues.apache.org/jira/browse/FLINK-4913) doesn't
>> mention any particular user or use case.
>>
>> I honestly care so much if we enable or disable it by default. But since
>> its the new default behavior of Flink 1.2. I'm against changing that in
>> Flink 1.2.1, that's why I proposed to add a flag to disable it in 1.2.1, so
>> that users upgrading from 1.2.0 to 1.2.1 don't notice it.
>>
>> On Fri, Feb 24, 2017 at 5:41 PM, Aljoscha Krettek <al...@apache.org>
>> wrote:
>>
>> Did any user have problems with the Flink 1.1 behaviour? If not, we could
>> disable it again, by default, and add a flag for adding the user jar to all
>> the classpaths.
>>
>> On Fri, 24 Feb 2017 at 14:50 Robert Metzger <rm...@apache.org> wrote:
>>
>> I agree with you Gyula, this change is dangerous. I have seen another case
>> from a user with Hadoop dependencies that crashed in Flink 1.2.0 that
>> didn't in 1.1.x
>>
>> I wonder if we should introduce a config flag for Flink 1.2.1 to disable
>> the behavior if needed.
>>
>> On Fri, Feb 24, 2017 at 2:27 PM, Ufuk Celebi <uc...@apache.org> wrote:
>>
>> > On Fri, Feb 24, 2017 at 11:05 AM, Gyula Fóra <gy...@gmail.com>
>> wrote:
>> > > I was not aware of this big change (I know it's my fault) but I am not
>> > sure
>> > > if I agree with the rationale.
>> >
>> > No comment on the actual issue from my side, but I strongly disagree
>> > that this is your fault. We should have covered this better in the
>> > release announcement in my opinion. Of course, this doesn't help now.
>> > ;-)
>> >
>> > – Ufuk
>> >
>>
>>
>>
Re: Frontend classpath issue
Posted by Stephan Ewen <se...@apache.org>.
I think we need to get away from the dynamic class loading as much as
possible. It breaks way to soon and causes easily class leaks.
I would be in favor if understanding how to fix this on the Flink side,
i.e., either:
- Having flags for disabling it optionally
- Having an option of "user code first" or "user code last" in the
classpath
- Shading Netty in Flink. I think Netty is a good candidate to be shaded,
actually.
On Mon, Mar 13, 2017 at 2:33 PM, Robert Metzger <rm...@apache.org> wrote:
> I think the change reduces the chances to run into classloading issues in
> case there's a bug in Flink (= it is using the wrong CL)
>
> I've filed a JIRA for the problem:
> https://issues.apache.org/jira/browse/FLINK-6031
>
> On Fri, Feb 24, 2017 at 9:29 PM, Gyula Fóra <gy...@gmail.com> wrote:
>
> > Hi,
> > I am wondering whether there is any scenario where the new way makes
> > anything better under normal circumstances.
> >
> > I can only see how it will break things in subtle ways.
> >
> > If you think there is any real benefit to the current approach I dont
> mind
> > having it as a default, otherwise I am in favor of reverting to the 1.1
> > default. (My logic is that the user will only observe a difference in
> > behavior when the new setup actually causes problems)
> >
> > Gyula
> >
> > On Fri, Feb 24, 2017, 17:53 Robert Metzger <rm...@apache.org> wrote:
> >
> >> The JIRA (https://issues.apache.org/jira/browse/FLINK-4913) doesn't
> >> mention any particular user or use case.
> >>
> >> I honestly care so much if we enable or disable it by default. But since
> >> its the new default behavior of Flink 1.2. I'm against changing that in
> >> Flink 1.2.1, that's why I proposed to add a flag to disable it in
> 1.2.1, so
> >> that users upgrading from 1.2.0 to 1.2.1 don't notice it.
> >>
> >> On Fri, Feb 24, 2017 at 5:41 PM, Aljoscha Krettek <al...@apache.org>
> >> wrote:
> >>
> >> Did any user have problems with the Flink 1.1 behaviour? If not, we
> could
> >> disable it again, by default, and add a flag for adding the user jar to
> all
> >> the classpaths.
> >>
> >> On Fri, 24 Feb 2017 at 14:50 Robert Metzger <rm...@apache.org>
> wrote:
> >>
> >> I agree with you Gyula, this change is dangerous. I have seen another
> case
> >> from a user with Hadoop dependencies that crashed in Flink 1.2.0 that
> >> didn't in 1.1.x
> >>
> >> I wonder if we should introduce a config flag for Flink 1.2.1 to disable
> >> the behavior if needed.
> >>
> >> On Fri, Feb 24, 2017 at 2:27 PM, Ufuk Celebi <uc...@apache.org> wrote:
> >>
> >> > On Fri, Feb 24, 2017 at 11:05 AM, Gyula Fóra <gy...@gmail.com>
> >> wrote:
> >> > > I was not aware of this big change (I know it's my fault) but I am
> not
> >> > sure
> >> > > if I agree with the rationale.
> >> >
> >> > No comment on the actual issue from my side, but I strongly disagree
> >> > that this is your fault. We should have covered this better in the
> >> > release announcement in my opinion. Of course, this doesn't help now.
> >> > ;-)
> >> >
> >> > – Ufuk
> >> >
> >>
> >>
> >>
>
Re: Frontend classpath issue
Posted by Stephan Ewen <se...@apache.org>.
I think we need to get away from the dynamic class loading as much as
possible. It breaks way to soon and causes easily class leaks.
I would be in favor if understanding how to fix this on the Flink side,
i.e., either:
- Having flags for disabling it optionally
- Having an option of "user code first" or "user code last" in the
classpath
- Shading Netty in Flink. I think Netty is a good candidate to be shaded,
actually.
On Mon, Mar 13, 2017 at 2:33 PM, Robert Metzger <rm...@apache.org> wrote:
> I think the change reduces the chances to run into classloading issues in
> case there's a bug in Flink (= it is using the wrong CL)
>
> I've filed a JIRA for the problem:
> https://issues.apache.org/jira/browse/FLINK-6031
>
> On Fri, Feb 24, 2017 at 9:29 PM, Gyula Fóra <gy...@gmail.com> wrote:
>
> > Hi,
> > I am wondering whether there is any scenario where the new way makes
> > anything better under normal circumstances.
> >
> > I can only see how it will break things in subtle ways.
> >
> > If you think there is any real benefit to the current approach I dont
> mind
> > having it as a default, otherwise I am in favor of reverting to the 1.1
> > default. (My logic is that the user will only observe a difference in
> > behavior when the new setup actually causes problems)
> >
> > Gyula
> >
> > On Fri, Feb 24, 2017, 17:53 Robert Metzger <rm...@apache.org> wrote:
> >
> >> The JIRA (https://issues.apache.org/jira/browse/FLINK-4913) doesn't
> >> mention any particular user or use case.
> >>
> >> I honestly care so much if we enable or disable it by default. But since
> >> its the new default behavior of Flink 1.2. I'm against changing that in
> >> Flink 1.2.1, that's why I proposed to add a flag to disable it in
> 1.2.1, so
> >> that users upgrading from 1.2.0 to 1.2.1 don't notice it.
> >>
> >> On Fri, Feb 24, 2017 at 5:41 PM, Aljoscha Krettek <al...@apache.org>
> >> wrote:
> >>
> >> Did any user have problems with the Flink 1.1 behaviour? If not, we
> could
> >> disable it again, by default, and add a flag for adding the user jar to
> all
> >> the classpaths.
> >>
> >> On Fri, 24 Feb 2017 at 14:50 Robert Metzger <rm...@apache.org>
> wrote:
> >>
> >> I agree with you Gyula, this change is dangerous. I have seen another
> case
> >> from a user with Hadoop dependencies that crashed in Flink 1.2.0 that
> >> didn't in 1.1.x
> >>
> >> I wonder if we should introduce a config flag for Flink 1.2.1 to disable
> >> the behavior if needed.
> >>
> >> On Fri, Feb 24, 2017 at 2:27 PM, Ufuk Celebi <uc...@apache.org> wrote:
> >>
> >> > On Fri, Feb 24, 2017 at 11:05 AM, Gyula Fóra <gy...@gmail.com>
> >> wrote:
> >> > > I was not aware of this big change (I know it's my fault) but I am
> not
> >> > sure
> >> > > if I agree with the rationale.
> >> >
> >> > No comment on the actual issue from my side, but I strongly disagree
> >> > that this is your fault. We should have covered this better in the
> >> > release announcement in my opinion. Of course, this doesn't help now.
> >> > ;-)
> >> >
> >> > – Ufuk
> >> >
> >>
> >>
> >>
>