You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Reuven Lax <re...@google.com> on 2019/04/26 18:10:14 UTC
:beam-sdks-java-io-hadoop-input-format:test is extremely flaky
I find I usually have to rerun Presubmit multiple times to get a green run,
and this test is one of the biggest culprits (though it's not the only
culprit). Does anyone know what's going on with it?
Reuven
Re: :beam-sdks-java-io-hadoop-input-format:test is extremely flaky
Posted by Alexey Romanenko <ar...@gmail.com>.
FYI: In the end, the module "hadoop-input-format” was removed in favour of using “hadoop-format” instead.
> On 29 Apr 2019, at 15:50, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>
> Agree, +1
>
> Regards
> JB
>
> On 29/04/2019 15:30, Ismaël Mejía wrote:
>> +1 to remove it on this release, this is a maintenance pain for no real reason.
>>
>> On Mon, Apr 29, 2019 at 3:06 PM Alexey Romanenko
>> <ar...@gmail.com> wrote:
>>>
>>> Despite the fact that after fixing an issue with ports allocation (thanks to Etienne!) for embedded Cassandra cluster (it’s used in hadoop-input-format and this was the main cause of flakiness) it's got much better, I’m 100% pro to remove this module since it’s already been deprecated for several last releases.
>>>
>>> PS: Just an observation when I was digging into PreCommit jobs results - “org.apache.beam.runners.dataflow.worker.fn.BeamFnControlServiceTest.testClientConnecting” fails quite often in the last time. Anyone works on this?
>>>
>>>> On 29 Apr 2019, at 14:19, Maximilian Michels <mx...@apache.org> wrote:
>>>>
>>>> I don't know what going on with it but I agree it's annoying.
>>>>
>>>> Came across https://jira.apache.org/jira/browse/BEAM-6247, maybe it is time to remove this module for the next release?
>>>>
>>>> -Max
>>>>
>>>> On 26.04.19 20:10, Reuven Lax wrote:
>>>>> I find I usually have to rerun Presubmit multiple times to get a green run, and this test is one of the biggest culprits (though it's not the only culprit). Does anyone know what's going on with it?
>>>>> Reuven
>>>
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
Re: :beam-sdks-java-io-hadoop-input-format:test is extremely flaky
Posted by Valentyn Tymofieiev <va...@google.com>.
BeamFnControlServiceTest is being worked on in
https://issues.apache.org/jira/browse/BEAM-5709.
On Mon, Apr 29, 2019 at 2:01 PM Reuven Lax <re...@google.com> wrote:
> yeah, that testClientConnecting test is also extremely flaky.
>
> On Mon, Apr 29, 2019 at 6:50 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Agree, +1
>>
>> Regards
>> JB
>>
>> On 29/04/2019 15:30, Ismaël Mejía wrote:
>> > +1 to remove it on this release, this is a maintenance pain for no real
>> reason.
>> >
>> > On Mon, Apr 29, 2019 at 3:06 PM Alexey Romanenko
>> > <ar...@gmail.com> wrote:
>> >>
>> >> Despite the fact that after fixing an issue with ports allocation
>> (thanks to Etienne!) for embedded Cassandra cluster (it’s used in
>> hadoop-input-format and this was the main cause of flakiness) it's got much
>> better, I’m 100% pro to remove this module since it’s already been
>> deprecated for several last releases.
>> >>
>> >> PS: Just an observation when I was digging into PreCommit jobs results
>> -
>> “org.apache.beam.runners.dataflow.worker.fn.BeamFnControlServiceTest.testClientConnecting”
>> fails quite often in the last time. Anyone works on this?
>> >>
>> >>> On 29 Apr 2019, at 14:19, Maximilian Michels <mx...@apache.org> wrote:
>> >>>
>> >>> I don't know what going on with it but I agree it's annoying.
>> >>>
>> >>> Came across https://jira.apache.org/jira/browse/BEAM-6247, maybe it
>> is time to remove this module for the next release?
>> >>>
>> >>> -Max
>> >>>
>> >>> On 26.04.19 20:10, Reuven Lax wrote:
>> >>>> I find I usually have to rerun Presubmit multiple times to get a
>> green run, and this test is one of the biggest culprits (though it's not
>> the only culprit). Does anyone know what's going on with it?
>> >>>> Reuven
>> >>
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>
Re: :beam-sdks-java-io-hadoop-input-format:test is extremely flaky
Posted by Reuven Lax <re...@google.com>.
yeah, that testClientConnecting test is also extremely flaky.
On Mon, Apr 29, 2019 at 6:50 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Agree, +1
>
> Regards
> JB
>
> On 29/04/2019 15:30, Ismaël Mejía wrote:
> > +1 to remove it on this release, this is a maintenance pain for no real
> reason.
> >
> > On Mon, Apr 29, 2019 at 3:06 PM Alexey Romanenko
> > <ar...@gmail.com> wrote:
> >>
> >> Despite the fact that after fixing an issue with ports allocation
> (thanks to Etienne!) for embedded Cassandra cluster (it’s used in
> hadoop-input-format and this was the main cause of flakiness) it's got much
> better, I’m 100% pro to remove this module since it’s already been
> deprecated for several last releases.
> >>
> >> PS: Just an observation when I was digging into PreCommit jobs results
> -
> “org.apache.beam.runners.dataflow.worker.fn.BeamFnControlServiceTest.testClientConnecting”
> fails quite often in the last time. Anyone works on this?
> >>
> >>> On 29 Apr 2019, at 14:19, Maximilian Michels <mx...@apache.org> wrote:
> >>>
> >>> I don't know what going on with it but I agree it's annoying.
> >>>
> >>> Came across https://jira.apache.org/jira/browse/BEAM-6247, maybe it
> is time to remove this module for the next release?
> >>>
> >>> -Max
> >>>
> >>> On 26.04.19 20:10, Reuven Lax wrote:
> >>>> I find I usually have to rerun Presubmit multiple times to get a
> green run, and this test is one of the biggest culprits (though it's not
> the only culprit). Does anyone know what's going on with it?
> >>>> Reuven
> >>
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: :beam-sdks-java-io-hadoop-input-format:test is extremely flaky
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Agree, +1
Regards
JB
On 29/04/2019 15:30, Ismaël Mejía wrote:
> +1 to remove it on this release, this is a maintenance pain for no real reason.
>
> On Mon, Apr 29, 2019 at 3:06 PM Alexey Romanenko
> <ar...@gmail.com> wrote:
>>
>> Despite the fact that after fixing an issue with ports allocation (thanks to Etienne!) for embedded Cassandra cluster (it’s used in hadoop-input-format and this was the main cause of flakiness) it's got much better, I’m 100% pro to remove this module since it’s already been deprecated for several last releases.
>>
>> PS: Just an observation when I was digging into PreCommit jobs results - “org.apache.beam.runners.dataflow.worker.fn.BeamFnControlServiceTest.testClientConnecting” fails quite often in the last time. Anyone works on this?
>>
>>> On 29 Apr 2019, at 14:19, Maximilian Michels <mx...@apache.org> wrote:
>>>
>>> I don't know what going on with it but I agree it's annoying.
>>>
>>> Came across https://jira.apache.org/jira/browse/BEAM-6247, maybe it is time to remove this module for the next release?
>>>
>>> -Max
>>>
>>> On 26.04.19 20:10, Reuven Lax wrote:
>>>> I find I usually have to rerun Presubmit multiple times to get a green run, and this test is one of the biggest culprits (though it's not the only culprit). Does anyone know what's going on with it?
>>>> Reuven
>>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: :beam-sdks-java-io-hadoop-input-format:test is extremely flaky
Posted by Ismaël Mejía <ie...@gmail.com>.
+1 to remove it on this release, this is a maintenance pain for no real reason.
On Mon, Apr 29, 2019 at 3:06 PM Alexey Romanenko
<ar...@gmail.com> wrote:
>
> Despite the fact that after fixing an issue with ports allocation (thanks to Etienne!) for embedded Cassandra cluster (it’s used in hadoop-input-format and this was the main cause of flakiness) it's got much better, I’m 100% pro to remove this module since it’s already been deprecated for several last releases.
>
> PS: Just an observation when I was digging into PreCommit jobs results - “org.apache.beam.runners.dataflow.worker.fn.BeamFnControlServiceTest.testClientConnecting” fails quite often in the last time. Anyone works on this?
>
> > On 29 Apr 2019, at 14:19, Maximilian Michels <mx...@apache.org> wrote:
> >
> > I don't know what going on with it but I agree it's annoying.
> >
> > Came across https://jira.apache.org/jira/browse/BEAM-6247, maybe it is time to remove this module for the next release?
> >
> > -Max
> >
> > On 26.04.19 20:10, Reuven Lax wrote:
> >> I find I usually have to rerun Presubmit multiple times to get a green run, and this test is one of the biggest culprits (though it's not the only culprit). Does anyone know what's going on with it?
> >> Reuven
>
Re: :beam-sdks-java-io-hadoop-input-format:test is extremely flaky
Posted by Alexey Romanenko <ar...@gmail.com>.
Despite the fact that after fixing an issue with ports allocation (thanks to Etienne!) for embedded Cassandra cluster (it’s used in hadoop-input-format and this was the main cause of flakiness) it's got much better, I’m 100% pro to remove this module since it’s already been deprecated for several last releases.
PS: Just an observation when I was digging into PreCommit jobs results - “org.apache.beam.runners.dataflow.worker.fn.BeamFnControlServiceTest.testClientConnecting” fails quite often in the last time. Anyone works on this?
> On 29 Apr 2019, at 14:19, Maximilian Michels <mx...@apache.org> wrote:
>
> I don't know what going on with it but I agree it's annoying.
>
> Came across https://jira.apache.org/jira/browse/BEAM-6247, maybe it is time to remove this module for the next release?
>
> -Max
>
> On 26.04.19 20:10, Reuven Lax wrote:
>> I find I usually have to rerun Presubmit multiple times to get a green run, and this test is one of the biggest culprits (though it's not the only culprit). Does anyone know what's going on with it?
>> Reuven
Re: :beam-sdks-java-io-hadoop-input-format:test is extremely flaky
Posted by Maximilian Michels <mx...@apache.org>.
I don't know what going on with it but I agree it's annoying.
Came across https://jira.apache.org/jira/browse/BEAM-6247, maybe it is
time to remove this module for the next release?
-Max
On 26.04.19 20:10, Reuven Lax wrote:
> I find I usually have to rerun Presubmit multiple times to get a green
> run, and this test is one of the biggest culprits (though it's not the
> only culprit). Does anyone know what's going on with it?
>
> Reuven