You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Wei-Chiu Chuang <we...@apache.org> on 2021/04/29 06:23:54 UTC
[DISCUSS] Looking for help fix flaky tests
Hello,
There are a number of bad tests currently.
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/490/testReport/
Most are HDFS UTs and are almostly failing consistently.
For example, Dynamometer test are failing because it fetches from a mirror
that no longer exists.
I am preparing the 3.3.1 release and these failing tests could be blockers
for the release work. Looking for some help.
Thanks all!
Weichiu
Re: [DISCUSS] Looking for help fix flaky tests
Posted by Takanobu Asanuma <ta...@gmail.com>.
There are two PRs for the Dynamometer test.
https://github.com/apache/hadoop/pull/2471 (HADOOP-17375)
https://github.com/apache/hadoop/pull/2622 (HADOOP-17472)
The problem is not only that the tarball doesn't exist. Please see Erik's
comment in #2622.
I will go over it again.
- Takanobu
2021年4月30日(金) 10:46 Wei-Chiu Chuang <we...@apache.org>:
> Thanks all!
>
> On Thu, Apr 29, 2021 at 3:08 PM Ayush Saxena <ay...@gmail.com> wrote:
>
> > Sorry the number was HADOOP-16524
> >
> > https://issues.apache.org/jira/browse/HADOOP-16524
> >
> > Sent from my iPhone
> >
> > > On 29-Apr-2021, at 12:35 PM, Xiaoqiao He <xq...@gmail.com> wrote:
> > >
> > >
> > > Thanks Wei-Chiu raise this discussion here. I would like to be involved
> > to fix them.
> > >
> > > > For example, Dynamometer test are failing because it fetches from a
> > mirror that no longer exists.
> > > For Dynamometer, I am not sure why not fetch resources from apache
> > directly, any load consideration? or we should redirect it to other
> > mirrors, but it could fail again if some others offline?
> > >
> > > To Ayush,
> > > > TestJournalNode is broken by HADOOP-16254
> > > It seems HADOOP-16254 not commit to trunk or other active branches from
> > now on IIRC. Any information I missed?
> > >
> > > Thanks. Regards.
> > >
> > >> On Thu, Apr 29, 2021 at 2:44 PM Nan <lo...@gmail.com> wrote:
> > >> I am interested in fixing those too. Let me take a look.
> > >>
> > >> On Wed, Apr 28, 2021, 11:39 PM Ayush Saxena <ay...@gmail.com>
> wrote:
> > >>
> > >> > I have a PR to fix a couple of HDFS ones as part of HDFS-15988.
> Give a
> > >> > check if it helps somewhere.
> > >> >
> > >> > The dynamometer one yep is trouble. There is a jira tracking that
> > AFAIK. I
> > >> > don’t recall the number though.
> > >> >
> > >> > TestJournalNode is broken by HADOOP-16254
> > >> >
> > >> > Other seems failing once in a while.
> > >> > Let me know if there is some blocking stuff from HDFS side, I will
> > attempt
> > >> > fixing.
> > >> >
> > >> > -Ayush
> > >> >
> > >> > > On 29-Apr-2021, at 11:54 AM, Wei-Chiu Chuang <we...@apache.org>
> > wrote:
> > >> > >
> > >> > >
> > >> > > Hello,
> > >> > >
> > >> > > There are a number of bad tests currently.
> > >> >
> >
> https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/490/testReport/
> > >> > >
> > >> > > Most are HDFS UTs and are almostly failing consistently.
> > >> > >
> > >> > > For example, Dynamometer test are failing because it fetches from
> a
> > >> > mirror that no longer exists.
> > >> > >
> > >> > > I am preparing the 3.3.1 release and these failing tests could be
> > >> > blockers for the release work. Looking for some help.
> > >> > >
> > >> > > Thanks all!
> > >> > > Weichiu
> > >> > >
> > >> > >
> > ---------------------------------------------------------------------
> > >> > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > >> > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> > >> >
> >
>
Re: [DISCUSS] Looking for help fix flaky tests
Posted by Wei-Chiu Chuang <we...@apache.org>.
Thanks all!
On Thu, Apr 29, 2021 at 3:08 PM Ayush Saxena <ay...@gmail.com> wrote:
> Sorry the number was HADOOP-16524
>
> https://issues.apache.org/jira/browse/HADOOP-16524
>
> Sent from my iPhone
>
> > On 29-Apr-2021, at 12:35 PM, Xiaoqiao He <xq...@gmail.com> wrote:
> >
> >
> > Thanks Wei-Chiu raise this discussion here. I would like to be involved
> to fix them.
> >
> > > For example, Dynamometer test are failing because it fetches from a
> mirror that no longer exists.
> > For Dynamometer, I am not sure why not fetch resources from apache
> directly, any load consideration? or we should redirect it to other
> mirrors, but it could fail again if some others offline?
> >
> > To Ayush,
> > > TestJournalNode is broken by HADOOP-16254
> > It seems HADOOP-16254 not commit to trunk or other active branches from
> now on IIRC. Any information I missed?
> >
> > Thanks. Regards.
> >
> >> On Thu, Apr 29, 2021 at 2:44 PM Nan <lo...@gmail.com> wrote:
> >> I am interested in fixing those too. Let me take a look.
> >>
> >> On Wed, Apr 28, 2021, 11:39 PM Ayush Saxena <ay...@gmail.com> wrote:
> >>
> >> > I have a PR to fix a couple of HDFS ones as part of HDFS-15988. Give a
> >> > check if it helps somewhere.
> >> >
> >> > The dynamometer one yep is trouble. There is a jira tracking that
> AFAIK. I
> >> > don’t recall the number though.
> >> >
> >> > TestJournalNode is broken by HADOOP-16254
> >> >
> >> > Other seems failing once in a while.
> >> > Let me know if there is some blocking stuff from HDFS side, I will
> attempt
> >> > fixing.
> >> >
> >> > -Ayush
> >> >
> >> > > On 29-Apr-2021, at 11:54 AM, Wei-Chiu Chuang <we...@apache.org>
> wrote:
> >> > >
> >> > >
> >> > > Hello,
> >> > >
> >> > > There are a number of bad tests currently.
> >> >
> https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/490/testReport/
> >> > >
> >> > > Most are HDFS UTs and are almostly failing consistently.
> >> > >
> >> > > For example, Dynamometer test are failing because it fetches from a
> >> > mirror that no longer exists.
> >> > >
> >> > > I am preparing the 3.3.1 release and these failing tests could be
> >> > blockers for the release work. Looking for some help.
> >> > >
> >> > > Thanks all!
> >> > > Weichiu
> >> > >
> >> > >
> ---------------------------------------------------------------------
> >> > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> >> > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> >> >
>
Re: [DISCUSS] Looking for help fix flaky tests
Posted by Ayush Saxena <ay...@gmail.com>.
Sorry the number was HADOOP-16524
https://issues.apache.org/jira/browse/HADOOP-16524
Sent from my iPhone
> On 29-Apr-2021, at 12:35 PM, Xiaoqiao He <xq...@gmail.com> wrote:
>
>
> Thanks Wei-Chiu raise this discussion here. I would like to be involved to fix them.
>
> > For example, Dynamometer test are failing because it fetches from a mirror that no longer exists.
> For Dynamometer, I am not sure why not fetch resources from apache directly, any load consideration? or we should redirect it to other mirrors, but it could fail again if some others offline?
>
> To Ayush,
> > TestJournalNode is broken by HADOOP-16254
> It seems HADOOP-16254 not commit to trunk or other active branches from now on IIRC. Any information I missed?
>
> Thanks. Regards.
>
>> On Thu, Apr 29, 2021 at 2:44 PM Nan <lo...@gmail.com> wrote:
>> I am interested in fixing those too. Let me take a look.
>>
>> On Wed, Apr 28, 2021, 11:39 PM Ayush Saxena <ay...@gmail.com> wrote:
>>
>> > I have a PR to fix a couple of HDFS ones as part of HDFS-15988. Give a
>> > check if it helps somewhere.
>> >
>> > The dynamometer one yep is trouble. There is a jira tracking that AFAIK. I
>> > don’t recall the number though.
>> >
>> > TestJournalNode is broken by HADOOP-16254
>> >
>> > Other seems failing once in a while.
>> > Let me know if there is some blocking stuff from HDFS side, I will attempt
>> > fixing.
>> >
>> > -Ayush
>> >
>> > > On 29-Apr-2021, at 11:54 AM, Wei-Chiu Chuang <we...@apache.org> wrote:
>> > >
>> > >
>> > > Hello,
>> > >
>> > > There are a number of bad tests currently.
>> > https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/490/testReport/
>> > >
>> > > Most are HDFS UTs and are almostly failing consistently.
>> > >
>> > > For example, Dynamometer test are failing because it fetches from a
>> > mirror that no longer exists.
>> > >
>> > > I am preparing the 3.3.1 release and these failing tests could be
>> > blockers for the release work. Looking for some help.
>> > >
>> > > Thanks all!
>> > > Weichiu
>> > >
>> > > ---------------------------------------------------------------------
>> > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>> >
Re: [DISCUSS] Looking for help fix flaky tests
Posted by Xiaoqiao He <xq...@gmail.com>.
Thanks Wei-Chiu raise this discussion here. I would like to be involved to
fix them.
> For example, Dynamometer test are failing because it fetches from a
mirror that no longer exists.
For Dynamometer, I am not sure why not fetch resources from apache
directly, any load consideration? or we should redirect it to other
mirrors, but it could fail again if some others offline?
To Ayush,
> TestJournalNode is broken by HADOOP-16254
It seems HADOOP-16254 not commit to trunk or other active branches from now
on IIRC. Any information I missed?
Thanks. Regards.
On Thu, Apr 29, 2021 at 2:44 PM Nan <lo...@gmail.com> wrote:
> I am interested in fixing those too. Let me take a look.
>
> On Wed, Apr 28, 2021, 11:39 PM Ayush Saxena <ay...@gmail.com> wrote:
>
> > I have a PR to fix a couple of HDFS ones as part of HDFS-15988. Give a
> > check if it helps somewhere.
> >
> > The dynamometer one yep is trouble. There is a jira tracking that AFAIK.
> I
> > don’t recall the number though.
> >
> > TestJournalNode is broken by HADOOP-16254
> >
> > Other seems failing once in a while.
> > Let me know if there is some blocking stuff from HDFS side, I will
> attempt
> > fixing.
> >
> > -Ayush
> >
> > > On 29-Apr-2021, at 11:54 AM, Wei-Chiu Chuang <we...@apache.org>
> wrote:
> > >
> > >
> > > Hello,
> > >
> > > There are a number of bad tests currently.
> >
> https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/490/testReport/
> > >
> > > Most are HDFS UTs and are almostly failing consistently.
> > >
> > > For example, Dynamometer test are failing because it fetches from a
> > mirror that no longer exists.
> > >
> > > I am preparing the 3.3.1 release and these failing tests could be
> > blockers for the release work. Looking for some help.
> > >
> > > Thanks all!
> > > Weichiu
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> >
>
Re: [DISCUSS] Looking for help fix flaky tests
Posted by Nan <lo...@gmail.com>.
I am interested in fixing those too. Let me take a look.
On Wed, Apr 28, 2021, 11:39 PM Ayush Saxena <ay...@gmail.com> wrote:
> I have a PR to fix a couple of HDFS ones as part of HDFS-15988. Give a
> check if it helps somewhere.
>
> The dynamometer one yep is trouble. There is a jira tracking that AFAIK. I
> don’t recall the number though.
>
> TestJournalNode is broken by HADOOP-16254
>
> Other seems failing once in a while.
> Let me know if there is some blocking stuff from HDFS side, I will attempt
> fixing.
>
> -Ayush
>
> > On 29-Apr-2021, at 11:54 AM, Wei-Chiu Chuang <we...@apache.org> wrote:
> >
> >
> > Hello,
> >
> > There are a number of bad tests currently.
> https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/490/testReport/
> >
> > Most are HDFS UTs and are almostly failing consistently.
> >
> > For example, Dynamometer test are failing because it fetches from a
> mirror that no longer exists.
> >
> > I am preparing the 3.3.1 release and these failing tests could be
> blockers for the release work. Looking for some help.
> >
> > Thanks all!
> > Weichiu
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
Re: [DISCUSS] Looking for help fix flaky tests
Posted by Ayush Saxena <ay...@gmail.com>.
I have a PR to fix a couple of HDFS ones as part of HDFS-15988. Give a check if it helps somewhere.
The dynamometer one yep is trouble. There is a jira tracking that AFAIK. I don’t recall the number though.
TestJournalNode is broken by HADOOP-16254
Other seems failing once in a while.
Let me know if there is some blocking stuff from HDFS side, I will attempt fixing.
-Ayush
> On 29-Apr-2021, at 11:54 AM, Wei-Chiu Chuang <we...@apache.org> wrote:
>
>
> Hello,
>
> There are a number of bad tests currently. https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/490/testReport/
>
> Most are HDFS UTs and are almostly failing consistently.
>
> For example, Dynamometer test are failing because it fetches from a mirror that no longer exists.
>
> I am preparing the 3.3.1 release and these failing tests could be blockers for the release work. Looking for some help.
>
> Thanks all!
> Weichiu
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org