You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Zhijie Shen <zs...@hortonworks.com> on 2014/09/01 08:40:15 UTC
Re: Hadoop 2.5.0 unit tests failures
Hi Rajat,
It is the situation that some test cases will have kinds of race
conditions, an fail intermittently. In most cases, contributors are working
on a linux box, such that the test case may implicitly make some assumption
that is not valid for other systems.
It is appreciated to report these test failures on Jira, with the
environment details and the exception tracing, such that the community can
investigate the test problems.
Thanks,
Zhijie
On Fri, Aug 29, 2014 at 10:06 AM, Rajat Jain <ra...@gmail.com> wrote:
> Hi,
>
> I wanted to know if all the unit tests pass in the hadoop-common project
> across various releases. I have never been able to get a clean run on my
> machine (Centos 6.5 / 4GB RAM / tried both Java 6 and Java 7). I have also
> attached the document which has the failures that I got while running the
> tests.
>
> I ran "mvn clean package install -DskipTests" to compile, and thereafter,
> ran "mvn test" from individual subprojects.
>
> In my company, we have forked Apache Hadoop 2.5.0 and we are planning to
> deploy a nightly unit test run to make sure we don't introduce any
> regressions. Is there a way to get a clean unit-test run, or should I
> disable these tests from our suite? I also read somewhere else that there
> are a few flaky tests as well.
>
> Any help is appreciated.
>
>
> https://docs.google.com/a/qubole.com/spreadsheets/d/1bKCclEA0u9VUZvykgaRj_gBqY4xMxTBIGPJml04TXtE/edit#gid=1215903400
>
> Thanks,
> Rajat
>
--
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Hadoop 2.5.0 unit tests failures
Posted by Ray Chiang <rc...@cloudera.com>.
Just as a quick follow up, you can also search the JIRAs to see which tests
are already known to be on the flakier side (e.g. race conditions like
Zhijie mentions, or some similar hard-to-replicate reason).
-Ray
On Sun, Aug 31, 2014 at 11:40 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
> Hi Rajat,
>
> It is the situation that some test cases will have kinds of race
> conditions, an fail intermittently. In most cases, contributors are working
> on a linux box, such that the test case may implicitly make some assumption
> that is not valid for other systems.
>
> It is appreciated to report these test failures on Jira, with the
> environment details and the exception tracing, such that the community can
> investigate the test problems.
>
> Thanks,
> Zhijie
>
>
> On Fri, Aug 29, 2014 at 10:06 AM, Rajat Jain <ra...@gmail.com> wrote:
>
>> Hi,
>>
>> I wanted to know if all the unit tests pass in the hadoop-common project
>> across various releases. I have never been able to get a clean run on my
>> machine (Centos 6.5 / 4GB RAM / tried both Java 6 and Java 7). I have also
>> attached the document which has the failures that I got while running the
>> tests.
>>
>> I ran "mvn clean package install -DskipTests" to compile, and thereafter,
>> ran "mvn test" from individual subprojects.
>>
>> In my company, we have forked Apache Hadoop 2.5.0 and we are planning to
>> deploy a nightly unit test run to make sure we don't introduce any
>> regressions. Is there a way to get a clean unit-test run, or should I
>> disable these tests from our suite? I also read somewhere else that there
>> are a few flaky tests as well.
>>
>> Any help is appreciated.
>>
>>
>> https://docs.google.com/a/qubole.com/spreadsheets/d/1bKCclEA0u9VUZvykgaRj_gBqY4xMxTBIGPJml04TXtE/edit#gid=1215903400
>>
>> Thanks,
>> Rajat
>>
>
>
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Hadoop 2.5.0 unit tests failures
Posted by Ray Chiang <rc...@cloudera.com>.
Just as a quick follow up, you can also search the JIRAs to see which tests
are already known to be on the flakier side (e.g. race conditions like
Zhijie mentions, or some similar hard-to-replicate reason).
-Ray
On Sun, Aug 31, 2014 at 11:40 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
> Hi Rajat,
>
> It is the situation that some test cases will have kinds of race
> conditions, an fail intermittently. In most cases, contributors are working
> on a linux box, such that the test case may implicitly make some assumption
> that is not valid for other systems.
>
> It is appreciated to report these test failures on Jira, with the
> environment details and the exception tracing, such that the community can
> investigate the test problems.
>
> Thanks,
> Zhijie
>
>
> On Fri, Aug 29, 2014 at 10:06 AM, Rajat Jain <ra...@gmail.com> wrote:
>
>> Hi,
>>
>> I wanted to know if all the unit tests pass in the hadoop-common project
>> across various releases. I have never been able to get a clean run on my
>> machine (Centos 6.5 / 4GB RAM / tried both Java 6 and Java 7). I have also
>> attached the document which has the failures that I got while running the
>> tests.
>>
>> I ran "mvn clean package install -DskipTests" to compile, and thereafter,
>> ran "mvn test" from individual subprojects.
>>
>> In my company, we have forked Apache Hadoop 2.5.0 and we are planning to
>> deploy a nightly unit test run to make sure we don't introduce any
>> regressions. Is there a way to get a clean unit-test run, or should I
>> disable these tests from our suite? I also read somewhere else that there
>> are a few flaky tests as well.
>>
>> Any help is appreciated.
>>
>>
>> https://docs.google.com/a/qubole.com/spreadsheets/d/1bKCclEA0u9VUZvykgaRj_gBqY4xMxTBIGPJml04TXtE/edit#gid=1215903400
>>
>> Thanks,
>> Rajat
>>
>
>
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Hadoop 2.5.0 unit tests failures
Posted by Ray Chiang <rc...@cloudera.com>.
Just as a quick follow up, you can also search the JIRAs to see which tests
are already known to be on the flakier side (e.g. race conditions like
Zhijie mentions, or some similar hard-to-replicate reason).
-Ray
On Sun, Aug 31, 2014 at 11:40 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
> Hi Rajat,
>
> It is the situation that some test cases will have kinds of race
> conditions, an fail intermittently. In most cases, contributors are working
> on a linux box, such that the test case may implicitly make some assumption
> that is not valid for other systems.
>
> It is appreciated to report these test failures on Jira, with the
> environment details and the exception tracing, such that the community can
> investigate the test problems.
>
> Thanks,
> Zhijie
>
>
> On Fri, Aug 29, 2014 at 10:06 AM, Rajat Jain <ra...@gmail.com> wrote:
>
>> Hi,
>>
>> I wanted to know if all the unit tests pass in the hadoop-common project
>> across various releases. I have never been able to get a clean run on my
>> machine (Centos 6.5 / 4GB RAM / tried both Java 6 and Java 7). I have also
>> attached the document which has the failures that I got while running the
>> tests.
>>
>> I ran "mvn clean package install -DskipTests" to compile, and thereafter,
>> ran "mvn test" from individual subprojects.
>>
>> In my company, we have forked Apache Hadoop 2.5.0 and we are planning to
>> deploy a nightly unit test run to make sure we don't introduce any
>> regressions. Is there a way to get a clean unit-test run, or should I
>> disable these tests from our suite? I also read somewhere else that there
>> are a few flaky tests as well.
>>
>> Any help is appreciated.
>>
>>
>> https://docs.google.com/a/qubole.com/spreadsheets/d/1bKCclEA0u9VUZvykgaRj_gBqY4xMxTBIGPJml04TXtE/edit#gid=1215903400
>>
>> Thanks,
>> Rajat
>>
>
>
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Hadoop 2.5.0 unit tests failures
Posted by Ray Chiang <rc...@cloudera.com>.
Just as a quick follow up, you can also search the JIRAs to see which tests
are already known to be on the flakier side (e.g. race conditions like
Zhijie mentions, or some similar hard-to-replicate reason).
-Ray
On Sun, Aug 31, 2014 at 11:40 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
> Hi Rajat,
>
> It is the situation that some test cases will have kinds of race
> conditions, an fail intermittently. In most cases, contributors are working
> on a linux box, such that the test case may implicitly make some assumption
> that is not valid for other systems.
>
> It is appreciated to report these test failures on Jira, with the
> environment details and the exception tracing, such that the community can
> investigate the test problems.
>
> Thanks,
> Zhijie
>
>
> On Fri, Aug 29, 2014 at 10:06 AM, Rajat Jain <ra...@gmail.com> wrote:
>
>> Hi,
>>
>> I wanted to know if all the unit tests pass in the hadoop-common project
>> across various releases. I have never been able to get a clean run on my
>> machine (Centos 6.5 / 4GB RAM / tried both Java 6 and Java 7). I have also
>> attached the document which has the failures that I got while running the
>> tests.
>>
>> I ran "mvn clean package install -DskipTests" to compile, and thereafter,
>> ran "mvn test" from individual subprojects.
>>
>> In my company, we have forked Apache Hadoop 2.5.0 and we are planning to
>> deploy a nightly unit test run to make sure we don't introduce any
>> regressions. Is there a way to get a clean unit-test run, or should I
>> disable these tests from our suite? I also read somewhere else that there
>> are a few flaky tests as well.
>>
>> Any help is appreciated.
>>
>>
>> https://docs.google.com/a/qubole.com/spreadsheets/d/1bKCclEA0u9VUZvykgaRj_gBqY4xMxTBIGPJml04TXtE/edit#gid=1215903400
>>
>> Thanks,
>> Rajat
>>
>
>
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.