You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Mi...@emc.com on 2011/09/23 23:29:25 UTC
Long running tests failure
Folks,
When running TestLargeBlock and TestBalancer, which tend to take a long
time to run on my dev box, I get the following error:
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.6:test (default-test) on
project hadoop-hdfs: Error while executing forked tests.; nested exception
is
org.apache.maven.surefire.booter.shade.org.codehaus.plexus.util.cli.Command
LineTimeOutException: Error while executing external command, process
killed. Process timeout out after 900 seconds -> [Help 1]
How do you set the timeout to be 1 hour instead of default 900 seconds ?
(Other option is to buy a supercomputer as a dev environment for hadoop,
but thought I should try asking the dev list first ;-)
- milind
---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)
Re: Long running tests failure
Posted by Steve Loughran <st...@apache.org>.
On 24/09/11 00:15, Todd Lipcon wrote:
> I think TestLargeBlock fails if you have low disk space on your dev
> machine -- since the drive selection algorithm needs something like 5x
> block size free, and the test uses a 2G+ block size. I know it always
> fails on my laptop where I tend to have 5-6G free (SSD small but worth
> it!)
It's timing out on a machine of mine with 300+GB, so that isn't the
cause. Maybe slow disks/cpus/swapping triggers the timeout?
Re: Long running tests failure
Posted by Mi...@emc.com.
Yeah ! That's really crazy. I don;t have time to debug it though. I will
move my dev environment to linux.
:-(
- milind
On 9/29/11 4:12 AM, "Jeff MAURY" <je...@jeffmaury.com> wrote:
>I have met a very strange behaviour: as it seems the Hadoop build does not
>work on MacOS, I build an Ubuntu VM that I launched on my MBP and when I
>run
>maven on this VM, the LargeBlock test passed in 475s (compared to 2000s on
>native MacOS build)
>
>Regards
>Jeff
>
>
>On Thu, Sep 29, 2011 at 12:58 PM, Steve Loughran <st...@apache.org>
>wrote:
>
>> On 28/09/11 21:15, Jeff MAURY wrote:
>>
>>> On Wed, Sep 28, 2011 at 8:58 PM,<Mi...@emc.com> wrote:
>>>
>>> Cool! I commented out that line as well. Can I override that parameter
>>>> from commandline ? It will be less headache.
>>>>
>>>> This is theorically possible through the surefire.timeout property
>>>>but
>>> as
>>> soon as the property is defined in a POM (or a parent POM), it will
>>>have
>>> higher priority.
>>>
>>>
>> -Because Maven knows better than you what you want...
>>
>>
>
>
>--
>"Legacy code" often differs from its suggested alternative by actually
>working and scaling.
> - Bjarne Stroustrup
>
>http://www.jeffmaury.com
>http://riadiscuss.jeffmaury.com
>http://www.twitter.com/jeffmaury
Re: Long running tests failure
Posted by Jeff MAURY <je...@jeffmaury.com>.
I have met a very strange behaviour: as it seems the Hadoop build does not
work on MacOS, I build an Ubuntu VM that I launched on my MBP and when I run
maven on this VM, the LargeBlock test passed in 475s (compared to 2000s on
native MacOS build)
Regards
Jeff
On Thu, Sep 29, 2011 at 12:58 PM, Steve Loughran <st...@apache.org> wrote:
> On 28/09/11 21:15, Jeff MAURY wrote:
>
>> On Wed, Sep 28, 2011 at 8:58 PM,<Mi...@emc.com> wrote:
>>
>> Cool! I commented out that line as well. Can I override that parameter
>>> from commandline ? It will be less headache.
>>>
>>> This is theorically possible through the surefire.timeout property but
>> as
>> soon as the property is defined in a POM (or a parent POM), it will have
>> higher priority.
>>
>>
> -Because Maven knows better than you what you want...
>
>
--
"Legacy code" often differs from its suggested alternative by actually
working and scaling.
- Bjarne Stroustrup
http://www.jeffmaury.com
http://riadiscuss.jeffmaury.com
http://www.twitter.com/jeffmaury
Re: Long running tests failure
Posted by Steve Loughran <st...@apache.org>.
On 28/09/11 21:15, Jeff MAURY wrote:
> On Wed, Sep 28, 2011 at 8:58 PM,<Mi...@emc.com> wrote:
>
>> Cool! I commented out that line as well. Can I override that parameter
>> from commandline ? It will be less headache.
>>
> This is theorically possible through the surefire.timeout property but as
> soon as the property is defined in a POM (or a parent POM), it will have
> higher priority.
>
-Because Maven knows better than you what you want...
Re: Long running tests failure
Posted by Jeff MAURY <je...@jeffmaury.com>.
On Wed, Sep 28, 2011 at 8:58 PM, <Mi...@emc.com> wrote:
> Cool! I commented out that line as well. Can I override that parameter
> from commandline ? It will be less headache.
>
This is theorically possible through the surefire.timeout property but as
soon as the property is defined in a POM (or a parent POM), it will have
higher priority.
Regards
Jeff
>
> - milind
>
> On 9/28/11 3:48 AM, "Jeff MAURY" <je...@jeffmaury.com> wrote:
>
> >I faced the same issue on my brand new MPB (i7) laptop.
> >The Hadoop pom.xml are configured with a surefire plugin timeout of 900s.
> >I
> >commented it out (infinite timeout) and the TestLargeBlock last 2000s on
> >my
> >laptop. I have now a problem with another test
> >(org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade) which seems
> >to
> >last forever !!!
> >I will have a look to understand.
> >
> >Regards
> >Jeff
> >
> >
> >On Tue, Sep 27, 2011 at 8:12 PM, <Mi...@emc.com> wrote:
> >
> >> Disk space is not an issue on my MBP:
> >>
> >> Filesystem 1K-blocks Used Available Use% Mounted on
> >> /dev/disk0s2 488050672 198290788 289503884 41% /
> >>
> >>
> >>
> >>
> >> - milind
> >>
> >> On 9/23/11 4:15 PM, "Todd Lipcon" <to...@cloudera.com> wrote:
> >>
> >> >I think TestLargeBlock fails if you have low disk space on your dev
> >> >machine -- since the drive selection algorithm needs something like 5x
> >> >block size free, and the test uses a 2G+ block size. I know it always
> >> >fails on my laptop where I tend to have 5-6G free (SSD small but worth
> >> >it!)
> >> >
> >> >-Todd
> >> >
> >> >On Sat, Sep 24, 2011 at 6:29 AM, <Mi...@emc.com> wrote:
> >> >> Folks,
> >> >>
> >> >> When running TestLargeBlock and TestBalancer, which tend to take a
> >>long
> >> >> time to run on my dev box, I get the following error:
> >> >>
> >> >> [ERROR] Failed to execute goal
> >> >> org.apache.maven.plugins:maven-surefire-plugin:2.6:test
> >>(default-test)
> >> >>on
> >> >> project hadoop-hdfs: Error while executing forked tests.; nested
> >> >>exception
> >> >> is
> >> >>
> >>
> >>>>
> org.apache.maven.surefire.booter.shade.org.codehaus.plexus.util.cli.Com
> >>>>ma
> >> >>nd
> >> >> LineTimeOutException: Error while executing external command, process
> >> >> killed. Process timeout out after 900 seconds -> [Help 1]
> >> >>
> >> >> How do you set the timeout to be 1 hour instead of default 900
> >>seconds ?
> >> >>
> >> >>
> >> >> (Other option is to buy a supercomputer as a dev environment for
> >>hadoop,
> >> >> but thought I should try asking the dev list first ;-)
> >> >>
> >> >> - milind
> >> >>
> >> >> ---
> >> >> Milind Bhandarkar
> >> >> Greenplum Labs, EMC
> >> >> (Disclaimer: Opinions expressed in this email are those of the
> >>author,
> >> >>and
> >> >> do not necessarily represent the views of any organization, past or
> >> >> present, the author might be affiliated with.)
> >> >>
> >> >>
> >> >>
> >> >>
> >> >
> >> >
> >> >
> >> >--
> >> >Todd Lipcon
> >> >Software Engineer, Cloudera
> >> >
> >>
> >>
> >
> >
> >--
> >"Legacy code" often differs from its suggested alternative by actually
> >working and scaling.
> > - Bjarne Stroustrup
> >
> >http://www.jeffmaury.com
> >http://riadiscuss.jeffmaury.com
> >http://www.twitter.com/jeffmaury
>
>
--
"Legacy code" often differs from its suggested alternative by actually
working and scaling.
- Bjarne Stroustrup
http://www.jeffmaury.com
http://riadiscuss.jeffmaury.com
http://www.twitter.com/jeffmaury
Re: Long running tests failure
Posted by Mi...@emc.com.
Cool! I commented out that line as well. Can I override that parameter
from commandline ? It will be less headache.
- milind
On 9/28/11 3:48 AM, "Jeff MAURY" <je...@jeffmaury.com> wrote:
>I faced the same issue on my brand new MPB (i7) laptop.
>The Hadoop pom.xml are configured with a surefire plugin timeout of 900s.
>I
>commented it out (infinite timeout) and the TestLargeBlock last 2000s on
>my
>laptop. I have now a problem with another test
>(org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade) which seems
>to
>last forever !!!
>I will have a look to understand.
>
>Regards
>Jeff
>
>
>On Tue, Sep 27, 2011 at 8:12 PM, <Mi...@emc.com> wrote:
>
>> Disk space is not an issue on my MBP:
>>
>> Filesystem 1K-blocks Used Available Use% Mounted on
>> /dev/disk0s2 488050672 198290788 289503884 41% /
>>
>>
>>
>>
>> - milind
>>
>> On 9/23/11 4:15 PM, "Todd Lipcon" <to...@cloudera.com> wrote:
>>
>> >I think TestLargeBlock fails if you have low disk space on your dev
>> >machine -- since the drive selection algorithm needs something like 5x
>> >block size free, and the test uses a 2G+ block size. I know it always
>> >fails on my laptop where I tend to have 5-6G free (SSD small but worth
>> >it!)
>> >
>> >-Todd
>> >
>> >On Sat, Sep 24, 2011 at 6:29 AM, <Mi...@emc.com> wrote:
>> >> Folks,
>> >>
>> >> When running TestLargeBlock and TestBalancer, which tend to take a
>>long
>> >> time to run on my dev box, I get the following error:
>> >>
>> >> [ERROR] Failed to execute goal
>> >> org.apache.maven.plugins:maven-surefire-plugin:2.6:test
>>(default-test)
>> >>on
>> >> project hadoop-hdfs: Error while executing forked tests.; nested
>> >>exception
>> >> is
>> >>
>>
>>>>org.apache.maven.surefire.booter.shade.org.codehaus.plexus.util.cli.Com
>>>>ma
>> >>nd
>> >> LineTimeOutException: Error while executing external command, process
>> >> killed. Process timeout out after 900 seconds -> [Help 1]
>> >>
>> >> How do you set the timeout to be 1 hour instead of default 900
>>seconds ?
>> >>
>> >>
>> >> (Other option is to buy a supercomputer as a dev environment for
>>hadoop,
>> >> but thought I should try asking the dev list first ;-)
>> >>
>> >> - milind
>> >>
>> >> ---
>> >> Milind Bhandarkar
>> >> Greenplum Labs, EMC
>> >> (Disclaimer: Opinions expressed in this email are those of the
>>author,
>> >>and
>> >> do not necessarily represent the views of any organization, past or
>> >> present, the author might be affiliated with.)
>> >>
>> >>
>> >>
>> >>
>> >
>> >
>> >
>> >--
>> >Todd Lipcon
>> >Software Engineer, Cloudera
>> >
>>
>>
>
>
>--
>"Legacy code" often differs from its suggested alternative by actually
>working and scaling.
> - Bjarne Stroustrup
>
>http://www.jeffmaury.com
>http://riadiscuss.jeffmaury.com
>http://www.twitter.com/jeffmaury
Re: Long running tests failure
Posted by Jeff MAURY <je...@jeffmaury.com>.
I faced the same issue on my brand new MPB (i7) laptop.
The Hadoop pom.xml are configured with a surefire plugin timeout of 900s. I
commented it out (infinite timeout) and the TestLargeBlock last 2000s on my
laptop. I have now a problem with another test
(org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade) which seems to
last forever !!!
I will have a look to understand.
Regards
Jeff
On Tue, Sep 27, 2011 at 8:12 PM, <Mi...@emc.com> wrote:
> Disk space is not an issue on my MBP:
>
> Filesystem 1K-blocks Used Available Use% Mounted on
> /dev/disk0s2 488050672 198290788 289503884 41% /
>
>
>
>
> - milind
>
> On 9/23/11 4:15 PM, "Todd Lipcon" <to...@cloudera.com> wrote:
>
> >I think TestLargeBlock fails if you have low disk space on your dev
> >machine -- since the drive selection algorithm needs something like 5x
> >block size free, and the test uses a 2G+ block size. I know it always
> >fails on my laptop where I tend to have 5-6G free (SSD small but worth
> >it!)
> >
> >-Todd
> >
> >On Sat, Sep 24, 2011 at 6:29 AM, <Mi...@emc.com> wrote:
> >> Folks,
> >>
> >> When running TestLargeBlock and TestBalancer, which tend to take a long
> >> time to run on my dev box, I get the following error:
> >>
> >> [ERROR] Failed to execute goal
> >> org.apache.maven.plugins:maven-surefire-plugin:2.6:test (default-test)
> >>on
> >> project hadoop-hdfs: Error while executing forked tests.; nested
> >>exception
> >> is
> >>
> >>org.apache.maven.surefire.booter.shade.org.codehaus.plexus.util.cli.Comma
> >>nd
> >> LineTimeOutException: Error while executing external command, process
> >> killed. Process timeout out after 900 seconds -> [Help 1]
> >>
> >> How do you set the timeout to be 1 hour instead of default 900 seconds ?
> >>
> >>
> >> (Other option is to buy a supercomputer as a dev environment for hadoop,
> >> but thought I should try asking the dev list first ;-)
> >>
> >> - milind
> >>
> >> ---
> >> Milind Bhandarkar
> >> Greenplum Labs, EMC
> >> (Disclaimer: Opinions expressed in this email are those of the author,
> >>and
> >> do not necessarily represent the views of any organization, past or
> >> present, the author might be affiliated with.)
> >>
> >>
> >>
> >>
> >
> >
> >
> >--
> >Todd Lipcon
> >Software Engineer, Cloudera
> >
>
>
--
"Legacy code" often differs from its suggested alternative by actually
working and scaling.
- Bjarne Stroustrup
http://www.jeffmaury.com
http://riadiscuss.jeffmaury.com
http://www.twitter.com/jeffmaury
Re: Long running tests failure
Posted by Mi...@emc.com.
Disk space is not an issue on my MBP:
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/disk0s2 488050672 198290788 289503884 41% /
- milind
On 9/23/11 4:15 PM, "Todd Lipcon" <to...@cloudera.com> wrote:
>I think TestLargeBlock fails if you have low disk space on your dev
>machine -- since the drive selection algorithm needs something like 5x
>block size free, and the test uses a 2G+ block size. I know it always
>fails on my laptop where I tend to have 5-6G free (SSD small but worth
>it!)
>
>-Todd
>
>On Sat, Sep 24, 2011 at 6:29 AM, <Mi...@emc.com> wrote:
>> Folks,
>>
>> When running TestLargeBlock and TestBalancer, which tend to take a long
>> time to run on my dev box, I get the following error:
>>
>> [ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-surefire-plugin:2.6:test (default-test)
>>on
>> project hadoop-hdfs: Error while executing forked tests.; nested
>>exception
>> is
>>
>>org.apache.maven.surefire.booter.shade.org.codehaus.plexus.util.cli.Comma
>>nd
>> LineTimeOutException: Error while executing external command, process
>> killed. Process timeout out after 900 seconds -> [Help 1]
>>
>> How do you set the timeout to be 1 hour instead of default 900 seconds ?
>>
>>
>> (Other option is to buy a supercomputer as a dev environment for hadoop,
>> but thought I should try asking the dev list first ;-)
>>
>> - milind
>>
>> ---
>> Milind Bhandarkar
>> Greenplum Labs, EMC
>> (Disclaimer: Opinions expressed in this email are those of the author,
>>and
>> do not necessarily represent the views of any organization, past or
>> present, the author might be affiliated with.)
>>
>>
>>
>>
>
>
>
>--
>Todd Lipcon
>Software Engineer, Cloudera
>
Re: Long running tests failure
Posted by Todd Lipcon <to...@cloudera.com>.
I think TestLargeBlock fails if you have low disk space on your dev
machine -- since the drive selection algorithm needs something like 5x
block size free, and the test uses a 2G+ block size. I know it always
fails on my laptop where I tend to have 5-6G free (SSD small but worth
it!)
-Todd
On Sat, Sep 24, 2011 at 6:29 AM, <Mi...@emc.com> wrote:
> Folks,
>
> When running TestLargeBlock and TestBalancer, which tend to take a long
> time to run on my dev box, I get the following error:
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.6:test (default-test) on
> project hadoop-hdfs: Error while executing forked tests.; nested exception
> is
> org.apache.maven.surefire.booter.shade.org.codehaus.plexus.util.cli.Command
> LineTimeOutException: Error while executing external command, process
> killed. Process timeout out after 900 seconds -> [Help 1]
>
> How do you set the timeout to be 1 hour instead of default 900 seconds ?
>
>
> (Other option is to buy a supercomputer as a dev environment for hadoop,
> but thought I should try asking the dev list first ;-)
>
> - milind
>
> ---
> Milind Bhandarkar
> Greenplum Labs, EMC
> (Disclaimer: Opinions expressed in this email are those of the author, and
> do not necessarily represent the views of any organization, past or
> present, the author might be affiliated with.)
>
>
>
>
--
Todd Lipcon
Software Engineer, Cloudera