You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stdcxx.apache.org by Eric Lemings <Er...@roguewave.com> on 2008/03/07 20:13:07 UTC
STDCXX-536
FWIW, I was just reading the description for Jira issue STDCXX-536 and
the associated comments.
I believe this is basically what Travis has already stated in a round-
about manner but it seems to me that if a test is timing out, there are
two possibilities: 1. the test is caught in an infinite loop, or 2.)
the test requires too much time to completely execute. In either case,
the test needs to be rewritten to fix the infinite loop or to break
the test into smaller tests that can completely execute in a reasonably
short amount of time -- a couple of minutes at most on the slowest
supported platforms.
And aren't multithreaded programs (or tests) supposed to perform faster
-- not slower -- than their single-threaded counterparts? :)
Brad.
Re: STDCXX-536
Posted by Martin Sebor <se...@roguewave.com>.
Eric Lemings wrote:
> FWIW, I just noticed that mpfr <http://www.mpfr.org/> contains the
> following configuration option:
>
> --enable-tests-timeout=NUM enable timeout (NUM seconds) for test
> programs
> (NUM <= 9999) [default=no]; if enabled, env
> variable
> $MPFR_TESTS_TIMEOUT overrides NUM (0: no
> timeout).
Right. I thought we could do something similar.
Martin
>
> Brad.
>
>> -----Original Message-----
>> From: Martin Sebor [mailto:msebor@gmail.com] On Behalf Of Martin Sebor
>> Sent: Sunday, March 09, 2008 5:41 PM
>> To: dev@stdcxx.apache.org
>> Subject: Re: STDCXX-536
>>
>> Eric Lemings wrote:
>>> FWIW, I was just reading the description for Jira issue
>> STDCXX-536 and
>>> the associated comments.
>>>
>>> I believe this is basically what Travis has already stated
>> in a round-
>>> about manner but it seems to me that if a test is timing
>> out, there are
>>> two possibilities: 1. the test is caught in an infinite loop, or 2.)
>>> the test requires too much time to completely execute. In
>> either case,
>>> the test needs to be rewritten to fix the infinite loop or to break
>>> the test into smaller tests that can completely execute in
>> a reasonably
>>> short amount of time -- a couple of minutes at most on the slowest
>>> supported platforms.
>> Right, that's the conclusion we came to for the tests that can be
>> broken up. The downside of doing this is code duplication (either
>> source or object) between the small tests that share the same
>> scaffolding, which will increase compilation and/or link times,
>> and, of course, run times. With a large enough of number of these
>> little tests the consequences could be actually worse than the
>> status quo.
>>
>> The problem with most of our thread safety tests is that they are
>> written to do the sane fixed number of operations in parallel, and
>> this number takes a different amount of time depending on factors
>> such as the scheduler's time slice and the system load. Ideally,
>> the tests would take these parameters into consideration and run
>> for sufficiently long (but no longer) to exercise the functionality
>> with some desired percentage of parallelism/contention. To do that,
>> the tests would need to be able to time out.
>>
>>> And aren't multithreaded programs (or tests) supposed to
>> perform faster
>>> -- not slower -- than their single-threaded counterparts? :)
>> Only if they are written to avoid contention, which is the exact
>> opposite of what our thread safety tests are designed to do.
>>
>> Martin
>>
RE: STDCXX-536
Posted by Eric Lemings <Er...@roguewave.com>.
FWIW, I just noticed that mpfr <http://www.mpfr.org/> contains the
following configuration option:
--enable-tests-timeout=NUM enable timeout (NUM seconds) for test
programs
(NUM <= 9999) [default=no]; if enabled, env
variable
$MPFR_TESTS_TIMEOUT overrides NUM (0: no
timeout).
Brad.
> -----Original Message-----
> From: Martin Sebor [mailto:msebor@gmail.com] On Behalf Of Martin Sebor
> Sent: Sunday, March 09, 2008 5:41 PM
> To: dev@stdcxx.apache.org
> Subject: Re: STDCXX-536
>
> Eric Lemings wrote:
> > FWIW, I was just reading the description for Jira issue
> STDCXX-536 and
> > the associated comments.
> >
> > I believe this is basically what Travis has already stated
> in a round-
> > about manner but it seems to me that if a test is timing
> out, there are
> > two possibilities: 1. the test is caught in an infinite loop, or 2.)
> > the test requires too much time to completely execute. In
> either case,
> > the test needs to be rewritten to fix the infinite loop or to break
> > the test into smaller tests that can completely execute in
> a reasonably
> > short amount of time -- a couple of minutes at most on the slowest
> > supported platforms.
>
> Right, that's the conclusion we came to for the tests that can be
> broken up. The downside of doing this is code duplication (either
> source or object) between the small tests that share the same
> scaffolding, which will increase compilation and/or link times,
> and, of course, run times. With a large enough of number of these
> little tests the consequences could be actually worse than the
> status quo.
>
> The problem with most of our thread safety tests is that they are
> written to do the sane fixed number of operations in parallel, and
> this number takes a different amount of time depending on factors
> such as the scheduler's time slice and the system load. Ideally,
> the tests would take these parameters into consideration and run
> for sufficiently long (but no longer) to exercise the functionality
> with some desired percentage of parallelism/contention. To do that,
> the tests would need to be able to time out.
>
> >
> > And aren't multithreaded programs (or tests) supposed to
> perform faster
> > -- not slower -- than their single-threaded counterparts? :)
>
> Only if they are written to avoid contention, which is the exact
> opposite of what our thread safety tests are designed to do.
>
> Martin
>
Re: STDCXX-536
Posted by Martin Sebor <se...@roguewave.com>.
Eric Lemings wrote:
> FWIW, I was just reading the description for Jira issue STDCXX-536 and
> the associated comments.
>
> I believe this is basically what Travis has already stated in a round-
> about manner but it seems to me that if a test is timing out, there are
> two possibilities: 1. the test is caught in an infinite loop, or 2.)
> the test requires too much time to completely execute. In either case,
> the test needs to be rewritten to fix the infinite loop or to break
> the test into smaller tests that can completely execute in a reasonably
> short amount of time -- a couple of minutes at most on the slowest
> supported platforms.
Right, that's the conclusion we came to for the tests that can be
broken up. The downside of doing this is code duplication (either
source or object) between the small tests that share the same
scaffolding, which will increase compilation and/or link times,
and, of course, run times. With a large enough of number of these
little tests the consequences could be actually worse than the
status quo.
The problem with most of our thread safety tests is that they are
written to do the sane fixed number of operations in parallel, and
this number takes a different amount of time depending on factors
such as the scheduler's time slice and the system load. Ideally,
the tests would take these parameters into consideration and run
for sufficiently long (but no longer) to exercise the functionality
with some desired percentage of parallelism/contention. To do that,
the tests would need to be able to time out.
>
> And aren't multithreaded programs (or tests) supposed to perform faster
> -- not slower -- than their single-threaded counterparts? :)
Only if they are written to avoid contention, which is the exact
opposite of what our thread safety tests are designed to do.
Martin