You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@jmeter.apache.org by pb...@gmx.net on 2007/10/03 22:28:44 UTC

jmeter.bat: Special exit code when test plan run fails?

Hi all,

I was wondering if there is a special exit code for jmeter.[bat|sh] and the like when a test fails.

For example if I run this on Windows, the exit code is '0' although some tests failed:

  c:\jakarta-jmeter-2.3RC4\bin>jmeter.bat -n -t ..\extras\Test.jmx  -l shorttest.log
  Created the tree successfully
  Starting the test @ Wed Oct 03 13:10:00 PDT 2007 (1191442200884)
  Tidying up ...    @ Wed Oct 03 13:10:04 PDT 2007 (1191442204589)
  ... end of run

  c:\jakarta-jmeter-2.3RC4\bin>echo %errorlevel%
  0

The failure shows up nicely in the HTML report that I can generate with the XSLT stylesheet, but it would be extremely helpful if the command would return a non-zero exit code if a test fails. This allows me to better integrate JMeter in our test automation framework.

Thanks in advance,
Rico.


-- 
Psssst! Schon vom neuen GMX MultiMessenger gehört?
Der kanns mit allen: http://www.gmx.net/de/go/multimessenger

---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org


Re: jmeter.bat: Special exit code when test plan run fails?

Posted by pb...@gmx.net.
Ok, I see your point. Here is another idea to tackle this problem:

What about adding a "Set Exit Code" Post-processor?

The job of the post-processor would be to determine:
a) A test failure
b) The exit code to return (defined by user)

To determine a failure one could use assertion-results, i.e. I could have an assertion that makes sure my throughput is above a certain limit, if not it sets a failure variable. The post-processor checks for this variable and defines the exit code.

The main program loop would have to check and use the global exit code set by the post-processor as the last step before the program terminates.

-Rico
-------- Original-Nachricht --------
> Datum: Thu, 4 Oct 2007 13:25:37 +0100
> Von: sebb <se...@gmail.com>
> An: "JMeter Users List" <jm...@jakarta.apache.org>
> Betreff: Re: jmeter.bat: Special exit code when test plan run fails?

> Yes, but:
> * ensuring that the exit does not prevent other threads from finishing
> normally may not be trivial;
> * it would prevent the code from being used directly from Java.
> 
> Agreeing the codes is also not easy - what constitutes a test failure?
> 
> On 03/10/2007, pbm-rico@gmx.net <pb...@gmx.net> wrote:
> > Actually, returning an error code in Java is pretty straight forward:
> >
> > System.exit(123); // or whatever code you want.
> >
> > All you have to do is agree on the error code(s).
> >
> > -Rico
> >
> > -------- Original-Nachricht --------
> > > Datum: Wed, 3 Oct 2007 22:02:38 +0100
> > > Von: sebb <se...@gmail.com>
> > > An: "JMeter Users List" <jm...@jakarta.apache.org>
> > > Betreff: Re: jmeter.bat: Special exit code when test plan run fails?
> >
> > > This is a problem with Java applications.
> > >
> > > In general it is very difficult to return a meaningful code from Java
> > > applications.
> > >
> > >
> > > On 03/10/2007, pbm-rico@gmx.net <pb...@gmx.net> wrote:
> > > > Hi all,
> > > >
> > > > I was wondering if there is a special exit code for jmeter.[bat|sh]
> and
> > > the like when a test fails.
> > > >
> > > > For example if I run this on Windows, the exit code is '0' although
> some
> > > tests failed:
> > > >
> > > >  c:\jakarta-jmeter-2.3RC4\bin>jmeter.bat -n -t ..\extras\Test.jmx 
> -l
> > > shorttest.log
> > > >  Created the tree successfully
> > > >  Starting the test @ Wed Oct 03 13:10:00 PDT 2007 (1191442200884)
> > > >  Tidying up ...    @ Wed Oct 03 13:10:04 PDT 2007 (1191442204589)
> > > >  ... end of run
> > > >
> > > >  c:\jakarta-jmeter-2.3RC4\bin>echo %errorlevel%
> > > >  0
> > > >
> > > > The failure shows up nicely in the HTML report that I can generate
> with
> > > the XSLT stylesheet, but it would be extremely helpful if the command
> would
> > > return a non-zero exit code if a test fails. This allows me to better
> > > integrate JMeter in our test automation framework.
> > > >
> > > > Thanks in advance,
> > > > Rico.
> > > >
> > > >
> > > > --
> > > > Psssst! Schon vom neuen GMX MultiMessenger gehört?
> > > > Der kanns mit allen: http://www.gmx.net/de/go/multimessenger
> > > >
> > > >
> ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> >
> > --
> > Psssst! Schon vom neuen GMX MultiMessenger gehört?
> > Der kanns mit allen: http://www.gmx.net/de/go/multimessenger
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> >
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org

-- 
GMX FreeMail: 1 GB Postfach, 5 E-Mail-Adressen, 10 Free SMS.
Alle Infos und kostenlose Anmeldung: http://www.gmx.net/de/go/freemail

---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org


Re: jmeter.bat: Special exit code when test plan run fails?

Posted by sebb <se...@gmail.com>.
Yes, but:
* ensuring that the exit does not prevent other threads from finishing
normally may not be trivial;
* it would prevent the code from being used directly from Java.

Agreeing the codes is also not easy - what constitutes a test failure?

On 03/10/2007, pbm-rico@gmx.net <pb...@gmx.net> wrote:
> Actually, returning an error code in Java is pretty straight forward:
>
> System.exit(123); // or whatever code you want.
>
> All you have to do is agree on the error code(s).
>
> -Rico
>
> -------- Original-Nachricht --------
> > Datum: Wed, 3 Oct 2007 22:02:38 +0100
> > Von: sebb <se...@gmail.com>
> > An: "JMeter Users List" <jm...@jakarta.apache.org>
> > Betreff: Re: jmeter.bat: Special exit code when test plan run fails?
>
> > This is a problem with Java applications.
> >
> > In general it is very difficult to return a meaningful code from Java
> > applications.
> >
> >
> > On 03/10/2007, pbm-rico@gmx.net <pb...@gmx.net> wrote:
> > > Hi all,
> > >
> > > I was wondering if there is a special exit code for jmeter.[bat|sh] and
> > the like when a test fails.
> > >
> > > For example if I run this on Windows, the exit code is '0' although some
> > tests failed:
> > >
> > >  c:\jakarta-jmeter-2.3RC4\bin>jmeter.bat -n -t ..\extras\Test.jmx  -l
> > shorttest.log
> > >  Created the tree successfully
> > >  Starting the test @ Wed Oct 03 13:10:00 PDT 2007 (1191442200884)
> > >  Tidying up ...    @ Wed Oct 03 13:10:04 PDT 2007 (1191442204589)
> > >  ... end of run
> > >
> > >  c:\jakarta-jmeter-2.3RC4\bin>echo %errorlevel%
> > >  0
> > >
> > > The failure shows up nicely in the HTML report that I can generate with
> > the XSLT stylesheet, but it would be extremely helpful if the command would
> > return a non-zero exit code if a test fails. This allows me to better
> > integrate JMeter in our test automation framework.
> > >
> > > Thanks in advance,
> > > Rico.
> > >
> > >
> > > --
> > > Psssst! Schon vom neuen GMX MultiMessenger gehört?
> > > Der kanns mit allen: http://www.gmx.net/de/go/multimessenger
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
> --
> Psssst! Schon vom neuen GMX MultiMessenger gehört?
> Der kanns mit allen: http://www.gmx.net/de/go/multimessenger
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org


Re: jmeter.bat: Special exit code when test plan run fails?

Posted by pb...@gmx.net.
Actually, returning an error code in Java is pretty straight forward:

System.exit(123); // or whatever code you want.

All you have to do is agree on the error code(s).

-Rico

-------- Original-Nachricht --------
> Datum: Wed, 3 Oct 2007 22:02:38 +0100
> Von: sebb <se...@gmail.com>
> An: "JMeter Users List" <jm...@jakarta.apache.org>
> Betreff: Re: jmeter.bat: Special exit code when test plan run fails?

> This is a problem with Java applications.
> 
> In general it is very difficult to return a meaningful code from Java
> applications.
> 
> 
> On 03/10/2007, pbm-rico@gmx.net <pb...@gmx.net> wrote:
> > Hi all,
> >
> > I was wondering if there is a special exit code for jmeter.[bat|sh] and
> the like when a test fails.
> >
> > For example if I run this on Windows, the exit code is '0' although some
> tests failed:
> >
> >  c:\jakarta-jmeter-2.3RC4\bin>jmeter.bat -n -t ..\extras\Test.jmx  -l
> shorttest.log
> >  Created the tree successfully
> >  Starting the test @ Wed Oct 03 13:10:00 PDT 2007 (1191442200884)
> >  Tidying up ...    @ Wed Oct 03 13:10:04 PDT 2007 (1191442204589)
> >  ... end of run
> >
> >  c:\jakarta-jmeter-2.3RC4\bin>echo %errorlevel%
> >  0
> >
> > The failure shows up nicely in the HTML report that I can generate with
> the XSLT stylesheet, but it would be extremely helpful if the command would
> return a non-zero exit code if a test fails. This allows me to better
> integrate JMeter in our test automation framework.
> >
> > Thanks in advance,
> > Rico.
> >
> >
> > --
> > Psssst! Schon vom neuen GMX MultiMessenger gehört?
> > Der kanns mit allen: http://www.gmx.net/de/go/multimessenger
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> >
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org

-- 
Psssst! Schon vom neuen GMX MultiMessenger gehört?
Der kanns mit allen: http://www.gmx.net/de/go/multimessenger

---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org


Re: jmeter.bat: Special exit code when test plan run fails?

Posted by sebb <se...@gmail.com>.
This is a problem with Java applications.

In general it is very difficult to return a meaningful code from Java
applications.


On 03/10/2007, pbm-rico@gmx.net <pb...@gmx.net> wrote:
> Hi all,
>
> I was wondering if there is a special exit code for jmeter.[bat|sh] and the like when a test fails.
>
> For example if I run this on Windows, the exit code is '0' although some tests failed:
>
>  c:\jakarta-jmeter-2.3RC4\bin>jmeter.bat -n -t ..\extras\Test.jmx  -l shorttest.log
>  Created the tree successfully
>  Starting the test @ Wed Oct 03 13:10:00 PDT 2007 (1191442200884)
>  Tidying up ...    @ Wed Oct 03 13:10:04 PDT 2007 (1191442204589)
>  ... end of run
>
>  c:\jakarta-jmeter-2.3RC4\bin>echo %errorlevel%
>  0
>
> The failure shows up nicely in the HTML report that I can generate with the XSLT stylesheet, but it would be extremely helpful if the command would return a non-zero exit code if a test fails. This allows me to better integrate JMeter in our test automation framework.
>
> Thanks in advance,
> Rico.
>
>
> --
> Psssst! Schon vom neuen GMX MultiMessenger gehört?
> Der kanns mit allen: http://www.gmx.net/de/go/multimessenger
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org