You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@jmeter.apache.org by pb...@gmx.net on 2007/10/03 22:28:44 UTC
jmeter.bat: Special exit code when test plan run fails?
Hi all,
I was wondering if there is a special exit code for jmeter.[bat|sh] and the like when a test fails.
For example if I run this on Windows, the exit code is '0' although some tests failed:
c:\jakarta-jmeter-2.3RC4\bin>jmeter.bat -n -t ..\extras\Test.jmx -l shorttest.log
Created the tree successfully
Starting the test @ Wed Oct 03 13:10:00 PDT 2007 (1191442200884)
Tidying up ... @ Wed Oct 03 13:10:04 PDT 2007 (1191442204589)
... end of run
c:\jakarta-jmeter-2.3RC4\bin>echo %errorlevel%
0
The failure shows up nicely in the HTML report that I can generate with the XSLT stylesheet, but it would be extremely helpful if the command would return a non-zero exit code if a test fails. This allows me to better integrate JMeter in our test automation framework.
Thanks in advance,
Rico.
--
Psssst! Schon vom neuen GMX MultiMessenger gehört?
Der kanns mit allen: http://www.gmx.net/de/go/multimessenger
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
Re: jmeter.bat: Special exit code when test plan run fails?
Posted by pb...@gmx.net.
Ok, I see your point. Here is another idea to tackle this problem:
What about adding a "Set Exit Code" Post-processor?
The job of the post-processor would be to determine:
a) A test failure
b) The exit code to return (defined by user)
To determine a failure one could use assertion-results, i.e. I could have an assertion that makes sure my throughput is above a certain limit, if not it sets a failure variable. The post-processor checks for this variable and defines the exit code.
The main program loop would have to check and use the global exit code set by the post-processor as the last step before the program terminates.
-Rico
-------- Original-Nachricht --------
> Datum: Thu, 4 Oct 2007 13:25:37 +0100
> Von: sebb <se...@gmail.com>
> An: "JMeter Users List" <jm...@jakarta.apache.org>
> Betreff: Re: jmeter.bat: Special exit code when test plan run fails?
> Yes, but:
> * ensuring that the exit does not prevent other threads from finishing
> normally may not be trivial;
> * it would prevent the code from being used directly from Java.
>
> Agreeing the codes is also not easy - what constitutes a test failure?
>
> On 03/10/2007, pbm-rico@gmx.net <pb...@gmx.net> wrote:
> > Actually, returning an error code in Java is pretty straight forward:
> >
> > System.exit(123); // or whatever code you want.
> >
> > All you have to do is agree on the error code(s).
> >
> > -Rico
> >
> > -------- Original-Nachricht --------
> > > Datum: Wed, 3 Oct 2007 22:02:38 +0100
> > > Von: sebb <se...@gmail.com>
> > > An: "JMeter Users List" <jm...@jakarta.apache.org>
> > > Betreff: Re: jmeter.bat: Special exit code when test plan run fails?
> >
> > > This is a problem with Java applications.
> > >
> > > In general it is very difficult to return a meaningful code from Java
> > > applications.
> > >
> > >
> > > On 03/10/2007, pbm-rico@gmx.net <pb...@gmx.net> wrote:
> > > > Hi all,
> > > >
> > > > I was wondering if there is a special exit code for jmeter.[bat|sh]
> and
> > > the like when a test fails.
> > > >
> > > > For example if I run this on Windows, the exit code is '0' although
> some
> > > tests failed:
> > > >
> > > > c:\jakarta-jmeter-2.3RC4\bin>jmeter.bat -n -t ..\extras\Test.jmx
> -l
> > > shorttest.log
> > > > Created the tree successfully
> > > > Starting the test @ Wed Oct 03 13:10:00 PDT 2007 (1191442200884)
> > > > Tidying up ... @ Wed Oct 03 13:10:04 PDT 2007 (1191442204589)
> > > > ... end of run
> > > >
> > > > c:\jakarta-jmeter-2.3RC4\bin>echo %errorlevel%
> > > > 0
> > > >
> > > > The failure shows up nicely in the HTML report that I can generate
> with
> > > the XSLT stylesheet, but it would be extremely helpful if the command
> would
> > > return a non-zero exit code if a test fails. This allows me to better
> > > integrate JMeter in our test automation framework.
> > > >
> > > > Thanks in advance,
> > > > Rico.
> > > >
> > > >
> > > > --
> > > > Psssst! Schon vom neuen GMX MultiMessenger gehört?
> > > > Der kanns mit allen: http://www.gmx.net/de/go/multimessenger
> > > >
> > > >
> ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> >
> > --
> > Psssst! Schon vom neuen GMX MultiMessenger gehört?
> > Der kanns mit allen: http://www.gmx.net/de/go/multimessenger
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
--
GMX FreeMail: 1 GB Postfach, 5 E-Mail-Adressen, 10 Free SMS.
Alle Infos und kostenlose Anmeldung: http://www.gmx.net/de/go/freemail
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
Re: jmeter.bat: Special exit code when test plan run fails?
Posted by sebb <se...@gmail.com>.
Yes, but:
* ensuring that the exit does not prevent other threads from finishing
normally may not be trivial;
* it would prevent the code from being used directly from Java.
Agreeing the codes is also not easy - what constitutes a test failure?
On 03/10/2007, pbm-rico@gmx.net <pb...@gmx.net> wrote:
> Actually, returning an error code in Java is pretty straight forward:
>
> System.exit(123); // or whatever code you want.
>
> All you have to do is agree on the error code(s).
>
> -Rico
>
> -------- Original-Nachricht --------
> > Datum: Wed, 3 Oct 2007 22:02:38 +0100
> > Von: sebb <se...@gmail.com>
> > An: "JMeter Users List" <jm...@jakarta.apache.org>
> > Betreff: Re: jmeter.bat: Special exit code when test plan run fails?
>
> > This is a problem with Java applications.
> >
> > In general it is very difficult to return a meaningful code from Java
> > applications.
> >
> >
> > On 03/10/2007, pbm-rico@gmx.net <pb...@gmx.net> wrote:
> > > Hi all,
> > >
> > > I was wondering if there is a special exit code for jmeter.[bat|sh] and
> > the like when a test fails.
> > >
> > > For example if I run this on Windows, the exit code is '0' although some
> > tests failed:
> > >
> > > c:\jakarta-jmeter-2.3RC4\bin>jmeter.bat -n -t ..\extras\Test.jmx -l
> > shorttest.log
> > > Created the tree successfully
> > > Starting the test @ Wed Oct 03 13:10:00 PDT 2007 (1191442200884)
> > > Tidying up ... @ Wed Oct 03 13:10:04 PDT 2007 (1191442204589)
> > > ... end of run
> > >
> > > c:\jakarta-jmeter-2.3RC4\bin>echo %errorlevel%
> > > 0
> > >
> > > The failure shows up nicely in the HTML report that I can generate with
> > the XSLT stylesheet, but it would be extremely helpful if the command would
> > return a non-zero exit code if a test fails. This allows me to better
> > integrate JMeter in our test automation framework.
> > >
> > > Thanks in advance,
> > > Rico.
> > >
> > >
> > > --
> > > Psssst! Schon vom neuen GMX MultiMessenger gehört?
> > > Der kanns mit allen: http://www.gmx.net/de/go/multimessenger
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
> --
> Psssst! Schon vom neuen GMX MultiMessenger gehört?
> Der kanns mit allen: http://www.gmx.net/de/go/multimessenger
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
Re: jmeter.bat: Special exit code when test plan run fails?
Posted by pb...@gmx.net.
Actually, returning an error code in Java is pretty straight forward:
System.exit(123); // or whatever code you want.
All you have to do is agree on the error code(s).
-Rico
-------- Original-Nachricht --------
> Datum: Wed, 3 Oct 2007 22:02:38 +0100
> Von: sebb <se...@gmail.com>
> An: "JMeter Users List" <jm...@jakarta.apache.org>
> Betreff: Re: jmeter.bat: Special exit code when test plan run fails?
> This is a problem with Java applications.
>
> In general it is very difficult to return a meaningful code from Java
> applications.
>
>
> On 03/10/2007, pbm-rico@gmx.net <pb...@gmx.net> wrote:
> > Hi all,
> >
> > I was wondering if there is a special exit code for jmeter.[bat|sh] and
> the like when a test fails.
> >
> > For example if I run this on Windows, the exit code is '0' although some
> tests failed:
> >
> > c:\jakarta-jmeter-2.3RC4\bin>jmeter.bat -n -t ..\extras\Test.jmx -l
> shorttest.log
> > Created the tree successfully
> > Starting the test @ Wed Oct 03 13:10:00 PDT 2007 (1191442200884)
> > Tidying up ... @ Wed Oct 03 13:10:04 PDT 2007 (1191442204589)
> > ... end of run
> >
> > c:\jakarta-jmeter-2.3RC4\bin>echo %errorlevel%
> > 0
> >
> > The failure shows up nicely in the HTML report that I can generate with
> the XSLT stylesheet, but it would be extremely helpful if the command would
> return a non-zero exit code if a test fails. This allows me to better
> integrate JMeter in our test automation framework.
> >
> > Thanks in advance,
> > Rico.
> >
> >
> > --
> > Psssst! Schon vom neuen GMX MultiMessenger gehört?
> > Der kanns mit allen: http://www.gmx.net/de/go/multimessenger
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
--
Psssst! Schon vom neuen GMX MultiMessenger gehört?
Der kanns mit allen: http://www.gmx.net/de/go/multimessenger
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
Re: jmeter.bat: Special exit code when test plan run fails?
Posted by sebb <se...@gmail.com>.
This is a problem with Java applications.
In general it is very difficult to return a meaningful code from Java
applications.
On 03/10/2007, pbm-rico@gmx.net <pb...@gmx.net> wrote:
> Hi all,
>
> I was wondering if there is a special exit code for jmeter.[bat|sh] and the like when a test fails.
>
> For example if I run this on Windows, the exit code is '0' although some tests failed:
>
> c:\jakarta-jmeter-2.3RC4\bin>jmeter.bat -n -t ..\extras\Test.jmx -l shorttest.log
> Created the tree successfully
> Starting the test @ Wed Oct 03 13:10:00 PDT 2007 (1191442200884)
> Tidying up ... @ Wed Oct 03 13:10:04 PDT 2007 (1191442204589)
> ... end of run
>
> c:\jakarta-jmeter-2.3RC4\bin>echo %errorlevel%
> 0
>
> The failure shows up nicely in the HTML report that I can generate with the XSLT stylesheet, but it would be extremely helpful if the command would return a non-zero exit code if a test fails. This allows me to better integrate JMeter in our test automation framework.
>
> Thanks in advance,
> Rico.
>
>
> --
> Psssst! Schon vom neuen GMX MultiMessenger gehört?
> Der kanns mit allen: http://www.gmx.net/de/go/multimessenger
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: jmeter-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jmeter-user-help@jakarta.apache.org