You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jmeter-dev@jakarta.apache.org by Ilia Iourovitski <ii...@yahoo.com> on 2002/07/10 23:27:24 UTC
Re: JMeter variables and the beginnings of a JMeter scripting languag e
Isn't it better to embed Jython into JMeter like MAXQ
does.
http://www.bitmechanic.com/projects/maxq/
It will let users write real scripts. After all JMeter
about measurements. All tools like Silk Performer has
it. And it is much easier to debug jython script than
long set of tests.
Ilia
--- "Stover, Michael" <Mi...@usa.xerox.com>
wrote:
> Recently I've noticed increased interest in making
> JMeter react more
> dynamically during test runs. This seems only
> natural. The HTML Link
> Parser provides some support, but really only seems
> to wet the appetite for
> more powerful options.
>
> So, I'm starting on a more general solution and
> thought some of you might
> want to give some input.
>
> I would like to allow users to embed simple
> reference elements as values in
> config and sampler objects, and then have a separate
> place where the value
> of these referenced elements would be specified.
> Currently, people use a
> Default Config Object to set universal values such
> as DOMAIN and PORT. I
> think a reference replacement system might be more
> intuitive than the
> current overlay system. Rather than leaving the
> DOMAIN field blank and
> supplying a Default Config in the correct place in
> the tree, one would
> instead insert a reference - such as ${my_domain} -
> which would be replaced
> when the test is compiled. Elsewhere, a table of
> all reference +
> replacement values would allow the user to change
> all such values in one
> location.
>
> This has many advantages over the current situation
> - it's generic, meaning
> one can use this system for any protocol.
> Currently, there is an HTTP
> Defaults, and FTP Defaults, etc.
> It allows all such elements to have one centralized
> home which should be
> easier to manage.
>
> For such user-defined values, this is simple. I've
> taken the variable style
> from Ant - ${var-name} - because I like it. Much
> preferable to XML IMO. A
> bit more explicit than a Perl style as well.
> Thoughts on this format would
> be much appreciated. Anything simpler than what
> I've come up with that does
> the job.
>
> Certain built-in values would be equally simple:
> ${__threadNumber}
> ${__threadName}
> ${__timestamp}
> ${__counter(0,100,1)}
> ${__xmlFile(c:\jmeter\bin\users.xml,username)}
> ${__literal[[${ant-variable}]]}
>
> Well, some are not so simple, but I'm hoping you get
> the idea.
> ${__threadNumber} is replaced by the current
> thread's number. I'm using
> "__" to indicate a built-in value as opposed to a
> user-defined variable
> name.
>
> ${__counter(0,100,1)} - for those familiar with the
> HTTP Parameter Mask,
> this might look familiar. The first number gives
> the starting point, then
> the end point, and the last is the increment. I
> think it's another way of
> getting the same effect. By embedding this variable
> into an argument name
> or value, you can get a similar effect:
>
> NAME PARAMETER
> username user${__counter(0,100,1)}
>
> This would create arguments in the following manner:
> username=user0
> username=user1
> username=user2
> etc...
>
> Similarly, I see
> ${__xmlFile(c:\jmeter\bin\users.xml,username)} as a
> replacement for the User Parameter Modifier, though
> some complications have
> to be worked out.
>
> And ${__literal[[Anything at all]]} allows a user to
> use values that might
> otherwise conflict with this system. Although, that
> should be very rare,
> because variables not found in the table would be
> passed on as is.
>
> This is getting long, but I've implemented a more
> complex built-in variable
> that allows users to scrape values from responses
> using regular expressions.
> Right now, it takes the following form:
>
>
${__responseVariable(REGULAR_EXPRESSION,prefix$1sometext$2suffix,[ALL|#|RAND
> |0.###],[between])}
>
> Take it slow. From the beginning, "__" indicates a
> built-in variable.
> "responseVariable" is the name used to find the
> correct Java class to
> execute. Then there are parentheses with arguments
> - just like the
> __counter(0,100,1) example above. There are 4
> possible arguments, but only
> the first 2 are required (thus the square brackets
> showing optional args).
>
> The first argument is a regular expression, Perl
> style. This regular
> expression will be applied to the response data
> received from the previous
> request.
>
> The second argument provides a template for
> generating the string that will
> be substituted for this whole mess at run time. The
> variables ($1,$2) refer
> to regular expression groups (see Perl docs for more
> info).
>
> The next argument indicates which match, or matches,
> to use - if your
> regular expression finds more than one match in the
> data. ALL means use
> all. A number (starting from 1) indicates which
> value by it's order in the
> response. Thus, if you find 5 successful matches, a
> number of "3" means to
> use the third match. RAND means pick one at random.
> A float value is also
> a possible value.
>
> The fourth argument is a static string to go between
> matched values if ALL
> was chosen as the third argument value.
>
> Somewhat complex, though I envision a dialog
> allowing users to input these
> arguments in a friendly manner and then generating
> this string for them.
>
> -Mike
>
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>
__________________________________________________
Do You Yahoo!?
Sign up for SBC Yahoo! Dial - First Month Free
http://sbc.yahoo.com
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: JMeter variables and the beginnings of a JMeter scripting languag e
Posted by Mike Stover <ms...@apache.org>.
On 10 Jul 2002 at 19:13, Berin Loritsch wrote:
> > [snip]
>
> I agree with you to a point. Keeping in mind who your user
> is, we can provide the GUI that builds the script under the
> hood. Either that, or the tests are converted on the fly to
> the scripting language in question.
I could create some new components that allowed the JMeter user to choose different
implementations of JMeterThread. By doing so, new implementations of JMeterThread could
be written, including one that provided such bindings to Jython and the like. Thus, a user
could write arbitrary Jython code in the GUI, and then submit it to be run within a special
implementation of JMeterThread which provided access to all the other JMeter resources, like
listeners, controllers, timers, etc.
This way, there's no need to write a code generation engine (whew!), and the underlying
language of JMeter doesn't have to change. And better yet, I can pawn off the work involved
to someone who wants this feature - and who is more familiar with Jython.
I may have used a poor choice of words by referring to what I'm doing as a "scripting
language". In reality, I'm providing static variables and built-in functions for use within test
elements. These functions only provide substitute values within the configuration data - there
is no control of program flow, and no actions can be taken. I apologize for being misleading,
but I hadn't really thought it entirely through. I'm believe that what I'm trying to provide is very
different from what you and Llia are talking about.
-Mike
--
Michael Stover
mstover1@apache.org
Yahoo IM: mstover_ya
ICQ: 152975688
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: JMeter variables and the beginnings of a JMeter scripting languag e
Posted by Ilia Iourovitski <ii...@yahoo.com>.
Typically the following features are highly desirable:
1. Test/scenario skeleton generation. MAXQ does it
easily.
1: # imports
2: from com.bitmechanic.maxq import HttpTestCase
3: from junit.textui import TestRunner
4: from java.util import HashMap
5:
6: # defintition of test class
7: class MaxQTest(HttpTestCase):
8: def __init__(self):
9: HttpTestCase.__init__(self, "")
10:
11: def runTest(self):
12: self.get("http://www.google.com/")
13: self.assertEquals(200,
self.getResponse().getStatusCode())
14:
15:
self.get("http://www.google.com/images/logo.gif")
16: self.assertEquals(304,
self.getResponse().getStatusCode())
17:
18:
19: map = HashMap()
20: map.put("hl", "en")
21: map.put("q", "testing+tools")
22: self.get("http://www.google.com/search",
map)
23: self.assertEquals(200,
self.getResponse().getStatusCode())
24:
25: ##########################################
26:
27: # Code to load and run the test
28: test = MaxQTest()
29: test.runTest()
2. adding assertions.
3. Instrumenting for performance measurements.
adding
begin = System.currentTimeMillis()
self.post(self.url + self.path, map)
interval1 = System.currentTimeMillis() - begin
is boring.
4. wrting loader which can create n threads and run m
times.
5. reporting.
Looks like if JMeter API will be published, 3,4,5 can
be done easily. And if HTTPProxy can generate scripts
it will be great.
Thanks
Ilia
--- Mike Stover <ms...@apache.org> wrote:
> I would consider such an implementation as a new
> "protocol" for JMeter. Currently
> JMeter supports HTTP pretty well, also supports JDBC
> and FTP less thoroughly. New
> "protocols" have recently been added, such as SOAP
> and arbitrary Java objects.
>
> The value of being able to create Jython scripts
> that are then executed within the
> framework of JMeter is obvious - but it's not a
> replacement for the type of system I'm
> talking about. Not everyone who tests applications
> is a programmer. I'd like JMeter to
> be usable by such people. Furthermore, why would
> you want to write you're own regular
> expression routines and HTTP sampling routines if
> you didn't have to? If a Jython
> protocol were implemented, allowing people to write
> arbitrary Jython code to be
> "sampled", you'd have what you want, plus, you'd
> have access to some built-in JMeter
> values.
>
> -Mike
>
> On 10 Jul 2002 at 14:27, Ilia Iourovitski wrote:
>
> > Isn't it better to embed Jython into JMeter like
> MAXQ
> > does.
> > http://www.bitmechanic.com/projects/maxq/
> > It will let users write real scripts. After all
> JMeter
> > about measurements. All tools like Silk Performer
> has
> > it. And it is much easier to debug jython script
> than
> > long set of tests.
> >
> > Ilia
> >
> >
> > --- "Stover, Michael"
> <Mi...@usa.xerox.com>
> > wrote:
> > > Recently I've noticed increased interest in
> making
> > > JMeter react more
> > > dynamically during test runs. This seems only
> > > natural. The HTML Link
> > > Parser provides some support, but really only
> seems
> > > to wet the appetite for
> > > more powerful options.
> > >
> > > So, I'm starting on a more general solution and
> > > thought some of you might
> > > want to give some input.
> > >
> > > I would like to allow users to embed simple
> > > reference elements as values in
> > > config and sampler objects, and then have a
> separate
> > > place where the value
> > > of these referenced elements would be specified.
>
> > > Currently, people use a
> > > Default Config Object to set universal values
> such
> > > as DOMAIN and PORT. I
> > > think a reference replacement system might be
> more
> > > intuitive than the
> > > current overlay system. Rather than leaving the
> > > DOMAIN field blank and
> > > supplying a Default Config in the correct place
> in
> > > the tree, one would
> > > instead insert a reference - such as
> ${my_domain} -
> > > which would be replaced
> > > when the test is compiled. Elsewhere, a table
> of
> > > all reference +
> > > replacement values would allow the user to
> change
> > > all such values in one
> > > location.
> > >
> > > This has many advantages over the current
> situation
> > > - it's generic, meaning
> > > one can use this system for any protocol.
> > > Currently, there is an HTTP
> > > Defaults, and FTP Defaults, etc.
> > > It allows all such elements to have one
> centralized
> > > home which should be
> > > easier to manage.
> > >
> > > For such user-defined values, this is simple.
> I've
> > > taken the variable style
> > > from Ant - ${var-name} - because I like it.
> Much
> > > preferable to XML IMO. A
> > > bit more explicit than a Perl style as well.
> > > Thoughts on this format would
> > > be much appreciated. Anything simpler than what
> > > I've come up with that does
> > > the job.
> > >
> > > Certain built-in values would be equally simple:
> > > ${__threadNumber}
> > > ${__threadName}
> > > ${__timestamp}
> > > ${__counter(0,100,1)}
> > > ${__xmlFile(c:\jmeter\bin\users.xml,username)}
> > > ${__literal[[${ant-variable}]]}
> > >
> > > Well, some are not so simple, but I'm hoping you
> get
> > > the idea.
> > > ${__threadNumber} is replaced by the current
> > > thread's number. I'm using
> > > "__" to indicate a built-in value as opposed to
> a
> > > user-defined variable
> > > name.
> > >
> > > ${__counter(0,100,1)} - for those familiar with
> the
> > > HTTP Parameter Mask,
> > > this might look familiar. The first number
> gives
> > > the starting point, then
> > > the end point, and the last is the increment. I
> > > think it's another way of
> > > getting the same effect. By embedding this
> variable
> > > into an argument name
> > > or value, you can get a similar effect:
> > >
> > > NAME PARAMETER
> > > username user${__counter(0,100,1)}
> > >
> > > This would create arguments in the following
> manner:
> > > username=user0
> > > username=user1
> > > username=user2
> > > etc...
> > >
> > > Similarly, I see
> > > ${__xmlFile(c:\jmeter\bin\users.xml,username)}
> as a
> > > replacement for the User Parameter Modifier,
> though
> > > some complications have
> > > to be worked out.
> > >
> > > And ${__literal[[Anything at all]]} allows a
> user to
> > > use values that might
> > > otherwise conflict with this system. Although,
> that
> > > should be very rare,
> > > because variables not found in the table would
> be
> > > passed on as is.
> > >
> > > This is getting long, but I've implemented a
> more
> > > complex built-in variable
> > > that allows users to scrape values from
> responses
> > > using regular expressions.
> > > Right now, it takes the following form:
> > >
> > >
> >
>
${__responseVariable(REGULAR_EXPRESSION,prefix$1sometext$2suffix,[ALL|#|RAND
> > > |0.###],[between])}
> > >
> > > Take it slow. From the beginning, "__"
> indicates a
> > > built-in variable.
> > > "responseVariable" is the name used to find the
> > > correct Java class to
> > > execute. Then there are parentheses with
> arguments
> > > - just like the
> > > __counter(0,100,1) example above. There are 4
> > > possible arguments, but only
> > > the first 2 are required (thus the square
> brackets
> > > showing optional args).
> > >
> > > The first argument is a regular expression, Perl
> > > style. This regular
> > > expression will be applied to the response data
>
=== message truncated ===
__________________________________________________
Do You Yahoo!?
Sign up for SBC Yahoo! Dial - First Month Free
http://sbc.yahoo.com
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: JMeter variables and the beginnings of a JMeter scripting languag e
Posted by Mike Stover <ms...@apache.org>.
Hi Kevin,
I have made a 1.7.2 unstable release, but the documentation still needs some
bringing up to date. in particular, the components_reference page and the gif images.
The migration to avalon hasn't really happened. What I did was work out a more
complete separation of GUI code from test element code, and I made use of the Avalon
Configuration classes to save and load test scripts as XML. The new code is in the
"src_1" directory. "src" is obsolete.
The base interfaces are
org.apache.jmeter.testelement.TestElement and
org.apache.jmeter.gui.JMeterGUIComponent
But there are many abstract classes that make it easier to write Samplers, Visualizers,
controllers, assertions, etc.
I'm sure your password will still work.
-Mike
On 15 Jul 2002 at 21:04, kevin hammond wrote:
> Hi Mike and others,
>
> I have been out of the JMeter loop for awhile and
> would like to start contributing documentation and
> eventually code. (Guess I should first check to see
> that my login still works...)
>
> Let me know which areas of the documentation need
> updated. I have no problem digging through the code.
>
> Have we finished (started?) the migration to Jakarta
> Avalon (right?)? Before I left, that was the big
> thing we were talking about.
>
> Thanks,
> Kevin Hammond
>
> --- Mike Stover <ms...@apache.org> wrote:
> > Refining, fully documenting, and publishing a
> > carefully selected subset of the JMeter API
> > would indeed be a good thing. I'd love to have the
> > time (and ability!). I have great trouble
> > keeping the user documentation up-to-date in
> > addition to writing the code. Up till now, I have
> > generally relied on the brilliance of developers out
> > there to understand my crazy code
> > schemes - which has worked out ok. I will add it to
> > my mental todo list though.
> >
> > -Mike
> >
> > On 10 Jul 2002 at 16:27, Ilia Iourovitski wrote:
> >
> > > Looks like it is time to publish or refine JMeter
> > API
> > > for "advanced" users like M$ WEB stress tool does.
> >
> > > All IDE has limitations and I wonder if Avalon can
> > be
> > > used as generic component model like M$ COM.
> > >
> > > Ilia
> > > --- Berin Loritsch <bl...@apache.org> wrote:
> > > > > From: Mike Stover [mailto:mstover1@apache.org]
> >
> > > > >
> > > > > I would consider such an implementation as a
> > new
> > > > "protocol"
> > > > > for JMeter. Currently
> > > > > JMeter supports HTTP pretty well, also
> > supports
> > > > JDBC and FTP
> > > > > less thoroughly. New
> > > > > "protocols" have recently been added, such as
> > SOAP
> > > > and
> > > > > arbitrary Java objects.
> > > > >
> > > > > The value of being able to create Jython
> > scripts
> > > > that are
> > > > > then executed within the
> > > > > framework of JMeter is obvious - but it's not
> > a
> > > > replacement
> > > > > for the type of system I'm
> > > > > talking about. Not everyone who tests
> > > > applications is a
> > > > > programmer. I'd like JMeter to
> > > > > be usable by such people. Furthermore, why
> > would
> > > > you want to
> > > > > write you're own regular
> > > > > expression routines and HTTP sampling routines
> > if
> > > > you didn't
> > > > > have to? If a Jython
> > > > > protocol were implemented, allowing people to
> > > > write arbitrary
> > > > > Jython code to be
> > > > > "sampled", you'd have what you want, plus,
> > you'd
> > > > have access
> > > > > to some built-in JMeter
> > > > > values.
> > > >
> > > > I agree with you to a point. Keeping in mind
> > who
> > > > your user
> > > > is, we can provide the GUI that builds the
> > script
> > > > under the
> > > > hood. Either that, or the tests are converted
> > on
> > > > the fly to
> > > > the scripting language in question.
> > > >
> > > > Jython is one option, Scheme another (we all run
> > > > screaming), and
> > > > JavaScript is even another.
> > > >
> > > > The important thing is providing the bindings to
> > the
> > > > languages
> > > > in question. Generative programming is the
> > process
> > > > of creating
> > > > the script, or even java bytecode, as it is
> > needed.
> > > > If a
> > > > programmer wants native access to the scripting
> > > > language, they
> > > > are welcome to it. Otherwise, the test
> > > > configuration gets
> > > > transformed into the script at runtime.
> > > >
> > > > One thing that would really be neat to try is
> > the
> > > > JESS (Java
> > > > Expert System Shell). It is based on Scheme,
> > but it
> > > > allows
> > > > you to create an intelligent expert system based
> > on
> > > > the results
> > > > as they are sampled. It can be integrated quite
> > > > well.
> > > >
> > > > Since all developers have their pet languages,
> > we
> > > > should devise a
> > > > component that takes care of the process of
> > bindng
> > > > JMeter values
> > > > to the scripting engine. From that point on, it
> > is
> > > > just a matter
> > > > of executing the script.
> > > >
> > > >
> > > > --
> > > > To unsubscribe, e-mail:
> > > >
> > <ma...@jakarta.apache.org>
> > > > For additional commands, e-mail:
> > > > <ma...@jakarta.apache.org>
> > > >
> > >
> > >
> > > __________________________________________________
> > > Do You Yahoo!?
> > > Sign up for SBC Yahoo! Dial - First Month Free
> > > http://sbc.yahoo.com
> > >
> > > --
> > > To unsubscribe, e-mail:
> > <ma...@jakarta.apache.org>
> > > For additional commands, e-mail:
> > <ma...@jakarta.apache.org>
> > >
> >
> >
> >
> > --
> > Michael Stover
> > mstover1@apache.org
> > Yahoo IM: mstover_ya
> > ICQ: 152975688
> >
> > --
> > To unsubscribe, e-mail:
> > <ma...@jakarta.apache.org>
> > For additional commands, e-mail:
> > <ma...@jakarta.apache.org>
> >
>
>
> __________________________________________________
> Do You Yahoo!?
> Sign up for SBC Yahoo! Dial - First Month Free
> http://sbc.yahoo.com
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
Michael Stover
mstover1@apache.org
Yahoo IM: mstover_ya
ICQ: 152975688
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: JMeter variables and the beginnings of a JMeter scripting languag e
Posted by kevin hammond <ke...@yahoo.com>.
Hi Mike and others,
I have been out of the JMeter loop for awhile and
would like to start contributing documentation and
eventually code. (Guess I should first check to see
that my login still works...)
Let me know which areas of the documentation need
updated. I have no problem digging through the code.
Have we finished (started?) the migration to Jakarta
Avalon (right?)? Before I left, that was the big
thing we were talking about.
Thanks,
Kevin Hammond
--- Mike Stover <ms...@apache.org> wrote:
> Refining, fully documenting, and publishing a
> carefully selected subset of the JMeter API
> would indeed be a good thing. I'd love to have the
> time (and ability!). I have great trouble
> keeping the user documentation up-to-date in
> addition to writing the code. Up till now, I have
> generally relied on the brilliance of developers out
> there to understand my crazy code
> schemes - which has worked out ok. I will add it to
> my mental todo list though.
>
> -Mike
>
> On 10 Jul 2002 at 16:27, Ilia Iourovitski wrote:
>
> > Looks like it is time to publish or refine JMeter
> API
> > for "advanced" users like M$ WEB stress tool does.
>
> > All IDE has limitations and I wonder if Avalon can
> be
> > used as generic component model like M$ COM.
> >
> > Ilia
> > --- Berin Loritsch <bl...@apache.org> wrote:
> > > > From: Mike Stover [mailto:mstover1@apache.org]
>
> > > >
> > > > I would consider such an implementation as a
> new
> > > "protocol"
> > > > for JMeter. Currently
> > > > JMeter supports HTTP pretty well, also
> supports
> > > JDBC and FTP
> > > > less thoroughly. New
> > > > "protocols" have recently been added, such as
> SOAP
> > > and
> > > > arbitrary Java objects.
> > > >
> > > > The value of being able to create Jython
> scripts
> > > that are
> > > > then executed within the
> > > > framework of JMeter is obvious - but it's not
> a
> > > replacement
> > > > for the type of system I'm
> > > > talking about. Not everyone who tests
> > > applications is a
> > > > programmer. I'd like JMeter to
> > > > be usable by such people. Furthermore, why
> would
> > > you want to
> > > > write you're own regular
> > > > expression routines and HTTP sampling routines
> if
> > > you didn't
> > > > have to? If a Jython
> > > > protocol were implemented, allowing people to
> > > write arbitrary
> > > > Jython code to be
> > > > "sampled", you'd have what you want, plus,
> you'd
> > > have access
> > > > to some built-in JMeter
> > > > values.
> > >
> > > I agree with you to a point. Keeping in mind
> who
> > > your user
> > > is, we can provide the GUI that builds the
> script
> > > under the
> > > hood. Either that, or the tests are converted
> on
> > > the fly to
> > > the scripting language in question.
> > >
> > > Jython is one option, Scheme another (we all run
> > > screaming), and
> > > JavaScript is even another.
> > >
> > > The important thing is providing the bindings to
> the
> > > languages
> > > in question. Generative programming is the
> process
> > > of creating
> > > the script, or even java bytecode, as it is
> needed.
> > > If a
> > > programmer wants native access to the scripting
> > > language, they
> > > are welcome to it. Otherwise, the test
> > > configuration gets
> > > transformed into the script at runtime.
> > >
> > > One thing that would really be neat to try is
> the
> > > JESS (Java
> > > Expert System Shell). It is based on Scheme,
> but it
> > > allows
> > > you to create an intelligent expert system based
> on
> > > the results
> > > as they are sampled. It can be integrated quite
> > > well.
> > >
> > > Since all developers have their pet languages,
> we
> > > should devise a
> > > component that takes care of the process of
> bindng
> > > JMeter values
> > > to the scripting engine. From that point on, it
> is
> > > just a matter
> > > of executing the script.
> > >
> > >
> > > --
> > > To unsubscribe, e-mail:
> > >
> <ma...@jakarta.apache.org>
> > > For additional commands, e-mail:
> > > <ma...@jakarta.apache.org>
> > >
> >
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Sign up for SBC Yahoo! Dial - First Month Free
> > http://sbc.yahoo.com
> >
> > --
> > To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> > For additional commands, e-mail:
> <ma...@jakarta.apache.org>
> >
>
>
>
> --
> Michael Stover
> mstover1@apache.org
> Yahoo IM: mstover_ya
> ICQ: 152975688
>
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>
__________________________________________________
Do You Yahoo!?
Sign up for SBC Yahoo! Dial - First Month Free
http://sbc.yahoo.com
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: JMeter variables and the beginnings of a JMeter scripting languag e
Posted by Mike Stover <ms...@apache.org>.
Refining, fully documenting, and publishing a carefully selected subset of the JMeter API
would indeed be a good thing. I'd love to have the time (and ability!). I have great trouble
keeping the user documentation up-to-date in addition to writing the code. Up till now, I have
generally relied on the brilliance of developers out there to understand my crazy code
schemes - which has worked out ok. I will add it to my mental todo list though.
-Mike
On 10 Jul 2002 at 16:27, Ilia Iourovitski wrote:
> Looks like it is time to publish or refine JMeter API
> for "advanced" users like M$ WEB stress tool does.
> All IDE has limitations and I wonder if Avalon can be
> used as generic component model like M$ COM.
>
> Ilia
> --- Berin Loritsch <bl...@apache.org> wrote:
> > > From: Mike Stover [mailto:mstover1@apache.org]
> > >
> > > I would consider such an implementation as a new
> > "protocol"
> > > for JMeter. Currently
> > > JMeter supports HTTP pretty well, also supports
> > JDBC and FTP
> > > less thoroughly. New
> > > "protocols" have recently been added, such as SOAP
> > and
> > > arbitrary Java objects.
> > >
> > > The value of being able to create Jython scripts
> > that are
> > > then executed within the
> > > framework of JMeter is obvious - but it's not a
> > replacement
> > > for the type of system I'm
> > > talking about. Not everyone who tests
> > applications is a
> > > programmer. I'd like JMeter to
> > > be usable by such people. Furthermore, why would
> > you want to
> > > write you're own regular
> > > expression routines and HTTP sampling routines if
> > you didn't
> > > have to? If a Jython
> > > protocol were implemented, allowing people to
> > write arbitrary
> > > Jython code to be
> > > "sampled", you'd have what you want, plus, you'd
> > have access
> > > to some built-in JMeter
> > > values.
> >
> > I agree with you to a point. Keeping in mind who
> > your user
> > is, we can provide the GUI that builds the script
> > under the
> > hood. Either that, or the tests are converted on
> > the fly to
> > the scripting language in question.
> >
> > Jython is one option, Scheme another (we all run
> > screaming), and
> > JavaScript is even another.
> >
> > The important thing is providing the bindings to the
> > languages
> > in question. Generative programming is the process
> > of creating
> > the script, or even java bytecode, as it is needed.
> > If a
> > programmer wants native access to the scripting
> > language, they
> > are welcome to it. Otherwise, the test
> > configuration gets
> > transformed into the script at runtime.
> >
> > One thing that would really be neat to try is the
> > JESS (Java
> > Expert System Shell). It is based on Scheme, but it
> > allows
> > you to create an intelligent expert system based on
> > the results
> > as they are sampled. It can be integrated quite
> > well.
> >
> > Since all developers have their pet languages, we
> > should devise a
> > component that takes care of the process of bindng
> > JMeter values
> > to the scripting engine. From that point on, it is
> > just a matter
> > of executing the script.
> >
> >
> > --
> > To unsubscribe, e-mail:
> > <ma...@jakarta.apache.org>
> > For additional commands, e-mail:
> > <ma...@jakarta.apache.org>
> >
>
>
> __________________________________________________
> Do You Yahoo!?
> Sign up for SBC Yahoo! Dial - First Month Free
> http://sbc.yahoo.com
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
Michael Stover
mstover1@apache.org
Yahoo IM: mstover_ya
ICQ: 152975688
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: JMeter variables and the beginnings of a JMeter scripting languag e
Posted by Ilia Iourovitski <ii...@yahoo.com>.
Looks like it is time to publish or refine JMeter API
for "advanced" users like M$ WEB stress tool does.
All IDE has limitations and I wonder if Avalon can be
used as generic component model like M$ COM.
Ilia
--- Berin Loritsch <bl...@apache.org> wrote:
> > From: Mike Stover [mailto:mstover1@apache.org]
> >
> > I would consider such an implementation as a new
> "protocol"
> > for JMeter. Currently
> > JMeter supports HTTP pretty well, also supports
> JDBC and FTP
> > less thoroughly. New
> > "protocols" have recently been added, such as SOAP
> and
> > arbitrary Java objects.
> >
> > The value of being able to create Jython scripts
> that are
> > then executed within the
> > framework of JMeter is obvious - but it's not a
> replacement
> > for the type of system I'm
> > talking about. Not everyone who tests
> applications is a
> > programmer. I'd like JMeter to
> > be usable by such people. Furthermore, why would
> you want to
> > write you're own regular
> > expression routines and HTTP sampling routines if
> you didn't
> > have to? If a Jython
> > protocol were implemented, allowing people to
> write arbitrary
> > Jython code to be
> > "sampled", you'd have what you want, plus, you'd
> have access
> > to some built-in JMeter
> > values.
>
> I agree with you to a point. Keeping in mind who
> your user
> is, we can provide the GUI that builds the script
> under the
> hood. Either that, or the tests are converted on
> the fly to
> the scripting language in question.
>
> Jython is one option, Scheme another (we all run
> screaming), and
> JavaScript is even another.
>
> The important thing is providing the bindings to the
> languages
> in question. Generative programming is the process
> of creating
> the script, or even java bytecode, as it is needed.
> If a
> programmer wants native access to the scripting
> language, they
> are welcome to it. Otherwise, the test
> configuration gets
> transformed into the script at runtime.
>
> One thing that would really be neat to try is the
> JESS (Java
> Expert System Shell). It is based on Scheme, but it
> allows
> you to create an intelligent expert system based on
> the results
> as they are sampled. It can be integrated quite
> well.
>
> Since all developers have their pet languages, we
> should devise a
> component that takes care of the process of bindng
> JMeter values
> to the scripting engine. From that point on, it is
> just a matter
> of executing the script.
>
>
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>
__________________________________________________
Do You Yahoo!?
Sign up for SBC Yahoo! Dial - First Month Free
http://sbc.yahoo.com
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: JMeter variables and the beginnings of a JMeter scripting languag e
Posted by Berin Loritsch <bl...@apache.org>.
> From: Mike Stover [mailto:mstover1@apache.org]
>
> I would consider such an implementation as a new "protocol"
> for JMeter. Currently
> JMeter supports HTTP pretty well, also supports JDBC and FTP
> less thoroughly. New
> "protocols" have recently been added, such as SOAP and
> arbitrary Java objects.
>
> The value of being able to create Jython scripts that are
> then executed within the
> framework of JMeter is obvious - but it's not a replacement
> for the type of system I'm
> talking about. Not everyone who tests applications is a
> programmer. I'd like JMeter to
> be usable by such people. Furthermore, why would you want to
> write you're own regular
> expression routines and HTTP sampling routines if you didn't
> have to? If a Jython
> protocol were implemented, allowing people to write arbitrary
> Jython code to be
> "sampled", you'd have what you want, plus, you'd have access
> to some built-in JMeter
> values.
I agree with you to a point. Keeping in mind who your user
is, we can provide the GUI that builds the script under the
hood. Either that, or the tests are converted on the fly to
the scripting language in question.
Jython is one option, Scheme another (we all run screaming), and
JavaScript is even another.
The important thing is providing the bindings to the languages
in question. Generative programming is the process of creating
the script, or even java bytecode, as it is needed. If a
programmer wants native access to the scripting language, they
are welcome to it. Otherwise, the test configuration gets
transformed into the script at runtime.
One thing that would really be neat to try is the JESS (Java
Expert System Shell). It is based on Scheme, but it allows
you to create an intelligent expert system based on the results
as they are sampled. It can be integrated quite well.
Since all developers have their pet languages, we should devise a
component that takes care of the process of bindng JMeter values
to the scripting engine. From that point on, it is just a matter
of executing the script.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: JMeter variables and the beginnings of a JMeter scripting languag e
Posted by Mike Stover <ms...@apache.org>.
I would consider such an implementation as a new "protocol" for JMeter. Currently
JMeter supports HTTP pretty well, also supports JDBC and FTP less thoroughly. New
"protocols" have recently been added, such as SOAP and arbitrary Java objects.
The value of being able to create Jython scripts that are then executed within the
framework of JMeter is obvious - but it's not a replacement for the type of system I'm
talking about. Not everyone who tests applications is a programmer. I'd like JMeter to
be usable by such people. Furthermore, why would you want to write you're own regular
expression routines and HTTP sampling routines if you didn't have to? If a Jython
protocol were implemented, allowing people to write arbitrary Jython code to be
"sampled", you'd have what you want, plus, you'd have access to some built-in JMeter
values.
-Mike
On 10 Jul 2002 at 14:27, Ilia Iourovitski wrote:
> Isn't it better to embed Jython into JMeter like MAXQ
> does.
> http://www.bitmechanic.com/projects/maxq/
> It will let users write real scripts. After all JMeter
> about measurements. All tools like Silk Performer has
> it. And it is much easier to debug jython script than
> long set of tests.
>
> Ilia
>
>
> --- "Stover, Michael" <Mi...@usa.xerox.com>
> wrote:
> > Recently I've noticed increased interest in making
> > JMeter react more
> > dynamically during test runs. This seems only
> > natural. The HTML Link
> > Parser provides some support, but really only seems
> > to wet the appetite for
> > more powerful options.
> >
> > So, I'm starting on a more general solution and
> > thought some of you might
> > want to give some input.
> >
> > I would like to allow users to embed simple
> > reference elements as values in
> > config and sampler objects, and then have a separate
> > place where the value
> > of these referenced elements would be specified.
> > Currently, people use a
> > Default Config Object to set universal values such
> > as DOMAIN and PORT. I
> > think a reference replacement system might be more
> > intuitive than the
> > current overlay system. Rather than leaving the
> > DOMAIN field blank and
> > supplying a Default Config in the correct place in
> > the tree, one would
> > instead insert a reference - such as ${my_domain} -
> > which would be replaced
> > when the test is compiled. Elsewhere, a table of
> > all reference +
> > replacement values would allow the user to change
> > all such values in one
> > location.
> >
> > This has many advantages over the current situation
> > - it's generic, meaning
> > one can use this system for any protocol.
> > Currently, there is an HTTP
> > Defaults, and FTP Defaults, etc.
> > It allows all such elements to have one centralized
> > home which should be
> > easier to manage.
> >
> > For such user-defined values, this is simple. I've
> > taken the variable style
> > from Ant - ${var-name} - because I like it. Much
> > preferable to XML IMO. A
> > bit more explicit than a Perl style as well.
> > Thoughts on this format would
> > be much appreciated. Anything simpler than what
> > I've come up with that does
> > the job.
> >
> > Certain built-in values would be equally simple:
> > ${__threadNumber}
> > ${__threadName}
> > ${__timestamp}
> > ${__counter(0,100,1)}
> > ${__xmlFile(c:\jmeter\bin\users.xml,username)}
> > ${__literal[[${ant-variable}]]}
> >
> > Well, some are not so simple, but I'm hoping you get
> > the idea.
> > ${__threadNumber} is replaced by the current
> > thread's number. I'm using
> > "__" to indicate a built-in value as opposed to a
> > user-defined variable
> > name.
> >
> > ${__counter(0,100,1)} - for those familiar with the
> > HTTP Parameter Mask,
> > this might look familiar. The first number gives
> > the starting point, then
> > the end point, and the last is the increment. I
> > think it's another way of
> > getting the same effect. By embedding this variable
> > into an argument name
> > or value, you can get a similar effect:
> >
> > NAME PARAMETER
> > username user${__counter(0,100,1)}
> >
> > This would create arguments in the following manner:
> > username=user0
> > username=user1
> > username=user2
> > etc...
> >
> > Similarly, I see
> > ${__xmlFile(c:\jmeter\bin\users.xml,username)} as a
> > replacement for the User Parameter Modifier, though
> > some complications have
> > to be worked out.
> >
> > And ${__literal[[Anything at all]]} allows a user to
> > use values that might
> > otherwise conflict with this system. Although, that
> > should be very rare,
> > because variables not found in the table would be
> > passed on as is.
> >
> > This is getting long, but I've implemented a more
> > complex built-in variable
> > that allows users to scrape values from responses
> > using regular expressions.
> > Right now, it takes the following form:
> >
> >
> ${__responseVariable(REGULAR_EXPRESSION,prefix$1sometext$2suffix,[ALL|#|RAND
> > |0.###],[between])}
> >
> > Take it slow. From the beginning, "__" indicates a
> > built-in variable.
> > "responseVariable" is the name used to find the
> > correct Java class to
> > execute. Then there are parentheses with arguments
> > - just like the
> > __counter(0,100,1) example above. There are 4
> > possible arguments, but only
> > the first 2 are required (thus the square brackets
> > showing optional args).
> >
> > The first argument is a regular expression, Perl
> > style. This regular
> > expression will be applied to the response data
> > received from the previous
> > request.
> >
> > The second argument provides a template for
> > generating the string that will
> > be substituted for this whole mess at run time. The
> > variables ($1,$2) refer
> > to regular expression groups (see Perl docs for more
> > info).
> >
> > The next argument indicates which match, or matches,
> > to use - if your
> > regular expression finds more than one match in the
> > data. ALL means use
> > all. A number (starting from 1) indicates which
> > value by it's order in the
> > response. Thus, if you find 5 successful matches, a
> > number of "3" means to
> > use the third match. RAND means pick one at random.
> > A float value is also
> > a possible value.
> >
> > The fourth argument is a static string to go between
> > matched values if ALL
> > was chosen as the third argument value.
> >
> > Somewhat complex, though I envision a dialog
> > allowing users to input these
> > arguments in a friendly manner and then generating
> > this string for them.
> >
> > -Mike
> >
> > --
> > To unsubscribe, e-mail:
> > <ma...@jakarta.apache.org>
> > For additional commands, e-mail:
> > <ma...@jakarta.apache.org>
> >
>
> __________________________________________________
> Do You Yahoo!?
> Sign up for SBC Yahoo! Dial - First Month Free
> http://sbc.yahoo.com
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
--
Michael Stover
mstover1@apache.org
Yahoo IM: mstover_ya
ICQ: 152975688
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: JMeter variables and the beginnings of a JMeter scripting languag e
Posted by Berin Loritsch <bl...@apache.org>.
> From: Ilia Iourovitski [mailto:iiourov@yahoo.com]
>
> Isn't it better to embed Jython into JMeter like MAXQ
> does.
> http://www.bitmechanic.com/projects/maxq/
> It will let users write real scripts. After all JMeter
> about measurements. All tools like Silk Performer has
> it. And it is much easier to debug jython script than
> long set of tests.
I have to agree with Ilia. It would be a much better
use of your time to use an already developed scripting
language and merely provide hooks to bind to JMeter.
One big point that we need to make here is that as soon
as we start working with scripts, JMeter becomes more
a functional testing tool than a load testing tool.
There is a lot of need for functional testing. Another
point to tack on to that is that we can test how the
server degrades with more clients, but we can't trust
the numbers as actual hard numbers. We can only determine
trends.
That said, I think the best way to find the balance is
to #1) provide the scripting language bindings, and #2)
fine tune the reporting to reflect trends. For instance,
it would be really helpful to find the saturation curve
for a webapp. The scripting language can probably even
be used to cull the results and determine the ratio of
bad responses to good responses or other intelligent
data mining.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>