You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jmeter.apache.org by Rainer Jung <ra...@kippdata.de> on 2013/08/09 21:45:23 UTC
user.classpath, search_path and many jar files
I have a situation where the user.classpath contains many jar files
(about 100). Listing all of the jar files is tedious, difficult to
maintain and not in any place where one can configure user.classpath
possible or easy because of line length restrictions.
Looking at the code I saw that in some places jar files lying in
directories configured for class search are automatically also searched,
but sometimes not.
There are three cases:
- not auto-searching the jar files:
o.a.jmeter.JMeter start() calls updateClassLoader() calls updatePath()
for search_paths and user.classpath. updatePath() calls
NewDriver.addPath() for each path contained in those settings. Finally
NewDriver.addPath() adds them to the system property "java.class.path"
without listing all jar files contained in directories.
So the class loader itself does not know about such jar files.
- auto-searching such jar files in ClassFinder.findClassesThatExtend()
using search_paths for JMeter extension:
In findClassesThatExtend() a call to addJarsInPath() will automatically
add all jar files found in a configured directory.
- auto-searching jar files in user.classpath for JUnit setup:
In JUnitTestSamplerGui the user.classpath property is used to search for
classes extending Test or TestCase but the search is based on
ClassFinder, so will automatically search through jar files.
I propose to also add the jar files in the first case, the updatePath
setting up the class loader. I guess the only concern would be the speed
of class loading? IMHO if there are only few such jar files, then there
should be no problem and if there are many such jar files, then not
needing to configure them all is really helpful. So the remaining case
is if there are many jar files auto-detected which are not really
needed. That means it might make sense to make the behavior configurable
with default the old behavior.
Comments?
Regards,
Rainer
Re: user.classpath, search_path and many jar files
Posted by sebb <se...@gmail.com>.
On 12 August 2013 15:11, Philippe Mouawad <ph...@gmail.com> wrote:
> Regarding Property resolution, it comes from:
> TurnElementsOn#addNode:
>
> if (node instanceof TestElement && !(node instanceof TestPlan)) {
>
> I don't know why there is this condition :
>
> - !(node instanceof TestPlan)
>
> But it is the root cause of property not being translated.
>
>
> @sebb any idea why it exists ?
No, not offhand. But the TestPlan node is special.
The full answer might be in the SVN logs.
If so. it needs to be added as code comments so future maintainers can
understand what's happening.
> Regards
>
> Philippe
> On Mon, Aug 12, 2013 at 3:30 PM, Rainer Jung <ra...@kippdata.de>wrote:
>
>> On 12.08.2013 14:59, Philippe Mouawad wrote:
>> > Hello Rainer,
>> > So your patch only concerns Test Plan which do "Add directory or jar to
>> > classpath" ?
>> >
>> > If so, some notes about patch:
>> >
>> > - Shouldn't code be within try/catch ? as if a MalformedURLException
>> is
>> > thrown there are big chances that the rest won't work
>>
>> The NewDriver methods addURL() and addPath() have existing inconsistent
>> behavior. addPath() can throw the exception, so any call should be
>> inside try/catch, addURL() already does the try/catch internally so will
>> not throw it.
>>
>> > Regarding Bugzilla, I think it is better to open one for this
>> enhancement.
>>
>> Will do.
>>
>> Any idea about the property replacement:
>>
>> >> Now there's another thing that would be nice: being able to use a jmeter
>> >> property in the user defined class path in TestPlan.
>> >>
>> >> Unfortunately that property is used very early and the resolution of
>> >> properties returns with an "Not running version, return raw function
>> >> string" error from FunctionProperty.
>> >>
>> >> Is there a way to force a property replacement? Or how can we support
>> >> using a property in the user defined class path inside a test plan.
>> >>
>> >> The use case is somewhat clear: you want to be able to collect your
>> >> dependency jars in some directory. When you relocate the directory you
>> >> don't want to edit all our test plans referring that directory. Instead
>> >> use a property in the test plans and set it during jmeter startup to
>> >> point to your current jar file directory.
>> >>
>> >> Any hints would be helpful.
>>
>> Regards,
>>
>> Rainer
>>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
Re: user.classpath, search_path and many jar files
Posted by Philippe Mouawad <ph...@gmail.com>.
@Rainer, also not something about this enhancement, currently adding a JAR
or folder is not possible without first browsing, so adding afterwards a
property seems a bit weird as you cannot add a line and setup a path with
property without this browsing.
On Mon, Aug 12, 2013 at 4:11 PM, Philippe Mouawad <
philippe.mouawad@gmail.com> wrote:
> Regarding Property resolution, it comes from:
> TurnElementsOn#addNode:
>
> if (node instanceof TestElement && !(node instanceof TestPlan)) {
>
> I don't know why there is this condition :
>
> - !(node instanceof TestPlan)
>
> But it is the root cause of property not being translated.
>
>
> @sebb any idea why it exists ?
>
> Regards
>
> Philippe
> On Mon, Aug 12, 2013 at 3:30 PM, Rainer Jung <ra...@kippdata.de>wrote:
>
>> On 12.08.2013 14:59, Philippe Mouawad wrote:
>> > Hello Rainer,
>> > So your patch only concerns Test Plan which do "Add directory or jar to
>> > classpath" ?
>> >
>> > If so, some notes about patch:
>> >
>> > - Shouldn't code be within try/catch ? as if a MalformedURLException
>> is
>> > thrown there are big chances that the rest won't work
>>
>> The NewDriver methods addURL() and addPath() have existing inconsistent
>> behavior. addPath() can throw the exception, so any call should be
>> inside try/catch, addURL() already does the try/catch internally so will
>> not throw it.
>>
>> > Regarding Bugzilla, I think it is better to open one for this
>> enhancement.
>>
>> Will do.
>>
>> Any idea about the property replacement:
>>
>> >> Now there's another thing that would be nice: being able to use a
>> jmeter
>> >> property in the user defined class path in TestPlan.
>> >>
>> >> Unfortunately that property is used very early and the resolution of
>> >> properties returns with an "Not running version, return raw function
>> >> string" error from FunctionProperty.
>> >>
>> >> Is there a way to force a property replacement? Or how can we support
>> >> using a property in the user defined class path inside a test plan.
>> >>
>> >> The use case is somewhat clear: you want to be able to collect your
>> >> dependency jars in some directory. When you relocate the directory you
>> >> don't want to edit all our test plans referring that directory. Instead
>> >> use a property in the test plans and set it during jmeter startup to
>> >> point to your current jar file directory.
>> >>
>> >> Any hints would be helpful.
>>
>> Regards,
>>
>> Rainer
>>
>>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
>
>
--
Cordialement.
Philippe Mouawad.
Re: user.classpath, search_path and many jar files
Posted by Philippe Mouawad <ph...@gmail.com>.
Regarding Property resolution, it comes from:
TurnElementsOn#addNode:
if (node instanceof TestElement && !(node instanceof TestPlan)) {
I don't know why there is this condition :
- !(node instanceof TestPlan)
But it is the root cause of property not being translated.
@sebb any idea why it exists ?
Regards
Philippe
On Mon, Aug 12, 2013 at 3:30 PM, Rainer Jung <ra...@kippdata.de>wrote:
> On 12.08.2013 14:59, Philippe Mouawad wrote:
> > Hello Rainer,
> > So your patch only concerns Test Plan which do "Add directory or jar to
> > classpath" ?
> >
> > If so, some notes about patch:
> >
> > - Shouldn't code be within try/catch ? as if a MalformedURLException
> is
> > thrown there are big chances that the rest won't work
>
> The NewDriver methods addURL() and addPath() have existing inconsistent
> behavior. addPath() can throw the exception, so any call should be
> inside try/catch, addURL() already does the try/catch internally so will
> not throw it.
>
> > Regarding Bugzilla, I think it is better to open one for this
> enhancement.
>
> Will do.
>
> Any idea about the property replacement:
>
> >> Now there's another thing that would be nice: being able to use a jmeter
> >> property in the user defined class path in TestPlan.
> >>
> >> Unfortunately that property is used very early and the resolution of
> >> properties returns with an "Not running version, return raw function
> >> string" error from FunctionProperty.
> >>
> >> Is there a way to force a property replacement? Or how can we support
> >> using a property in the user defined class path inside a test plan.
> >>
> >> The use case is somewhat clear: you want to be able to collect your
> >> dependency jars in some directory. When you relocate the directory you
> >> don't want to edit all our test plans referring that directory. Instead
> >> use a property in the test plans and set it during jmeter startup to
> >> point to your current jar file directory.
> >>
> >> Any hints would be helpful.
>
> Regards,
>
> Rainer
>
>
--
Cordialement.
Philippe Mouawad.
Re: user.classpath, search_path and many jar files
Posted by Rainer Jung <ra...@kippdata.de>.
On 12.08.2013 14:59, Philippe Mouawad wrote:
> Hello Rainer,
> So your patch only concerns Test Plan which do "Add directory or jar to
> classpath" ?
>
> If so, some notes about patch:
>
> - Shouldn't code be within try/catch ? as if a MalformedURLException is
> thrown there are big chances that the rest won't work
The NewDriver methods addURL() and addPath() have existing inconsistent
behavior. addPath() can throw the exception, so any call should be
inside try/catch, addURL() already does the try/catch internally so will
not throw it.
> Regarding Bugzilla, I think it is better to open one for this enhancement.
Will do.
Any idea about the property replacement:
>> Now there's another thing that would be nice: being able to use a jmeter
>> property in the user defined class path in TestPlan.
>>
>> Unfortunately that property is used very early and the resolution of
>> properties returns with an "Not running version, return raw function
>> string" error from FunctionProperty.
>>
>> Is there a way to force a property replacement? Or how can we support
>> using a property in the user defined class path inside a test plan.
>>
>> The use case is somewhat clear: you want to be able to collect your
>> dependency jars in some directory. When you relocate the directory you
>> don't want to edit all our test plans referring that directory. Instead
>> use a property in the test plans and set it during jmeter startup to
>> point to your current jar file directory.
>>
>> Any hints would be helpful.
Regards,
Rainer
Re: user.classpath, search_path and many jar files
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello Rainer,
So your patch only concerns Test Plan which do "Add directory or jar to
classpath" ?
If so, some notes about patch:
- Shouldn't code be within try/catch ? as if a MalformedURLException is
thrown there are big chances that the rest won't work
Regarding Bugzilla, I think it is better to open one for this enhancement.
Regards
Philippe
On Mon, Aug 12, 2013 at 2:13 PM, Rainer Jung <ra...@kippdata.de>wrote:
> On 11.08.2013 23:40, Rainer Jung wrote:
> > On 11.08.2013 21:45, Philippe Mouawad wrote:
> >> On Fri, Aug 9, 2013 at 9:45 PM, Rainer Jung <ra...@kippdata.de>
> wrote:
> >>
> >>> I have a situation where the user.classpath contains many jar files
> >>> (about 100). Listing all of the jar files is tedious, difficult to
> >>> maintain and not in any place where one can configure user.classpath
> >>> possible or easy because of line length restrictions.
> >>>
> >>> Looking at the code I saw that in some places jar files lying in
> >>> directories configured for class search are automatically also
> searched,
> >>> but sometimes not.
> >>>
> >>> There are three cases:
> >>>
> >>> - not auto-searching the jar files:
> >>>
> >>> o.a.jmeter.JMeter start() calls updateClassLoader() calls updatePath()
> >>> for search_paths and user.classpath. updatePath() calls
> >>> NewDriver.addPath() for each path contained in those settings. Finally
> >>> NewDriver.addPath() adds them to the system property "java.class.path"
> >>> without listing all jar files contained in directories.
> >>>
> >>> So the class loader itself does not know about such jar files.
> >>>
> >>> - auto-searching such jar files in ClassFinder.findClassesThatExtend()
> >>> using search_paths for JMeter extension:
> >>>
> >>> In findClassesThatExtend() a call to addJarsInPath() will automatically
> >>> add all jar files found in a configured directory.
> >>>
> >>> - auto-searching jar files in user.classpath for JUnit setup:
> >>>
> >>> In JUnitTestSamplerGui the user.classpath property is used to search
> for
> >>> classes extending Test or TestCase but the search is based on
> >>> ClassFinder, so will automatically search through jar files.
> >>>
> >>> I propose to also add the jar files in the first case, the updatePath
> >>> setting up the class loader. I guess the only concern would be the
> speed
> >>> of class loading? IMHO if there are only few such jar files, then there
> >>> should be no problem and if there are many such jar files, then not
> >>> needing to configure them all is really helpful. So the remaining case
> >>> is if there are many jar files auto-detected which are not really
> >>> needed. That means it might make sense to make the behavior
> configurable
> >>> with default the old behavior.
> >>>
> >>> Comments?
> >>>
> >> Not clear for me. Could you illustrate with an example ? thanks
> >
> > Not really tested, but you might get the idea:
> >
> >
> http://people.apache.org/~rjung/patches/jmeter-user_classpath-load_jars.patch
> >
> > The added code is similar to the handling of jars in the method
> > addJarsInPath() called from findClassesThatExtend() of class ClassFinder.
> >
> > Example is a jmeter extension (here a java sampler) that has lots of
> > dependencies. All of those (about 100) jars would need to be added to
> > user.classpath.
> >
> > If we agree to include all jars located in some path component of
> > user.classpath automatically, then one could simply put the 100 jar
> > files in some directory, e.g. /my/sample/dependencies, and set
> >
> > user.classpath=/my/sample/dependencies
> >
> > For search_paths this already works, but not for user.classpath.
>
> Sorry my patch was rubbish. There are two types of user.classpath. I
> changed the impl for the user.classpath given in the properties file.
> But the entries there are also added to the system classpath, so you
> don't want your extension dependencies there.
>
> The correct patch is here:
>
>
> http://people.apache.org/~rjung/patches/jmeter-user_classpath-load_jars-v2.patch
>
> It adds the jar search to addURL() (called by TestPlan) instead of
> addPath() (called by the main JMeter class).
>
> addURL() only adds items to the loader, addPath() also to the system
> classpath.
>
> In the meantime I tested the patch and it works correct for me.
>
> Now there's another thing that would be nice: being able to use a jmeter
> property in the user defined class path in TestPlan.
>
> Unfortunately that property is used very early and the resolution of
> properties returns with an "Not running version, return raw function
> string" error from FunctionProperty.
>
> Is there a way to force a property replacement? Or how can we support
> using a property in the user defined class path inside a test plan.
>
> The use case is somewhat clear: you want to be able to collect your
> dependency jars in some directory. When you relocate the directory you
> don't want to edit all our test plans referring that directory. Instead
> use a property in the test plans and set it during jmeter startup to
> point to your current jar file directory.
>
> Any hints would be helpful.
>
> What about the patch for automatic jar file exclusion? Any objections?
> Should I file a bugzilla before committing (it seems we now do it for
> nearly every change)?
>
> Regards,
>
> Rainer
>
--
Cordialement.
Philippe Mouawad.
Re: user.classpath, search_path and many jar files
Posted by Rainer Jung <ra...@kippdata.de>.
On 11.08.2013 23:40, Rainer Jung wrote:
> On 11.08.2013 21:45, Philippe Mouawad wrote:
>> On Fri, Aug 9, 2013 at 9:45 PM, Rainer Jung <ra...@kippdata.de> wrote:
>>
>>> I have a situation where the user.classpath contains many jar files
>>> (about 100). Listing all of the jar files is tedious, difficult to
>>> maintain and not in any place where one can configure user.classpath
>>> possible or easy because of line length restrictions.
>>>
>>> Looking at the code I saw that in some places jar files lying in
>>> directories configured for class search are automatically also searched,
>>> but sometimes not.
>>>
>>> There are three cases:
>>>
>>> - not auto-searching the jar files:
>>>
>>> o.a.jmeter.JMeter start() calls updateClassLoader() calls updatePath()
>>> for search_paths and user.classpath. updatePath() calls
>>> NewDriver.addPath() for each path contained in those settings. Finally
>>> NewDriver.addPath() adds them to the system property "java.class.path"
>>> without listing all jar files contained in directories.
>>>
>>> So the class loader itself does not know about such jar files.
>>>
>>> - auto-searching such jar files in ClassFinder.findClassesThatExtend()
>>> using search_paths for JMeter extension:
>>>
>>> In findClassesThatExtend() a call to addJarsInPath() will automatically
>>> add all jar files found in a configured directory.
>>>
>>> - auto-searching jar files in user.classpath for JUnit setup:
>>>
>>> In JUnitTestSamplerGui the user.classpath property is used to search for
>>> classes extending Test or TestCase but the search is based on
>>> ClassFinder, so will automatically search through jar files.
>>>
>>> I propose to also add the jar files in the first case, the updatePath
>>> setting up the class loader. I guess the only concern would be the speed
>>> of class loading? IMHO if there are only few such jar files, then there
>>> should be no problem and if there are many such jar files, then not
>>> needing to configure them all is really helpful. So the remaining case
>>> is if there are many jar files auto-detected which are not really
>>> needed. That means it might make sense to make the behavior configurable
>>> with default the old behavior.
>>>
>>> Comments?
>>>
>> Not clear for me. Could you illustrate with an example ? thanks
>
> Not really tested, but you might get the idea:
>
> http://people.apache.org/~rjung/patches/jmeter-user_classpath-load_jars.patch
>
> The added code is similar to the handling of jars in the method
> addJarsInPath() called from findClassesThatExtend() of class ClassFinder.
>
> Example is a jmeter extension (here a java sampler) that has lots of
> dependencies. All of those (about 100) jars would need to be added to
> user.classpath.
>
> If we agree to include all jars located in some path component of
> user.classpath automatically, then one could simply put the 100 jar
> files in some directory, e.g. /my/sample/dependencies, and set
>
> user.classpath=/my/sample/dependencies
>
> For search_paths this already works, but not for user.classpath.
Sorry my patch was rubbish. There are two types of user.classpath. I
changed the impl for the user.classpath given in the properties file.
But the entries there are also added to the system classpath, so you
don't want your extension dependencies there.
The correct patch is here:
http://people.apache.org/~rjung/patches/jmeter-user_classpath-load_jars-v2.patch
It adds the jar search to addURL() (called by TestPlan) instead of
addPath() (called by the main JMeter class).
addURL() only adds items to the loader, addPath() also to the system
classpath.
In the meantime I tested the patch and it works correct for me.
Now there's another thing that would be nice: being able to use a jmeter
property in the user defined class path in TestPlan.
Unfortunately that property is used very early and the resolution of
properties returns with an "Not running version, return raw function
string" error from FunctionProperty.
Is there a way to force a property replacement? Or how can we support
using a property in the user defined class path inside a test plan.
The use case is somewhat clear: you want to be able to collect your
dependency jars in some directory. When you relocate the directory you
don't want to edit all our test plans referring that directory. Instead
use a property in the test plans and set it during jmeter startup to
point to your current jar file directory.
Any hints would be helpful.
What about the patch for automatic jar file exclusion? Any objections?
Should I file a bugzilla before committing (it seems we now do it for
nearly every change)?
Regards,
Rainer
Re: user.classpath, search_path and many jar files
Posted by Rainer Jung <ra...@kippdata.de>.
On 11.08.2013 21:45, Philippe Mouawad wrote:
> On Fri, Aug 9, 2013 at 9:45 PM, Rainer Jung <ra...@kippdata.de> wrote:
>
>> I have a situation where the user.classpath contains many jar files
>> (about 100). Listing all of the jar files is tedious, difficult to
>> maintain and not in any place where one can configure user.classpath
>> possible or easy because of line length restrictions.
>>
>> Looking at the code I saw that in some places jar files lying in
>> directories configured for class search are automatically also searched,
>> but sometimes not.
>>
>> There are three cases:
>>
>> - not auto-searching the jar files:
>>
>> o.a.jmeter.JMeter start() calls updateClassLoader() calls updatePath()
>> for search_paths and user.classpath. updatePath() calls
>> NewDriver.addPath() for each path contained in those settings. Finally
>> NewDriver.addPath() adds them to the system property "java.class.path"
>> without listing all jar files contained in directories.
>>
>> So the class loader itself does not know about such jar files.
>>
>> - auto-searching such jar files in ClassFinder.findClassesThatExtend()
>> using search_paths for JMeter extension:
>>
>> In findClassesThatExtend() a call to addJarsInPath() will automatically
>> add all jar files found in a configured directory.
>>
>> - auto-searching jar files in user.classpath for JUnit setup:
>>
>> In JUnitTestSamplerGui the user.classpath property is used to search for
>> classes extending Test or TestCase but the search is based on
>> ClassFinder, so will automatically search through jar files.
>>
>> I propose to also add the jar files in the first case, the updatePath
>> setting up the class loader. I guess the only concern would be the speed
>> of class loading? IMHO if there are only few such jar files, then there
>> should be no problem and if there are many such jar files, then not
>> needing to configure them all is really helpful. So the remaining case
>> is if there are many jar files auto-detected which are not really
>> needed. That means it might make sense to make the behavior configurable
>> with default the old behavior.
>>
>> Comments?
>>
> Not clear for me. Could you illustrate with an example ? thanks
Not really tested, but you might get the idea:
http://people.apache.org/~rjung/patches/jmeter-user_classpath-load_jars.patch
The added code is similar to the handling of jars in the method
addJarsInPath() called from findClassesThatExtend() of class ClassFinder.
Example is a jmeter extension (here a java sampler) that has lots of
dependencies. All of those (about 100) jars would need to be added to
user.classpath.
If we agree to include all jars located in some path component of
user.classpath automatically, then one could simply put the 100 jar
files in some directory, e.g. /my/sample/dependencies, and set
user.classpath=/my/sample/dependencies
For search_paths this already works, but not for user.classpath.
Regards,
Rainer
Re: user.classpath, search_path and many jar files
Posted by Philippe Mouawad <ph...@gmail.com>.
On Fri, Aug 9, 2013 at 9:45 PM, Rainer Jung <ra...@kippdata.de> wrote:
> I have a situation where the user.classpath contains many jar files
> (about 100). Listing all of the jar files is tedious, difficult to
> maintain and not in any place where one can configure user.classpath
> possible or easy because of line length restrictions.
>
> Looking at the code I saw that in some places jar files lying in
> directories configured for class search are automatically also searched,
> but sometimes not.
>
> There are three cases:
>
> - not auto-searching the jar files:
>
> o.a.jmeter.JMeter start() calls updateClassLoader() calls updatePath()
> for search_paths and user.classpath. updatePath() calls
> NewDriver.addPath() for each path contained in those settings. Finally
> NewDriver.addPath() adds them to the system property "java.class.path"
> without listing all jar files contained in directories.
>
> So the class loader itself does not know about such jar files.
>
> - auto-searching such jar files in ClassFinder.findClassesThatExtend()
> using search_paths for JMeter extension:
>
> In findClassesThatExtend() a call to addJarsInPath() will automatically
> add all jar files found in a configured directory.
>
> - auto-searching jar files in user.classpath for JUnit setup:
>
> In JUnitTestSamplerGui the user.classpath property is used to search for
> classes extending Test or TestCase but the search is based on
> ClassFinder, so will automatically search through jar files.
>
> I propose to also add the jar files in the first case, the updatePath
> setting up the class loader. I guess the only concern would be the speed
> of class loading? IMHO if there are only few such jar files, then there
> should be no problem and if there are many such jar files, then not
> needing to configure them all is really helpful. So the remaining case
> is if there are many jar files auto-detected which are not really
> needed. That means it might make sense to make the behavior configurable
> with default the old behavior.
>
> Comments?
>
> Not clear for me. Could you illustrate with an example ? thanks
> Regards,
>
> Rainer
>
--
Cordialement.
Philippe Mouawad.