You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tuscany.apache.org by Mike Edwards <mi...@gmail.com> on 2009/05/10 12:24:59 UTC
** WARNING ** node-impl2 can cause hard to diagnose errors with OSGi
Folks,
A warning - after I chased my tail for a day with a bug that was very hard to pin down.
Turned up running NodeLauncherTestCase in node-launcher-equinox after I had made a change in a
completely different module - Assembly.
What I started getting were NoClassDefFound errors for ContributionScanner when trying to load the
ContributionContentProcessor from NodeImpl in the startup phase. The worst aspect was that these
errors were intermittent - they happened often, but not every time - and trying to use the debugger
in Eclipse usually made the code work - but not always.
Eventually, I managed to get the ClassLoader code to stop at the point where it was throwing the
exception and discovered to my surprise that the Bundle doing the loading was not node-impl but
rather node-impl2. Removing the node-impl2 module from my system solved the problem.
Now, node-impl2 isn't part of the mainline build so you'd expect it not to get used. And in general
this is true. However, the class loading under OSGi is different - and if node-impl2 is there in
your modules directory, it is fair game for loading under OSGi for some configurations. And for
whatever reason, it screws up the classloading.
Yours, Mike
Re: ** WARNING ** node-impl2 can cause hard to diagnose errors with
OSGi
Posted by Mike Edwards <mi...@gmail.com>.
ant elder wrote:
>
>
> On Sun, May 10, 2009 at 11:24 AM, Mike Edwards
> <mike.edwards.inglenook@gmail.com
> <ma...@gmail.com>> wrote:
>
> Folks,
>
> A warning - after I chased my tail for a day with a bug that was
> very hard to pin down.
>
> Turned up running NodeLauncherTestCase in node-launcher-equinox
> after I had made a change in a completely different module - Assembly.
>
> What I started getting were NoClassDefFound errors for
> ContributionScanner when trying to load the
> ContributionContentProcessor from NodeImpl in the startup phase.
> The worst aspect was that these errors were intermittent - they
> happened often, but not every time - and trying to use the debugger
> in Eclipse usually made the code work - but not always.
>
> Eventually, I managed to get the ClassLoader code to stop at the
> point where it was throwing the exception and discovered to my
> surprise that the Bundle doing the loading was not node-impl but
> rather node-impl2. Removing the node-impl2 module from my system
> solved the problem.
>
> Now, node-impl2 isn't part of the mainline build so you'd expect it
> not to get used. And in general this is true. However, the class
> loading under OSGi is different - and if node-impl2 is there in your
> modules directory, it is fair game for loading under OSGi for some
> configurations. And for whatever reason, it screws up the classloading.
>
>
> Yours, Mike
>
>
> This is not specific to node-impl2 and its come up before -
> http://apache.markmail.org/message/jflhtsrgwx7w4uy5
>
> It seems like a design flaw in how node-launcher-equinox works. It
> doesn't seem right that when its being run from in the maven build it
> just uses anything it happens to find in the directory above where it
> is. If it wants to work as part or the maven build then it needs to use
> the modules that are part of that build, anything else is inevitably
> going to cause this type of confusion. Working the way it is now its
> also including jars that are in the build but not built until after
> node-launcher-equinox, so when the tests run its testing what the
> previous build run did not what the current build run is creating.
>
> One easy way to fix this may be to remove this way of working from the
> node-launcher-equinox module and move it to in the distribution itests
> so its testing what we actually intend to part of the build.
>
> As an FYI to anyone trying to understand whats doing this see
> NodeLauncherUtil.runtimeClasspathEntries in the node-launcher-equinox
> module.
>
> ...ant
>
Ant,
It's going to be a bit more involved than that.
There is also the running of OSGi stuff when in development mode under Eclipse. Then it DOES seem
to make sense to use the materials in the target\classes directories, since you want the code to
pick up what you're working on without the need for a full build. This is what happens today and
for me, it works well.
But THEN - you will have to remember to remove node-impl2 from your installation (which I what I
ended up doing), since you get it if you do a full checkout. Having to remember something like this
is tedious.
Yours, Mike.
Re: ** WARNING ** node-impl2 can cause hard to diagnose errors with
OSGi
Posted by ant elder <an...@gmail.com>.
On Sun, May 10, 2009 at 11:24 AM, Mike Edwards <
mike.edwards.inglenook@gmail.com> wrote:
> Folks,
>
> A warning - after I chased my tail for a day with a bug that was very hard
> to pin down.
>
> Turned up running NodeLauncherTestCase in node-launcher-equinox after I had
> made a change in a completely different module - Assembly.
>
> What I started getting were NoClassDefFound errors for ContributionScanner
> when trying to load the ContributionContentProcessor from NodeImpl in the
> startup phase. The worst aspect was that these errors were intermittent -
> they happened often, but not every time - and trying to use the debugger in
> Eclipse usually made the code work - but not always.
>
> Eventually, I managed to get the ClassLoader code to stop at the point
> where it was throwing the exception and discovered to my surprise that the
> Bundle doing the loading was not node-impl but rather node-impl2. Removing
> the node-impl2 module from my system solved the problem.
>
> Now, node-impl2 isn't part of the mainline build so you'd expect it not to
> get used. And in general this is true. However, the class loading under
> OSGi is different - and if node-impl2 is there in your modules directory, it
> is fair game for loading under OSGi for some configurations. And for
> whatever reason, it screws up the classloading.
>
>
> Yours, Mike
>
This is not specific to node-impl2 and its come up before -
http://apache.markmail.org/message/jflhtsrgwx7w4uy5
It seems like a design flaw in how node-launcher-equinox works. It doesn't
seem right that when its being run from in the maven build it just uses
anything it happens to find in the directory above where it is. If it wants
to work as part or the maven build then it needs to use the modules that are
part of that build, anything else is inevitably going to cause this type of
confusion. Working the way it is now its also including jars that are in the
build but not built until after node-launcher-equinox, so when the tests run
its testing what the previous build run did not what the current build run
is creating.
One easy way to fix this may be to remove this way of working from the
node-launcher-equinox module and move it to in the distribution itests so
its testing what we actually intend to part of the build.
As an FYI to anyone trying to understand whats doing this see
NodeLauncherUtil.runtimeClasspathEntries in the node-launcher-equinox
module.
...ant
Re: ** WARNING ** node-impl2 can cause hard to diagnose errors with
OSGi
Posted by Simon Laws <si...@googlemail.com>.
On Sun, May 10, 2009 at 9:40 PM, Raymond Feng <en...@gmail.com> wrote:
> node-impl2 has conflicts with node-impl from the OSGi perspective because
> both of them export "overlapping" packages. In Eclipse, all the PDE projects
> in the workspace contributes to the OSGi bundle resolution (requiredPlugins
> classpath container). This is so called "split packages" in OSGi and that's
> probably root cause of the problem. You should either load node-impl or
> node-impl2 but not both in your workspace. We should remove node-impl2 from
> the main buid if it is there to avoid further confusions.
>
> Thanks,
> Raymond
> --------------------------------------------------
> From: "Mike Edwards" <mi...@gmail.com>
> Sent: Sunday, May 10, 2009 3:24 AM
> To: "tuscany-dev" <tu...@ws.apache.org>
> Subject: ** WARNING ** node-impl2 can cause hard to diagnose errors with
> OSGi
>
>> Folks,
>>
>> A warning - after I chased my tail for a day with a bug that was very hard
>> to pin down.
>>
>> Turned up running NodeLauncherTestCase in node-launcher-equinox after I
>> had made a change in a completely different module - Assembly.
>>
>> What I started getting were NoClassDefFound errors for ContributionScanner
>> when trying to load the ContributionContentProcessor from NodeImpl in the
>> startup phase. The worst aspect was that these errors were intermittent -
>> they happened often, but not every time - and trying to use the debugger in
>> Eclipse usually made the code work - but not always.
>>
>> Eventually, I managed to get the ClassLoader code to stop at the point
>> where it was throwing the exception and discovered to my surprise that the
>> Bundle doing the loading was not node-impl but rather node-impl2. Removing
>> the node-impl2 module from my system solved the problem.
>>
>> Now, node-impl2 isn't part of the mainline build so you'd expect it not to
>> get used. And in general this is true. However, the class loading under
>> OSGi is different - and if node-impl2 is there in your modules directory, it
>> is fair game for loading under OSGi for some configurations. And for
>> whatever reason, it screws up the classloading.
>>
>>
>> Yours, Mike
>
>
Would someone be able to articulate what different behaviours are now
in place for loading the correct modules in order to run SCA
contributions in the various different environments we support, e.g.
command line, Eclipse dev, Eclipse plugin, mvn, ant, webapp, embedded.
This was a topic of much discussion some months ago when we debated
the merits of various different launcher approaches. Where are we now?
Yes, I could look at the code but if someone happens to know it would
save some time. I'd like to get a launcher page up to date, e.g. [1],
for how we expect people to use the runtime.
Simon
[1] http://cwiki.apache.org/confluence/display/TUSCANYWIKI/Tuscany+Runtime+Launching
Re: ** WARNING ** node-impl2 can cause hard to diagnose errors with OSGi
Posted by Raymond Feng <en...@gmail.com>.
There is code in EquinoxHost.start():
// Test if the configuration/config.ini or osgi.bundles has
been set
// If yes, try to avoid discovery of bundles
if (bundleLocations == null) {
if (props.getProperty("osgi.bundles") != null) {
bundleLocations = Collections.emptySet();
} else {
String config =
props.getProperty(LocationManager.PROP_CONFIG_AREA);
File ini = new File(config, "config.ini");
if (ini.isFile()) {
Properties iniProps = new Properties();
iniProps.load(new FileInputStream(ini));
if (iniProps.getProperty("osgi.bundles") !=
null) {
bundleLocations = Collections.emptySet();
}
}
}
}
If the bundleLocations is not null, then the findBundleLocations() will not
try to discover the bundles.
Can you debug to see if the test is expected?
Thanks,
Raymond
--------------------------------------------------
From: "ant elder" <an...@apache.org>
Sent: Tuesday, June 09, 2009 3:20 AM
To: <de...@tuscany.apache.org>
Subject: Re: ** WARNING ** node-impl2 can cause hard to diagnose errors with
OSGi
> Digging in this some more I couldn't get it to work using the
> config.ini created in the distribution, is that what you mean by ATM
> it uses a list of files not the osgi.bundles format?
>
> Debugging through what happens isn't the problem caused by line 431 in
> EquinoxHost which calls
>
> bundleLocations = runtimeClasspathEntries(true, false,
> true);
>
> where the last "true" means useModulesDirectory so simply changing
> that to be false should fix this problem when running a maven build?
>
> ...ant
>
>
> On Wed, Jun 3, 2009 at 4:46 PM, Raymond Feng<en...@gmail.com> wrote:
>> We use the same syntax as Equinox:
>> http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/runtime-options.html
>>
>> For -configuration, please see the
>> <distro>/all/target/features/configuration/config.ini.
>>
>> java org.apache.tuscany.sca.node.equinox.launcher.NodeLauncher
>> -configuration <configuration folder>
>> java org.apache.tuscany.sca.node.equinox.launcher.NodeLauncher -bundles
>> <bundles> (ATM, we support a list of files. I'll fix it to use the
>> Equinox
>> osgi.bundles format)
>>
>> Thanks,
>> Raymond
>> --------------------------------------------------
>> From: "ant elder" <an...@apache.org>
>> Sent: Wednesday, June 03, 2009 2:22 AM
>> To: "Raymond Feng" <en...@gmail.com>
>> Cc: <de...@tuscany.apache.org>
>> Subject: Re: ** WARNING ** node-impl2 can cause hard to diagnose errors
>> with
>> OSGi
>>
>>> On Sat, May 23, 2009 at 6:09 PM, Raymond Feng <en...@gmail.com>
>>> wrote:
>>>>
>>>> There are already two options to control the discovery:
>>>>
>>>> 1) -configuration (or osgi.configuration.area system property):
>>>> Pointing
>>>> to
>>>> an equinox configuration folder that contains a config.ini which lists
>>>> all
>>>> the bundles
>>>> 2) -bundles: Pointing to a list of files (we can enhance it to support
>>>> URLs)
>>>> for OSGi bundles
>>>>
>>>> Thanks,
>>>> Raymond
>>>
>>> Do we have and doc on these or examples showing how to use them?
>>>
>>> ...ant
>>
>>
Re: ** WARNING ** node-impl2 can cause hard to diagnose errors with
OSGi
Posted by ant elder <an...@apache.org>.
Digging in this some more I couldn't get it to work using the
config.ini created in the distribution, is that what you mean by ATM
it uses a list of files not the osgi.bundles format?
Debugging through what happens isn't the problem caused by line 431 in
EquinoxHost which calls
bundleLocations = runtimeClasspathEntries(true, false, true);
where the last "true" means useModulesDirectory so simply changing
that to be false should fix this problem when running a maven build?
...ant
On Wed, Jun 3, 2009 at 4:46 PM, Raymond Feng<en...@gmail.com> wrote:
> We use the same syntax as Equinox:
> http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/misc/runtime-options.html
>
> For -configuration, please see the
> <distro>/all/target/features/configuration/config.ini.
>
> java org.apache.tuscany.sca.node.equinox.launcher.NodeLauncher
> -configuration <configuration folder>
> java org.apache.tuscany.sca.node.equinox.launcher.NodeLauncher -bundles
> <bundles> (ATM, we support a list of files. I'll fix it to use the Equinox
> osgi.bundles format)
>
> Thanks,
> Raymond
> --------------------------------------------------
> From: "ant elder" <an...@apache.org>
> Sent: Wednesday, June 03, 2009 2:22 AM
> To: "Raymond Feng" <en...@gmail.com>
> Cc: <de...@tuscany.apache.org>
> Subject: Re: ** WARNING ** node-impl2 can cause hard to diagnose errors with
> OSGi
>
>> On Sat, May 23, 2009 at 6:09 PM, Raymond Feng <en...@gmail.com> wrote:
>>>
>>> There are already two options to control the discovery:
>>>
>>> 1) -configuration (or osgi.configuration.area system property): Pointing
>>> to
>>> an equinox configuration folder that contains a config.ini which lists
>>> all
>>> the bundles
>>> 2) -bundles: Pointing to a list of files (we can enhance it to support
>>> URLs)
>>> for OSGi bundles
>>>
>>> Thanks,
>>> Raymond
>>
>> Do we have and doc on these or examples showing how to use them?
>>
>> ...ant
>
>
Re: ** WARNING ** node-impl2 can cause hard to diagnose errors with
OSGi
Posted by ant elder <an...@apache.org>.
On Sat, May 23, 2009 at 6:09 PM, Raymond Feng <en...@gmail.com> wrote:
> There are already two options to control the discovery:
>
> 1) -configuration (or osgi.configuration.area system property): Pointing to
> an equinox configuration folder that contains a config.ini which lists all
> the bundles
> 2) -bundles: Pointing to a list of files (we can enhance it to support URLs)
> for OSGi bundles
>
> Thanks,
> Raymond
Do we have and doc on these or examples showing how to use them?
...ant
Re: ** WARNING ** node-impl2 can cause hard to diagnose errors with
OSGi
Posted by ant elder <an...@apache.org>.
Wonderful, can we update the node-launcher-equinox tests to use these
by default then? Do we have any examples of those types of config so I
can do this, or doc on what to do?
...ant
On Sat, May 23, 2009 at 6:09 PM, Raymond Feng <en...@gmail.com> wrote:
> There are already two options to control the discovery:
>
> 1) -configuration (or osgi.configuration.area system property): Pointing to
> an equinox configuration folder that contains a config.ini which lists all
> the bundles
> 2) -bundles: Pointing to a list of files (we can enhance it to support URLs)
> for OSGi bundles
>
> Thanks,
> Raymond
> --------------------------------------------------
> From: "ant elder" <an...@gmail.com>
> Sent: Saturday, May 23, 2009 12:50 AM
> To: <de...@tuscany.apache.org>
> Subject: Re: ** WARNING ** node-impl2 can cause hard to diagnose errors with
> OSGi
>
>> On Mon, May 11, 2009 at 10:28 AM, ant elder <an...@gmail.com> wrote:
>>
>>> (2) We need to find a better long term solution for
>>> node-launcher-equinox as having it work by just finding modules in the
>>> parent directory is only going to continue to make the build unstable.
>>> Maven is the canonical build for Tuscany so module code needs to play
>>> properly with that. It also doesn't seem great for the modularity
>>> story to say that while Tuscany is designed to be really modular and
>>> composable but actually we must have only a single impl of each
>>> function as part of the build - and this isn't some theoretical
>>> academic discussion as we do have multiple implementations of things
>>> in 1.x which will be brought into 2.x so we need to avoid random
>>> indeterminate combinations of the modules being used in different
>>> peoples builds.
>>>
>>> I guess we need some configuration mechanism for node-launcher-equinox
>>> to tell it which modules to use, does anyone else have any
>>> suggestions?
>>>
>>
>> I've just wasted some time tracking down yet another one of these
>> build problems caused by node-launcher-equinox finding things not in
>> the build so I'm still keen to find a better approach. I wonder if
>> the more declarative approach to system service discovery thats being
>> discussed in that other ML thread could help this.
>>
>> ..ant
>
>
Re: ** WARNING ** node-impl2 can cause hard to diagnose errors with OSGi
Posted by Raymond Feng <en...@gmail.com>.
There are already two options to control the discovery:
1) -configuration (or osgi.configuration.area system property): Pointing to
an equinox configuration folder that contains a config.ini which lists all
the bundles
2) -bundles: Pointing to a list of files (we can enhance it to support URLs)
for OSGi bundles
Thanks,
Raymond
--------------------------------------------------
From: "ant elder" <an...@gmail.com>
Sent: Saturday, May 23, 2009 12:50 AM
To: <de...@tuscany.apache.org>
Subject: Re: ** WARNING ** node-impl2 can cause hard to diagnose errors with
OSGi
> On Mon, May 11, 2009 at 10:28 AM, ant elder <an...@gmail.com> wrote:
>
>> (2) We need to find a better long term solution for
>> node-launcher-equinox as having it work by just finding modules in the
>> parent directory is only going to continue to make the build unstable.
>> Maven is the canonical build for Tuscany so module code needs to play
>> properly with that. It also doesn't seem great for the modularity
>> story to say that while Tuscany is designed to be really modular and
>> composable but actually we must have only a single impl of each
>> function as part of the build - and this isn't some theoretical
>> academic discussion as we do have multiple implementations of things
>> in 1.x which will be brought into 2.x so we need to avoid random
>> indeterminate combinations of the modules being used in different
>> peoples builds.
>>
>> I guess we need some configuration mechanism for node-launcher-equinox
>> to tell it which modules to use, does anyone else have any
>> suggestions?
>>
>
> I've just wasted some time tracking down yet another one of these
> build problems caused by node-launcher-equinox finding things not in
> the build so I'm still keen to find a better approach. I wonder if
> the more declarative approach to system service discovery thats being
> discussed in that other ML thread could help this.
>
> ..ant
Re: ** WARNING ** node-impl2 can cause hard to diagnose errors with
OSGi
Posted by ant elder <an...@gmail.com>.
On Mon, May 11, 2009 at 10:28 AM, ant elder <an...@gmail.com> wrote:
> (2) We need to find a better long term solution for
> node-launcher-equinox as having it work by just finding modules in the
> parent directory is only going to continue to make the build unstable.
> Maven is the canonical build for Tuscany so module code needs to play
> properly with that. It also doesn't seem great for the modularity
> story to say that while Tuscany is designed to be really modular and
> composable but actually we must have only a single impl of each
> function as part of the build - and this isn't some theoretical
> academic discussion as we do have multiple implementations of things
> in 1.x which will be brought into 2.x so we need to avoid random
> indeterminate combinations of the modules being used in different
> peoples builds.
>
> I guess we need some configuration mechanism for node-launcher-equinox
> to tell it which modules to use, does anyone else have any
> suggestions?
>
I've just wasted some time tracking down yet another one of these
build problems caused by node-launcher-equinox finding things not in
the build so I'm still keen to find a better approach. I wonder if
the more declarative approach to system service discovery thats being
discussed in that other ML thread could help this.
..ant
Re: ** WARNING ** node-impl2 can cause hard to diagnose errors with
OSGi
Posted by ant elder <an...@gmail.com>.
On Sun, May 10, 2009 at 9:40 PM, Raymond Feng <en...@gmail.com> wrote:
>
> node-impl2 has conflicts with node-impl from the OSGi perspective because both of them export "overlapping" packages. In Eclipse, all the PDE projects in the workspace contributes to the OSGi bundle resolution (requiredPlugins classpath container). This is so called "split packages" in OSGi and that's probably root cause of the problem. You should either load node-impl or node-impl2 but not both in your workspace. We should remove node-impl2 from the main buid if it is there to avoid further confusions.
>
There's two things here, (1) a short term fix, and (2) in the long
term finding a better way for node-launcher-equinox to work.
(1) lets just get rid of node-impl2 as now the changes can be merged
back into node-impl. There is general agreement on the "Node usage of
the workspace and implementation-node modules" thread that we need to
refactor this and i already said last week on that thread i was going
to do this, so it seems fine now to make this change and we can
continue refining there as required.
(2) We need to find a better long term solution for
node-launcher-equinox as having it work by just finding modules in the
parent directory is only going to continue to make the build unstable.
Maven is the canonical build for Tuscany so module code needs to play
properly with that. It also doesn't seem great for the modularity
story to say that while Tuscany is designed to be really modular and
composable but actually we must have only a single impl of each
function as part of the build - and this isn't some theoretical
academic discussion as we do have multiple implementations of things
in 1.x which will be brought into 2.x so we need to avoid random
indeterminate combinations of the modules being used in different
peoples builds.
I guess we need some configuration mechanism for node-launcher-equinox
to tell it which modules to use, does anyone else have any
suggestions?
...ant
Re: ** WARNING ** node-impl2 can cause hard to diagnose errors with OSGi
Posted by Raymond Feng <en...@gmail.com>.
node-impl2 has conflicts with node-impl from the OSGi perspective because
both of them export "overlapping" packages. In Eclipse, all the PDE projects
in the workspace contributes to the OSGi bundle resolution (requiredPlugins
classpath container). This is so called "split packages" in OSGi and that's
probably root cause of the problem. You should either load node-impl or
node-impl2 but not both in your workspace. We should remove node-impl2 from
the main buid if it is there to avoid further confusions.
Thanks,
Raymond
--------------------------------------------------
From: "Mike Edwards" <mi...@gmail.com>
Sent: Sunday, May 10, 2009 3:24 AM
To: "tuscany-dev" <tu...@ws.apache.org>
Subject: ** WARNING ** node-impl2 can cause hard to diagnose errors with
OSGi
> Folks,
>
> A warning - after I chased my tail for a day with a bug that was very hard
> to pin down.
>
> Turned up running NodeLauncherTestCase in node-launcher-equinox after I
> had made a change in a completely different module - Assembly.
>
> What I started getting were NoClassDefFound errors for ContributionScanner
> when trying to load the ContributionContentProcessor from NodeImpl in the
> startup phase. The worst aspect was that these errors were intermittent -
> they happened often, but not every time - and trying to use the debugger
> in Eclipse usually made the code work - but not always.
>
> Eventually, I managed to get the ClassLoader code to stop at the point
> where it was throwing the exception and discovered to my surprise that the
> Bundle doing the loading was not node-impl but rather node-impl2.
> Removing the node-impl2 module from my system solved the problem.
>
> Now, node-impl2 isn't part of the mainline build so you'd expect it not to
> get used. And in general this is true. However, the class loading under
> OSGi is different - and if node-impl2 is there in your modules directory,
> it is fair game for loading under OSGi for some configurations. And for
> whatever reason, it screws up the classloading.
>
>
> Yours, Mike