You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jdo-dev@db.apache.org by Martin Zaun <Ma...@Sun.COM> on 2005/09/16 16:30:26 UTC
status update: ri11 enhancer support for jdk 1.5
Hi Craig,
here's the current status of testing my RI11 enhancer changes for
Java 5 support (running jdk 1.5.0_05):
maven.compile.source = 1.3 1.3 1.5
maven.compile.target = 1.3 1.5 1.5
-------------------------------------------------------------------
tck11 OK OK OK
junit tests (maven build) OK OK 2 errors
runtest.list/security OK OK 1+5 errors
multiJVMtests error error error
ClassNotFoundException: com.sun.jndi.fscontext.RefFSContextFactory
So, the basic 1.5 classfile support seems to be working (target=1.5),
but there are errors when compiling the ri11 and junit tests with
source=1.5, into which I started looking.
More details on the errors below.
Martin
----------------------------------------------------------------------
Failing JUnit tests:
[junit] [ERROR] TEST org.apache.jdo.test.Test_JDOModel FAILED
[junit] [ERROR] TEST org.apache.jdo.test.Test_Query FAILED
How can I get more information on these failures?
I tried modifying the log level in ri11/test/conf/
logging.properties and simplelog.properties
but with no success.
----------------------------------------------------------------------
JDO RI test suite, runtest.list and runtest.security:
Tests run: 149, Failures: 1, Errors: 5
Details below.
----------------------------------------------------------------------
[java] RUN Test_JDOModel.testDefaults FAILURE
[java] There was 1 failure:
[java] 1) testDefaults(org.apache.jdo.test.Test_JDOModel)junit.framework.Com
parisonFailure: Wrong list of persistentSerializableFieldNumbers of class org.ap
ache.jdo.pc.empdept.PCFullTimeEmployee expected:<..., 10...> but was:<......>
[java] at org.apache.jdo.test.Test_JDOModel.verifyPCFullTimeEmployee(Te
st_JDOModel.java:304)
...
[java] Caused by: java.util.MissingResourceException: Can't find resource fo
r bundle java.util.PropertyResourceBundle, key ERR_MultipleJavaField
----------------------------------------------------------------------
[java] RUN Test_EmpDeptSerialization.test ERROR
[java] 1) test(org.apache.jdo.test.Test_EmpDeptSerialization)java.lang.Excep
tionInInitializerError
[java] at org.apache.jdo.test.Test_EmpDeptSerialization.createObjects(T
est_EmpDeptSerialization.java:105)
----------------------------------------------------------------------
[java] RUN Test_Inheritance.test ERROR
[java] 2) test(org.apache.jdo.test.Test_Inheritance)java.lang.NoClassDefFoun
dError
[java] at org.apache.jdo.test.EmpDeptSupport.createObjects(EmpDeptSuppo
rt.java:76)
----------------------------------------------------------------------
[java] RUN Test_StringOID.test ERROR
[java] 3) test(org.apache.jdo.test.Test_StringOID)java.lang.NoClassDefFoundE
rror
[java] at org.apache.jdo.test.EmpDeptSupport.createObjects(EmpDeptSuppo
rt.java:76)
----------------------------------------------------------------------
[java] RUN Test_Query.testMemoryQueries ERROR
[java] 4) testMemoryQueries(org.apache.jdo.test.Test_Query)java.lang.Excepti
onInInitializerError
[java] at org.apache.jdo.test.Test_Query.createObjects(Test_Query.java:
344)
...
[java] Caused by: java.util.MissingResourceException: Can't find resource fo
r bundle java.util.PropertyResourceBundle, key ERR_MultipleJavaField
----------------------------------------------------------------------
[java] RUN Test_Query.testExtentQueries ERROR
[java] 5) testExtentQueries(org.apache.jdo.test.Test_Query)java.lang.NoClass
DefFoundError
[java] at org.apache.jdo.test.Test_Query.createObjects(Test_Query.java:
344)
----------------------------------------------------------------------
JDO RI test suite, runtest.multiJVMtests:
2 errors, even with maven.compile.source,target = 1.3
[java] RUN Test_FetchInserted.test ERROR
[java] 1) test(org.apache.jdo.test.Test_FetchInserted)javax.naming.NoInitial
ContextException: Cannot instantiate class: com.sun.jndi.fscontext.RefFSContextF
actory [Root exception is java.lang.ClassNotFoundException: com.sun.jndi.fsconte
xt.RefFSContextFactory]
This sun-specific JDK class seems to be missing or renamed in
JDK 1.5. I guess we only have to change the entry in
ri11/test/conf/jndi.properties. If anyone knows, let me know.
----------------------------------------------------------------------
queries and class registration issue [was: Re: status update: ri11
enhancer support for jdk 1.5]
Posted by Martin Zaun <Ma...@Sun.COM>.
Craig,
here's a summary of the discussion Michael and I had on the issue of
"queries and class registration" (as exposed by the 1.5 jvm). Please,
review and comment on our assumptions and solution approach.
1. JDO requires the application to register the "target" class and
relevant subclasses (!) before executing a polymorphic query,
navigation, getExtent (with subclasses), or getObjectById.
Now, at least this is what I think to remember from JDO1: Some JDO
implementations have no means to know the "class universe", i.e.,
all subclasses of a class or the PC class for an Oid class. So,
the getExtent method and queries are executed based on the set of
PC classes currently known to the runtime. This requires the
application to register (load+initialize) the relevant classes
before the above methods and queries are being used.
However, I couldn't find that provision in the jdo-2005-09-09.pdf
spec document (we grepped for a number of keywords).
Have I overlooked something or has this requirement upon the
application been dropped in the JDO2 spec?
2. On the other hand, a JDO implementation may actively load and
initialize PC classes.
Section 12.5 (Class Loaders) states explicitly:
JDO requires access to class instances in several situations
where the class instance is not provided explicitly. In these
cases, the only information available to the implementation is
the name of the class. To resolve class names to class
instances, JDO implementations will use
Class.forName(String name, boolean initialize, ClassLoader loader)
with up to three loaders. The initialize parameter can be
either true or false depending on the implementation.
So, in general, the user must be aware that a JDO implementation
may load and initialize classes, that is, execute static
initializers, in a sometimes, perhaps, unexpected order.
3. For classes/types used in queries, such as in navigational
expressions ("a.b.c"), the question arises if the application
is responsible for having them loaded before query compilation
or execution, or if a JDO implementation may load and initialize
them when compiling or executing a query.
I think it's OK for a JDO implementation to initialize those
classes, since this results from them being used in a query, that
is, the user has given a hint to use these classes.
This is the approach Michael has implemented, which solves the
JVM 1.5 class registration issue.
From a hotspot at London Heathrow,
Martin
Re: status update: ri11 enhancer support for jdk 1.5
Posted by Michael Bouschen <mb...@spree.de>.
Hi Martin,
I checked in the java model changes in ri11 and core20/runtime20
(revision 292681). From my point of view you can go ahead with your
enhancner support for jdk 1.5 changes :-).
Regards Michael
> Hi Martin,
>
> thanks for the comments!
>
> About the issue of a class argument of a wrong classloader passed to the
> JavaModel method getJavaType: I agree the code should use the class
> loader from the class object. The only issue is that the call
> clazz.getClassLoader() might result in a SecurityException, so I need to
> put this call into a doPrivileged block. I already have a convenience
> method in RuntimeJavaModelFactory, I just need to move this method to a
> class that is accessible in both places.
>
> Regards Michael
>
>>
>> Hi Michael,
>> thanks for the patch, only a few cosmetic comments inline, below.
>> I'll send out a summary of our discussion on the jdk 1.5 class
>> registration problem and the chosen approach later.
>> Martin
>>
>> Michael Bouschen wrote:
>>
>>> Hi Martin,
>>>
>>> attached you find a patch for the JDOModel implementation. It
>>> initializes a class instance if the model instance runs with the
>>> initialize=true flag. It also fixes the MissingResourceException. I
>>> could successfully run the ri tests in a workspace including your
>>> enhancer plus my JDOModel changes.
>>> I would port the JDOModel changes to core20 and check them in, if you
>>> agree with the changes.
>>>
>>> Regards Michael
>>>
>>>>
>>>> Michael,
>>>> attached is a stable subset of my ri11 enhancer
>>>> changes that produce the problem with Test_Query.
>>>> Unzip in trunk and run 'maven build' from ri11.
>>>> Talk to you next week,
>>>> Martin
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> Index:
>>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/runtime/Bundle.properties
>>>
>>> ===================================================================
>>> ---
>>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/runtime/Bundle.properties
>>> (Revision 292403)
>>> +++
>>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/runtime/Bundle.properties
>>> (Arbeitskopie)
>>> @@ -62,12 +62,3 @@
>>> #NOI18N
>>> ERR_CannotSetJDOModel=Cannot set JDOModel for JavaModel instance.
>>>
>>> -#
>>> -# RuntimeJavaType
>>> -#
>>> -
>>> -# {0} - error location (class.method)
>>> -# {1} - implementation method name
>>> -# {2} - field name
>>> -#NOI18N
>>> -ERR_MultipleJavaField={0}: multiple JavaField ''{1}'' for class '{2}''.
>>> Index:
>>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/reflection/ReflectionJavaModel.java
>>>
>>> ===================================================================
>>> ---
>>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/reflection/ReflectionJavaModel.java
>>> (Revision 292403)
>>> +++
>>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/reflection/ReflectionJavaModel.java
>>> (Arbeitskopie)
>>> @@ -87,8 +87,8 @@
>>> // loaded, Class.forName will load the class which
>>> // calls RegisterClassListener.registerClass.
>>> // This will create a new JavaType entry in the
>>> cache.
>>> - javaType = getJavaType(Class.forName(name,
>>> initialize, -
>>> classLoader));
>>> + javaType = getJavaTypeInternal(
>>> + Class.forName(name, initialize, classLoader));
>>> }
>>> catch (ClassNotFoundException ex) {
>>> // cannot find class => return null
>>> @@ -121,15 +121,20 @@
>>> */
>>> public JavaType getJavaType(Class clazz)
>>> {
>>> - String name = clazz.getName();
>>> - synchronized (types) {
>>> - JavaType javaType = (JavaType)types.get(name);
>>> - if (javaType == null) {
>>> - javaType = createJavaType(clazz);
>>> - types.put(name, javaType);
>>> + if (clazz == null)
>>> + return null;
>>> + + if (initialize) {
>>> + try {
>>> + // make sure the class is initialized
>>> + Class.forName(clazz.getName(), initialize,
>>> classLoader);
>>
>>
>>
>> Since this is a public method, there's a chance for a bug that we're
>> called here with a class argument of a wrong classloader, I think.
>> If that happened, we'd load and initialize the class in the wrong
>> place. So, I'd change this line, making an assumption explicit, to:
>> Class.forName(clazz.getName(), initialize,
>> clazz.getClassLoader());
>> Or, if we allowed for assertions, we could keep the line but just add:
>> assert (classLoader == clazz.getClassLoader());
>>
>>> }
>>> - return javaType;
>>> + catch (ClassNotFoundException ex) {
>>> + // ignore
>>
>>
>>
>> I'd add:
>> // ignore, since class has already been loaded
>>
>>> + }
>>> }
>>> +
>>> + return getJavaTypeInternal(clazz);
>>> }
>>>
>>> /**
>>> @@ -170,6 +175,28 @@
>>> return classLoader;
>>> }
>>>
>>> + /**
>>> + * The method returns the JavaType instance for the type name of
>>> the
>>> + * specified class object. It first checks the cache and if
>>> there is no
>>> + * entry for the type name in the cache then it creates a new
>>> JavaType
>>> + * instance for the specified Class object.
>>> + * @param clazz the Class instance representing the type
>>> + * @return a JavaType instance for the name of the specified class
>>> + * object or <code>null</code> if not present in this model
>>> instance.
>>> + */
>>> + protected JavaType getJavaTypeInternal(Class clazz)
>>> + {
>>> + String name = clazz.getName();
>>> + synchronized (types) {
>>> + JavaType javaType = (JavaType)types.get(name);
>>> + if (javaType == null) {
>>> + javaType = createJavaType(clazz);
>>> + types.put(name, javaType);
>>> + }
>>> + return javaType;
>>> + }
>>> + }
>>> +
>>> /** * Creates a new JavaType instance for the specified
>>> Class object.
>>> * This method provides a hook such that ReflectionJavaModel
>>> subclasses
>>> Index:
>>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/Bundle.properties
>>> ===================================================================
>>> ---
>>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/Bundle.properties
>>> (Revision 292403)
>>> +++
>>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/Bundle.properties
>>> (Arbeitskopie)
>>> @@ -27,7 +27,7 @@
>>> EXC_ClassLoadingError=Error during loading of class ''{0}'': {1}.
>>>
>>> #
>>> -# ReflectionJavaType
>>> +# BaseReflectionJavaType
>>> #
>>>
>>> # {0} - error location (class.method)
>>> @@ -35,6 +35,16 @@
>>> ERR_InvalidNullClassInstance={0}: specified Class instance is null.
>>>
>>> #
>>> +# ReflectionJavaType
>>> +#
>>> +
>>> +# {0} - error location (class.method)
>>> +# {1} - implementation method name
>>> +# {2} - field name
>>> +#NOI18N
>>> +ERR_MultipleJavaField={0}: multiple JavaField ''{1}'' for class
>>> ''{2}''.
>>> +
>>> +#
>>> # BaseReflectionJavaField
>>> #
>>> # {0} - class name
>>
>>
>
>
--
Michael Bouschen Tech@Spree Engineering GmbH
mailto:mbo.tech@spree.de http://www.tech.spree.de/
Tel.:++49/30/235 520-33 Buelowstr. 66
Fax.:++49/30/2175 2012 D-10783 Berlin
Re: class loader issue in java model - ri11 security manager tests
failing
Posted by Michael Bouschen <mb...@spree.de>.
Hi Martin,
thanks for testing! All your remarks make sense. I changed the code
accordingly and checked it in.
Regards Michael
>
> Michael,
> I tested your JDO1 patch with JDK 1.4 in a clean workspace, and
> all security manager tests run fine.
>
> I mostly looked at the code additions to RuntimeJavaModel.java,
> which are well documented (no further questions on the code).
> Also, the getJavaModel() and getDeclaringJavaModelFactory()
> changes look good. I was surprised about the amount of changes,
> but to me the classloading code has gained clarity.
>
> Only true cosmetics:
> - 'instacne' misspelled twice in RuntimeJavaModel
> - "class instance" might be ambiguous, consider "Class instance"
> since the comments refer to an instance of java.lang.Class
> - The 'true' value passed to the two invocations of forNamePrivileged()
> leaves one guessing, though the code comments explain the intent.
> Consider:
> final boolean initialize = true;
> RuntimeJavaModelFactory.forNamePrivileged(..., initialize,...)
> - In getJavaType(Class), somewhere after
> if (javaType == null) { ...
> I'd expected a comment (or assertion) about the special handling
> of pre-defined types, i.e., that 'javaType' cannot be a PreDefined
> type. Correct? The javadoc on that class already has it, so it's
> not a real lack of information there, just redundancy.
>
> Martin
>
> Michael Bouschen wrote:
>
>>
>> I have implemented the JavaModel changes as we discussed. Now the
>> Class.forName is called in a doPrivileged block to avoid the
>> SecurityException.
>> The other change is that I moved all the code that checks for the
>> right class loader to the RuntimeJavaModel. A JavaType lookup by name
>> first gets the class instance using the class loader bound to the
>> model instance. Then it gets the class loader from the class instance
>> and delegates to JavaModel bound to this class loader. This makes
>> sure that a JavaModel handles only classes that are loaded by th
>> class loader bound to the JavaModel. There is one exception: all
>> JavaModel instances know about all the PredefinedType instances
>> (JavaType instances for primitive types, Java wrapper class,
>> java.util classes etc.).
>>
>> Attached find two patches created from the trunk(!) directory:
>> - JDO1-JavaModel.patch: ri11 changes. I tested this running the ri11
>> tests and tck11 in a jdk1.5 environment (I'm using the enhancer
>> changes you send over).
>> - JDO2-JavaModel.patch: the corresponding changes for JDO2. This
>> changes classes in core20, enhancer20, runtime20 and fostore20. I run
>> the fostore20 tests with jdk 1.4.
>>
>> Please let me know what you think.
>
--
Michael Bouschen Tech@Spree Engineering GmbH
mailto:mbo.tech@spree.de http://www.tech.spree.de/
Tel.:++49/30/235 520-33 Buelowstr. 66
Fax.:++49/30/2175 2012 D-10783 Berlin
Re: class loader issue in java model - ri11 security manager tests
failing
Posted by Martin Zaun <Ma...@Sun.COM>.
Michael,
I tested your JDO1 patch with JDK 1.4 in a clean workspace, and
all security manager tests run fine.
I mostly looked at the code additions to RuntimeJavaModel.java,
which are well documented (no further questions on the code).
Also, the getJavaModel() and getDeclaringJavaModelFactory()
changes look good. I was surprised about the amount of changes,
but to me the classloading code has gained clarity.
Only true cosmetics:
- 'instacne' misspelled twice in RuntimeJavaModel
- "class instance" might be ambiguous, consider "Class instance"
since the comments refer to an instance of java.lang.Class
- The 'true' value passed to the two invocations of forNamePrivileged()
leaves one guessing, though the code comments explain the intent.
Consider:
final boolean initialize = true;
RuntimeJavaModelFactory.forNamePrivileged(..., initialize,...)
- In getJavaType(Class), somewhere after
if (javaType == null) { ...
I'd expected a comment (or assertion) about the special handling
of pre-defined types, i.e., that 'javaType' cannot be a PreDefined
type. Correct? The javadoc on that class already has it, so it's
not a real lack of information there, just redundancy.
Martin
Michael Bouschen wrote:
>
> I have implemented the JavaModel changes as we discussed. Now the
> Class.forName is called in a doPrivileged block to avoid the
> SecurityException.
> The other change is that I moved all the code that checks for the right
> class loader to the RuntimeJavaModel. A JavaType lookup by name first
> gets the class instance using the class loader bound to the model
> instance. Then it gets the class loader from the class instance and
> delegates to JavaModel bound to this class loader. This makes sure that
> a JavaModel handles only classes that are loaded by th class loader
> bound to the JavaModel. There is one exception: all JavaModel instances
> know about all the PredefinedType instances (JavaType instances for
> primitive types, Java wrapper class, java.util classes etc.).
>
> Attached find two patches created from the trunk(!) directory:
> - JDO1-JavaModel.patch: ri11 changes. I tested this running the ri11
> tests and tck11 in a jdk1.5 environment (I'm using the enhancer changes
> you send over).
> - JDO2-JavaModel.patch: the corresponding changes for JDO2. This changes
> classes in core20, enhancer20, runtime20 and fostore20. I run the
> fostore20 tests with jdk 1.4.
>
> Please let me know what you think.
Re: class loader issue in java model - ri11 security manager tests
failing
Posted by Michael Bouschen <mb...@spree.de>.
Hi Martin,
I have implemented the JavaModel changes as we discussed. Now the
Class.forName is called in a doPrivileged block to avoid the
SecurityException.
The other change is that I moved all the code that checks for the right
class loader to the RuntimeJavaModel. A JavaType lookup by name first
gets the class instance using the class loader bound to the model
instance. Then it gets the class loader from the class instance and
delegates to JavaModel bound to this class loader. This makes sure that
a JavaModel handles only classes that are loaded by th class loader
bound to the JavaModel. There is one exception: all JavaModel instances
know about all the PredefinedType instances (JavaType instances for
primitive types, Java wrapper class, java.util classes etc.).
Attached find two patches created from the trunk(!) directory:
- JDO1-JavaModel.patch: ri11 changes. I tested this running the ri11
tests and tck11 in a jdk1.5 environment (I'm using the enhancer changes
you send over).
- JDO2-JavaModel.patch: the corresponding changes for JDO2. This changes
classes in core20, enhancer20, runtime20 and fostore20. I run the
fostore20 tests with jdk 1.4.
Please let me know what you think.
Regards Michael
>
> Hi Michael,
>
> unfortunately, the fix for the class registration problem, as we
> discussed it and as it was checked in, causes most of the ri11
> security manager junit tests to fail (138 out of 149 tests).
>
> The symptom is an AccessControlException ("getClassLoader") within
> method Class.forName(), called from ReflectionJavaModel.java:130.
>
> The issue is exposed by an almost trivial change that we discussed
> (see below): When calling Class.forName() to initialize a class, we
> decided to use the class' classloader instead of the one stored in
> the JavaModel instance. As you pointed out, we then must fetch the
> classloader in a doPrivileged block. (This block succeeds, the
> exception is really raised within Class.forName()).
>
> Though the code assumes that the stored and the fetched classloader
> are always the same here, we discussed that if they're not, this
> would result in an internal error that is very hard to track down,
> since the consequences might show up later and elsewhere depending
> on the order in which classes are registered. We considered to
> guard against this case with an assertion or an explicit check.
>
> It turns out the stored and the fetched classloader may differ for
> two reasons, I think.
>
> First, clazz.getClassLoader() returns null for a number of classes
> like Date and ArrayList. Craig has pointed out that this is the
> specified behaviour for classes loaded by the bootstrap classloader.
>
> The javadoc on Class.forName() says that if the loader argument is
> null (and other conditions apply), the security manager is checked
> for permission ("getClassLoader") to access the bootstrap class
> loader. This check fails.
>
> When using the stored classloader, which is always non-null, instead
> of the fetched one as argument to clazz.forName(), this method does
> not issue a "getClassLoader" request, and all tests security manager
> junit tests pass then.
>
> But I'm not sure this is the right solution either. Essentially,
> we'd represent Date, ArrayList etc. a multiple times in the Java
> model - in each model instance per application classloader - while
> they're only represented once in the JVM, by the bootstrap loader.
>
> There might be another case to consider why the stored and the
> fetched classloader may differ: An application that uses multiple
> classloaders forming a parent-child hierarchy, it may happen that
> a PC class is loaded by a child classloader while it's superclasses
> or the persistent field types are loaded by a parent classloader.
>
> In this case, we probably do not want to have the type universe be
> duplicated in each java model instance (i.e. per classloader), but
> rather represent a class in the java model only as many times as
> it's loaded in the VM, that is, per "owning" classloader.
>
> Now, I do not know if we're already doing this (class represented in
> java model only for "owning" classloader), but it seems to me that
> the implicit assumption in ReflectionJavaModel.getJavaType() that the
> stored and the fetched classloader are always the same does not hold.
>
> That's why I'm not sure that always using the stored classloader
> (instead of the fetched one) would be a general solution either.
>
> Any thoughts?
>
> Sorry for the long email. If you'd rather like to discuss details
> over phone, I'd have some time on Thursday morning (until 9:30am) or
> on Friday during or after our JDO phone con. Craig and I discussed
> this issue briefly today.
>
> Thx,
> Martin
>
--
Michael Bouschen Tech@Spree Engineering GmbH
mailto:mbo.tech@spree.de http://www.tech.spree.de/
Tel.:++49/30/235 520-33 Buelowstr. 66
Fax.:++49/30/2175 2012 D-10783 Berlin
Re: class loader issue in java model - ri11 security manager tests
failing
Posted by Martin Zaun <Ma...@Sun.COM>.
Michael,
Michael Bouschen wrote On 10/12/05 12:18,:
> thanks for the info! Maybe it makes sense to discuss this over the
> phone. I have a meeting from 9-10am PDT tomorrow, so we could talk
> before 9am (which is a little early for you) or on Friday.
Thursday 7:30-9 am would be OK for me. Please, call me at home.
Thx,
Martin
Re: class loader issue in java model - ri11 security manager tests
failing
Posted by Michael Bouschen <mb...@spree.de>.
Hi Martin,
thanks for the info! Maybe it makes sense to discuss this over the
phone. I have a meeting from 9-10am PDT tomorrow, so we could talk
before 9am (which is a little early for you) or on Friday.
Regards Michael
>
> Hi Michael,
>
> unfortunately, the fix for the class registration problem, as we
> discussed it and as it was checked in, causes most of the ri11
> security manager junit tests to fail (138 out of 149 tests).
>
> The symptom is an AccessControlException ("getClassLoader") within
> method Class.forName(), called from ReflectionJavaModel.java:130.
>
> The issue is exposed by an almost trivial change that we discussed
> (see below): When calling Class.forName() to initialize a class, we
> decided to use the class' classloader instead of the one stored in
> the JavaModel instance. As you pointed out, we then must fetch the
> classloader in a doPrivileged block. (This block succeeds, the
> exception is really raised within Class.forName()).
>
> Though the code assumes that the stored and the fetched classloader
> are always the same here, we discussed that if they're not, this
> would result in an internal error that is very hard to track down,
> since the consequences might show up later and elsewhere depending
> on the order in which classes are registered. We considered to
> guard against this case with an assertion or an explicit check.
>
> It turns out the stored and the fetched classloader may differ for
> two reasons, I think.
>
> First, clazz.getClassLoader() returns null for a number of classes
> like Date and ArrayList. Craig has pointed out that this is the
> specified behaviour for classes loaded by the bootstrap classloader.
>
> The javadoc on Class.forName() says that if the loader argument is
> null (and other conditions apply), the security manager is checked
> for permission ("getClassLoader") to access the bootstrap class
> loader. This check fails.
>
> When using the stored classloader, which is always non-null, instead
> of the fetched one as argument to clazz.forName(), this method does
> not issue a "getClassLoader" request, and all tests security manager
> junit tests pass then.
>
> But I'm not sure this is the right solution either. Essentially,
> we'd represent Date, ArrayList etc. a multiple times in the Java
> model - in each model instance per application classloader - while
> they're only represented once in the JVM, by the bootstrap loader.
>
> There might be another case to consider why the stored and the
> fetched classloader may differ: An application that uses multiple
> classloaders forming a parent-child hierarchy, it may happen that
> a PC class is loaded by a child classloader while it's superclasses
> or the persistent field types are loaded by a parent classloader.
>
> In this case, we probably do not want to have the type universe be
> duplicated in each java model instance (i.e. per classloader), but
> rather represent a class in the java model only as many times as
> it's loaded in the VM, that is, per "owning" classloader.
>
> Now, I do not know if we're already doing this (class represented in
> java model only for "owning" classloader), but it seems to me that
> the implicit assumption in ReflectionJavaModel.getJavaType() that the
> stored and the fetched classloader are always the same does not hold.
>
> That's why I'm not sure that always using the stored classloader
> (instead of the fetched one) would be a general solution either.
>
> Any thoughts?
>
> Sorry for the long email. If you'd rather like to discuss details
> over phone, I'd have some time on Thursday morning (until 9:30am) or
> on Friday during or after our JDO phone con. Craig and I discussed
> this issue briefly today.
>
> Thx,
> Martin
>
> Michael Bouschen wrote:
>
>>
>> About the issue of a class argument of a wrong classloader passed to
>> the JavaModel method getJavaType: I agree the code should use the
>> class loader from the class object. The only issue is that the call
>> clazz.getClassLoader() might result in a SecurityException, so I need
>> to put this call into a doPrivileged block. I already have a
>> convenience method in RuntimeJavaModelFactory, I just need to move
>> this method to a class that is accessible in both places.
>>
>>>
>>> Michael Bouschen wrote:
>>>
>>>> Hi Martin,
>>>>
>>>> attached you find a patch for the JDOModel implementation. It
>>>> initializes a class instance if the model instance runs with the
>>>> initialize=true flag. It also fixes the MissingResourceException. I
>>>> could successfully run the ri tests in a workspace including your
>>>> enhancer plus my JDOModel changes.
>>>> I would port the JDOModel changes to core20 and check them in, if
>>>> you agree with the changes.
>>>>
>>>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/reflection/ReflectionJavaModel.java
>>>> public JavaType getJavaType(Class clazz)
>>>> {
>>>> - String name = clazz.getName();
>>>> - synchronized (types) {
>>>> - JavaType javaType = (JavaType)types.get(name);
>>>> - if (javaType == null) {
>>>> - javaType = createJavaType(clazz);
>>>> - types.put(name, javaType);
>>>> + if (clazz == null)
>>>> + return null;
>>>> + + if (initialize) {
>>>> + try {
>>>> + // make sure the class is initialized
>>>> + Class.forName(clazz.getName(), initialize,
>>>> classLoader);
>>>
>>>
>>>
>>> Since this is a public method, there's a chance for a bug that we're
>>> called here with a class argument of a wrong classloader, I think.
>>> If that happened, we'd load and initialize the class in the wrong
>>> place. So, I'd change this line, making an assumption explicit, to:
>>> Class.forName(clazz.getName(), initialize,
>>> clazz.getClassLoader());
>>> Or, if we allowed for assertions, we could keep the line but just add:
>>> assert (classLoader == clazz.getClassLoader());
>>>
>>>> }
--
Michael Bouschen Tech@Spree Engineering GmbH
mailto:mbo.tech@spree.de http://www.tech.spree.de/
Tel.:++49/30/235 520-33 Buelowstr. 66
Fax.:++49/30/2175 2012 D-10783 Berlin
Re: class loader issue in java model - ri11 security manager tests
failing
Posted by Martin Zaun <Ma...@Sun.COM>.
Martin Zaun wrote:
>
> Hi Michael, ...
Sorry, I meant Michael B. :)
Martin
class loader issue in java model - ri11 security manager tests failing
Posted by Martin Zaun <Ma...@Sun.COM>.
Hi Michael,
unfortunately, the fix for the class registration problem, as we
discussed it and as it was checked in, causes most of the ri11
security manager junit tests to fail (138 out of 149 tests).
The symptom is an AccessControlException ("getClassLoader") within
method Class.forName(), called from ReflectionJavaModel.java:130.
The issue is exposed by an almost trivial change that we discussed
(see below): When calling Class.forName() to initialize a class, we
decided to use the class' classloader instead of the one stored in
the JavaModel instance. As you pointed out, we then must fetch the
classloader in a doPrivileged block. (This block succeeds, the
exception is really raised within Class.forName()).
Though the code assumes that the stored and the fetched classloader
are always the same here, we discussed that if they're not, this
would result in an internal error that is very hard to track down,
since the consequences might show up later and elsewhere depending
on the order in which classes are registered. We considered to
guard against this case with an assertion or an explicit check.
It turns out the stored and the fetched classloader may differ for
two reasons, I think.
First, clazz.getClassLoader() returns null for a number of classes
like Date and ArrayList. Craig has pointed out that this is the
specified behaviour for classes loaded by the bootstrap classloader.
The javadoc on Class.forName() says that if the loader argument is
null (and other conditions apply), the security manager is checked
for permission ("getClassLoader") to access the bootstrap class
loader. This check fails.
When using the stored classloader, which is always non-null, instead
of the fetched one as argument to clazz.forName(), this method does
not issue a "getClassLoader" request, and all tests security manager
junit tests pass then.
But I'm not sure this is the right solution either. Essentially,
we'd represent Date, ArrayList etc. a multiple times in the Java
model - in each model instance per application classloader - while
they're only represented once in the JVM, by the bootstrap loader.
There might be another case to consider why the stored and the
fetched classloader may differ: An application that uses multiple
classloaders forming a parent-child hierarchy, it may happen that
a PC class is loaded by a child classloader while it's superclasses
or the persistent field types are loaded by a parent classloader.
In this case, we probably do not want to have the type universe be
duplicated in each java model instance (i.e. per classloader), but
rather represent a class in the java model only as many times as
it's loaded in the VM, that is, per "owning" classloader.
Now, I do not know if we're already doing this (class represented in
java model only for "owning" classloader), but it seems to me that
the implicit assumption in ReflectionJavaModel.getJavaType() that the
stored and the fetched classloader are always the same does not hold.
That's why I'm not sure that always using the stored classloader
(instead of the fetched one) would be a general solution either.
Any thoughts?
Sorry for the long email. If you'd rather like to discuss details
over phone, I'd have some time on Thursday morning (until 9:30am) or
on Friday during or after our JDO phone con. Craig and I discussed
this issue briefly today.
Thx,
Martin
Michael Bouschen wrote:
>
> About the issue of a class argument of a wrong classloader passed to the
> JavaModel method getJavaType: I agree the code should use the class
> loader from the class object. The only issue is that the call
> clazz.getClassLoader() might result in a SecurityException, so I need to
> put this call into a doPrivileged block. I already have a convenience
> method in RuntimeJavaModelFactory, I just need to move this method to a
> class that is accessible in both places.
>>
>> Michael Bouschen wrote:
>>
>>> Hi Martin,
>>>
>>> attached you find a patch for the JDOModel implementation. It
>>> initializes a class instance if the model instance runs with the
>>> initialize=true flag. It also fixes the MissingResourceException. I
>>> could successfully run the ri tests in a workspace including your
>>> enhancer plus my JDOModel changes.
>>> I would port the JDOModel changes to core20 and check them in, if you
>>> agree with the changes.
>>>
>>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/reflection/ReflectionJavaModel.java
>>> public JavaType getJavaType(Class clazz)
>>> {
>>> - String name = clazz.getName();
>>> - synchronized (types) {
>>> - JavaType javaType = (JavaType)types.get(name);
>>> - if (javaType == null) {
>>> - javaType = createJavaType(clazz);
>>> - types.put(name, javaType);
>>> + if (clazz == null)
>>> + return null;
>>> + + if (initialize) {
>>> + try {
>>> + // make sure the class is initialized
>>> + Class.forName(clazz.getName(), initialize,
>>> classLoader);
>>
>>
>> Since this is a public method, there's a chance for a bug that we're
>> called here with a class argument of a wrong classloader, I think.
>> If that happened, we'd load and initialize the class in the wrong
>> place. So, I'd change this line, making an assumption explicit, to:
>> Class.forName(clazz.getName(), initialize,
>> clazz.getClassLoader());
>> Or, if we allowed for assertions, we could keep the line but just add:
>> assert (classLoader == clazz.getClassLoader());
>>
>>> }
Re: status update: ri11 enhancer support for jdk 1.5
Posted by Michael Bouschen <mb...@spree.de>.
Hi Martin,
thanks for the comments!
About the issue of a class argument of a wrong classloader passed to the
JavaModel method getJavaType: I agree the code should use the class
loader from the class object. The only issue is that the call
clazz.getClassLoader() might result in a SecurityException, so I need to
put this call into a doPrivileged block. I already have a convenience
method in RuntimeJavaModelFactory, I just need to move this method to a
class that is accessible in both places.
Regards Michael
>
> Hi Michael,
> thanks for the patch, only a few cosmetic comments inline, below.
> I'll send out a summary of our discussion on the jdk 1.5 class
> registration problem and the chosen approach later.
> Martin
>
> Michael Bouschen wrote:
>
>> Hi Martin,
>>
>> attached you find a patch for the JDOModel implementation. It
>> initializes a class instance if the model instance runs with the
>> initialize=true flag. It also fixes the MissingResourceException. I
>> could successfully run the ri tests in a workspace including your
>> enhancer plus my JDOModel changes.
>> I would port the JDOModel changes to core20 and check them in, if you
>> agree with the changes.
>>
>> Regards Michael
>>
>>>
>>> Michael,
>>> attached is a stable subset of my ri11 enhancer
>>> changes that produce the problem with Test_Query.
>>> Unzip in trunk and run 'maven build' from ri11.
>>> Talk to you next week,
>>> Martin
>>
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> Index:
>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/runtime/Bundle.properties
>>
>> ===================================================================
>> ---
>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/runtime/Bundle.properties
>> (Revision 292403)
>> +++
>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/runtime/Bundle.properties
>> (Arbeitskopie)
>> @@ -62,12 +62,3 @@
>> #NOI18N
>> ERR_CannotSetJDOModel=Cannot set JDOModel for JavaModel instance.
>>
>> -#
>> -# RuntimeJavaType
>> -#
>> -
>> -# {0} - error location (class.method)
>> -# {1} - implementation method name
>> -# {2} - field name
>> -#NOI18N
>> -ERR_MultipleJavaField={0}: multiple JavaField ''{1}'' for class '{2}''.
>> Index:
>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/reflection/ReflectionJavaModel.java
>>
>> ===================================================================
>> ---
>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/reflection/ReflectionJavaModel.java
>> (Revision 292403)
>> +++
>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/reflection/ReflectionJavaModel.java
>> (Arbeitskopie)
>> @@ -87,8 +87,8 @@
>> // loaded, Class.forName will load the class which
>> // calls RegisterClassListener.registerClass.
>> // This will create a new JavaType entry in the
>> cache.
>> - javaType = getJavaType(Class.forName(name,
>> initialize, -
>> classLoader));
>> + javaType = getJavaTypeInternal(
>> + Class.forName(name, initialize, classLoader));
>> }
>> catch (ClassNotFoundException ex) {
>> // cannot find class => return null
>> @@ -121,15 +121,20 @@
>> */
>> public JavaType getJavaType(Class clazz)
>> {
>> - String name = clazz.getName();
>> - synchronized (types) {
>> - JavaType javaType = (JavaType)types.get(name);
>> - if (javaType == null) {
>> - javaType = createJavaType(clazz);
>> - types.put(name, javaType);
>> + if (clazz == null)
>> + return null;
>> + + if (initialize) {
>> + try {
>> + // make sure the class is initialized
>> + Class.forName(clazz.getName(), initialize,
>> classLoader);
>
>
> Since this is a public method, there's a chance for a bug that we're
> called here with a class argument of a wrong classloader, I think.
> If that happened, we'd load and initialize the class in the wrong
> place. So, I'd change this line, making an assumption explicit, to:
> Class.forName(clazz.getName(), initialize,
> clazz.getClassLoader());
> Or, if we allowed for assertions, we could keep the line but just add:
> assert (classLoader == clazz.getClassLoader());
>
>> }
>> - return javaType;
>> + catch (ClassNotFoundException ex) {
>> + // ignore
>
>
> I'd add:
> // ignore, since class has already been loaded
>
>> + }
>> }
>> +
>> + return getJavaTypeInternal(clazz);
>> }
>>
>> /**
>> @@ -170,6 +175,28 @@
>> return classLoader;
>> }
>>
>> + /**
>> + * The method returns the JavaType instance for the type name of
>> the
>> + * specified class object. It first checks the cache and if
>> there is no
>> + * entry for the type name in the cache then it creates a new
>> JavaType
>> + * instance for the specified Class object.
>> + * @param clazz the Class instance representing the type
>> + * @return a JavaType instance for the name of the specified class
>> + * object or <code>null</code> if not present in this model
>> instance.
>> + */
>> + protected JavaType getJavaTypeInternal(Class clazz)
>> + {
>> + String name = clazz.getName();
>> + synchronized (types) {
>> + JavaType javaType = (JavaType)types.get(name);
>> + if (javaType == null) {
>> + javaType = createJavaType(clazz);
>> + types.put(name, javaType);
>> + }
>> + return javaType;
>> + }
>> + }
>> +
>> /** * Creates a new JavaType instance for the specified
>> Class object.
>> * This method provides a hook such that ReflectionJavaModel
>> subclasses
>> Index:
>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/Bundle.properties
>> ===================================================================
>> ---
>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/Bundle.properties
>> (Revision 292403)
>> +++
>> trunk/ri11/src/java/org/apache/jdo/impl/model/java/Bundle.properties
>> (Arbeitskopie)
>> @@ -27,7 +27,7 @@
>> EXC_ClassLoadingError=Error during loading of class ''{0}'': {1}.
>>
>> #
>> -# ReflectionJavaType
>> +# BaseReflectionJavaType
>> #
>>
>> # {0} - error location (class.method)
>> @@ -35,6 +35,16 @@
>> ERR_InvalidNullClassInstance={0}: specified Class instance is null.
>>
>> #
>> +# ReflectionJavaType
>> +#
>> +
>> +# {0} - error location (class.method)
>> +# {1} - implementation method name
>> +# {2} - field name
>> +#NOI18N
>> +ERR_MultipleJavaField={0}: multiple JavaField ''{1}'' for class
>> ''{2}''.
>> +
>> +#
>> # BaseReflectionJavaField
>> #
>> # {0} - class name
>
--
Michael Bouschen Tech@Spree Engineering GmbH
mailto:mbo.tech@spree.de http://www.tech.spree.de/
Tel.:++49/30/235 520-33 Buelowstr. 66
Fax.:++49/30/2175 2012 D-10783 Berlin
Re: status update: ri11 enhancer support for jdk 1.5
Posted by Martin Zaun <Ma...@Sun.COM>.
Hi Michael,
thanks for the patch, only a few cosmetic comments inline, below.
I'll send out a summary of our discussion on the jdk 1.5 class
registration problem and the chosen approach later.
Martin
Michael Bouschen wrote:
> Hi Martin,
>
> attached you find a patch for the JDOModel implementation. It
> initializes a class instance if the model instance runs with the
> initialize=true flag. It also fixes the MissingResourceException. I
> could successfully run the ri tests in a workspace including your
> enhancer plus my JDOModel changes.
> I would port the JDOModel changes to core20 and check them in, if you
> agree with the changes.
>
> Regards Michael
>
>>
>> Michael,
>> attached is a stable subset of my ri11 enhancer
>> changes that produce the problem with Test_Query.
>> Unzip in trunk and run 'maven build' from ri11.
>> Talk to you next week,
>> Martin
>
>
>
>
>
> ------------------------------------------------------------------------
>
> Index: trunk/ri11/src/java/org/apache/jdo/impl/model/java/runtime/Bundle.properties
> ===================================================================
> --- trunk/ri11/src/java/org/apache/jdo/impl/model/java/runtime/Bundle.properties (Revision 292403)
> +++ trunk/ri11/src/java/org/apache/jdo/impl/model/java/runtime/Bundle.properties (Arbeitskopie)
> @@ -62,12 +62,3 @@
> #NOI18N
> ERR_CannotSetJDOModel=Cannot set JDOModel for JavaModel instance.
>
> -#
> -# RuntimeJavaType
> -#
> -
> -# {0} - error location (class.method)
> -# {1} - implementation method name
> -# {2} - field name
> -#NOI18N
> -ERR_MultipleJavaField={0}: multiple JavaField ''{1}'' for class '{2}''.
> Index: trunk/ri11/src/java/org/apache/jdo/impl/model/java/reflection/ReflectionJavaModel.java
> ===================================================================
> --- trunk/ri11/src/java/org/apache/jdo/impl/model/java/reflection/ReflectionJavaModel.java (Revision 292403)
> +++ trunk/ri11/src/java/org/apache/jdo/impl/model/java/reflection/ReflectionJavaModel.java (Arbeitskopie)
> @@ -87,8 +87,8 @@
> // loaded, Class.forName will load the class which
> // calls RegisterClassListener.registerClass.
> // This will create a new JavaType entry in the cache.
> - javaType = getJavaType(Class.forName(name, initialize,
> - classLoader));
> + javaType = getJavaTypeInternal(
> + Class.forName(name, initialize, classLoader));
> }
> catch (ClassNotFoundException ex) {
> // cannot find class => return null
> @@ -121,15 +121,20 @@
> */
> public JavaType getJavaType(Class clazz)
> {
> - String name = clazz.getName();
> - synchronized (types) {
> - JavaType javaType = (JavaType)types.get(name);
> - if (javaType == null) {
> - javaType = createJavaType(clazz);
> - types.put(name, javaType);
> + if (clazz == null)
> + return null;
> +
> + if (initialize) {
> + try {
> + // make sure the class is initialized
> + Class.forName(clazz.getName(), initialize, classLoader);
Since this is a public method, there's a chance for a bug that we're
called here with a class argument of a wrong classloader, I think.
If that happened, we'd load and initialize the class in the wrong
place. So, I'd change this line, making an assumption explicit, to:
Class.forName(clazz.getName(), initialize, clazz.getClassLoader());
Or, if we allowed for assertions, we could keep the line but just add:
assert (classLoader == clazz.getClassLoader());
> }
> - return javaType;
> + catch (ClassNotFoundException ex) {
> + // ignore
I'd add:
// ignore, since class has already been loaded
> + }
> }
> +
> + return getJavaTypeInternal(clazz);
> }
>
> /**
> @@ -170,6 +175,28 @@
> return classLoader;
> }
>
> + /**
> + * The method returns the JavaType instance for the type name of the
> + * specified class object. It first checks the cache and if there is no
> + * entry for the type name in the cache then it creates a new JavaType
> + * instance for the specified Class object.
> + * @param clazz the Class instance representing the type
> + * @return a JavaType instance for the name of the specified class
> + * object or <code>null</code> if not present in this model instance.
> + */
> + protected JavaType getJavaTypeInternal(Class clazz)
> + {
> + String name = clazz.getName();
> + synchronized (types) {
> + JavaType javaType = (JavaType)types.get(name);
> + if (javaType == null) {
> + javaType = createJavaType(clazz);
> + types.put(name, javaType);
> + }
> + return javaType;
> + }
> + }
> +
> /**
> * Creates a new JavaType instance for the specified Class object.
> * This method provides a hook such that ReflectionJavaModel subclasses
> Index: trunk/ri11/src/java/org/apache/jdo/impl/model/java/Bundle.properties
> ===================================================================
> --- trunk/ri11/src/java/org/apache/jdo/impl/model/java/Bundle.properties (Revision 292403)
> +++ trunk/ri11/src/java/org/apache/jdo/impl/model/java/Bundle.properties (Arbeitskopie)
> @@ -27,7 +27,7 @@
> EXC_ClassLoadingError=Error during loading of class ''{0}'': {1}.
>
> #
> -# ReflectionJavaType
> +# BaseReflectionJavaType
> #
>
> # {0} - error location (class.method)
> @@ -35,6 +35,16 @@
> ERR_InvalidNullClassInstance={0}: specified Class instance is null.
>
> #
> +# ReflectionJavaType
> +#
> +
> +# {0} - error location (class.method)
> +# {1} - implementation method name
> +# {2} - field name
> +#NOI18N
> +ERR_MultipleJavaField={0}: multiple JavaField ''{1}'' for class ''{2}''.
> +
> +#
> # BaseReflectionJavaField
> #
> # {0} - class name
Re: status update: ri11 enhancer support for jdk 1.5
Posted by Martin Zaun <Ma...@Sun.COM>.
Michael,
Craig and I had an in-depth discussion this afternoon on the
order-of-class-registration problem. Here's a summary:
1) The query's semantic analysis requires java metadata (i.e.
name/type/existence of fields/classes), which we obtain by
reflection.
Reflection requires a class to be loaded and linked but not
neccessarily initialized (=static initializer executed).
The vmspec only mandates that initialization happens before
- an instance of the class is created,
- a static method of the class is invoked, or
- a nonconstant static field is used (read or assigned).
2) Apparently, there's a difference between source=1.4 and 1.5
on when class initialization happens. Source=1.5 seems to
result in a more deferred scheme.
This seems to explain the difference in the order of
registration of PC classes that we observe.
3) Does the query's semantic analysis require genuine JDO
metadata ?
JDO allows for in-memory query on even non-PC classes, if
I recall correctly. So, that would suggest that for query
compilation, we shouldn't require JDO metadata, while for
query execution we may need JDO (and mapping) metadata.
4) However, during/for the semantic analysis both models, Java
and JDO, seem to be populated (e.g JavaField and JDOField).
While JDOField has a PersistenceModifier status that covers
non-managed fields, a JDOClass instance always represents a
PC class. Is that a problem if we consider queries over
non-PC classes?
5) Depending on the answer to question 3), if the semantic
analysis needs all classes that are "touched" in a query
(navigational expressions) have registered, the query
compilation could enforce class initialization by calling
Class.forName().
As a hack, we changed QueryImpl.setClass() to ensure the
the target class is initialized:
try {
Class.forName(cls.getName(), true, cls.getClassLoader());
} catch (ClassNotFoundException ex) {
throw new RuntimeException(ex);
}
This hack fixes Query_Test with source=1.5 (but wouldn't
cover navigational expressions).
6) Depending on the answer to question 3), if the semantic
analysis does NOT need the classes to have registered, we
may want consider allowing for a partially populated java
model (JavaFields) upon class registration.
For this, we'd have to change
ReflectionJavaField.createJavaField()
to not throwing an exception. Something like this:
String name = jdoField.getName();
synchronized(javaFields) {
JavaField javaField = (JavaField)javaFields.get(name);
if (javaField == null) {
javaField = new ReflectionJavaField(jdoField, type, this);
javaFields.put(name, javaField);
}
return javaField;
}
7) Another option to consider is not populating the Java
model during semantic analysis (but class registration
only).
> - For a test, I changed ReflectionJavaType.getDeclaredJavaField()
> not to store the JavaField field in the private HashMap
> 'javaFields', and Test_Query runs fine then. But this doesn't
> look like the right approach to me.
This looks odd to me for we'd create JavaFields that we
wouldn't store in the java model.
Thanks,
Martin
Re: status update: ri11 enhancer support for jdk 1.5
Posted by Martin Zaun <Ma...@Sun.COM>.
Michael,
after I had a closer look into Test_Query, I figured that you
might want to see a bit more stacktrace context around the
interesting points. So, more details below.
Martin
Martin Zaun wrote:
>
> - Checking the point of time when PC classes register with the
> runtime, it's different with source=1.5 versus 1.4!
>
> Here's the order with source=1.5, marking when 'deptid' get's
> inserted for the first time:
>
> +++ jdoClass.getName() = org.apache.jdo.pc.PCBase
> +++ jdoClass.getName() = org.apache.jdo.pc.PCDepartment1
> +++ jdoClass.getName() = org.apache.jdo.pc.PCInsurance1
> +++ jdoClass.getName() = org.apache.jdo.pc.PCEmployee1
> +++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Person
> +++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Project
> !!! ReflectionJavaType.getDeclaredJavaField() inserts 'deptid'
> as part of QueryImpl.compile(QueryImpl.java:454)
The full stacktrace at this point (line numbers +/- printlns):
at org.apache.jdo.impl.model.java.reflection.ReflectionJavaType.getDeclaredJavaField(ReflectionJavaType.java:241)
at org.apache.jdo.impl.model.java.reflection.ReflectionJavaType.getJavaField(ReflectionJavaType.java:177)
at org.apache.jdo.impl.jdoql.jdoqlc.Semantic.analyseDefinedIdentifier(Semantic.java:306)
at org.apache.jdo.impl.jdoql.jdoqlc.Semantic.identifier(Semantic.java:3328)
at org.apache.jdo.impl.jdoql.jdoqlc.Semantic.primary(Semantic.java:2839)
at org.apache.jdo.impl.jdoql.jdoqlc.Semantic.exprNoCheck(Semantic.java:1883)
at org.apache.jdo.impl.jdoql.jdoqlc.Semantic.expression(Semantic.java:1768)
at org.apache.jdo.impl.jdoql.jdoqlc.Semantic.relationalExpr(Semantic.java:2145)
at org.apache.jdo.impl.jdoql.jdoqlc.Semantic.exprNoCheck(Semantic.java:1822)
at org.apache.jdo.impl.jdoql.jdoqlc.Semantic.expression(Semantic.java:1768)
at org.apache.jdo.impl.jdoql.jdoqlc.Semantic.filter(Semantic.java:1264)
at org.apache.jdo.impl.jdoql.jdoqlc.Semantic.query(Semantic.java:1077)
at org.apache.jdo.impl.jdoql.jdoqlc.JDOQLC.semanticCheck(JDOQLC.java:261)
at org.apache.jdo.impl.jdoql.QueryImpl.compile(QueryImpl.java:454)
at org.apache.jdo.test.query.UnsupportedTest.unsupported001(UnsupportedTest.java:72)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.jdo.test.query.NegativeTest.runTest(NegativeTest.java:134)
at org.apache.jdo.test.query.NegativeTest.runAll(NegativeTest.java:93)
at org.apache.jdo.test.Test_Query.runQueryTestSuite(Test_Query.java:198)
at org.apache.jdo.test.Test_Query.testNegativeTests(Test_Query.java:117)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at junit.framework.TestCase.runTest(TestCase.java:154)
at org.apache.jdo.test.util.AbstractTest.runBare(AbstractTest.java:264)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:325)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:536)
> +++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Company
> +++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Department
At this point, in the registering of class Department, an attempt
is made to throw a ModelFatalException(ERR_MultipleJavaField):
at org.apache.jdo.impl.model.java.reflection.ReflectionJavaType.createJavaField(ReflectionJavaType.java:206)
at org.apache.jdo.impl.model.java.runtime.RegisterClassListener.updateJDOField(RegisterClassListener.java:173)
at org.apache.jdo.impl.model.java.runtime.RegisterClassListener.updateJDOClass(RegisterClassListener.java:154)
at org.apache.jdo.impl.model.java.runtime.RegisterClassListener.registerClass(RegisterClassListener.java:83)
at javax.jdo.spi.JDOImplHelper.registerClass(JDOImplHelper.java:288)
at org.apache.jdo.pc.xempdept.Department.<clinit>(Department.java)
Which causes this java.lang.ExceptionInInitializerError:
at org.apache.jdo.test.Test_Query.createObjects(Test_Query.java:346)
at org.apache.jdo.test.Test_Query.testMemoryQueries(Test_Query.java:129)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at junit.framework.TestCase.runTest(TestCase.java:154)
at org.apache.jdo.test.util.AbstractTest.runBare(AbstractTest.java:264)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:325)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:536)
>
> In contrast, here's the order with source=1.4, marking when
> 'deptid' get's inserted for the first time:
>
> +++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Person
> +++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Company
> +++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Department
> !!! ReflectionJavaType.createJavaField() inserts 'deptid'
> as part of Class.forName()
The full stacktrace at this point:
at org.apache.jdo.impl.model.java.reflection.ReflectionJavaType.createJavaField(ReflectionJavaType.java:213)
at org.apache.jdo.impl.model.java.runtime.RegisterClassListener.updateJDOField(RegisterClassListener.java:173)
at org.apache.jdo.impl.model.java.runtime.RegisterClassListener.updateJDOClass(RegisterClassListener.java:154)
at org.apache.jdo.impl.model.java.runtime.RegisterClassListener.registerClass(RegisterClassListener.java:83)
at javax.jdo.spi.JDOImplHelper.registerClass(JDOImplHelper.java:288)
at org.apache.jdo.pc.xempdept.Department.<clinit>(Department.java)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:164)
at org.apache.jdo.pc.xempdept.Person.sunjdo$classForName$(Person.java)
at org.apache.jdo.pc.xempdept.Employee.<clinit>(Employee.java)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:164)
at org.apache.jdo.test.query.QueryErrorTest.class$(QueryErrorTest.java:71)
at org.apache.jdo.test.query.QueryErrorTest.queryError003(QueryErrorTest.java:125)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.jdo.test.query.NegativeTest.runTest(NegativeTest.java:134)
at org.apache.jdo.test.query.NegativeTest.runAll(NegativeTest.java:93)
at org.apache.jdo.test.Test_Query.runQueryTestSuite(Test_Query.java:198)
at org.apache.jdo.test.Test_Query.testNegativeTests(Test_Query.java:117)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at junit.framework.TestCase.runTest(TestCase.java:154)
at org.apache.jdo.test.util.AbstractTest.runBare(AbstractTest.java:264)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:325)
at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:536)
> +++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Insurance
> +++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Employee
> +++ jdoClass.getName() = org.apache.jdo.pc.PCBase
> +++ jdoClass.getName() = org.apache.jdo.pc.PCDepartment1
> +++ jdoClass.getName() = org.apache.jdo.pc.PCInsurance1
> +++ jdoClass.getName() = org.apache.jdo.pc.PCEmployee1
> +++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Project
> +++ jdoClass.getName() = org.apache.jdo.pc.xempdept.FullTimeEmployee
> +++ jdoClass.getName() = org.apache.jdo.pc.xempdept.PartTimeEmployee
> +++ jdoClass.getName() = org.apache.jdo.pc.xempdept.PrimitiveTypes
Test succeeds.
Re: status update: ri11 enhancer support for jdk 1.5
Posted by Martin Zaun <Ma...@Sun.COM>.
Michael,
Craig found the cause of the MissingResourceException.
> Michael Bouschen wrote:
>>
>> some remarks:
>> - I'm surprised about the MissingResourceException for the key
>> ERR_MultipleJavaField. Is there any difference in the bundle lookup in
>> Java 5?
BaseReflectionJavaType loads
org/apache/jdo/impl/model/java/Bundle.properties
which doesn't have an entry for ERR_MultipleJavaField.
Instead, ERR_MultipleJavaField is declared in the subdir
org/apache/jdo/impl/model/java/runtime/Bundle.properties
However, these two bundles seem to overlap a lot.
Maybe, we should go with only the parent bundle file ?
Martin
Re: status update: ri11 enhancer support for jdk 1.5
Posted by Martin Zaun <Ma...@Sun.COM>.
Michael, Craig,
A status update on the RI11 enhancer changes for Java 5 support:
As it looks like, the junit test errors are not caused by the
enhancer changes but by order-of-class-loading issues with the
tests or model/query, which just haven't been exposed before.
I still don't have the all the pieces, but here are some facts:
- The junit tests errors occur when compiling with source=1.5,
while source=1.4 and target=1.5 runs fine.
- I've compared the disassembled, enhanced classes obtained by
source=1.5 against 1.4: They're practically equivalent. (One
difference, though, is the use of the unsynchronized class
StringBuilder instead of StringBuffer).
- The root cause for Test_Query to fail with source=1.5 is a
ModelFatalException("ERR_MultipleJavaField"). (Actually, this
exception is preempted by a MissingResourceException).
This happens during registration of class Department, when the
JavaModel complains that the field 'deptid' has already been
registered. The line numbers might be inacurate due to added
printlns:
at org.apache.jdo.impl.model.java.reflection.ReflectionJavaType.createJavaField(ReflectionJavaType.java:206)
at org.apache.jdo.impl.model.java.runtime.RegisterClassListener.updateJDOField(RegisterClassListener.java:173)
at org.apache.jdo.impl.model.java.runtime.RegisterClassListener.updateJDOClass(RegisterClassListener.java:154)
at org.apache.jdo.impl.model.java.runtime.RegisterClassListener.registerClass(RegisterClassListener.java:83)
at javax.jdo.spi.JDOImplHelper.registerClass(JDOImplHelper.java:288)
at org.apache.jdo.pc.xempdept.Department.<clinit>(Department.java)
- Turns out that with source=1.5, it turns out that the 'deptid'
field has been registred before with the model during a
query.compile():
at org.apache.jdo.impl.model.java.reflection.ReflectionJavaType.getDeclaredJavaField(ReflectionJavaType.java:241)
at org.apache.jdo.impl.model.java.reflection.ReflectionJavaType.getJavaField(ReflectionJavaType.java:177)
at org.apache.jdo.impl.jdoql.jdoqlc.Semantic.analyseDefinedIdentifier(Semantic.java:306)
...
at org.apache.jdo.impl.jdoql.jdoqlc.Semantic.query(Semantic.java:1077)
at org.apache.jdo.impl.jdoql.jdoqlc.JDOQLC.semanticCheck(JDOQLC.java:261)
at org.apache.jdo.impl.jdoql.QueryImpl.compile(QueryImpl.java:454)
...
at org.apache.jdo.test.Test_Query.runQueryTestSuite(Test_Query.java:198)
- For a test, I changed ReflectionJavaType.getDeclaredJavaField()
not to store the JavaField field in the private HashMap
'javaFields', and Test_Query runs fine then. But this doesn't
look like the right approach to me.
- Checking the point of time when PC classes register with the
runtime, it's different with source=1.5 versus 1.4!
Here's the order with source=1.5, marking when 'deptid' get's
inserted for the first time:
+++ jdoClass.getName() = org.apache.jdo.pc.PCBase
+++ jdoClass.getName() = org.apache.jdo.pc.PCDepartment1
+++ jdoClass.getName() = org.apache.jdo.pc.PCInsurance1
+++ jdoClass.getName() = org.apache.jdo.pc.PCEmployee1
+++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Person
+++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Project
!!! ReflectionJavaType.getDeclaredJavaField() inserts 'deptid'
as part of QueryImpl.compile(QueryImpl.java:454)
+++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Company
+++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Department
In contrast, here's the order with source=1.4, marking when
'deptid' get's inserted for the first time:
+++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Person
+++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Company
+++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Department
!!! ReflectionJavaType.createJavaField() inserts 'deptid'
as part of Class.forName()
+++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Insurance
+++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Employee
+++ jdoClass.getName() = org.apache.jdo.pc.PCBase
+++ jdoClass.getName() = org.apache.jdo.pc.PCDepartment1
+++ jdoClass.getName() = org.apache.jdo.pc.PCInsurance1
+++ jdoClass.getName() = org.apache.jdo.pc.PCEmployee1
+++ jdoClass.getName() = org.apache.jdo.pc.xempdept.Project
+++ jdoClass.getName() = org.apache.jdo.pc.xempdept.FullTimeEmployee
+++ jdoClass.getName() = org.apache.jdo.pc.xempdept.PartTimeEmployee
+++ jdoClass.getName() = org.apache.jdo.pc.xempdept.PrimitiveTypes
Michael, any comments?
Is it an issue with the test, the model not being tolerant
about already added fields, or query.compile() populating
the java model?
Michael Bouschen wrote:
>
> some remarks:
> - I'm surprised about the MissingResourceException for the key
> ERR_MultipleJavaField. Is there any difference in the bundle lookup in
> Java 5?
It's strange. I don't have an answer to that yet.
Thanks,
Martin
Re: status update: ri11 enhancer support for jdk 1.5
Posted by Michael Bouschen <mb...@spree.de>.
Hi Martin,
some remarks:
- About the ClassNotFoundException ...RefFSContextFactory:
You need to copy the jndi jars fscontext.jar and providerutil.jar to
trunk/lib/ext. The issue is that we cannot download the jndi jars
automatically, because you need to accept the license agreement. Have a
look at trunk/README.txt, section "JNDI implementation".
There is no need to change the jndi property in project.properties.
- Maven calls the junit tests as part of the build goal in the ri11
project. The test output is stored under target/test-results with a log
file per test class.
- I'm surprised about the MissingResourceException for the key
ERR_MultipleJavaField. Is there any difference in the bundle lookup in
Java 5?
I hope this helps.
Regards Michael
>
> Hi Craig,
>
> here's the current status of testing my RI11 enhancer changes for
> Java 5 support (running jdk 1.5.0_05):
>
> maven.compile.source = 1.3 1.3 1.5
> maven.compile.target = 1.3 1.5 1.5
> -------------------------------------------------------------------
> tck11 OK OK OK
> junit tests (maven build) OK OK 2 errors
> runtest.list/security OK OK 1+5 errors
> multiJVMtests error error error
> ClassNotFoundException: com.sun.jndi.fscontext.RefFSContextFactory
>
> So, the basic 1.5 classfile support seems to be working (target=1.5),
> but there are errors when compiling the ri11 and junit tests with
> source=1.5, into which I started looking.
>
> More details on the errors below.
>
> Martin
>
> ----------------------------------------------------------------------
>
> Failing JUnit tests:
>
> [junit] [ERROR] TEST org.apache.jdo.test.Test_JDOModel FAILED
> [junit] [ERROR] TEST org.apache.jdo.test.Test_Query FAILED
>
> How can I get more information on these failures?
>
> I tried modifying the log level in ri11/test/conf/
> logging.properties and simplelog.properties
> but with no success.
>
> ----------------------------------------------------------------------
>
> JDO RI test suite, runtest.list and runtest.security:
>
> Tests run: 149, Failures: 1, Errors: 5
>
> Details below.
>
> ----------------------------------------------------------------------
>
> [java] RUN Test_JDOModel.testDefaults FAILURE
>
> [java] There was 1 failure:
> [java] 1)
> testDefaults(org.apache.jdo.test.Test_JDOModel)junit.framework.Com
> parisonFailure: Wrong list of persistentSerializableFieldNumbers of
> class org.ap
> ache.jdo.pc.empdept.PCFullTimeEmployee expected:<..., 10...> but
> was:<......>
> [java] at
> org.apache.jdo.test.Test_JDOModel.verifyPCFullTimeEmployee(Te
> st_JDOModel.java:304)
> ...
> [java] Caused by: java.util.MissingResourceException: Can't find
> resource fo
> r bundle java.util.PropertyResourceBundle, key ERR_MultipleJavaField
>
> ----------------------------------------------------------------------
>
> [java] RUN Test_EmpDeptSerialization.test ERROR
>
> [java] 1)
> test(org.apache.jdo.test.Test_EmpDeptSerialization)java.lang.Excep
> tionInInitializerError
> [java] at
> org.apache.jdo.test.Test_EmpDeptSerialization.createObjects(T
> est_EmpDeptSerialization.java:105)
>
> ----------------------------------------------------------------------
>
> [java] RUN Test_Inheritance.test ERROR
>
> [java] 2)
> test(org.apache.jdo.test.Test_Inheritance)java.lang.NoClassDefFoun
> dError
> [java] at
> org.apache.jdo.test.EmpDeptSupport.createObjects(EmpDeptSuppo
> rt.java:76)
>
> ----------------------------------------------------------------------
>
> [java] RUN Test_StringOID.test ERROR
>
> [java] 3)
> test(org.apache.jdo.test.Test_StringOID)java.lang.NoClassDefFoundE
> rror
> [java] at
> org.apache.jdo.test.EmpDeptSupport.createObjects(EmpDeptSuppo
> rt.java:76)
>
> ----------------------------------------------------------------------
>
> [java] RUN Test_Query.testMemoryQueries ERROR
>
> [java] 4)
> testMemoryQueries(org.apache.jdo.test.Test_Query)java.lang.Excepti
> onInInitializerError
> [java] at
> org.apache.jdo.test.Test_Query.createObjects(Test_Query.java:
> 344)
> ...
> [java] Caused by: java.util.MissingResourceException: Can't find
> resource fo
> r bundle java.util.PropertyResourceBundle, key ERR_MultipleJavaField
>
> ----------------------------------------------------------------------
>
> [java] RUN Test_Query.testExtentQueries ERROR
>
> [java] 5)
> testExtentQueries(org.apache.jdo.test.Test_Query)java.lang.NoClass
> DefFoundError
> [java] at
> org.apache.jdo.test.Test_Query.createObjects(Test_Query.java:
> 344)
>
> ----------------------------------------------------------------------
>
> JDO RI test suite, runtest.multiJVMtests:
>
> 2 errors, even with maven.compile.source,target = 1.3
>
> [java] RUN Test_FetchInserted.test ERROR
>
> [java] 1)
> test(org.apache.jdo.test.Test_FetchInserted)javax.naming.NoInitial
> ContextException: Cannot instantiate class:
> com.sun.jndi.fscontext.RefFSContextF
> actory [Root exception is java.lang.ClassNotFoundException:
> com.sun.jndi.fsconte
> xt.RefFSContextFactory]
>
>
> This sun-specific JDK class seems to be missing or renamed in
> JDK 1.5. I guess we only have to change the entry in
> ri11/test/conf/jndi.properties. If anyone knows, let me know.
>
> ----------------------------------------------------------------------
--
Michael Bouschen Tech@Spree Engineering GmbH
mailto:mbo.tech@spree.de http://www.tech.spree.de/
Tel.:++49/30/235 520-33 Buelowstr. 66
Fax.:++49/30/2175 2012 D-10783 Berlin