Persistence in Merlin, Part One [long]

[Niclas Hedhman <ni...@hedhman.org>]

Gang,

I, Andreas and Peter Neubauer was chatting yesterday about a standard 
persistence mechanism in Merlin.

This Part One will provide an introduction to JDO to those who have no 
previous experience, and look at the basic principles involved.
A second Part will cover what are the concerns regarding classloading, 
hot-deployments and reloading of components using persistence.
A third Part will cover the implementation of such system, and how to use it. 
Perhaps I'll skip this part and just do it, together with some examples.


Introduction to JDO.
====================
Personally, I am favour JDO very much, since I think it has a fairly clean API 
and is a proven concept with many implementations of various quality. 
There is an argument to put an additional layer on top, so we could map 
additional persistence strategies to tool bag.
The only additional system I can see in the near future would be Hibernate, 
and I am wondering if it isn't better to make a JDO styled (obs not JDO, 
since that implies a lot of compatibility constraints) wrapper around 
Hibernate instead.

JDO is trying to promise "transparent persistence" in that classes that can be 
persisted doesn't normally need any special programming. Any POJO *should* 
do, but there are some restrictions (see below).


Now, JDO looks roughly like this (ascii art warning);

+---------------------------+
| PersistenceManagerFactory |
+---------------------------+
             1                     +-------------+
             |        +-----------1| Transaction |
             |        |            +-------------+
             *        1        
   +--------------------+          +--------+
   | PersistenceManager |1--------*| Extent |
   +--------------------+          +--------+
             1        1
             |        |            +-------+
             |        +-----------*| Query |
             *                     +-------+
   +--------------------+
   | PersistenceCapable |
   +--------------------+


The PersistenceManagerFactory is the bootstrap singleton, which I think we 
don't need to engage at all.
The PersistenceManager is the central guy in the system. In our terms it will 
be a Service, and the implementation comes from the JDO implementation 
vendor.
The PersistenceCapable interface are implemented for classes to be storeable 
with a primary key, both to allow queries as well as being decomposed when 
referenced by other PersistenceCapable objects.
Transaction is an interface with methods as expected.
Query is an interface with methods as expected.
Extent is a logical view of all the objects of a particular class that exists 
in the store. (don't worry about this for a second)


Now, the PersistenceCapable interface is the marker for whether a class should 
be decomposed into its fields or saved serialized when stored. However, JDO 
doesn't require you to implement this interface yourself in the source code. 
Instead, the compiled class will be modified at bytecode level to include 
this. This is done by a tool called the JDO Enhancer, which is supplied by 
the JDO implementation vendor. However, there is a requirement by JDO that 
all JDO Enhancers are binary compatible.
Furthermore, since classes may have public, package protected or protected 
fields/members, the JDO Enhancer will need to modify the bytecodes for all 
classes that access these fields as well.

Restrictions placed on PersistenceCapable classes;
 *  A No-Arg constructor, but not necessarily public.
 *  PersistenceCapable can inherit from non-persistence capable classes, but 
there are some exceptions;
    - Not from some native implemented classes, such as Thread and Socket.
    - The fields in the superclass will not be persisted.
    - All classes in the inheritence tree must use the same JDO identity type 
or, if using application identity type, then they must use the same identity 
class.
 *  Immutable datatypes supported required by spec; All primitives and their 
wrappers, String, BigInteger, BigDecimal, Number and Locale.
 *  Mutable datatypes supported required by spec; Date and HashSet. (This I a 
bit odd, but luckily many implementations support the entire Collections 
API.)
 *  Arrays are a troubled chapter. Since one can't subclass or replace arrays, 
JDO impls can't detect when new values are written into the array. So one 
need to either assign the array field/member again, or mark the array dirty 
through an JDOHelper utility class.
 *  Fields of non-PersistenceCapable classes and user-defined interfaces will 
be handled in an implementation dependent manner, most impls seems to 
serialize and not decomposed even if the object assigned to the field is 
PersistenceCapable. 

Yes these are restrictions, but are they unbearable. IMHO, after doing a 
couple of apps is that they are "reasonable".

Ok, let's continue;
Just like EJBs, JDO can callback to the object instance for certain 
operations, jdoPostLoad, jdoPreStore, jdoPreDelete, jdoPreClear.
However, unlike EJBs, this is optional, and done by implementing an interface 
called InstanceCallbacks.

I'm skipping the explaination of JDO identity types and identity classes, for 
now, as I feel it is not important in this context (maybe a lot of the above 
is not important in the context as well, but...)


JDO requires Metadata for the PersistenceCapable classes. That must be defined 
in XML according to a DTD.
There are basically three purposes for this metadata;
 * Identification of PersistenceCapable classes.
 * Override of default JDO behaviour, such as linking fields to existing DB 
schemas.
 * Provision of information that the implementation can not figure out from 
reflecting the class, i.e. JDO impl extensions.

The amount of metadata that can be defined is massive, but not very much is 
required.
The JDO implementation I use, have javadoc tags to place the metadata in the 
source, and a tool generates the XML. I am not sure, whether this is a 
standard or Solarmetrics own construct.

The XML metadata can either be one file for the entire application, in which 
case it must be located as a resource "package.jdo". Or it can be placed in 
the same package as the PersistentCapable class and named "<classname>.jdo". 
The packaging requires the file to be reachable by the class' classloader.


I think this concludes the principals of JDO.

There are so much features in JDO, and I don't think it is appropriate to get 
into them here. Instead, I would like to conclude;

I think that the whole JDO API/SPI can be 'considered' services in the Merlin 
component model, and that we can wrap JDO implementations as Merlin 
components, similarily to the Jetty wrapping done for the Http facility.

I also don't think that the Persistence mechanism is a Facility, i.e. it 
doesn't need to reach the Merlin composition model. And we should create a 
Magic plugin that creates the JDO metadata as well as running the bytecode 
enhancer (which I assume an OSS version is available).

And I strongly suspect that some solid classloader analysis and deployment 
strategy is required. Perhaps leading to a proposal for "System Services", 
which are not facilities.


Cheers
Niclas
-- 
   +------//-------------------+
  / http://www.bali.ac        /
 / http://niclas.hedhman.org / 
+------//-------------------+


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@avalon.apache.org
For additional commands, e-mail: dev-help@avalon.apache.org