You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ojb-dev@db.apache.org by Brian McCallister <mc...@forthillcompany.com> on 2004/08/01 18:00:57 UTC
Layer Design [was Release OJB 1.0.01]
I am of the opinion that too much functionality has probably been
pushed into the PB right now in order to compensate for the lack of
exposed functionality in the ODMG, and the immaturity of the OTM.
I'd like to see a PB style API with the transaction and dirtying
capabilities. Along those lines I have been doing a lot of work on a
library to handle the object state tracking and manipulation, which I
wanted to get fully hooked up with PB callbacks before bringing it up
as a serious option.
Too late =)
Short form of what I have right now is a library to which persistent
instances can be attached, which tracks the CLEAN/NEW/DELETED/DIRTY
states of objects and collections, provides callbacks on state changes,
and is capable of rolling back purely from the object model. There are
no dependencies other than JDK 1.2 or 1.3 (should be 1.2 but I haven't
actually tested it there). There is also, to a lesser degree, graph
dependency information available in terms of who references a given
object, etc. This information, combined with the pk/fk knoweldge
directly, or a topological sort knowing how we map fk's in general,
would allow for statement re-ordering. It also handles various JDO2
things like attach/detach in a compliant manner.
My thinking in terms of design is that one or two layers sit under
this. The "one" would be the PB directly. The "two" would be an
isolation-level handling layer, then the PB.
I *think* the internal design of this stuff is easier to follow than
the OTM internals right now, mostly as it doesn't attempt to do as
much. Isolation is a separate concern from state tracking, in my
thinking. With this, the PB would need to provide the direct ability to
do CRUD, to manipulate the contents of collections, and finally, to
allow manipulation of M:N implementors somewhat. All of which exists =)
In terms of using it, the integration could be done 100% via callback
registration on the existing PB. The o.a.o.broker.dirty.* stuff is
basically a proof-of-concept on the callback hooks from the OJB side
(registering instances, etc). Instead of using the
tightly-coupled-to-PB stuff in said package now, the callbacks would
register to a tx/unit in this library, passing through the isolation
layer if we want that, and hopefully we could then go ahead and expose
the PB with full OTM/ODMG style abilities.
The default for state tracking right now is reflective comparison,
though it is abstracted out such that using a bytecode enhancement
strategy would work very well (was designed very carefully to allow
this) and would increase performance a lot -- no graph traversal with
fieldwise comparisons needed at commit time.
Anyway, I don't have anywhere good to put this code right now, and
didn't want to add it to our repository until it was decided if we even
wanted to use it. Source xref's are up at
http://kasparov.skife.org/grafolia/xref/index.html under my personal
namespace and listed as copyright myself, but is has never been
released and is intended for checkin to OJB =)
Should I go ahead and add this, or drop it?
-Brian
On Jul 31, 2004, at 5:17 AM, Armin Waibel wrote:
> [off-topic]
> This is a more general problem of current odmg/otm design, they can't
> use the auto-xxx features of the PB-api. For odmg (and otm as far as I
> know) all reference auto-update/delete settings have to be 'false'.
> It's a shame, because PB-api can handle all kinds of reference/linking
> in a correct way.
> But current top-level implementations rebuild this functionality
> instead of using the PB-api resources. So the PB-api lacks of
> functionality and extendable design or the top-level implementations
> "use wrong design".
> Think we should discuss this.
>
> Some time ago I think about a callback/listener method in the internal
> PB-api (in 1.1 we will extend PB interface to deal with internal
> functionality - for the present this new interface called
> PersistenceBrokerExt and extend PersistenceBroker).
> Maybe a special action listener. E.g. the transaction class of the
> top-level api (TL-api) implements this listener and on commit the
> TL-api pass the first persistent object to the PB-api (store/delete).
> But before perform the delete/store call the PB-api notify the
> listener. If the listener allows the operation the PB store/delete
> call will be done and if the persistent object has references the
> PB-api handle these referenced objects dependent on the auto-xxx
> settings. But for each referenced object store/delete call the
> listener will be asked again. If the listener dislike the operation
> the PB-api doesn't store/delete and the cascade operation leave out
> this object.
> This way the TL-api can decide (dirty check, locking check, ...) which
> persistent object should be stored/deleted without dealing with FK
> settings/linking/references.
>
> Alternative will be to move all TL-api services like dirty-detection,
> locking, ... to the kernel, then the TL-api can use these services.
> This means that OTM and kernel will be merged in some way.
>
> regards,
> Armin
>
>
> > jakob
> >
> > Jakob Braeuchi schrieb:
> >
> >> hi armin,
> >>
> >> i was debugging NativeIdentifierTest#testReferenceInsertUpdateODMG
> for quite some time and unfortunately i do not see a solution to this
> problem :(
> >>
> >> we have a mainObject with a 1:1 relationship to a SingleReference.
> during tx.commit the mainObj is save first, storeReferences does NOT
> save this singleRef because of auto-settings set to false. afterStore
> of mainObj the db-generated key is put into the object. i tried to
> call setReferenceFKs in afterStore, but this is useless because
> singleRef is not saved and still carries the dummy key !
> >>
> >> so, imo we cannot insert the mainObj referencing a non existent
> singleRef !
> >> evene if we could update all the fks in the mainObj the row in the
> database is still wrong, because it contains to dummy key of the
> singleRef:
> >>
> >> INSERT INTO NATIVE_MAIN_OBJECT (REF_ID,NAME) VALUES
> ('-2','testReferenceInsert_main_1091213387781')
> >>
> >> this actually only works because there are no referential
> constraints.
> >>
> >> jakob
> >>
> >>
> >> Armin Waibel schrieb:
> >>
> >>> Jakob Braeuchi wrote:
> >>>
> >>>> hi armin,
> >>>>
> >>>> are you solving this issues for 1.0.1 ?
> >>>>
> >>>
> >>> I will try to do so, but concurrently I'm working on the 1.1 stuff
> so any help is much appreciated.
> >>>
> >>> Armin
> >>>
> >>>> jakob
> >>>>
> >>>> Armin Waibel schrieb:
> >>>>
> >>>>> Hi all,
> >>>>>
> >>>>> Brian McCallister wrote:
> >>>>>
> >>>>>> We have a number of user-requested bug fixes in the
> OJB_1_0_RELEASE branch at the moment. I'd like to push a release.
> >>>>>>
> >>>>>
> >>>>> There are some open issues (2 failing odmg-test cases,
> CollectionTest) and a serious bug in FK assignment when odmg-api was
> used with native DB Identity columns (SequenceManagerNativeImpl).
> NativeIdentifierTest, on insert of objects with 1:1 references.
> Problem occurs in TransactionImpl#lock(Object obj, int lockMode) line
> 235.
> >>>>>
> >>>>> When using native Identity columns, OJB use a temporary dummy
> value for created OJB Identity objects of new pc objects (negative
> long values are used as dummy values for Identity columns), so the FK
> assignment is only valid after store of the referenced object.
> >>>>> TransactionImpl#lock assign the FK before the referenced object
> was written to DB.
> >>>>>
> >>>>>
> >>>>> regards,
> >>>>> Armin
> >>>>>
> >>>>>> +1/0/-1
> >>>>>>
> >>>>>> Vote will end Sunday, Aug 1.
> >>>>>>
> >>>>>> -Brian
> >>>>>>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-dev-unsubscribe@db.apache.org
> For additional commands, e-mail: ojb-dev-help@db.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-dev-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-dev-help@db.apache.org
Re: Layer Design [was Release OJB 1.0.01]
Posted by Jakob Braeuchi <jb...@gmx.ch>.
hi brian,
Brian McCallister schrieb:
> On Aug 1, 2004, at 12:48 PM, Thomas Dudziak wrote:
>
>> How does that work together with caches, esp. the two-level cache that
>> Armin works on ?
>
>
> The caching Armin is doing is PB level, and that is where caching
> belongs, in my mind. Object state tracking, and object level
> transactions can, and should, work independently of the relational
> backend (which the PB bridges). We are really talking about the design
> of higher level persistence API's above the O/R mapping layer. PB does
> O/R mapping beautifully. People want higher level API's built on that
> though; annoying things, those users, grrr ;-)
it will be hard to remove existing functionality from pb-api because it's
already used. but i also prefer a clean layering.
>
>> I'm not entirely sure how much of that is visible to the outside (e.g.
>> the OJB users) anyway, and if it usable by PB, ODMG and JDO, then I'm
>> all for adding it to OJB.
>
>
> Visibility:
> From JDO: none
> From ODMG: none
> From PB: 100%
>
> The stack I see for JDO is either:
>
>
> state management: JDO -> Grafolia -> Lonely -> PB -> RDBMS
>
> (where Grafolia is the working name I have been using for state
> tracking, and Lonely is the working name I have not done any work on,
> but have conceptual model of, for isolation). This looks like a lot of
> layers, but is, I suspect, going to be a lot less code than the current
> OTM which does it all.
>
> or
>
> state management: JDO -> OTM -> PB -> RDBMS
>
>
> Where the OTM handles state and isolation together like now, as well as
> understanding OQL, etc.
i do not know how many users actually work with otm, so maybe we should replace
it completely with the new components.
>
> We keep re-implementing concerns in multiple levels (auto-* vs
> otm-dependent vs odmg reqs, etc) and we should probably stop this
> now-ish =) I tend to think that the auto-retrieve (getting the data)
> belongs in the PB layer , auto-delete and auto-update (deciding what to
> change) belong in the graph management layer (ODMG, OTM, Grafolia).
>
removing auto-update/delete from pb-api could make this api quite unusable ?
jakob
>
> Speaking of JDO, a similar split to break dependencies, which is what I
> have interpreted Thomas D. to be doing with the new OQL and JDOQL
> compilers (btw, Thomas D, go join the JDO2 EG -- because of NDA I cannot
> tell you why, though). Having arbitrary syntax compiled to an AST we
> understand, which then can be directly assembled into a PB query will be
> very very nice as adding a query language simply means implementing a
> parser which will generate our AST.
>
> -Brian
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-dev-unsubscribe@db.apache.org
> For additional commands, e-mail: ojb-dev-help@db.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-dev-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-dev-help@db.apache.org
Re: Layer Design [was Release OJB 1.0.01]
Posted by Brian McCallister <mc...@forthillcompany.com>.
On Aug 1, 2004, at 12:48 PM, Thomas Dudziak wrote:
> How does that work together with caches, esp. the two-level cache that
> Armin works on ?
The caching Armin is doing is PB level, and that is where caching
belongs, in my mind. Object state tracking, and object level
transactions can, and should, work independently of the relational
backend (which the PB bridges). We are really talking about the design
of higher level persistence API's above the O/R mapping layer. PB does
O/R mapping beautifully. People want higher level API's built on that
though; annoying things, those users, grrr ;-)
> I'm not entirely sure how much of that is visible to the outside (e.g.
> the OJB users) anyway, and if it usable by PB, ODMG and JDO, then I'm
> all for adding it to OJB.
Visibility:
From JDO: none
From ODMG: none
From PB: 100%
The stack I see for JDO is either:
state management: JDO -> Grafolia -> Lonely -> PB -> RDBMS
(where Grafolia is the working name I have been using for state
tracking, and Lonely is the working name I have not done any work on,
but have conceptual model of, for isolation). This looks like a lot of
layers, but is, I suspect, going to be a lot less code than the current
OTM which does it all.
or
state management: JDO -> OTM -> PB -> RDBMS
Where the OTM handles state and isolation together like now, as well as
understanding OQL, etc.
We keep re-implementing concerns in multiple levels (auto-* vs
otm-dependent vs odmg reqs, etc) and we should probably stop this
now-ish =) I tend to think that the auto-retrieve (getting the data)
belongs in the PB layer , auto-delete and auto-update (deciding what to
change) belong in the graph management layer (ODMG, OTM, Grafolia).
Speaking of JDO, a similar split to break dependencies, which is what I
have interpreted Thomas D. to be doing with the new OQL and JDOQL
compilers (btw, Thomas D, go join the JDO2 EG -- because of NDA I
cannot tell you why, though). Having arbitrary syntax compiled to an
AST we understand, which then can be directly assembled into a PB query
will be very very nice as adding a query language simply means
implementing a parser which will generate our AST.
-Brian
---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-dev-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-dev-help@db.apache.org
Re: Layer Design [was Release OJB 1.0.01]
Posted by Thomas Dudziak <to...@first.fhg.de>.
Brian McCallister wrote:
> I am of the opinion that too much functionality has probably been pushed
> into the PB right now in order to compensate for the lack of exposed
> functionality in the ODMG, and the immaturity of the OTM.
>
> I'd like to see a PB style API with the transaction and dirtying
> capabilities. Along those lines I have been doing a lot of work on a
> library to handle the object state tracking and manipulation, which I
> wanted to get fully hooked up with PB callbacks before bringing it up as
> a serious option.
>
> Too late =)
>
> Short form of what I have right now is a library to which persistent
> instances can be attached, which tracks the CLEAN/NEW/DELETED/DIRTY
> states of objects and collections, provides callbacks on state changes,
> and is capable of rolling back purely from the object model. There are
> no dependencies other than JDK 1.2 or 1.3 (should be 1.2 but I haven't
> actually tested it there). There is also, to a lesser degree, graph
> dependency information available in terms of who references a given
> object, etc. This information, combined with the pk/fk knoweldge
> directly, or a topological sort knowing how we map fk's in general,
> would allow for statement re-ordering. It also handles various JDO2
> things like attach/detach in a compliant manner.
>
> My thinking in terms of design is that one or two layers sit under this.
> The "one" would be the PB directly. The "two" would be an
> isolation-level handling layer, then the PB.
>
> I *think* the internal design of this stuff is easier to follow than the
> OTM internals right now, mostly as it doesn't attempt to do as much.
> Isolation is a separate concern from state tracking, in my thinking.
> With this, the PB would need to provide the direct ability to do CRUD,
> to manipulate the contents of collections, and finally, to allow
> manipulation of M:N implementors somewhat. All of which exists =)
>
> In terms of using it, the integration could be done 100% via callback
> registration on the existing PB. The o.a.o.broker.dirty.* stuff is
> basically a proof-of-concept on the callback hooks from the OJB side
> (registering instances, etc). Instead of using the tightly-coupled-to-PB
> stuff in said package now, the callbacks would register to a tx/unit in
> this library, passing through the isolation layer if we want that, and
> hopefully we could then go ahead and expose the PB with full OTM/ODMG
> style abilities.
>
> The default for state tracking right now is reflective comparison,
> though it is abstracted out such that using a bytecode enhancement
> strategy would work very well (was designed very carefully to allow
> this) and would increase performance a lot -- no graph traversal with
> fieldwise comparisons needed at commit time.
>
> Anyway, I don't have anywhere good to put this code right now, and
> didn't want to add it to our repository until it was decided if we even
> wanted to use it. Source xref's are up at
> http://kasparov.skife.org/grafolia/xref/index.html under my personal
> namespace and listed as copyright myself, but is has never been released
> and is intended for checkin to OJB =)
How does that work together with caches, esp. the two-level cache that
Armin works on ?
> Should I go ahead and add this, or drop it?
I'm not entirely sure how much of that is visible to the outside (e.g.
the OJB users) anyway, and if it usable by PB, ODMG and JDO, then I'm
all for adding it to OJB.
Tom
---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-dev-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-dev-help@db.apache.org