Fetch configuration

[Craig L Russell <Cr...@Sun.COM>]

Please see this message posted on the openjpa list:
http://n2.nabble.com/Fetch-Group-questions-tc534861.html

One of the comments is particularly relevant to JDO:
2.
http://openjpa.apache.org/docs/latest/manual/manual.html#ref_guide_fetch_impl
says that "Even when a direct relation is not eagerly fetched,
OpenJPA selects the foreign key columns and caches the values. This way
when you do traverse the relation, OpenJPA can often find the related
object in its cache, or at least avoid joins when loading the related
object from the database." - that's very kind, but may be we have an
Entity with lots of foreign key columns and I KNOW I only want a list
showing a few attributes, and I KNOW that the direct relations won't be
traversed (say because the Entites are detached, may be). Is there any
way to configure OpenJPA to NOT eagerly fetch direct relations, unless
they are explicitly part of the Fetch Group? Actually what I'd want is
to be able to by default not include foreign key columns, but have a way
to explicitly include them - which is not the same as having the field
as part of the FetchGroup (that will lead to a JOIN, I may just want to
control whether or not to selects the foreign key column).

Currently we don't have the notion in JDO of fetching key values of  
relationships, just fetching the targets of these keys.

Would it be useful to consider adding to the JDO specification the  
ability to specify fetching the keys for relationships and not the  
relationships themselves? This would probably include adding some  
metadata to fetch-group and to FetchGroup.

One thing to kick around, is changing the meaning of recursion- 
depth==0 so changing:

-1 recurse indefinitely
1 just fetch the target of this field
n fetch the target and recurse n times
0 just fetch the identifier (foreign key value) for the field but not  
the field itself

This is a "spare bit" that is currently not particularly useful, since  
the specified behavior is identical if "the member is not specified in  
the FetchGroup" and "the member is specified with a recursion-depth of  
0".

Craig

Craig L Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!

Re: Fetch configuration

[Matthew Adams <ma...@matthewadams.me>]

On Wed, Jul 14, 2010 at 12:29 PM, Craig L Russell
<cr...@oracle.com>wrote:

>
> On Jul 14, 2010, at 7:53 AM, Matthew Adams wrote:
>
> Seems like a good idea.  Could foreign keys be considered the same as basic
>> fields for the purposes of the default fetch group?  That is, fetched by
>> default unless the user says don't fetch?
>>
>
> That's where I would start. We need to decide whether this is just a
> relational database issue and use mapping metadata or if it's a general
> datastore issue and use jdo metadata. In other words, we could put columns
> into the fetch group or define a new jdo metadata concept to describe "the
> many-to-one artifacts that aren't visible as fields but are required to
> instantiate relationships".

Understood.


>
>> I assume that if an object is loaded using a fetch group that doesn't
>> include foreign keys and the foreign key field is dereferenced, the impl
>> would go back to the datastore and load the referenced object's foreign
>> key
>> via the referencing object's key (simple lazy loading).
>>
>
> Actually, what I expect the implementation to do is to load the referenced
> object directly using a join of the primary table (where t0.pk = ?) with
> the referenced table (and t0.fk = t1.pk). One round trip to the datastore
> instead of two.
>
That's actually what I had in mind; I would also expect an impl to do it in
one trip.


>
> But a compliant implementation can eagerly load the foreign keys today,
> since there's nothing in the spec to prohibit it.
>
> I think this would be a good feature to discuss via JIRA. There are
> specification issues as well as technical issues to work through.
>
>
> What's the
>> behavior of a JDO impl that supports change of identity if I load an
>> object
>> that uses application identity with a fetch group that doesn't include
>> foreign keys of direct relationships, I then change the identity value of
>> the referencing object, and then I dereference a direct relationship that
>> was not loaded?  The impl would not have a way to get back to the
>> datastore
>> to get the referenced object because the referencing object's key has been
>> changed.
>>
>
> I'd expect that the behavior would depend on whether the change of pk was
> flushed or not. If not flushed, the original pk is still visible in the
> datastore. If flushed, the changed pk is visible (along with all the
> consequent fk's that refer to the changed pk).
>
>
> It seems to me that a dumb implementation should throw
>> JDOUserException; a smart impl could cache the original value of the
>> referencing object's key if changed during the transaction and use the
>> cached id value to go get the referenced foreign key & object.  I could
>> see
>> this as being a JDO option:
>> javax.jdo.option.LazilyLoadDirectRelationshipWithoutLoadedForeignKey.
>>
>
> I think that an impl that supports change of pk should also be able to
> handle this case without being explicit about it.
>
> Sounds reasonable to me.

Re: Fetch configuration

[Craig L Russell <cr...@oracle.com>]

On Jul 14, 2010, at 7:53 AM, Matthew Adams wrote:

> Seems like a good idea.  Could foreign keys be considered the same  
> as basic
> fields for the purposes of the default fetch group?  That is,  
> fetched by
> default unless the user says don't fetch?

That's where I would start. We need to decide whether this is just a  
relational database issue and use mapping metadata or if it's a  
general datastore issue and use jdo metadata. In other words, we could  
put columns into the fetch group or define a new jdo metadata concept  
to describe "the many-to-one artifacts that aren't visible as fields  
but are required to instantiate relationships".
>
> I assume that if an object is loaded using a fetch group that doesn't
> include foreign keys and the foreign key field is dereferenced, the  
> impl
> would go back to the datastore and load the referenced object's  
> foreign key
> via the referencing object's key (simple lazy loading).

Actually, what I expect the implementation to do is to load the  
referenced object directly using a join of the primary table (where  
t0.pk = ?) with the referenced table (and t0.fk = t1.pk). One round  
trip to the datastore instead of two.

But a compliant implementation can eagerly load the foreign keys  
today, since there's nothing in the spec to prohibit it.

I think this would be a good feature to discuss via JIRA. There are  
specification issues as well as technical issues to work through.

> What's the
> behavior of a JDO impl that supports change of identity if I load an  
> object
> that uses application identity with a fetch group that doesn't include
> foreign keys of direct relationships, I then change the identity  
> value of
> the referencing object, and then I dereference a direct relationship  
> that
> was not loaded?  The impl would not have a way to get back to the  
> datastore
> to get the referenced object because the referencing object's key  
> has been
> changed.

I'd expect that the behavior would depend on whether the change of pk  
was flushed or not. If not flushed, the original pk is still visible  
in the datastore. If flushed, the changed pk is visible (along with  
all the consequent fk's that refer to the changed pk).

> It seems to me that a dumb implementation should throw
> JDOUserException; a smart impl could cache the original value of the
> referencing object's key if changed during the transaction and use the
> cached id value to go get the referenced foreign key & object.  I  
> could see
> this as being a JDO option:
> javax.jdo.option.LazilyLoadDirectRelationshipWithoutLoadedForeignKey.

I think that an impl that supports change of pk should also be able to  
handle this case without being explicit about it.

Craig
>
> Thoughts?
>
>
> On Wed, Jul 14, 2010 at 2:37 AM, Andy Jefferson  
> <an...@datanucleus.org>wrote:
>
>> Post from 25 August 2008 by Craig.
>>
>>>
>> http://openjpa.apache.org/docs/latest/manual/manual.html#ref_guide_fetch_im
>>> pl says that "Even when a direct relation is not eagerly fetched,
>>> OpenJPA selects the foreign key columns and caches the values.  
>>> This way
>>> when you do traverse the relation, OpenJPA can often find the  
>>> related
>>> object in its cache, or at least avoid joins when loading the  
>>> related
>>> object from the database." - that's very kind, but may be we have an
>>> Entity with lots of foreign key columns and I KNOW I only want a  
>>> list
>>> showing a few attributes, and I KNOW that the direct relations  
>>> won't be
>>> traversed (say because the Entites are detached, may be). Is there  
>>> any
>>> way to configure OpenJPA to NOT eagerly fetch direct relations,  
>>> unless
>>> they are explicitly part of the Fetch Group? Actually what I'd  
>>> want is
>>> to be able to by default not include foreign key columns, but have  
>>> a way
>>> to explicitly include them - which is not the same as having the  
>>> field
>>> as part of the FetchGroup (that will lead to a JOIN, I may just  
>>> want to
>>> control whether or not to selects the foreign key column).
>>>
>>> Currently we don't have the notion in JDO of fetching key values of
>>> relationships, just fetching the targets of these keys.
>>> Would it be useful to consider adding to the JDO specification the
>>> ability to specify fetching the keys for relationships and not the
>>> relationships themselves? This would probably include adding some
>>> metadata to fetch-group and to FetchGroup.
>>
>> Just seen a need for this also, bit late I know ;-)
>> Perhaps add something in JDO3.1 ?
>>
>>
>> --
>> Andy
>> DataNucleus (http://www.datanucleus.org)
>>
>
>
>
> -- 
> mailto:matthew@matthewadams.me
> skype:matthewadams12
> yahoo:matthewadams
> aol:matthewadams12
> google-talk:matthewadams12@gmail.com<google-talk%3Amatthewadams12@gmail.com 
> >
> msn:matthew@matthewadams.me <ms...@matthewadams.me>
> http://matthewadams.me
> http://www.linkedin.com/in/matthewadams

Craig L Russell
Architect, Oracle
http://db.apache.org/jdo
408 276-5638 mailto:Craig.Russell@oracle.com
P.S. A good JDO? O, Gasp!

Re: Fetch configuration

[Matthew Adams <ma...@matthewadams.me>]

Seems like a good idea.  Could foreign keys be considered the same as basic
fields for the purposes of the default fetch group?  That is, fetched by
default unless the user says don't fetch?

I assume that if an object is loaded using a fetch group that doesn't
include foreign keys and the foreign key field is dereferenced, the impl
would go back to the datastore and load the referenced object's foreign key
via the referencing object's key (simple lazy loading).  What's the
behavior of a JDO impl that supports change of identity if I load an object
that uses application identity with a fetch group that doesn't include
foreign keys of direct relationships, I then change the identity value of
the referencing object, and then I dereference a direct relationship that
was not loaded?  The impl would not have a way to get back to the datastore
to get the referenced object because the referencing object's key has been
changed.  It seems to me that a dumb implementation should throw
JDOUserException; a smart impl could cache the original value of the
referencing object's key if changed during the transaction and use the
cached id value to go get the referenced foreign key & object.  I could see
this as being a JDO option:
javax.jdo.option.LazilyLoadDirectRelationshipWithoutLoadedForeignKey.

Thoughts?


On Wed, Jul 14, 2010 at 2:37 AM, Andy Jefferson <an...@datanucleus.org>wrote:

> Post from 25 August 2008 by Craig.
>
> >
> http://openjpa.apache.org/docs/latest/manual/manual.html#ref_guide_fetch_im
> > pl says that "Even when a direct relation is not eagerly fetched,
> > OpenJPA selects the foreign key columns and caches the values. This way
> > when you do traverse the relation, OpenJPA can often find the related
> > object in its cache, or at least avoid joins when loading the related
> > object from the database." - that's very kind, but may be we have an
> > Entity with lots of foreign key columns and I KNOW I only want a list
> > showing a few attributes, and I KNOW that the direct relations won't be
> > traversed (say because the Entites are detached, may be). Is there any
> > way to configure OpenJPA to NOT eagerly fetch direct relations, unless
> > they are explicitly part of the Fetch Group? Actually what I'd want is
> > to be able to by default not include foreign key columns, but have a way
> > to explicitly include them - which is not the same as having the field
> > as part of the FetchGroup (that will lead to a JOIN, I may just want to
> > control whether or not to selects the foreign key column).
> >
> > Currently we don't have the notion in JDO of fetching key values of
> > relationships, just fetching the targets of these keys.
> > Would it be useful to consider adding to the JDO specification the
> > ability to specify fetching the keys for relationships and not the
> > relationships themselves? This would probably include adding some
> > metadata to fetch-group and to FetchGroup.
>
> Just seen a need for this also, bit late I know ;-)
> Perhaps add something in JDO3.1 ?
>
>
> --
> Andy
> DataNucleus (http://www.datanucleus.org)
>



-- 
mailto:matthew@matthewadams.me
skype:matthewadams12
yahoo:matthewadams
aol:matthewadams12
google-talk:matthewadams12@gmail.com<go...@gmail.com>
msn:matthew@matthewadams.me <ms...@matthewadams.me>
http://matthewadams.me
http://www.linkedin.com/in/matthewadams

Re: Fetch configuration

[Andy Jefferson <an...@datanucleus.org>]

Post from 25 August 2008 by Craig.

> http://openjpa.apache.org/docs/latest/manual/manual.html#ref_guide_fetch_im
> pl says that "Even when a direct relation is not eagerly fetched,
> OpenJPA selects the foreign key columns and caches the values. This way
> when you do traverse the relation, OpenJPA can often find the related
> object in its cache, or at least avoid joins when loading the related
> object from the database." - that's very kind, but may be we have an
> Entity with lots of foreign key columns and I KNOW I only want a list
> showing a few attributes, and I KNOW that the direct relations won't be
> traversed (say because the Entites are detached, may be). Is there any
> way to configure OpenJPA to NOT eagerly fetch direct relations, unless
> they are explicitly part of the Fetch Group? Actually what I'd want is
> to be able to by default not include foreign key columns, but have a way
> to explicitly include them - which is not the same as having the field
> as part of the FetchGroup (that will lead to a JOIN, I may just want to
> control whether or not to selects the foreign key column).
> 
> Currently we don't have the notion in JDO of fetching key values of
> relationships, just fetching the targets of these keys.
> Would it be useful to consider adding to the JDO specification the
> ability to specify fetching the keys for relationships and not the
> relationships themselves? This would probably include adding some
> metadata to fetch-group and to FetchGroup.

Just seen a need for this also, bit late I know ;-)
Perhaps add something in JDO3.1 ?


-- 
Andy
DataNucleus (http://www.datanucleus.org)