Re: JDO2 §12.7.2: fetch-depth only for "recursive fetch group references"?

[Craig Russell <Cr...@Sun.COM>]

Hi Jörg,

The term recursive is a bit misleading for fetch-depth, and I  
apologize for the confusion.

The fetch-depth attribute is intended to avoid accidental loading of  
the entire database (or at least a large portion of it)  into memory  
due to inadvertent loading of instances that themselves have fetch  
groups that load instances that have fetch groups and so forth. I  
referred to this as recursion simply because of fetched instances  
having other fetched instances.

The examples could be improved to include the cases that I intended  
to cover.

While self-referencing fields are indeed covered by the fetch-depth,  
other fields are covered as well. Think of the fetch-depth as the  
maximum number of joins in the SQL statement used to fetch the object  
graph in one statement.

On Sep 28, 2005, at 3:29 AM, Jörg von Frantzius wrote:

> [resending this because I haven't seen it coming back from jdo- 
> dev@db.apache.org]
>
> Hi Andy,
>
> I'm seeing some problems with this current JPOX approach to fetch- 
> depth,
> and I hope I can make them understandable.
>
> Andy Jefferson schrieb:
>
>
>>> I read the spec that a self-referencing relationship (meaning the  
>>> field
>>> type is the same class that declares the field) is a good use  
>>> case for
>>> the fetch-depth attribute. But I think the use of fetch-depth is not
>>> restricted to self-referencing relationships. Am I wrong here?
>>>
>>>
>>
>> I agree. I interpret fetch-depth as being for "recursive  
>> references" (as the spec says). This encompasses
>> 1. self-referencing (A->A->A->... )
>> 2. circular-referencing (e.g A->B->C->A->B->C->.....)
>>
>
> With fetch-depth=1, how do you discover the circularity while obeying
> the fetch-depth? (if the associations are all 1:n, you can't fetch the
> whole path in one go, you'd need more than one fetch) And by the  
> way, do
> you mean circularity on class level or on object level (you don't do
> static analysis of circles in the object model, do you?)

Let's take a specific example of the case above.

class A {
B myB; // default fetch-depth 1
}

class B {
A myA; // default fetch-depth 1
Set<C> myCs; // fetch-depth 2
}

class C {
B myB; // default fetch-depth 1
A myA; // fetch-depth 3
}

Now, if you query class A, you will get all of the A instances that  
satisfy the query plus all of the B instances directly referenced by  
the A instances. You get no C instances because the fetch-depth of  
A.myB limits the results to the directly reachable B instances.

If you query class B, you get

1. all of the B instances that satisfy the query;
2. plus all of the A instances directly reachable from B via the myA  
field (B->A) STOP
3. plus all of the C instances reachable via the B.myCs field (B->C)  
continue
4. plus all of the B instances reachable via the C.myB field in the C  
instances in 3 (B->C->B) STOP
5. plus all of the A instances reachable via the C.myA field in the C  
instances in 3 (B->C->A) STOP

If you query class C, you get

1.All of the C instances that satisfy the query
2. plus all of the B instances directly reachable via the C.myB field  
in the C instances in 1 (C->B) STOP
3. plus all of the A instances directly reachable via the C.myA field  
in the C instances in 1 (C->A) continue
4. plus all of the B instances directly reachable via the A.myB field  
in the A instances in 3 (C->A->B) continue
5. plus all of the A instances directly reachable via the B.myA field  
in the B instances in 4 (C->A->B->A) STOP
6.  plus all of the C instances directly reachable via the B.myCs  
field in the B instances in 4 (C->A->B->C) STOP

There is no requirement that a single database operation will be used  
to fetch the data. An iterative datastore access strategy is also  
spec-compliant.

Does this help clarify? I will put an example like this into the  
specification...

Craig

>
>
>> This is what JPOX aims to support.
>> The examples in the JDO2 spec are, incidentally, only of the self- 
>> referencing type.
>>
>
> Apart from the above problem, this approach makes it really hard  
> for the
> user to predict what gets fetched. Just imagine case 2 without the
> circle (no C->A): if the user wanted to detach an A object, with the
> number of C objects reachable being really really huge. How could he
> avoid detaching that A object without detaching a big number of C  
> objects?
>
> In my opinion things would be so much simpler if fetch-depth  
> applied to
> all kinds of fields, and the control over the size of the fetched  
> object
> graph would be much better.
>
> -- 
> __________________________________________________________
> Dipl.-Inf. Jörg von Frantzius  |            artnology GmbH
>                                |                Milastr. 4
> Tel +49 (0)30 4435 099 26      |              10437 Berlin
> Fax +49 (0)30 4435 099 99      |  http://www.artnology.com
> _______________________________|__________________________
>
>
>

Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!

Re: JDO2 §12.7.2: fetch-depth only for "recursive fetch group references"?

[Jörg von Frantzius <jo...@artnology.com>]

Hi Craig,

thanks a lot for your explanations and the elaborate example. It indeed 
helped clarify for me, but I'm afraid not for Andy Jefferson, and he's 
got the last word on the implementation  ;)

The example you elaborated on contains a circle of references, which in 
my opinion is a special case (and it is one of the cases that Andy 
indeed already sees covered). It would be nice if the examples could 
include a simpler case, like the same example only without the reference 
C->A, /so we don't have a circle/.

I think that's the more common case, and that's exactly what Andy 
doesn't see covered (although it can equally lead to large parts of the 
database being loaded).

Regards,
Jörg

Craig Russell schrieb:

> Hi Jörg,
>
> The term recursive is a bit misleading for fetch-depth, and I 
> apologize for the confusion.
>
> The fetch-depth attribute is intended to avoid accidental loading of 
> the entire database (or at least a large portion of it)  into memory 
> due to inadvertent loading of instances that themselves have fetch 
> groups that load instances that have fetch groups and so forth. I 
> referred to this as recursion simply because of fetched instances 
> having other fetched instances.
>
> The examples could be improved to include the cases that I intended to 
> cover. 
>
> While self-referencing fields are indeed covered by the fetch-depth, 
> other fields are covered as well. Think of the fetch-depth as the 
> maximum number of joins in the SQL statement used to fetch the object 
> graph in one statement. 
>
> On Sep 28, 2005, at 3:29 AM, Jörg von Frantzius wrote:
>
>> [resending this because I haven't seen it coming back from 
>> jdo-dev@db.apache.org <ma...@db.apache.org>]
>>
>> Hi Andy,
>>
>> I'm seeing some problems with this current JPOX approach to fetch-depth,
>> and I hope I can make them understandable.
>>
>> Andy Jefferson schrieb:
>>
>>
>>>> I read the spec that a self-referencing relationship (meaning the field
>>>> type is the same class that declares the field) is a good use case for
>>>> the fetch-depth attribute. But I think the use of fetch-depth is not
>>>> restricted to self-referencing relationships. Am I wrong here?
>>>>
>>>>    
>>>>
>>>>
>>>
>>> I agree. I interpret fetch-depth as being for "recursive references" 
>>> (as the spec says). This encompasses
>>> 1. self-referencing (A->A->A->... )
>>> 2. circular-referencing (e.g A->B->C->A->B->C->.....)
>>>
>>
>> With fetch-depth=1, how do you discover the circularity while obeying
>> the fetch-depth? (if the associations are all 1:n, you can't fetch the
>> whole path in one go, you'd need more than one fetch) And by the way, do
>> you mean circularity on class level or on object level (you don't do
>> static analysis of circles in the object model, do you?)
>
>
> Let's take a specific example of the case above.
>
> class A {
> B myB; // default fetch-depth 1
> }
>
> class B {
> A myA; // default fetch-depth 1
> Set<C> myCs; // fetch-depth 2
> }
>
> class C {
> B myB; // default fetch-depth 1
> A myA; // fetch-depth 3
> }
>
> Now, if you query class A, you will get all of the A instances that 
> satisfy the query plus all of the B instances directly referenced by 
> the A instances. You get no C instances because the fetch-depth of 
> A.myB limits the results to the directly reachable B instances.
>
> If you query class B, you get 
>
> 1. all of the B instances that satisfy the query; 
> 2. plus all of the A instances directly reachable from B via the myA 
> field (B->A) STOP
> 3. plus all of the C instances reachable via the B.myCs field (B->C) 
> continue
> 4. plus all of the B instances reachable via the C.myB field in the C 
> instances in 3 (B->C->B) STOP
> 5. plus all of the A instances reachable via the C.myA field in the C 
> instances in 3 (B->C->A) STOP
>
> If you query class C, you get
>
> 1.All of the C instances that satisfy the query
> 2. plus all of the B instances directly reachable via the C.myB field 
> in the C instances in 1 (C->B) STOP
> 3. plus all of the A instances directly reachable via the C.myA field 
> in the C instances in 1 (C->A) continue
> 4. plus all of the B instances directly reachable via the A.myB field 
> in the A instances in 3 (C->A->B) continue
> 5. plus all of the A instances directly reachable via the B.myA field 
> in the B instances in 4 (C->A->B->A) STOP
> 6.  plus all of the C instances directly reachable via the B.myCs 
> field in the B instances in 4 (C->A->B->C) STOP
>
> There is no requirement that a single database operation will be used 
> to fetch the data. An iterative datastore access strategy is also 
> spec-compliant.
>
> Does this help clarify? I will put an example like this into the 
> specification...
>
> Craig
>
>>
>>
>>> This is what JPOX aims to support.
>>> The examples in the JDO2 spec are, incidentally, only of the 
>>> self-referencing type.
>>>
>>
>> Apart from the above problem, this approach makes it really hard for the
>> user to predict what gets fetched. Just imagine case 2 without the
>> circle (no C->A): if the user wanted to detach an A object, with the
>> number of C objects reachable being really really huge. How could he
>> avoid detaching that A object without detaching a big number of C 
>> objects?
>>
>> In my opinion things would be so much simpler if fetch-depth applied to
>> all kinds of fields, and the control over the size of the fetched object
>> graph would be much better.
>>
>> -- 
>> __________________________________________________________
>> Dipl.-Inf. Jörg von Frantzius  |            artnology GmbH
>>                                |                Milastr. 4
>> Tel +49 (0)30 4435 099 26      |              10437 Berlin
>> Fax +49 (0)30 4435 099 99      |  http://www.artnology.com
>> _______________________________|__________________________
>>
>>
>>
>
> Craig Russell
>
> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
>
> 408 276-5638 mailto:Craig.Russell@sun.com
>
> P.S. A good JDO? O, Gasp!
>
>


-- 
__________________________________________________________
Dipl.-Inf. Jörg von Frantzius  |            artnology GmbH
                               |                Milastr. 4
Tel +49 (0)30 4435 099 26      |              10437 Berlin
Fax +49 (0)30 4435 099 99      |  http://www.artnology.com
_______________________________|__________________________