RMI and Proxied collections and references

[Tomasz Wysocki <t....@amg.net.pl>]

Hello,
I've been using OJB in a small web-application project, with a success.
It is performing nicely and a simple PB API is great ...
I'm considering, if OJB would be good for more complicated multi-tier 
application used in distributed environment.

Suppose we have some remote service returning results that contain 
persistent classes instances there is a problem if proxied references or 
collections are used.
PersistenceBroker field of CollectionProxy is transient, so instance 
will serialize correctly, and the same is true for IndirectionHandler.
But what if a service is truly remote, ie. there is no access to 
database on client side. There is no chance that PersistenceBroker 
brought up on a client side (by means of PBKey handle) could materialize 
proxied collections or references. There even is no much sense in it, 
cause client can be many network hops away from server's database.
I think this problem could be resolved by providing a configuration 
option that would turn-on proxy materializing triggered by serialization 
of persistent classes. This would result in fully materialized model 
transfered to a client, and transparent proxy handling for serialization.

What do you think?

BTW. I don't want to use c/s mode.

Re: RMI and Proxied collections and references

[Thomas Mahler <th...@web.de>]

Hi again,

Tomasz Wysocki wrote:
> Thomas Mahler wrote:
> 
>> The PB c/s mode has the same fine grained call structure as the 
>> singlevm PB. Currently there is no way to cut down the nuber of 
>> network calls for c/s mode. The only thing you can do is to change the 
>> "auto-retrieve" and "proxy" settings to change the granularity of 
>> loaded object graphs.
>>
> 
> Does auto-retrieve and proxy settings combine to produce meaningful 
> results?

auto-retrieve="false" : the reference-attribute is not touched and 
remains null. If the reference-attribute is needed later it must be 
loaded explicitely by a call to PB.loadReference("attribute_name").

auto-retrieve="true", proxy="false" : load the reference-attribute and 
fill it with "real" data.

auto-retrieve="true", proxy="true" : don't load the real data, but fill 
the reference-attribute with a proxy.

> 
>>
>> This is the most easy solution and would safe us a lot of work.
>> The big problem: it could result in transferring rather large object 
>> graphs to the client! The only remedy I see is some kind of 
>> "tranportable proxies".
>>
> 
> So transportable proxies can use whatever c/s mode is currently in use, 
>  servlets, ejbs or plain sockets?

correct!

>>> I don't know how many windows are broken in c/s mode, cause I don't 
>>> use it, and don't have any plans to do so.

all changes to PB API after 0.9.6 must be made accessible.

>>>
>>> The whole idea of c/s mode seems OK, just like database servers are ok.
>>> They can be clustered, they serve and cache data, provide 
>>> synchronization and transaction management. So c/s mode is just like 
>>> an object database.
>>
>>
>>
>> correct, that was my initial idea. the only problem: it is a lot of 
>> work to maintain singlevm an c/s in parallel!
> 
> 
> Here exposes my lack knowledge on OJB internals. But I think that it is 
> a matter of refactoring to separate remote features such as invocation 
> and cluster synchronization from core functionality, and not dropping 
> such an amount of work already done. Then changes in core PB would not 
> affect c/s mode build on top.

The c/s mode is already build that way. The server side component is 
just a simple delegation mechanism into a singlevm broker.
The rest is only prc infrastructure. For the socket server and for the 
servlet we have to implement the call mechanisms.

So it's not *that* much work. The worst thing is the test cycle. The 
complete Junit tests for PB take 20 minutes or so...

>>
>>> But just like them, c/s mode is also an another tier, which may not 
>>> be needed in a particular application, and would only impact the 
>>> performance.
>>
>>
>>
>> That's why I'm questioning the whole c/s mode. But I feel that we need 
>> some kind of "transportable proxies".
>>
> 
> "remote" proxy behaviour should be than configured on class descriptor 
> and collection basis not as global OJB option. Say, proxy="true|false" 
> remote-proxy="true|false". Question is how big 'codequake' this will 
> produce.

Won't be difficult to implement.

> 
>>
>>> The only thing lacking in this picture is a persistence cache 
>>> implementation that would behave correctly when application is 
>>> clustered, and that leads us to c/s mode or EJB Entity beans.
>>
>>
>>
>> I don't agree here. OJB provides a JCS cache that may be clustered. 
>> ALso the optimistic and pessimistic locking features are ready for 
>> clustering even without c/s mode!
>>
> 
> I forgot about JCS. Only thing bad about it, it has so many 
> config-options ;-). Is there an updated tutorial on using it with OJB?

I'm not quite sure. But I think Matthew once posted a howto?

> 
> Then at the second sentence c/s mode idea slipped me. How come that 
> pessimistic locking == synchronization doesn't require a form of remote 
> communication == c/s mode in clustered environment. Does it employ a db?

correct! For running the OMDG layer in a cluster we provide a Persistent 
LockMap that is maintained n a database table. (In SingleVM mode the 
lock Map is kept in memory).

>>
>> Maybe the PB SessionBean could be a solution:
>> if we are running container managed transactions (CTM) a remote proxy 
>> acesssing the PB SessionBean could act within the same server side 
>> transaction boundary as the main transaction.
> 
> 
> I cannot see how would that be possible. After transaction was commited 
> it was commited. And a new remote call to PB SessionBean from a remote 
> proxy will begin a new transaction (if marked as 'Required'). Or am I 
> missing an idea here?

If the tx is committed we are lost, correct. But as long as we are 
running within the same server side tx everything is fine...

> 
>>
>>> Looks like I'm demanding to much from OJB, but even EJB doesn't 
>>> support such behaviour like returning Local Entity Beans as a result 
>>> of a Remote Call.
>>
>>
>>
>> Yes that's really a tough problem. But as several users faced this 
>> kind of problem we should provide a solution.
>>
> 
> There is a "value object" pattern that is used in those situations. This 
> pattern we try to avoid, to lessen then burden of maintaining several 
> model representations namely : db schema, business objects, value 
> objects, html forms. It is a great advantage if business object are 
> POJOs, and not some framework components.

I agree. One of OJB guidelines is to support POJOs and not require 
special user coded infrastructure (like VOs).

> 
> Thanks for many insights, that were troubling me from some time now.

Thanks too. This discussion helped me to see clearer wrt. to the c/s 
mode and the remote proxy issue.

cu,
Thomas


>     Tomasz Wysocki
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: ojb-user-help@jakarta.apache.org
> 
> 

Re: RMI and Proxied collections and references

[Tomasz Wysocki <t....@amg.net.pl>]

Thomas Mahler wrote:

> The PB c/s mode has the same fine grained call structure as the singlevm 
> PB. Currently there is no way to cut down the nuber of network calls for 
> c/s mode. The only thing you can do is to change the "auto-retrieve" and 
> "proxy" settings to change the granularity of loaded object graphs.
> 

Does auto-retrieve and proxy settings combine to produce meaningful results?

> 
> This is the most easy solution and would safe us a lot of work.
> The big problem: it could result in transferring rather large object 
> graphs to the client! The only remedy I see is some kind of 
> "tranportable proxies".
> 

So transportable proxies can use whatever c/s mode is currently in use, 
  servlets, ejbs or plain sockets?

>> I don't know how many windows are broken in c/s mode, cause I don't 
>> use it, and don't have any plans to do so.
>>
>> The whole idea of c/s mode seems OK, just like database servers are ok.
>> They can be clustered, they serve and cache data, provide 
>> synchronization and transaction management. So c/s mode is just like 
>> an object database.
> 
> 
> correct, that was my initial idea. the only problem: it is a lot of work 
> to maintain singlevm an c/s in parallel!

Here exposes my lack knowledge on OJB internals. But I think that it is 
a matter of refactoring to separate remote features such as invocation 
and cluster synchronization from core functionality, and not dropping 
such an amount of work already done. Then changes in core PB would not 
affect c/s mode build on top.
> 
>> But just like them, c/s mode is also an another tier, which may not be 
>> needed in a particular application, and would only impact the 
>> performance.
> 
> 
> That's why I'm questioning the whole c/s mode. But I feel that we need 
> some kind of "transportable proxies".
> 

"remote" proxy behaviour should be than configured on class descriptor 
and collection basis not as global OJB option. Say, proxy="true|false" 
remote-proxy="true|false". Question is how big 'codequake' this will 
produce.

> 
>> The only thing lacking in this picture is a persistence cache 
>> implementation that would behave correctly when application is 
>> clustered, and that leads us to c/s mode or EJB Entity beans.
> 
> 
> I don't agree here. OJB provides a JCS cache that may be clustered. ALso 
> the optimistic and pessimistic locking features are ready for clustering 
> even without c/s mode!
> 

I forgot about JCS. Only thing bad about it, it has so many 
config-options ;-). Is there an updated tutorial on using it with OJB?

Then at the second sentence c/s mode idea slipped me. How come that 
pessimistic locking == synchronization doesn't require a form of remote 
communication == c/s mode in clustered environment. Does it employ a db?

> 
> Maybe the PB SessionBean could be a solution:
> if we are running container managed transactions (CTM) a remote proxy 
> acesssing the PB SessionBean could act within the same server side 
> transaction boundary as the main transaction.

I cannot see how would that be possible. After transaction was commited 
it was commited. And a new remote call to PB SessionBean from a remote 
proxy will begin a new transaction (if marked as 'Required'). Or am I 
missing an idea here?

> 
>> Looks like I'm demanding to much from OJB, but even EJB doesn't 
>> support such behaviour like returning Local Entity Beans as a result 
>> of a Remote Call.
> 
> 
> Yes that's really a tough problem. But as several users faced this kind 
> of problem we should provide a solution.
> 

There is a "value object" pattern that is used in those situations. This 
pattern we try to avoid, to lessen then burden of maintaining several 
model representations namely : db schema, business objects, value 
objects, html forms. It is a great advantage if business object are 
POJOs, and not some framework components.

Thanks for many insights, that were troubling me from some time now.

	Tomasz Wysocki

Re: RMI and Proxied collections and references

[Thomas Mahler <th...@web.de>]

Hi again,

Tomasz Wysocki wrote:
<snip>

> Yes, you're right. C/S mode in this case would defer materialization 
> further to the moment when client really needs data. However, in a 
> distributed environment there is also a need to minimize number of trips 
> to the server, to decrease the network overhead.
> In other words, configuration should provide means to change this 
> behaviour, so it will match needs of particular application.

I agree! That's why I said that c/s mode can cause problems if used 
carelessly. You will need some configuration tuning to make c/s apps 
performant. The biggest problem I see is access granularity.
With a typical EJB session bean it is easy to define the granularity of 
remote access programmatically.
The PB c/s mode has the same fine grained call structure as the singlevm 
PB. Currently there is no way to cut down the nuber of network calls for 
c/s mode. The only thing you can do is to change the "auto-retrieve" and 
"proxy" settings to change the granularity of loaded object graphs.

>>
>> Yes, this could be an elegant solution! AFAIK there are some callbacks 
>> in the serialization api that could help to implement this trick.
>> Only disatvantage I see: persistent classes must be derived from a 
>> base class implementing these callbacks?
>>
> 
> It is a proxy that serializes not a persistent object itself, so there 
> is no special need to implement callbacks by application developer on 
> persistent classes.
> 
> I was thinking about writeReplace rather than writeObject as Phil 
> Warrick suggested. So say we have a CollectionProxy class that on 
> serialization does materialize ArrayList and designates it to be its 
> replacement in a stream.

So it would be code that could reside in the OJB proxy classes. User 
code won't be affected. cool!

>>
>> Are you willing to work on this? I will review your stuff and 
>> integrate it into the codebase!
> 
> 
> I don't have a code yet, just thoughts.
> 
>>
>>> BTW. I don't want to use c/s mode.
>>
>>
>>
>> I see!
>> We are currently discussing to drop the c/s mode altogether. The only 
>> *real* benefit the c/s mode had was the support for network-enabled 
>> proxies.
>> If we can replace this feature with your solution the last reason for 
>> c/s mode has gone and we could drop it without problems!
> 
> 
> As I see it, there are two approaches.
> 1. The whole object model is materialized and no further calls are made 
> to the server.

This is the most easy solution and would safe us a lot of work.
The big problem: it could result in transferring rather large object 
graphs to the client! The only remedy I see is some kind of 
"tranportable proxies".

> 2. The proxies are transported as handles to a client side and 
> materialize instances when client needs data. And those proxies are 
> providing functionality resembling that of remote objects.

This is the current c/s mode solution: "transportable proxies"

> I don't know how many windows are broken in c/s mode, cause I don't use 
> it, and don't have any plans to do so.
> 
> The whole idea of c/s mode seems OK, just like database servers are ok.
> They can be clustered, they serve and cache data, provide 
> synchronization and transaction management. So c/s mode is just like an 
> object database.

correct, that was my initial idea. the only problem: it is a lot of work 
to maintain singlevm an c/s in parallel!

>  From my perspective c/s mode combined with proxies is a form of 
> replacement for remote EJB Entity Beans which provide the same 
> functionality. 

correct, that was my original idea!

> But just like them, c/s mode is also an another tier, 
> which may not be needed in a particular application, and would only 
> impact the performance.

That's why I'm questioning the whole c/s mode. But I feel that we need 
some kind of "transportable proxies".

>  From what I've learned, using combination of EJB Session Beans and 
> direct JDBC to access data is the most performant and robust solution in 
> most cases. Then, I don't mind using a simple persistence layer that 
> would isolate me from pecularities of a data storage, which PB API is 
> really best for.

I agree. This approach works fine for most cases.

> The only thing lacking in this picture is a persistence cache 
> implementation that would behave correctly when application is 
> clustered, and that leads us to c/s mode or EJB Entity beans.

I don't agree here. OJB provides a JCS cache that may be clustered. ALso 
the optimistic and pessimistic locking features are ready for clustering 
even without c/s mode!

> Matthew Baird pointed out that tx handling in most appservers is done 
> before serialization layer. Transaction commit will take place before a 
> serialization will occur, which leads to one *main* transaction on 
> remote call and then several transactions to materialize lazy objects, 
> just before send to client. Still in some appliactions, it is better 
> than serveral remote calls to retrieve those objects later.
> 
> This tx boundary problem arises from the fact that persistent objects 
> are not managed by application server container, ( and we don't want 
> them to be ). Only Jboss would be so flexible so that you could mess 
> with EJB interceptor stack and invokers to perform serialization earlier 
> than transaction handling, but surely this would impact other things.
> 

Maybe the PB SessionBean could be a solution:
if we are running container managed transactions (CTM) a remote proxy 
acesssing the PB SessionBean could act within the same server side 
transaction boundary as the main transaction.

> Looks like I'm demanding to much from OJB, but even EJB doesn't support 
> such behaviour like returning Local Entity Beans as a result of a Remote 
> Call.

Yes that's really a tough problem. But as several users faced this kind 
of problem we should provide a solution.

> 
> The idea of an EJB bean implementing PersistenceBroker interface seems 
> like a replacement for c/s mode.

It's not exactly a replacement. It's one of the c/s mode 
implementations. We currently have:
- a socket based PB server
- a PB servlet
- a PB Session Bean
All three components implement the PB interface.

cheers,
Thomas


> Cheers Tomasz ;-)
> 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: ojb-user-help@jakarta.apache.org
> 
> 

Re: RMI and Proxied collections and references

[Tomasz Wysocki <t....@amg.net.pl>]

Hi again,

Thomas Mahler wrote:

>> But what if a service is truly remote, ie. there is no access to 
>> database on client side. There is no chance that PersistenceBroker 
>> brought up on a client side (by means of PBKey handle) could 
>> materialize proxied collections or references. 
> 
> 
> This is correct in the singlevm mode only. If you were running your app 
> in c/s mode proxies would use the PersistenceBrokerClient to materialize 
>  references on demand!

Yes, you're right. C/S mode in this case would defer materialization 
further to the moment when client really needs data. However, in a 
distributed environment there is also a need to minimize number of trips 
to the server, to decrease the network overhead.
In other words, configuration should provide means to change this 
behaviour, so it will match needs of particular application.

> 
> Yes, this could be an elegant solution! AFAIK there are some callbacks 
> in the serialization api that could help to implement this trick.
> Only disatvantage I see: persistent classes must be derived from a base 
> class implementing these callbacks?
> 

It is a proxy that serializes not a persistent object itself, so there 
is no special need to implement callbacks by application developer on 
persistent classes.

I was thinking about writeReplace rather than writeObject as Phil 
Warrick suggested. So say we have a CollectionProxy class that on 
serialization does materialize ArrayList and designates it to be its 
replacement in a stream.

> 
> Are you willing to work on this? I will review your stuff and integrate 
> it into the codebase!

I don't have a code yet, just thoughts.

> 
>> BTW. I don't want to use c/s mode.
> 
> 
> I see!
> We are currently discussing to drop the c/s mode altogether. The only 
> *real* benefit the c/s mode had was the support for network-enabled 
> proxies.
> If we can replace this feature with your solution the last reason for 
> c/s mode has gone and we could drop it without problems!

As I see it, there are two approaches.
1. The whole object model is materialized and no further calls are made 
to the server.
2. The proxies are transported as handles to a client side and 
materialize instances when client needs data. And those proxies are 
providing functionality resembling that of remote objects.

I don't know how many windows are broken in c/s mode, cause I don't use 
it, and don't have any plans to do so.

The whole idea of c/s mode seems OK, just like database servers are ok.
They can be clustered, they serve and cache data, provide 
synchronization and transaction management. So c/s mode is just like an 
object database.
 From my perspective c/s mode combined with proxies is a form of 
replacement for remote EJB Entity Beans which provide the same 
functionality. But just like them, c/s mode is also an another tier, 
which may not be needed in a particular application, and would only 
impact the performance.

 From what I've learned, using combination of EJB Session Beans and 
direct JDBC to access data is the most performant and robust solution in 
most cases. Then, I don't mind using a simple persistence layer that 
would isolate me from pecularities of a data storage, which PB API is 
really best for.

The only thing lacking in this picture is a persistence cache 
implementation that would behave correctly when application is 
clustered, and that leads us to c/s mode or EJB Entity beans.

Matthew Baird pointed out that tx handling in most appservers is done 
before serialization layer. Transaction commit will take place before a 
serialization will occur, which leads to one *main* transaction on 
remote call and then several transactions to materialize lazy objects, 
just before send to client. Still in some appliactions, it is better 
than serveral remote calls to retrieve those objects later.

This tx boundary problem arises from the fact that persistent objects 
are not managed by application server container, ( and we don't want 
them to be ). Only Jboss would be so flexible so that you could mess 
with EJB interceptor stack and invokers to perform serialization earlier 
than transaction handling, but surely this would impact other things.

Looks like I'm demanding to much from OJB, but even EJB doesn't support 
such behaviour like returning Local Entity Beans as a result of a Remote 
Call.

The idea of an EJB bean implementing PersistenceBroker interface seems 
like a replacement for c/s mode.

Cheers Tomasz ;-)

Re: RMI and Proxied collections and references

[Phil Warrick <wa...@bmed.mcgill.ca>]

Thomas Mahler wrote:
>> I think this problem could be resolved by providing a configuration 
>> option that would turn-on proxy materializing triggered by 
>> serialization of persistent classes. This would result in fully 
>> materialized model transfered to a client, and transparent proxy 
>> handling for serialization.
> 
> 
> Yes, this could be an elegant solution! AFAIK there are some callbacks 
> in the serialization api that could help to implement this trick.
> Only disatvantage I see: persistent classes must be derived from a base 
> class implementing these callbacks?
> 

Any Serializable class can override these two functions for special 
handling during serialization/deserialization:

  private void writeObject(java.io.ObjectOutputStream out)
      throws IOException
  private void readObject(java.io.ObjectInputStream in)
      throws IOException, ClassNotFoundException;

So the only base assumed is the Serialization interface.
I do this when I want to ensure that certain proxies (or transient 
objects) are materialized during serialization or deserialization.  I 
too do not use c/s (I do use ejb with jboss) and I want the flexibility 
to choose how heavy my serialized object is under different conditions.

I'm interested to see where this all leads...

Phil

Re: RMI and Proxied collections and references

[Thomas Mahler <th...@web.de>]

Hi again Tomasz

Tomasz Wysocki wrote:
> Hello,
> I've been using OJB in a small web-application project, with a success.
> It is performing nicely and a simple PB API is great ...
> I'm considering, if OJB would be good for more complicated multi-tier 
> application used in distributed environment.
> 
> Suppose we have some remote service returning results that contain 
> persistent classes instances there is a problem if proxied references or 
> collections are used.
> PersistenceBroker field of CollectionProxy is transient, so instance 
> will serialize correctly, and the same is true for IndirectionHandler.
> But what if a service is truly remote, ie. there is no access to 
> database on client side. There is no chance that PersistenceBroker 
> brought up on a client side (by means of PBKey handle) could materialize 
> proxied collections or references. 

This is correct in the singlevm mode only. If you were running your app 
in c/s mode proxies would use the PersistenceBrokerClient to materialize 
  references on demand!

> There even is no much sense in it, 
> cause client can be many network hops away from server's database.

IMO it makes sense if your app is running in c/s mode.
But I agree this solution might produce a lot of problems if used 
carelessly!

> I think this problem could be resolved by providing a configuration 
> option that would turn-on proxy materializing triggered by serialization 
> of persistent classes. This would result in fully materialized model 
> transfered to a client, and transparent proxy handling for serialization.

Yes, this could be an elegant solution! AFAIK there are some callbacks 
in the serialization api that could help to implement this trick.
Only disatvantage I see: persistent classes must be derived from a base 
class implementing these callbacks?

> 
> What do you think?

Are you willing to work on this? I will review your stuff and integrate 
it into the codebase!

> BTW. I don't want to use c/s mode.

I see!
We are currently discussing to drop the c/s mode altogether. The only 
*real* benefit the c/s mode had was the support for network-enabled proxies.
If we can replace this feature with your solution the last reason for 
c/s mode has gone and we could drop it without problems!

cheers,
Thomas


> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: ojb-user-help@jakarta.apache.org
> 
>