openjpa-slices

[Fernando Padilla <fe...@alum.mit.edu>]

So I was wondering if anyone is actively maintaining the slices 
implementation?  My company, we are going to be doing a big bet on 
sharding and OpenJPA, so I would probably be giving feedback, ideas and 
patches towards the slices..

but I wanted to know who might be accepting/reviewing such discussions, 
so they don't fall on deaf ears..

Re: openjpa-slices

[Fernando Padilla <fe...@alum.mit.edu>]

cool.

Another idea/question.  The DistributedConnection/DistributedStatement 
classes all still do sequential execution of the queries against the 
database.  And I wanted to know if there was a technical reason not to 
move them to use the "ParallelExecutor" design pattern used in the 
DistributedStoreQuery?  ( aside from not being truly needed and adding 
code bloat, etc etc )



Pinaki Poddar wrote:
> Hi,
> 
>> So I was wondering if anyone is actively maintaining the slices
>> implementation? 
> 
> Yes. I have still not severed the umbilical cord with Slice :)
> 
>> My company, we are going to be doing a big bet on sharding and OpenJPA, so
>> I would probably be giving feedback, ideas and patches towards the
>> slices..
> 
> You are most welcome. Undoubtedly Slice will benefit from experiences of
> usage in practical setting. 
> 
>> but I wanted to know who might be accepting/reviewing such discussions, so
>> they don't fall on deaf ears..
> 
> You should post to this OpenJPA mailing list. Besides me, other OpenJPA
> community members do have significant insight on distributed architecture
> and as a community we do welcome new ideas and practical usage of our
> software.
> 

Re: openjpa-slices

[Pinaki Poddar <pp...@apache.org>]

Hi,

> So I was wondering if anyone is actively maintaining the slices
> implementation? 

Yes. I have still not severed the umbilical cord with Slice :)

> My company, we are going to be doing a big bet on sharding and OpenJPA, so
> I would probably be giving feedback, ideas and patches towards the
> slices..

You are most welcome. Undoubtedly Slice will benefit from experiences of
usage in practical setting. 

> but I wanted to know who might be accepting/reviewing such discussions, so
> they don't fall on deaf ears..

You should post to this OpenJPA mailing list. Besides me, other OpenJPA
community members do have significant insight on distributed architecture
and as a community we do welcome new ideas and practical usage of our
software.

-- 
View this message in context: http://n2.nabble.com/openjpa-slices-tp1560094p1562231.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.

Re: openjpa-slices

[Robert Burrell Donkin <ro...@gmail.com>]

On 11/21/08, Fernando Padilla <fe...@alum.mit.edu> wrote:
> So I was wondering if anyone is actively maintaining the slices
> implementation?

Apache uses a cloud or collective development model - no one is
assigned to particular code, committers organize themselves around
their current interests and the interests of fellow developers.

> My company, we are going to be doing a big bet on
> sharding and OpenJPA, so I would probably be giving feedback, ideas and
> patches towards the slices..

Cool

If you plan to contribute a number of patchs, I encourage you to file
a CLA (and CCLA if appropriate). Browse http://www.apache.org/dev  for
more information.

> but I wanted to know who might be accepting/reviewing such discussions,
> so they don't fall on deaf ears..

The patches might be picked up by any committer: what's more important
is connecting with the broader development community on list. So this
post is a good first step.

- Robert

Re: openjpa-slices

[Fernando Padilla <fe...@alum.mit.edu>]

Yeah.  I was riding in today and I was thinking about the Replication 
feature.  I was thinking the same thing about how the Replication slices 
and the Distribution slices look a lot a like.. and I was wondering 
maybe I didn't see the motivating use-case for the replication feature.

So for me to understand it better.. can you share some use-cases, users 
of the Replication feature.. was it a feature needed/used by someone, or 
was it just a neat feature to add.. just wondering.  I bet it's useful, 
but I guess from my app, I don't see a use for it just yet.


Pinaki Poddar wrote:
> Hi,
>> I have finished walking through the whole slices code, and it looks good.
> 
> "Proof of the pudding is in eating" :)
> 
>> I think we need to have a plugin to figure out the slice targets the
>> queries run against. Currently this is > supported through query "hints",
>> but I think that is a really bad choice, 
> 
> The "bad choice" is made to ensure JPA API stability. So far, one of the
> strong design constraints in Slice is to place minimal demand to an existing
> JPA application and Query hints being a JPA concept, we decided to leverage
> the same for narrowing the target slices for a query. But more on this
> follows...
> 
>> when really the ReplicationPolicy is the one that should decide which
>> classes are replicated and how...
> 
> An alternative design possibility is to reinterpret DistributionPolicy
> itself and deprecate ReplicationPolicy and @Replicated altogether. The
> modified DistributionPolicy method should return an array or list of slice
> names -- with the semantics that return value of more than one slices imply
> that the input entity is to be replicated.
> If we go by that route of modifying DistributionPolicy then it can also
> incorporate the Query targets as per your suggestions.
> 

Re: openjpa-slices

[Pinaki Poddar <pp...@apache.org>]

Hi,
> I have finished walking through the whole slices code, and it looks good.

"Proof of the pudding is in eating" :)

> I think we need to have a plugin to figure out the slice targets the
> queries run against. Currently this is > supported through query "hints",
> but I think that is a really bad choice, 

The "bad choice" is made to ensure JPA API stability. So far, one of the
strong design constraints in Slice is to place minimal demand to an existing
JPA application and Query hints being a JPA concept, we decided to leverage
the same for narrowing the target slices for a query. But more on this
follows...

> when really the ReplicationPolicy is the one that should decide which
> classes are replicated and how...

An alternative design possibility is to reinterpret DistributionPolicy
itself and deprecate ReplicationPolicy and @Replicated altogether. The
modified DistributionPolicy method should return an array or list of slice
names -- with the semantics that return value of more than one slices imply
that the input entity is to be replicated.
If we go by that route of modifying DistributionPolicy then it can also
incorporate the Query targets as per your suggestions.

-- 
View this message in context: http://n2.nabble.com/openjpa-slices-tp1560094p1562321.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.

Re: openjpa-slices

[Fernando Padilla <fe...@alum.mit.edu>]

I have finished walking through the whole slices code, and it looks 
good. Aside from minimal janitorial/code clean, there is one feature 
that I will want to get implemented.

I think we need to have a plugin to figure out the slice targets the 
queries run against.  So that application code might be able to know 
better about the queries and determine that any particular query might 
only get results on one or two slices..  (sharding has less benefits if 
most queries are forced to execute against all slices).

Currently this is supported through query "hints", but I think that is a 
really bad choice, since hints would be hard-coded, and you force you to 
do a full code change and rebuild, if you wanted to change the number or 
names of slices, etc etc.. (way too intimate and spaghetti code)


I propose to add another plugin, or extend the DistributionPolicy to 
return possible query targets, when you pass in a query, etc etc.. (i 
would also pass in the FecthConfiguration to allow application centric 
hints to help the code decide).


Any comments? ideas? brain-storms??





ps - Along the lines of mixing configurations with code.. the 
replication feature is driven by the @Replicated annotation in a class, 
when really the ReplicationPolicy is the one that should decide which 
classes are replicated and how.. it seems like a pain to have to 
coordinate both code annotation, and configuration level code..  So I 
propose to remove @Replicated annotation, and just expose another 
isReplicated method to the ReplicationPolicy.





Fernando Padilla wrote:
> So I was wondering if anyone is actively maintaining the slices 
> implementation?  My company, we are going to be doing a big bet on 
> sharding and OpenJPA, so I would probably be giving feedback, ideas and 
> patches towards the slices..
> 
> but I wanted to know who might be accepting/reviewing such discussions, 
> so they don't fall on deaf ears..