You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-dev@jackrabbit.apache.org by Francesco Mari <ma...@gmail.com> on 2015/12/07 12:44:09 UTC

Segment Store modularization

Hi all,

I started looking into moving the Segment Store and other classes closely
related to its implementation to a new bundle, independent from oak-core. I
compiled a list of backwards and forward dependencies in [1].

By looking at this list, I immediately recognise two tasks. First, all the
backward dependencies should be removed. Second, it should be evaluated if
some packages should be exposed to allow the segment store to work as an
independent bundle.

Looking forward to your opinions.

[1]: https://wiki.apache.org/jackrabbit/frm/SegmentStoreModularization

Re: Segment Store modularization

Posted by Francesco Mari <ma...@gmail.com>.
I opened OAK-3745 and attached a patch to it. It's really trivial, but it
would be better if somebody would look over it.

2015-12-08 11:08 GMT+01:00 Francesco Mari <ma...@gmail.com>:

> 2015-12-08 10:43 GMT+01:00 Michael Dürig <md...@apache.org>:
>
>>
>>
>> On 8.12.15 10:38 , Francesco Mari wrote:
>>
>>> 2015-12-08 10:32 GMT+01:00 Michael Dürig <md...@apache.org>:
>>>
>>> IMO SNFE should be exported so upstream projects can depend on it.
>>>>
>>>>> Otherwise there is no value in throwing a specific exception in the
>>>>>> first
>>>>>> place.
>>>>>>
>>>>>>
>>>>>> My goal is to move the Segment Store into its own bundle without
>>>>>> having
>>>>>>
>>>>> circular dependencies between this new bundle and oak-core. I could
>>>>> have
>>>>> tried to create two bundles - one with the exported API of the Segment
>>>>> Store and one with its implementation - but I prefer not to go this
>>>>> way at
>>>>> the moment. Defining a proper Segment Store API seems to require a
>>>>> refactoring way deeper than the one I'm doing, and I'm not sure if we
>>>>> want
>>>>> to go head first into this task, given the current changes currently in
>>>>> progress on the Segment Store.
>>>>>
>>>>>
>>>> Right, makes sense. Can we come up with a different way of (somewhat)
>>>> reliable conveying a SNFE up the stack so interested parties could hook
>>>> into it?
>>>>
>>>
>>>
>>> How about introducing a new unchecked exception in the
>>> org.apache.jackrabbit.oak.api package? I propose to call this exception
>>> something like InvalidRepositoryStateException, that implementations of
>>> the
>>> ContentRepository API can throw in case of inconsistencies. This
>>> exceptions
>>> can be checked up in the call stack (like in ValueImpl) and wrapped into
>>> checked exceptions when needed.
>>>
>>
>> +1. And use some kind of a code to signal more details of the cause.
>> Similar to org.apache.jackrabbit.oak.api.CommitFailedException, only better
>> ;-) That is, instead out of band documenting these codes (aka Wiki) make
>> them an enum.
>
>
> Another proposal that came up offline between Michael and me is not to add
> any kind of code to InvalidRepositoryStateException. Implementations of the
> ContentRepository can subclass the exception and expose those subclasses as
> part of their public API. This leaves the choice to the ContentRepository
> implementation about which failures can be handled by the caller and which
> ones are considered to be unrecoverable.
>
>
>>
>>
>> Michael
>>
>
>

Re: Segment Store modularization

Posted by Francesco Mari <ma...@gmail.com>.
2015-12-08 10:43 GMT+01:00 Michael Dürig <md...@apache.org>:

>
>
> On 8.12.15 10:38 , Francesco Mari wrote:
>
>> 2015-12-08 10:32 GMT+01:00 Michael Dürig <md...@apache.org>:
>>
>> IMO SNFE should be exported so upstream projects can depend on it.
>>>
>>>> Otherwise there is no value in throwing a specific exception in the
>>>>> first
>>>>> place.
>>>>>
>>>>>
>>>>> My goal is to move the Segment Store into its own bundle without having
>>>>>
>>>> circular dependencies between this new bundle and oak-core. I could have
>>>> tried to create two bundles - one with the exported API of the Segment
>>>> Store and one with its implementation - but I prefer not to go this way
>>>> at
>>>> the moment. Defining a proper Segment Store API seems to require a
>>>> refactoring way deeper than the one I'm doing, and I'm not sure if we
>>>> want
>>>> to go head first into this task, given the current changes currently in
>>>> progress on the Segment Store.
>>>>
>>>>
>>> Right, makes sense. Can we come up with a different way of (somewhat)
>>> reliable conveying a SNFE up the stack so interested parties could hook
>>> into it?
>>>
>>
>>
>> How about introducing a new unchecked exception in the
>> org.apache.jackrabbit.oak.api package? I propose to call this exception
>> something like InvalidRepositoryStateException, that implementations of
>> the
>> ContentRepository API can throw in case of inconsistencies. This
>> exceptions
>> can be checked up in the call stack (like in ValueImpl) and wrapped into
>> checked exceptions when needed.
>>
>
> +1. And use some kind of a code to signal more details of the cause.
> Similar to org.apache.jackrabbit.oak.api.CommitFailedException, only better
> ;-) That is, instead out of band documenting these codes (aka Wiki) make
> them an enum.


Another proposal that came up offline between Michael and me is not to add
any kind of code to InvalidRepositoryStateException. Implementations of the
ContentRepository can subclass the exception and expose those subclasses as
part of their public API. This leaves the choice to the ContentRepository
implementation about which failures can be handled by the caller and which
ones are considered to be unrecoverable.


>
>
> Michael
>

Re: Segment Store modularization

Posted by Michael Dürig <md...@apache.org>.

On 8.12.15 10:38 , Francesco Mari wrote:
> 2015-12-08 10:32 GMT+01:00 Michael Dürig <md...@apache.org>:
>
>> IMO SNFE should be exported so upstream projects can depend on it.
>>>> Otherwise there is no value in throwing a specific exception in the first
>>>> place.
>>>>
>>>>
>>>> My goal is to move the Segment Store into its own bundle without having
>>> circular dependencies between this new bundle and oak-core. I could have
>>> tried to create two bundles - one with the exported API of the Segment
>>> Store and one with its implementation - but I prefer not to go this way at
>>> the moment. Defining a proper Segment Store API seems to require a
>>> refactoring way deeper than the one I'm doing, and I'm not sure if we want
>>> to go head first into this task, given the current changes currently in
>>> progress on the Segment Store.
>>>
>>
>> Right, makes sense. Can we come up with a different way of (somewhat)
>> reliable conveying a SNFE up the stack so interested parties could hook
>> into it?
>
>
> How about introducing a new unchecked exception in the
> org.apache.jackrabbit.oak.api package? I propose to call this exception
> something like InvalidRepositoryStateException, that implementations of the
> ContentRepository API can throw in case of inconsistencies. This exceptions
> can be checked up in the call stack (like in ValueImpl) and wrapped into
> checked exceptions when needed.

+1. And use some kind of a code to signal more details of the cause. 
Similar to org.apache.jackrabbit.oak.api.CommitFailedException, only 
better ;-) That is, instead out of band documenting these codes (aka 
Wiki) make them an enum.

Michael

Re: Segment Store modularization

Posted by Francesco Mari <ma...@gmail.com>.
2015-12-08 10:32 GMT+01:00 Michael Dürig <md...@apache.org>:

> IMO SNFE should be exported so upstream projects can depend on it.
>>> Otherwise there is no value in throwing a specific exception in the first
>>> place.
>>>
>>>
>>> My goal is to move the Segment Store into its own bundle without having
>> circular dependencies between this new bundle and oak-core. I could have
>> tried to create two bundles - one with the exported API of the Segment
>> Store and one with its implementation - but I prefer not to go this way at
>> the moment. Defining a proper Segment Store API seems to require a
>> refactoring way deeper than the one I'm doing, and I'm not sure if we want
>> to go head first into this task, given the current changes currently in
>> progress on the Segment Store.
>>
>
> Right, makes sense. Can we come up with a different way of (somewhat)
> reliable conveying a SNFE up the stack so interested parties could hook
> into it?


How about introducing a new unchecked exception in the
org.apache.jackrabbit.oak.api package? I propose to call this exception
something like InvalidRepositoryStateException, that implementations of the
ContentRepository API can throw in case of inconsistencies. This exceptions
can be checked up in the call stack (like in ValueImpl) and wrapped into
checked exceptions when needed.


>
>
> Michael
>

Re: Segment Store modularization

Posted by Julian Sedding <js...@gmail.com>.
I agree with Francesco. SNFE should be an implementation detail of the
Segment bundle. If any code outside of this module depends on SNFE in
order to handle it differently, I would consider that a leaked
abstraction. The special handling should instead be moved into the
Segment bundle (which may not be trivial and could require API
changes/additions).

IMHO, that's how modularization can help drive good APIs. Together
with baselining + impprt/export packages, violations of module
boundaries become visible.

Regards
Julian

On Tue, Dec 8, 2015 at 10:32 AM, Michael Dürig <md...@apache.org> wrote:
>>> IMO SNFE should be exported so upstream projects can depend on it.
>>> Otherwise there is no value in throwing a specific exception in the first
>>> place.
>>>
>>>
>> My goal is to move the Segment Store into its own bundle without having
>> circular dependencies between this new bundle and oak-core. I could have
>> tried to create two bundles - one with the exported API of the Segment
>> Store and one with its implementation - but I prefer not to go this way at
>> the moment. Defining a proper Segment Store API seems to require a
>> refactoring way deeper than the one I'm doing, and I'm not sure if we want
>> to go head first into this task, given the current changes currently in
>> progress on the Segment Store.
>
>
> Right, makes sense. Can we come up with a different way of (somewhat)
> reliable conveying a SNFE up the stack so interested parties could hook into
> it?
>
> Michael

Re: Segment Store modularization

Posted by Michael Dürig <md...@apache.org>.
>> IMO SNFE should be exported so upstream projects can depend on it.
>> Otherwise there is no value in throwing a specific exception in the first
>> place.
>>
>>
> My goal is to move the Segment Store into its own bundle without having
> circular dependencies between this new bundle and oak-core. I could have
> tried to create two bundles - one with the exported API of the Segment
> Store and one with its implementation - but I prefer not to go this way at
> the moment. Defining a proper Segment Store API seems to require a
> refactoring way deeper than the one I'm doing, and I'm not sure if we want
> to go head first into this task, given the current changes currently in
> progress on the Segment Store.

Right, makes sense. Can we come up with a different way of (somewhat) 
reliable conveying a SNFE up the stack so interested parties could hook 
into it?

Michael

Re: Segment Store modularization

Posted by Francesco Mari <ma...@gmail.com>.
2015-12-07 18:09 GMT+01:00 Michael Dürig <md...@apache.org>:

>
>
> On 7.12.15 5:01 , Francesco Mari wrote:
>
>>   * org.apache.jackrabbit.oak.plugins.value.ValueImpl [0] : this looks
>>> like
>>> >something we can fix easily, just fix the Segment Property State impl to
>>> >catch potential SNFEs instead of the generic ValueImpl
>>> >
>>> >
>>>
>> This is OAK-3740. I don't know if this approach works.
>> PropertyState#getValue() doesn't declare checked exception, and
>> RepositoryException is a checked exception. What about documenting in
>> PropertyState#getValue() that implementers can throw an
>> IllegalStateException to signal inconsistencies in the underlying code? In
>> this case, ValueImpl could be implemented to always wrap
>> IllegalStateExceptions in a RepositoryException. Moreover,
>> SegemntNotFoundException already is a subclass of IllegalStateException.
>>
>>
> IMO SNFE should be exported so upstream projects can depend on it.
> Otherwise there is no value in throwing a specific exception in the first
> place.
>
>
My goal is to move the Segment Store into its own bundle without having
circular dependencies between this new bundle and oak-core. I could have
tried to create two bundles - one with the exported API of the Segment
Store and one with its implementation - but I prefer not to go this way at
the moment. Defining a proper Segment Store API seems to require a
refactoring way deeper than the one I'm doing, and I'm not sure if we want
to go head first into this task, given the current changes currently in
progress on the Segment Store.


> Michael
>
> PS: the revenge of checked exceptions ;-)

Re: Segment Store modularization

Posted by Michael Dürig <md...@apache.org>.

On 7.12.15 5:01 , Francesco Mari wrote:
>>   * org.apache.jackrabbit.oak.plugins.value.ValueImpl [0] : this looks like
>> >something we can fix easily, just fix the Segment Property State impl to
>> >catch potential SNFEs instead of the generic ValueImpl
>> >
>> >
> This is OAK-3740. I don't know if this approach works.
> PropertyState#getValue() doesn't declare checked exception, and
> RepositoryException is a checked exception. What about documenting in
> PropertyState#getValue() that implementers can throw an
> IllegalStateException to signal inconsistencies in the underlying code? In
> this case, ValueImpl could be implemented to always wrap
> IllegalStateExceptions in a RepositoryException. Moreover,
> SegemntNotFoundException already is a subclass of IllegalStateException.
>

IMO SNFE should be exported so upstream projects can depend on it. 
Otherwise there is no value in throwing a specific exception in the 
first place.

Michael

PS: the revenge of checked exceptions ;-)

Re: Segment Store modularization

Posted by Francesco Mari <ma...@gmail.com>.
Thanks for your review!

2015-12-07 15:34 GMT+01:00 Alex Parvulescu <al...@gmail.com>:

> thanks for the writeup, good stuff!
>
> I'd like to pick up some of the back references you mentioned for further
> discussion:
>
>   * org.apache.jackrabbit.oak.plugins.value.ValueImpl [0] : this looks like
> something we can fix easily, just fix the Segment Property State impl to
> catch potential SNFEs instead of the generic ValueImpl
>
>
This is OAK-3740. I don't know if this approach works.
PropertyState#getValue() doesn't declare checked exception, and
RepositoryException is a checked exception. What about documenting in
PropertyState#getValue() that implementers can throw an
IllegalStateException to signal inconsistencies in the underlying code? In
this case, ValueImpl could be implemented to always wrap
IllegalStateExceptions in a RepositoryException. Moreover,
SegemntNotFoundException already is a subclass of IllegalStateException.


>   * AbstractCheckpointMBean looks like a utility class to reduce code
> duplication between both SegmentMk and DocumentMk. I'm actually wondering
> if this reference is not an artifact of the refactoring and if we could
> simply reference the abstract class name instead of the specific
> SegmentCheckpointMBean class for the 'name' field [1].
>

This is OAK-3741. I will try to parameterize the type of the MBean,
adapting the different implementations when needed. Since I count only two
subclasses of AbstractCheckpointMBean, this shouldn't be a lot of work.


>
>  * FileStoreBackup & FileStoreRestore these are specific plugins which are
> not big enough to have their own module, so options are: maintain a
> 'plugins' package for SegmentMK bits in the new oak-core-segment bundle, or
> move the plugins under the *segment package, which would defeat the point
> of having plugin-like code in the first place.
>

This is OAK-3742. I will try to introduce a new interface to represent the
construction of a NodeStore to use during the backup and restore
operations. This way, I could be able to remove the direct dependency on
the Segment Store from the implementation of those two classes. I don't
have any further detail yet, I need to play a bit with the code and see how
it goes.


>
> alex
>
>
> [0]
>
> https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/value/ValueImpl.java#L379
> [1]
>
> https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/util/AbstractCheckpointMBean.java#L62
>
>
>
> On Mon, Dec 7, 2015 at 12:44 PM, Francesco Mari <ma...@gmail.com>
> wrote:
>
> > Hi all,
> >
> > I started looking into moving the Segment Store and other classes closely
> > related to its implementation to a new bundle, independent from
> oak-core. I
> > compiled a list of backwards and forward dependencies in [1].
> >
> > By looking at this list, I immediately recognise two tasks. First, all
> the
> > backward dependencies should be removed. Second, it should be evaluated
> if
> > some packages should be exposed to allow the segment store to work as an
> > independent bundle.
> >
> > Looking forward to your opinions.
> >
> > [1]: https://wiki.apache.org/jackrabbit/frm/SegmentStoreModularization
> >
>

Re: Segment Store modularization

Posted by Alex Parvulescu <al...@gmail.com>.
thanks for the writeup, good stuff!

I'd like to pick up some of the back references you mentioned for further
discussion:

  * org.apache.jackrabbit.oak.plugins.value.ValueImpl [0] : this looks like
something we can fix easily, just fix the Segment Property State impl to
catch potential SNFEs instead of the generic ValueImpl

  * AbstractCheckpointMBean looks like a utility class to reduce code
duplication between both SegmentMk and DocumentMk. I'm actually wondering
if this reference is not an artifact of the refactoring and if we could
simply reference the abstract class name instead of the specific
SegmentCheckpointMBean class for the 'name' field [1].

 * FileStoreBackup & FileStoreRestore these are specific plugins which are
not big enough to have their own module, so options are: maintain a
'plugins' package for SegmentMK bits in the new oak-core-segment bundle, or
move the plugins under the *segment package, which would defeat the point
of having plugin-like code in the first place.

alex


[0]
https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/value/ValueImpl.java#L379
[1]
https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/util/AbstractCheckpointMBean.java#L62



On Mon, Dec 7, 2015 at 12:44 PM, Francesco Mari <ma...@gmail.com>
wrote:

> Hi all,
>
> I started looking into moving the Segment Store and other classes closely
> related to its implementation to a new bundle, independent from oak-core. I
> compiled a list of backwards and forward dependencies in [1].
>
> By looking at this list, I immediately recognise two tasks. First, all the
> backward dependencies should be removed. Second, it should be evaluated if
> some packages should be exposed to allow the segment store to work as an
> independent bundle.
>
> Looking forward to your opinions.
>
> [1]: https://wiki.apache.org/jackrabbit/frm/SegmentStoreModularization
>