You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-dev@jackrabbit.apache.org by Angela Schreiber <an...@adobe.com> on 2017/04/12 09:21:31 UTC
[m12] Effort to Improve Modularisation of Oak
Hi
As mentioned my Marcel this morning [0] we had some offline discussions
related to the oak-blob-azure module and how we could independently
release it. While we didn't see a satisfying solution for the 1.6 branch,
we concluded that we should pick up the modularisation discussion for
address this in the near future.
Consequently a group of oak devs started to work on a PoC on how to
improve modularisation of Oak (in particular oak-core). As we managed to
get rid of the dependency of oak-blob-azure (and oak-segment-tar for that
matter) from oak-core with a reasonable effort, we would like move forward
with this in oak-trunk.
For that matter I created a new epic "Modularisation of Oak" (OAK-6069 at
[1]) and added/linked a initial bunch of issues spotted during the
workshop and earlier. For the 'oak-blob-azure' topic I create a dedicated
task OAK-6073 [2], where I will also add some detailed summary of the
initial effort. The latter can also be looked at on a github fork at [3].
Kind regards
Angela
[0]
http://markmail.org/message/neoiyv5qsffo424e?q=azure+list:org%2Eapache%2Eja
ckrabbit%2Eoak-dev+from:%22Marcel+Reutegger%22&page=1
[1] https://issues.apache.org/jira/browse/OAK-6069
[2] https://issues.apache.org/jira/browse/OAK-6073
[3] https://github.com/mreutegg/jackrabbit-oak/tree/m12n.
Re: [m12] Effort to Improve Modularisation of Oak
Posted by Davide Giannella <da...@apache.org>.
On 19/04/2017 08:14, Michael Drig wrote:
> 3) Further oak-example and oak-exercise: the former already has sub
> modules. Maybe we can rename it to oak-getting-started (or similar)
> and move oak-exercise into the renamed one.
+1. Please also check the deployment of each module. I don't think we
need it to be deployed to maven repository for the examples.
D.
Re: [m12] Effort to Improve Modularisation of Oak
Posted by Angela Schreiber <an...@adobe.com>.
hi michael
i created OAK-6101 <https://issues.apache.org/jira/browse/OAK-6101> as
task of OAK-6069 to track any work wrt consistent naming of oak modules.
will ping davide there wrt renaming of oak-commons-run
kind regards
angela
On 19/04/17 16:33, "Michael Dürig" <md...@apache.org> wrote:
>
>
>On 19.04.17 16:26, Angela Schreiber wrote:
>>> I'm actually reluctant about 1) and 2) as renaming established modules
>>> have quite a ripple effect. As with 3) we already have sub-modules in
>>> one place we should probably start a discussion of switching to a
>>> hierarchical module structure.
>> makes sense to me.
>>
>>> To address 1) and 2) once the main
>>> modularisation effort stabilised.
>> to be honest, i think we have more work ahead of us when looking at what
>> remains in oak-core right now.
>> i guess we would need another discussion here how we want to proceed
>>with
>> various plugins (mainly to comply with jcr), document nodestores, query
>> and security.
>
>Ack, let's leave it at that. I'm mostly fine with the naming and
>renaming existing modules is really not worth the trouble.
>
>Michael
Re: [m12] Effort to Improve Modularisation of Oak
Posted by Michael Dürig <md...@apache.org>.
On 19.04.17 16:26, Angela Schreiber wrote:
>> I'm actually reluctant about 1) and 2) as renaming established modules
>> have quite a ripple effect. As with 3) we already have sub-modules in
>> one place we should probably start a discussion of switching to a
>> hierarchical module structure.
> makes sense to me.
>
>> To address 1) and 2) once the main
>> modularisation effort stabilised.
> to be honest, i think we have more work ahead of us when looking at what
> remains in oak-core right now.
> i guess we would need another discussion here how we want to proceed with
> various plugins (mainly to comply with jcr), document nodestores, query
> and security.
Ack, let's leave it at that. I'm mostly fine with the naming and
renaming existing modules is really not worth the trouble.
Michael
Re: [m12] Effort to Improve Modularisation of Oak
Posted by Angela Schreiber <an...@adobe.com>.
>>
>> Sure... what modules do you think should be renamed? You mentioned
>> oak-commons-run... anything else?
>
>Apart from renaming oak-commons-run to oak-run-commons there is:
>
>1) oak-authentication-* instead of oak-auth-* as this would be inline
>with oak-authorization-*.
while i like the explicit naming i am not sure it's worth the trouble
given the fact that the 'auth' modules have been around for such a long
time.
>
>2) Also it is not obvious that oak-segment-tar and oak-store-spi are
>related. From that POC oak-segment-tar should be something like
>oak-store-segment.
same here. if renaming was an option for those mainly working on
segment-tar, i would suggest you create a jira issue.
>
>3) Further oak-example and oak-exercise: the former already has sub
>modules. Maybe we can rename it to oak-getting-started (or similar) and
>move oak-exercise into the renamed one.
fine with me. whoever takes care of this: note that the documentation for
oak security contains quite some references to the exercises.
>
>I'm actually reluctant about 1) and 2) as renaming established modules
>have quite a ripple effect. As with 3) we already have sub-modules in
>one place we should probably start a discussion of switching to a
>hierarchical module structure.
makes sense to me.
>To address 1) and 2) once the main
>modularisation effort stabilised.
to be honest, i think we have more work ahead of us when looking at what
remains in oak-core right now.
i guess we would need another discussion here how we want to proceed with
various plugins (mainly to comply with jcr), document nodestores, query
and security.
kind regards
angela
>
>Michael
>
>>
>> Kind regards
>> Angela
>>
>> On 18/04/17 08:57, "Michael Dürig" <md...@apache.org> wrote:
>>
>>>
>>>
>>> On 13.04.17 15:52, Angela Schreiber wrote:
>>>> {quote}
>>>> I would suggest to go with a naming scheme that reflects how modules
>>>> would be grouped together in a hierarchical structure as much as
>>>> possible for now. E.g. rename oak-commons-run to oak-run-commons.
>>>> {quote}
>>>>
>>>> I would like to address this separately as it would further expand the
>>>> scope of OAK-6073, which will be open for review over the weekend.
>>>>After
>>>> that I would suggest that we incorporate the refactoring into
>>>>oak-trunk.
>>>
>>> Works for me, but let's address it quickly afterwards so that those
>>> "intermediate" module names do not get a chance to "stick around".
>>>
>>> Michael
>>
Re: [m12] Effort to Improve Modularisation of Oak
Posted by Michael Dürig <md...@apache.org>.
On 18.04.17 14:51, Angela Schreiber wrote:
> Hi Michael
>
> Sure... what modules do you think should be renamed? You mentioned
> oak-commons-run... anything else?
Apart from renaming oak-commons-run to oak-run-commons there is:
1) oak-authentication-* instead of oak-auth-* as this would be inline
with oak-authorization-*.
2) Also it is not obvious that oak-segment-tar and oak-store-spi are
related. From that POC oak-segment-tar should be something like
oak-store-segment.
3) Further oak-example and oak-exercise: the former already has sub
modules. Maybe we can rename it to oak-getting-started (or similar) and
move oak-exercise into the renamed one.
I'm actually reluctant about 1) and 2) as renaming established modules
have quite a ripple effect. As with 3) we already have sub-modules in
one place we should probably start a discussion of switching to a
hierarchical module structure. To address 1) and 2) once the main
modularisation effort stabilised.
Michael
>
> Kind regards
> Angela
>
> On 18/04/17 08:57, "Michael Drig" <md...@apache.org> wrote:
>
>>
>>
>> On 13.04.17 15:52, Angela Schreiber wrote:
>>> {quote}
>>> I would suggest to go with a naming scheme that reflects how modules
>>> would be grouped together in a hierarchical structure as much as
>>> possible for now. E.g. rename oak-commons-run to oak-run-commons.
>>> {quote}
>>>
>>> I would like to address this separately as it would further expand the
>>> scope of OAK-6073, which will be open for review over the weekend. After
>>> that I would suggest that we incorporate the refactoring into oak-trunk.
>>
>> Works for me, but let's address it quickly afterwards so that those
>> "intermediate" module names do not get a chance to "stick around".
>>
>> Michael
>
Re: [m12] Effort to Improve Modularisation of Oak
Posted by Angela Schreiber <an...@adobe.com>.
Hi Michael
Sure... what modules do you think should be renamed? You mentioned
oak-commons-run... anything else?
Kind regards
Angela
On 18/04/17 08:57, "Michael Dürig" <md...@apache.org> wrote:
>
>
>On 13.04.17 15:52, Angela Schreiber wrote:
>> {quote}
>> I would suggest to go with a naming scheme that reflects how modules
>> would be grouped together in a hierarchical structure as much as
>> possible for now. E.g. rename oak-commons-run to oak-run-commons.
>> {quote}
>>
>> I would like to address this separately as it would further expand the
>> scope of OAK-6073, which will be open for review over the weekend. After
>> that I would suggest that we incorporate the refactoring into oak-trunk.
>
>Works for me, but let's address it quickly afterwards so that those
>"intermediate" module names do not get a chance to "stick around".
>
>Michael
Re: [m12] Effort to Improve Modularisation of Oak
Posted by Angela Schreiber <an...@adobe.com>.
Hi
Just a quick heads up: the refactoring (without the extra renaming
proposed by Michael) has been committed at revision 1791779.
Did my best to avoid breaking anything, checking individual modifications
as well as running the build including ITs.
I would still ask everyone to quickly verify if anything seems to be
broken in their area of expertise; in particular those parts (segment-tar
and document store) that have been worked on recently. I will leave
OAK-6073 <https://issues.apache.org/jira/browse/OAK-6073> open for another
day before resolving it; just in case we still need additional fixes.
Thanks and kind regards
angela
On 18/04/17 08:57, "Michael Dürig" <md...@apache.org> wrote:
>
>
>On 13.04.17 15:52, Angela Schreiber wrote:
>> {quote}
>> I would suggest to go with a naming scheme that reflects how modules
>> would be grouped together in a hierarchical structure as much as
>> possible for now. E.g. rename oak-commons-run to oak-run-commons.
>> {quote}
>>
>> I would like to address this separately as it would further expand the
>> scope of OAK-6073, which will be open for review over the weekend. After
>> that I would suggest that we incorporate the refactoring into oak-trunk.
>
>Works for me, but let's address it quickly afterwards so that those
>"intermediate" module names do not get a chance to "stick around".
>
>Michael
Re: [m12] Effort to Improve Modularisation of Oak
Posted by Michael Dürig <md...@apache.org>.
On 13.04.17 15:52, Angela Schreiber wrote:
> {quote}
> I would suggest to go with a naming scheme that reflects how modules
> would be grouped together in a hierarchical structure as much as
> possible for now. E.g. rename oak-commons-run to oak-run-commons.
> {quote}
>
> I would like to address this separately as it would further expand the
> scope of OAK-6073, which will be open for review over the weekend. After
> that I would suggest that we incorporate the refactoring into oak-trunk.
Works for me, but let's address it quickly afterwards so that those
"intermediate" module names do not get a chance to "stick around".
Michael
Re: [m12] Effort to Improve Modularisation of Oak
Posted by Robert Munteanu <ro...@apache.org>.
Hi Matt,
On Thu, 2017-04-13 at 09:07 -0600, Matt Ryan wrote:
> Great work so far to everyone involved in this effort.
>
> I'm under the impression that this refactoring will constitute a
> change in
> the public API contract of Oak.��In reading the links it seems to
> hint at
> this but whether or not this will actually result in a public API
> change
> isn't explicitly called out.
>
> So my first question:��Is my impression accurate here?
It is accurate to a certain degree. There is no change for applications
consuming Oak via the JCR API ( obviously ) or via the Oak API (
ContentRepository / ContentSession / Tree etc ).
There can be changes to applications hooking into Oak's SPIs, for
instance Sling which sets up an Oak content repository instance via
OSGi.
So in my view most consumers will not be affected.
>
> If so, the follow-up question is, do we have plans to retain backward
> API
> compatibility between 1.6 and 1.8 such that any current code relying
> on 1.6
> API will continue to work with the 1.8 release?
>
> If backward compatibility is in the plan let's make sure it is
> identified
> in the plan; so far I haven't noticed this but surely I could have
> missed
> it.
When we first approached the m12n task we decided to first make it work
as it should. I am not sure with the scale of changes that we
can/should offer backwards compatibility. I'd rather adapt a 'wait and
see' approach, and handle each case individually. There may be just a
small number of backwards compatible shims we need provide, rather than
making everything backwards compatible.
Instead, I view this as an opportunity to really make things backwards
compatible from now on :-)
>
> If backward compatibility is not in the plan, the next question I ask
> is
> whether the next release should actually be Oak 2.0 instead of Oak
> 1.8?��If
> semantic versioning matters it seems to me this change would qualify
> for a
> major release bump.
In my view the Oak version is a 'marketing' version, denoting stability
and feature increments, rather than backwards compatibility to
implementors.
I think that packages should be semantically versioned and - even more
- that modules should be independently versioned. With that, the Oak
version would become even less relevant to these changes.
Thanks,
Robert
>
>
> My intent isn't as much to lobby for strict adherence to semantic
> versioning so much as to make sure we are mindful of what appears to
> be a
> change in the public API contract and that we have a plan to manage
> it.
>
>
> -MR
>
>
> On Thu, Apr 13, 2017 at 7:52 AM, Angela Schreiber <an...@adobe.com>
> wrote:
>
> > Hi Michael and Oak-Devs
> >
> > Just a quick update: I added more details to OAK-6073 included a
> > summary
> > of the effects on oak-core and oak-commons as well as listing
> > changes to
> > non-test dependencies of most modules.
> >
> > I would like to encourage you to look at the fork and the summary
> > and
> > provide feedback if you spot any problems or concerns.
> >
> > Regarding
> >
> > {quote}
> > I would suggest to go with a naming scheme that reflects how
> > modules
> > would be grouped together in a hierarchical structure as much as
> > possible for now. E.g. rename oak-commons-run to oak-run-commons.
> > {quote}
> >
> > I would like to address this separately as it would further expand
> > the
> > scope of OAK-6073, which will be open for review over the weekend.
> > After
> > that I would suggest that we incorporate the refactoring into oak-
> > trunk.
> >
> > Kind regards
> > Angela
> >
> >
> >
> > On 13/04/17 12:17, "Michael D�rig" <md...@apache.org> wrote:
> >
> > >
> > > > I try to describe the changes proposed by the PoC in
> > > >
> > > > https://na01.safelinks.protection.outlook.com/?url=
> >
> > https%3A%2F%2Fissues.a
> > > > pache.org%2Fjira%2Fbrowse%2FOAK-6073%3FfocusedCommentId%
> >
> > 3D15965623%26pa&d
> > > > ata=02%7C01%7C%7Cd062470b1185427e793608d48256
> >
> > 5535%7Cfa7b1b5a7b34438794aed
> > > > 2c178decee1%7C0%7C0%7C636276754686224956&sdata=
> >
> > W7sTOFt4hmoew%2BMR7K%2F45I
> > > > PvOAOSQPsaGKhWfMUWOuI%3D&reserved=0
> > > >
> > > > ge=com.atlassian.jira.plugin.system.issuetabpanels:
> >
> > comment-tabpanel#comme
> > > > nt
> > > > -15965623.
> > > > Additionally added some step-by-step instruction on how we
> > > > proceeded.
> > > >
> > >
> > > Thanks, this is very valuable!
> > >
> > >
> > > > When we first looked at it on Tuesday last week, I thought that
> > > > we would
> > > > end the exercise with a "tried hard but failed" summary. So, I
> > > > am quite
> > > > pleased that actually ended up with a working PoC.
> > >
> > > So am I, I'm impressed by the progress here!
> > >
> > >
> > > > Looking back I thing the biggest issues are
> > > >
> > > > - putting everything in oak-core was obviously convenient but
> > > > it turned
> > > > out to be impossible to protect against boundary violations
> > > > - packages sometimes contain classes that not really belong
> > > > together,
> > > > e.g.
> > > > � - org.apache.jackrabbit.oak.spi.lifecycle containing
> > > > OakInitializer
> > > > � - oak.apache.jackrabbit.oak.spi.whiteboard containing classes
> > > > that
> > > > should be located with the corresponding feature (e.g. user
> > > > mgt, index)
> > > > - impl specific methods that are not defined by an API contract
> > > > such as
> > > > e.g. ValueImpl.getBlob, ValueImpl.getOakString... this was
> > > > actually the
> > > > only place where we added a new interface and modified existing
> > > > code
> > > > - somehow i got the impression that we didn't manage to make
> > > > consistent
> > > > decision wrt package naming
> > > > � - what should go into a 'spi' package?
> > > > � - what should go into 'plugins'-something? and how is that
> > > > different
> > > > from spi? (and what is e.g. the diff between spi.blob and
> > > > plugins.blob?)
> > > > � - when do we create a new package space oak.somethingnew and
> > > > how are
> > > > those packages intended to be used.
> > > >
> > > > Moving forward I think it would help a lot if we had a common
> > > > understanding here and come up with some description what is
> > > > used for
> > > > what.
> > > > Maybe we also need to take a closer look when adding new stuff
> > > > to
> > > > oak-core
> > > > while moving forward.
> > >
> > > I think this is a discussion we need to take up again in the
> > > aftermath
> > > of this restructuring. For now I think it is best to create JIRA
> > > issue
> > > for those things you had to somehow work around or leave out.
> > >
> > >
> > > >
> > > > > Quick wins?
> > > >
> > > > Well... for me the biggest win is the fact that 'oak-blob-
> > > > azure',
> > > > 'oak-blob-cloud' and 'oak-segment-tar' no longer would depend
> > > > on
> > > > oak-core.
> > >
> > > +1, I'm specially happy with the result for oak-segment-tar. We
> > > already
> > > tried last year to make this module more independent but had to
> > > revert
> > > eventually.
> > >
> > >
> > > > > Looking at the list of modules, its size and the names, did
> > > > > you
> > > > > consider
> > > > > switching to a hierarchical module structure?
> > > >
> > > > No, we didn't discuss that.
> > > >
> > > > > Or could this make sense later on?
> > > >
> > > > I don't have any strong preference here. We had some discussion
> > > > on how
> > > > we
> > > > should align the svn structure in general and what would be the
> > > > best
> > > > when
> > > > we want to start releasing modules individually.
> > > >
> > > > > Otherwise can we come up with a naming scheme that implies
> > > > > grouping (e.g. node store implementations, blob store
> > > > > implementations,
> > > > > etc.)
> > > >
> > > > Sure, makes a lot of sense to me. :-)
> > >
> > > I would suggest to go with a naming scheme that reflects how
> > > modules
> > > would be grouped together in a hierarchical structure as much as
> > > possible for now. E.g. rename oak-commons-run to oak-run-commons.
> > >
> > > Michael
> > > >
> > > > >
> > > > > Re. oak-base and oak-commons, these are probably separated to
> > > > > avoid
> > > > > circular dependencies. Is there anyway to otherwise clarify
> > > > > the
> > > > > difference between the two? I.e. if I implement a new class,
> > > > > which
> > > > > module it should go into? Would oak-base be something like
> > > > > oak-core-spi
> > > > > or even oak-spi? This would nicely dual the oak-store-spi
> > > > > module.
> > > >
> > > > Exactly... and actually I like the name 'oak-core-spi' a lot
> > > > better. In
> > > > OAK-6073 I stated that IMO that module might be a tmp solution
> > > > as it
> > > > currently contains a somewhat loose collection of packages that
> > > > were in
> > > > 'oak-core' and didn't really fit into 'oak-commons' from my
> > > > point of
> > > > view.
> > > > After all I wanted to avoid converting 'oak-commons' into a
> > > > second
> > > > 'oak-core' :-).
> > > > That module is the one with the least consistency IMO. But
> > > > things may
> > > > clarify if we move on... I definitely would love to move
> > > > oak.spi.security
> > > > and oak.security.* out of oak-core... but that probably
> > > > requires a
> > > > second
> > > > round *wishful thinking*.
> > > >
> > > > >
> > > > > Is there plans to move document/rdb stores to separate
> > > > > modules or is
> > > > > this beyond the current scope?
> > > >
> > > > I guess that would be a natural step as we move on... but
> > > > during the
> > > > last
> > > > week we didn't look into this.
> > > >
> > > > Kind regards
> > > > Angela
> > > >
> > > > PS: will attach simplified picture to OAK-6073 to illustrated
> > > > the big
> > > > picture.
> > > >
> > > > >
> > > > > Michael
> > > > >
> > > > > On 12.04.17 11:21, Angela Schreiber wrote:
> > > > > > Hi
> > > > > >
> > > > > > As mentioned my Marcel this morning [0] we had some offline
> > > > > > discussions
> > > > > > related to the oak-blob-azure module and how we could
> > > > > > independently
> > > > > > release it. While we didn't see a satisfying solution for
> > > > > > the 1.6
> > > > > > branch,
> > > > > > we concluded that we should pick up the modularisation
> > > > > > discussion for
> > > > > > address this in the near future.
> > > > > >
> > > > > > Consequently a group of oak devs started to work on a PoC
> > > > > > on how to
> > > > > > improve modularisation of Oak (in particular oak-core). As
> > > > > > we managed
> > > > > > to
> > > > > > get rid of the dependency of oak-blob-azure (and oak-
> > > > > > segment-tar for
> > > > > > that
> > > > > > matter) from oak-core with a reasonable effort, we would
> > > > > > like move
> > > > > > forward
> > > > > > with this in oak-trunk.
> > > > > >
> > > > > > For that matter I created a new epic "Modularisation of
> > > > > > Oak" (OAK-6069
> > > > > > at
> > > > > > [1]) and added/linked a initial bunch of issues spotted
> > > > > > during the
> > > > > > workshop and earlier. For the 'oak-blob-azure' topic I
> > > > > > create a
> > > > > > dedicated
> > > > > > task OAK-6073 [2], where I will also add some detailed
> > > > > > summary of the
> > > > > > initial effort. The latter can also be looked at on a
> > > > > > github fork at
> > > > > > [3].
> > > > > >
> > > > > > Kind regards
> > > > > > Angela
> > > > > >
> > > > > > [0]
> > > > > >
> > > > > >
> > > > > > https://na01.safelinks.protection.outlook.com/?url=
> >
> > http%3A%2F%2Fmarkmai
> > > > > > l.
> > > > > >
> > > > > > org%2Fmessage%2Fneoiyv5qsffo424e%3Fq%3Dazure%2Blist%3Aorg%
> >
> > 252Eapache%25
> > > > > > 2E
> > > > > >
> > > > > > ja&data=02%7C01%7C%7Cdcbd2fbcce2d43eed98208d4819a
> >
> > eae0%7Cfa7b1b5a7b34438
> > > > > > 79
> > > > > >
> > > > > > 4aed2c178decee1%7C0%7C0%7C636275949762279890&sdata=
> >
> > sSEdgDegV4Sigh5%2B5R
> > > > > > 1y
> > > > > > uZQiMUY%2F7pOmvBhxojSpDV8%3D&reserved=0
> > > > > > ckrabbit%2Eoak-dev+from:%22Marcel+Reutegger%22&page=1
> > > > > > [1]
> > > > > >
> > > > > > https://na01.safelinks.protection.outlook.com/?url=
> >
> > https%3A%2F%2Fissues
> > > > > > .a
> > > > > >
> > > > > > pache.org%2Fjira%2Fbrowse%2FOAK-6069&data=02%7C01%7C%
> >
> > 7Cdcbd2fbcce2d43ee
> > > > > > d9
> > > > > >
> > > > > > 8208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178de
> >
> > cee1%7C0%7C0%7C63627594976
> > > > > > 22
> > > > > > 89898&sdata=Gxu0MBcOz7zuobBRVSpofWBDJTV36T60bLH3Wmn1v5Q%3D&
> > > > > > reserved=0
> > > > > > [2]
> > > > > >
> > > > > > https://na01.safelinks.protection.outlook.com/?url=
> >
> > https%3A%2F%2Fissues
> > > > > > .a
> > > > > >
> > > > > > pache.org%2Fjira%2Fbrowse%2FOAK-6073&data=02%7C01%7C%
> >
> > 7Cdcbd2fbcce2d43ee
> > > > > > d9
> > > > > >
> > > > > > 8208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178de
> >
> > cee1%7C0%7C0%7C63627594976
> > > > > > 22
> > > > > > 89898&sdata=Xmxhr0gndLBTf4UvWiM50j7Q71yt13wpENm5SAeTVwo%3D&
> > > > > > reserved=0
> > > > > > [3]
> > > > > >
> > > > > > https://na01.safelinks.protection.outlook.com/?url=
> >
> > https%3A%2F%2Fgithub
> > > > > > .c
> > > > > >
> > > > > > om%2Fmreutegg%2Fjackrabbit-
> > > > > > oak%2Ftree%2Fm12n&data=02%7C01%7C%
> >
> > 7Cdcbd2fbc
> > > > > > ce
> > > > > >
> > > > > > 2d43eed98208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178de
> >
> > cee1%7C0%7C0%7C636
> > > > > > 27
> > > > > >
> > > > > > 5949762289898&sdata=7gBCJs5NTQkAfvE6X0aAHSjaJsXlze
> >
> > 9bPrTxBRFxXZE%3D&rese
> > > > > > rv
> > > > > > ed=0.
> > > > > >
> >
> >
Re: [m12] Effort to Improve Modularisation of Oak
Posted by Matt Ryan <os...@mvryan.org>.
Great work so far to everyone involved in this effort.
I'm under the impression that this refactoring will constitute a change in
the public API contract of Oak. In reading the links it seems to hint at
this but whether or not this will actually result in a public API change
isn't explicitly called out.
So my first question: Is my impression accurate here?
If so, the follow-up question is, do we have plans to retain backward API
compatibility between 1.6 and 1.8 such that any current code relying on 1.6
API will continue to work with the 1.8 release?
If backward compatibility is in the plan let's make sure it is identified
in the plan; so far I haven't noticed this but surely I could have missed
it.
If backward compatibility is not in the plan, the next question I ask is
whether the next release should actually be Oak 2.0 instead of Oak 1.8? If
semantic versioning matters it seems to me this change would qualify for a
major release bump.
My intent isn't as much to lobby for strict adherence to semantic
versioning so much as to make sure we are mindful of what appears to be a
change in the public API contract and that we have a plan to manage it.
-MR
On Thu, Apr 13, 2017 at 7:52 AM, Angela Schreiber <an...@adobe.com> wrote:
> Hi Michael and Oak-Devs
>
> Just a quick update: I added more details to OAK-6073 included a summary
> of the effects on oak-core and oak-commons as well as listing changes to
> non-test dependencies of most modules.
>
> I would like to encourage you to look at the fork and the summary and
> provide feedback if you spot any problems or concerns.
>
> Regarding
>
> {quote}
> I would suggest to go with a naming scheme that reflects how modules
> would be grouped together in a hierarchical structure as much as
> possible for now. E.g. rename oak-commons-run to oak-run-commons.
> {quote}
>
> I would like to address this separately as it would further expand the
> scope of OAK-6073, which will be open for review over the weekend. After
> that I would suggest that we incorporate the refactoring into oak-trunk.
>
> Kind regards
> Angela
>
>
>
> On 13/04/17 12:17, "Michael Dürig" <md...@apache.org> wrote:
>
> >
> >> I try to describe the changes proposed by the PoC in
> >>
> >>https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fissues.a
> >>pache.org%2Fjira%2Fbrowse%2FOAK-6073%3FfocusedCommentId%
> 3D15965623%26pa&d
> >>ata=02%7C01%7C%7Cd062470b1185427e793608d48256
> 5535%7Cfa7b1b5a7b34438794aed
> >>2c178decee1%7C0%7C0%7C636276754686224956&sdata=
> W7sTOFt4hmoew%2BMR7K%2F45I
> >>PvOAOSQPsaGKhWfMUWOuI%3D&reserved=0
> >>
> >>ge=com.atlassian.jira.plugin.system.issuetabpanels:
> comment-tabpanel#comme
> >>nt
> >> -15965623.
> >> Additionally added some step-by-step instruction on how we proceeded.
> >>
> >
> >Thanks, this is very valuable!
> >
> >
> >> When we first looked at it on Tuesday last week, I thought that we would
> >> end the exercise with a "tried hard but failed" summary. So, I am quite
> >> pleased that actually ended up with a working PoC.
> >
> >So am I, I'm impressed by the progress here!
> >
> >
> >> Looking back I thing the biggest issues are
> >>
> >> - putting everything in oak-core was obviously convenient but it turned
> >> out to be impossible to protect against boundary violations
> >> - packages sometimes contain classes that not really belong together,
> >>e.g.
> >> - org.apache.jackrabbit.oak.spi.lifecycle containing OakInitializer
> >> - oak.apache.jackrabbit.oak.spi.whiteboard containing classes that
> >> should be located with the corresponding feature (e.g. user mgt, index)
> >> - impl specific methods that are not defined by an API contract such as
> >> e.g. ValueImpl.getBlob, ValueImpl.getOakString... this was actually the
> >> only place where we added a new interface and modified existing code
> >> - somehow i got the impression that we didn't manage to make consistent
> >> decision wrt package naming
> >> - what should go into a 'spi' package?
> >> - what should go into 'plugins'-something? and how is that different
> >> from spi? (and what is e.g. the diff between spi.blob and plugins.blob?)
> >> - when do we create a new package space oak.somethingnew and how are
> >> those packages intended to be used.
> >>
> >> Moving forward I think it would help a lot if we had a common
> >> understanding here and come up with some description what is used for
> >>what.
> >> Maybe we also need to take a closer look when adding new stuff to
> >>oak-core
> >> while moving forward.
> >
> >I think this is a discussion we need to take up again in the aftermath
> >of this restructuring. For now I think it is best to create JIRA issue
> >for those things you had to somehow work around or leave out.
> >
> >
> >>
> >>> Quick wins?
> >>
> >> Well... for me the biggest win is the fact that 'oak-blob-azure',
> >> 'oak-blob-cloud' and 'oak-segment-tar' no longer would depend on
> >>oak-core.
> >
> >+1, I'm specially happy with the result for oak-segment-tar. We already
> >tried last year to make this module more independent but had to revert
> >eventually.
> >
> >
> >>> Looking at the list of modules, its size and the names, did you
> >>>consider
> >>> switching to a hierarchical module structure?
> >>
> >> No, we didn't discuss that.
> >>
> >>> Or could this make sense later on?
> >>
> >> I don't have any strong preference here. We had some discussion on how
> >>we
> >> should align the svn structure in general and what would be the best
> >>when
> >> we want to start releasing modules individually.
> >>
> >>> Otherwise can we come up with a naming scheme that implies
> >>> grouping (e.g. node store implementations, blob store implementations,
> >>> etc.)
> >>
> >> Sure, makes a lot of sense to me. :-)
> >
> >I would suggest to go with a naming scheme that reflects how modules
> >would be grouped together in a hierarchical structure as much as
> >possible for now. E.g. rename oak-commons-run to oak-run-commons.
> >
> >Michael
> >>
> >>>
> >>> Re. oak-base and oak-commons, these are probably separated to avoid
> >>> circular dependencies. Is there anyway to otherwise clarify the
> >>> difference between the two? I.e. if I implement a new class, which
> >>> module it should go into? Would oak-base be something like oak-core-spi
> >>> or even oak-spi? This would nicely dual the oak-store-spi module.
> >>
> >> Exactly... and actually I like the name 'oak-core-spi' a lot better. In
> >> OAK-6073 I stated that IMO that module might be a tmp solution as it
> >> currently contains a somewhat loose collection of packages that were in
> >> 'oak-core' and didn't really fit into 'oak-commons' from my point of
> >>view.
> >> After all I wanted to avoid converting 'oak-commons' into a second
> >> 'oak-core' :-).
> >> That module is the one with the least consistency IMO. But things may
> >> clarify if we move on... I definitely would love to move
> >>oak.spi.security
> >> and oak.security.* out of oak-core... but that probably requires a
> >>second
> >> round *wishful thinking*.
> >>
> >>>
> >>> Is there plans to move document/rdb stores to separate modules or is
> >>> this beyond the current scope?
> >>
> >> I guess that would be a natural step as we move on... but during the
> >>last
> >> week we didn't look into this.
> >>
> >> Kind regards
> >> Angela
> >>
> >> PS: will attach simplified picture to OAK-6073 to illustrated the big
> >> picture.
> >>
> >>>
> >>> Michael
> >>>
> >>> On 12.04.17 11:21, Angela Schreiber wrote:
> >>>> Hi
> >>>>
> >>>> As mentioned my Marcel this morning [0] we had some offline
> >>>>discussions
> >>>> related to the oak-blob-azure module and how we could independently
> >>>> release it. While we didn't see a satisfying solution for the 1.6
> >>>> branch,
> >>>> we concluded that we should pick up the modularisation discussion for
> >>>> address this in the near future.
> >>>>
> >>>> Consequently a group of oak devs started to work on a PoC on how to
> >>>> improve modularisation of Oak (in particular oak-core). As we managed
> >>>>to
> >>>> get rid of the dependency of oak-blob-azure (and oak-segment-tar for
> >>>> that
> >>>> matter) from oak-core with a reasonable effort, we would like move
> >>>> forward
> >>>> with this in oak-trunk.
> >>>>
> >>>> For that matter I created a new epic "Modularisation of Oak" (OAK-6069
> >>>> at
> >>>> [1]) and added/linked a initial bunch of issues spotted during the
> >>>> workshop and earlier. For the 'oak-blob-azure' topic I create a
> >>>> dedicated
> >>>> task OAK-6073 [2], where I will also add some detailed summary of the
> >>>> initial effort. The latter can also be looked at on a github fork at
> >>>> [3].
> >>>>
> >>>> Kind regards
> >>>> Angela
> >>>>
> >>>> [0]
> >>>>
> >>>>
> >>>>https://na01.safelinks.protection.outlook.com/?url=
> http%3A%2F%2Fmarkmai
> >>>>l.
> >>>>
> >>>>org%2Fmessage%2Fneoiyv5qsffo424e%3Fq%3Dazure%2Blist%3Aorg%
> 252Eapache%25
> >>>>2E
> >>>>
> >>>>ja&data=02%7C01%7C%7Cdcbd2fbcce2d43eed98208d4819a
> eae0%7Cfa7b1b5a7b34438
> >>>>79
> >>>>
> >>>>4aed2c178decee1%7C0%7C0%7C636275949762279890&sdata=
> sSEdgDegV4Sigh5%2B5R
> >>>>1y
> >>>> uZQiMUY%2F7pOmvBhxojSpDV8%3D&reserved=0
> >>>> ckrabbit%2Eoak-dev+from:%22Marcel+Reutegger%22&page=1
> >>>> [1]
> >>>>
> >>>>https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fissues
> >>>>.a
> >>>>
> >>>>pache.org%2Fjira%2Fbrowse%2FOAK-6069&data=02%7C01%7C%
> 7Cdcbd2fbcce2d43ee
> >>>>d9
> >>>>
> >>>>8208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178de
> cee1%7C0%7C0%7C63627594976
> >>>>22
> >>>> 89898&sdata=Gxu0MBcOz7zuobBRVSpofWBDJTV36T60bLH3Wmn1v5Q%3D&reserved=0
> >>>> [2]
> >>>>
> >>>>https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fissues
> >>>>.a
> >>>>
> >>>>pache.org%2Fjira%2Fbrowse%2FOAK-6073&data=02%7C01%7C%
> 7Cdcbd2fbcce2d43ee
> >>>>d9
> >>>>
> >>>>8208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178de
> cee1%7C0%7C0%7C63627594976
> >>>>22
> >>>> 89898&sdata=Xmxhr0gndLBTf4UvWiM50j7Q71yt13wpENm5SAeTVwo%3D&reserved=0
> >>>> [3]
> >>>>
> >>>>https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fgithub
> >>>>.c
> >>>>
> >>>>om%2Fmreutegg%2Fjackrabbit-oak%2Ftree%2Fm12n&data=02%7C01%7C%
> 7Cdcbd2fbc
> >>>>ce
> >>>>
> >>>>2d43eed98208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178de
> cee1%7C0%7C0%7C636
> >>>>27
> >>>>
> >>>>5949762289898&sdata=7gBCJs5NTQkAfvE6X0aAHSjaJsXlze
> 9bPrTxBRFxXZE%3D&rese
> >>>>rv
> >>>> ed=0.
> >>>>
> >>
>
>
Re: [m12] Effort to Improve Modularisation of Oak
Posted by Angela Schreiber <an...@adobe.com>.
Hi Michael and Oak-Devs
Just a quick update: I added more details to OAK-6073 included a summary
of the effects on oak-core and oak-commons as well as listing changes to
non-test dependencies of most modules.
I would like to encourage you to look at the fork and the summary and
provide feedback if you spot any problems or concerns.
Regarding
{quote}
I would suggest to go with a naming scheme that reflects how modules
would be grouped together in a hierarchical structure as much as
possible for now. E.g. rename oak-commons-run to oak-run-commons.
{quote}
I would like to address this separately as it would further expand the
scope of OAK-6073, which will be open for review over the weekend. After
that I would suggest that we incorporate the refactoring into oak-trunk.
Kind regards
Angela
On 13/04/17 12:17, "Michael Dürig" <md...@apache.org> wrote:
>
>> I try to describe the changes proposed by the PoC in
>>
>>https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.a
>>pache.org%2Fjira%2Fbrowse%2FOAK-6073%3FfocusedCommentId%3D15965623%26pa&d
>>ata=02%7C01%7C%7Cd062470b1185427e793608d482565535%7Cfa7b1b5a7b34438794aed
>>2c178decee1%7C0%7C0%7C636276754686224956&sdata=W7sTOFt4hmoew%2BMR7K%2F45I
>>PvOAOSQPsaGKhWfMUWOuI%3D&reserved=0
>>
>>ge=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comme
>>nt
>> -15965623.
>> Additionally added some step-by-step instruction on how we proceeded.
>>
>
>Thanks, this is very valuable!
>
>
>> When we first looked at it on Tuesday last week, I thought that we would
>> end the exercise with a "tried hard but failed" summary. So, I am quite
>> pleased that actually ended up with a working PoC.
>
>So am I, I'm impressed by the progress here!
>
>
>> Looking back I thing the biggest issues are
>>
>> - putting everything in oak-core was obviously convenient but it turned
>> out to be impossible to protect against boundary violations
>> - packages sometimes contain classes that not really belong together,
>>e.g.
>> - org.apache.jackrabbit.oak.spi.lifecycle containing OakInitializer
>> - oak.apache.jackrabbit.oak.spi.whiteboard containing classes that
>> should be located with the corresponding feature (e.g. user mgt, index)
>> - impl specific methods that are not defined by an API contract such as
>> e.g. ValueImpl.getBlob, ValueImpl.getOakString... this was actually the
>> only place where we added a new interface and modified existing code
>> - somehow i got the impression that we didn't manage to make consistent
>> decision wrt package naming
>> - what should go into a 'spi' package?
>> - what should go into 'plugins'-something? and how is that different
>> from spi? (and what is e.g. the diff between spi.blob and plugins.blob?)
>> - when do we create a new package space oak.somethingnew and how are
>> those packages intended to be used.
>>
>> Moving forward I think it would help a lot if we had a common
>> understanding here and come up with some description what is used for
>>what.
>> Maybe we also need to take a closer look when adding new stuff to
>>oak-core
>> while moving forward.
>
>I think this is a discussion we need to take up again in the aftermath
>of this restructuring. For now I think it is best to create JIRA issue
>for those things you had to somehow work around or leave out.
>
>
>>
>>> Quick wins?
>>
>> Well... for me the biggest win is the fact that 'oak-blob-azure',
>> 'oak-blob-cloud' and 'oak-segment-tar' no longer would depend on
>>oak-core.
>
>+1, I'm specially happy with the result for oak-segment-tar. We already
>tried last year to make this module more independent but had to revert
>eventually.
>
>
>>> Looking at the list of modules, its size and the names, did you
>>>consider
>>> switching to a hierarchical module structure?
>>
>> No, we didn't discuss that.
>>
>>> Or could this make sense later on?
>>
>> I don't have any strong preference here. We had some discussion on how
>>we
>> should align the svn structure in general and what would be the best
>>when
>> we want to start releasing modules individually.
>>
>>> Otherwise can we come up with a naming scheme that implies
>>> grouping (e.g. node store implementations, blob store implementations,
>>> etc.)
>>
>> Sure, makes a lot of sense to me. :-)
>
>I would suggest to go with a naming scheme that reflects how modules
>would be grouped together in a hierarchical structure as much as
>possible for now. E.g. rename oak-commons-run to oak-run-commons.
>
>Michael
>>
>>>
>>> Re. oak-base and oak-commons, these are probably separated to avoid
>>> circular dependencies. Is there anyway to otherwise clarify the
>>> difference between the two? I.e. if I implement a new class, which
>>> module it should go into? Would oak-base be something like oak-core-spi
>>> or even oak-spi? This would nicely dual the oak-store-spi module.
>>
>> Exactly... and actually I like the name 'oak-core-spi' a lot better. In
>> OAK-6073 I stated that IMO that module might be a tmp solution as it
>> currently contains a somewhat loose collection of packages that were in
>> 'oak-core' and didn't really fit into 'oak-commons' from my point of
>>view.
>> After all I wanted to avoid converting 'oak-commons' into a second
>> 'oak-core' :-).
>> That module is the one with the least consistency IMO. But things may
>> clarify if we move on... I definitely would love to move
>>oak.spi.security
>> and oak.security.* out of oak-core... but that probably requires a
>>second
>> round *wishful thinking*.
>>
>>>
>>> Is there plans to move document/rdb stores to separate modules or is
>>> this beyond the current scope?
>>
>> I guess that would be a natural step as we move on... but during the
>>last
>> week we didn't look into this.
>>
>> Kind regards
>> Angela
>>
>> PS: will attach simplified picture to OAK-6073 to illustrated the big
>> picture.
>>
>>>
>>> Michael
>>>
>>> On 12.04.17 11:21, Angela Schreiber wrote:
>>>> Hi
>>>>
>>>> As mentioned my Marcel this morning [0] we had some offline
>>>>discussions
>>>> related to the oak-blob-azure module and how we could independently
>>>> release it. While we didn't see a satisfying solution for the 1.6
>>>> branch,
>>>> we concluded that we should pick up the modularisation discussion for
>>>> address this in the near future.
>>>>
>>>> Consequently a group of oak devs started to work on a PoC on how to
>>>> improve modularisation of Oak (in particular oak-core). As we managed
>>>>to
>>>> get rid of the dependency of oak-blob-azure (and oak-segment-tar for
>>>> that
>>>> matter) from oak-core with a reasonable effort, we would like move
>>>> forward
>>>> with this in oak-trunk.
>>>>
>>>> For that matter I created a new epic "Modularisation of Oak" (OAK-6069
>>>> at
>>>> [1]) and added/linked a initial bunch of issues spotted during the
>>>> workshop and earlier. For the 'oak-blob-azure' topic I create a
>>>> dedicated
>>>> task OAK-6073 [2], where I will also add some detailed summary of the
>>>> initial effort. The latter can also be looked at on a github fork at
>>>> [3].
>>>>
>>>> Kind regards
>>>> Angela
>>>>
>>>> [0]
>>>>
>>>>
>>>>https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmarkmai
>>>>l.
>>>>
>>>>org%2Fmessage%2Fneoiyv5qsffo424e%3Fq%3Dazure%2Blist%3Aorg%252Eapache%25
>>>>2E
>>>>
>>>>ja&data=02%7C01%7C%7Cdcbd2fbcce2d43eed98208d4819aeae0%7Cfa7b1b5a7b34438
>>>>79
>>>>
>>>>4aed2c178decee1%7C0%7C0%7C636275949762279890&sdata=sSEdgDegV4Sigh5%2B5R
>>>>1y
>>>> uZQiMUY%2F7pOmvBhxojSpDV8%3D&reserved=0
>>>> ckrabbit%2Eoak-dev+from:%22Marcel+Reutegger%22&page=1
>>>> [1]
>>>>
>>>>https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues
>>>>.a
>>>>
>>>>pache.org%2Fjira%2Fbrowse%2FOAK-6069&data=02%7C01%7C%7Cdcbd2fbcce2d43ee
>>>>d9
>>>>
>>>>8208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C63627594976
>>>>22
>>>> 89898&sdata=Gxu0MBcOz7zuobBRVSpofWBDJTV36T60bLH3Wmn1v5Q%3D&reserved=0
>>>> [2]
>>>>
>>>>https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues
>>>>.a
>>>>
>>>>pache.org%2Fjira%2Fbrowse%2FOAK-6073&data=02%7C01%7C%7Cdcbd2fbcce2d43ee
>>>>d9
>>>>
>>>>8208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C63627594976
>>>>22
>>>> 89898&sdata=Xmxhr0gndLBTf4UvWiM50j7Q71yt13wpENm5SAeTVwo%3D&reserved=0
>>>> [3]
>>>>
>>>>https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub
>>>>.c
>>>>
>>>>om%2Fmreutegg%2Fjackrabbit-oak%2Ftree%2Fm12n&data=02%7C01%7C%7Cdcbd2fbc
>>>>ce
>>>>
>>>>2d43eed98208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636
>>>>27
>>>>
>>>>5949762289898&sdata=7gBCJs5NTQkAfvE6X0aAHSjaJsXlze9bPrTxBRFxXZE%3D&rese
>>>>rv
>>>> ed=0.
>>>>
>>
Re: [m12] Effort to Improve Modularisation of Oak
Posted by Michael Dürig <md...@apache.org>.
> I try to describe the changes proposed by the PoC in
> https://issues.apache.org/jira/browse/OAK-6073?focusedCommentId=15965623&pa
> ge=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment
> -15965623.
> Additionally added some step-by-step instruction on how we proceeded.
>
Thanks, this is very valuable!
> When we first looked at it on Tuesday last week, I thought that we would
> end the exercise with a "tried hard but failed" summary. So, I am quite
> pleased that actually ended up with a working PoC.
So am I, I'm impressed by the progress here!
> Looking back I thing the biggest issues are
>
> - putting everything in oak-core was obviously convenient but it turned
> out to be impossible to protect against boundary violations
> - packages sometimes contain classes that not really belong together, e.g.
> - org.apache.jackrabbit.oak.spi.lifecycle containing OakInitializer
> - oak.apache.jackrabbit.oak.spi.whiteboard containing classes that
> should be located with the corresponding feature (e.g. user mgt, index)
> - impl specific methods that are not defined by an API contract such as
> e.g. ValueImpl.getBlob, ValueImpl.getOakString... this was actually the
> only place where we added a new interface and modified existing code
> - somehow i got the impression that we didn't manage to make consistent
> decision wrt package naming
> - what should go into a 'spi' package?
> - what should go into 'plugins'-something? and how is that different
> from spi? (and what is e.g. the diff between spi.blob and plugins.blob?)
> - when do we create a new package space oak.somethingnew and how are
> those packages intended to be used.
>
> Moving forward I think it would help a lot if we had a common
> understanding here and come up with some description what is used for what.
> Maybe we also need to take a closer look when adding new stuff to oak-core
> while moving forward.
I think this is a discussion we need to take up again in the aftermath
of this restructuring. For now I think it is best to create JIRA issue
for those things you had to somehow work around or leave out.
>
>> Quick wins?
>
> Well... for me the biggest win is the fact that 'oak-blob-azure',
> 'oak-blob-cloud' and 'oak-segment-tar' no longer would depend on oak-core.
+1, I'm specially happy with the result for oak-segment-tar. We already
tried last year to make this module more independent but had to revert
eventually.
>> Looking at the list of modules, its size and the names, did you consider
>> switching to a hierarchical module structure?
>
> No, we didn't discuss that.
>
>> Or could this make sense later on?
>
> I don't have any strong preference here. We had some discussion on how we
> should align the svn structure in general and what would be the best when
> we want to start releasing modules individually.
>
>> Otherwise can we come up with a naming scheme that implies
>> grouping (e.g. node store implementations, blob store implementations,
>> etc.)
>
> Sure, makes a lot of sense to me. :-)
I would suggest to go with a naming scheme that reflects how modules
would be grouped together in a hierarchical structure as much as
possible for now. E.g. rename oak-commons-run to oak-run-commons.
Michael
>
>>
>> Re. oak-base and oak-commons, these are probably separated to avoid
>> circular dependencies. Is there anyway to otherwise clarify the
>> difference between the two? I.e. if I implement a new class, which
>> module it should go into? Would oak-base be something like oak-core-spi
>> or even oak-spi? This would nicely dual the oak-store-spi module.
>
> Exactly... and actually I like the name 'oak-core-spi' a lot better. In
> OAK-6073 I stated that IMO that module might be a tmp solution as it
> currently contains a somewhat loose collection of packages that were in
> 'oak-core' and didn't really fit into 'oak-commons' from my point of view.
> After all I wanted to avoid converting 'oak-commons' into a second
> 'oak-core' :-).
> That module is the one with the least consistency IMO. But things may
> clarify if we move on... I definitely would love to move oak.spi.security
> and oak.security.* out of oak-core... but that probably requires a second
> round *wishful thinking*.
>
>>
>> Is there plans to move document/rdb stores to separate modules or is
>> this beyond the current scope?
>
> I guess that would be a natural step as we move on... but during the last
> week we didn't look into this.
>
> Kind regards
> Angela
>
> PS: will attach simplified picture to OAK-6073 to illustrated the big
> picture.
>
>>
>> Michael
>>
>> On 12.04.17 11:21, Angela Schreiber wrote:
>>> Hi
>>>
>>> As mentioned my Marcel this morning [0] we had some offline discussions
>>> related to the oak-blob-azure module and how we could independently
>>> release it. While we didn't see a satisfying solution for the 1.6
>>> branch,
>>> we concluded that we should pick up the modularisation discussion for
>>> address this in the near future.
>>>
>>> Consequently a group of oak devs started to work on a PoC on how to
>>> improve modularisation of Oak (in particular oak-core). As we managed to
>>> get rid of the dependency of oak-blob-azure (and oak-segment-tar for
>>> that
>>> matter) from oak-core with a reasonable effort, we would like move
>>> forward
>>> with this in oak-trunk.
>>>
>>> For that matter I created a new epic "Modularisation of Oak" (OAK-6069
>>> at
>>> [1]) and added/linked a initial bunch of issues spotted during the
>>> workshop and earlier. For the 'oak-blob-azure' topic I create a
>>> dedicated
>>> task OAK-6073 [2], where I will also add some detailed summary of the
>>> initial effort. The latter can also be looked at on a github fork at
>>> [3].
>>>
>>> Kind regards
>>> Angela
>>>
>>> [0]
>>>
>>> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmarkmail.
>>> org%2Fmessage%2Fneoiyv5qsffo424e%3Fq%3Dazure%2Blist%3Aorg%252Eapache%252E
>>> ja&data=02%7C01%7C%7Cdcbd2fbcce2d43eed98208d4819aeae0%7Cfa7b1b5a7b3443879
>>> 4aed2c178decee1%7C0%7C0%7C636275949762279890&sdata=sSEdgDegV4Sigh5%2B5R1y
>>> uZQiMUY%2F7pOmvBhxojSpDV8%3D&reserved=0
>>> ckrabbit%2Eoak-dev+from:%22Marcel+Reutegger%22&page=1
>>> [1]
>>> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.a
>>> pache.org%2Fjira%2Fbrowse%2FOAK-6069&data=02%7C01%7C%7Cdcbd2fbcce2d43eed9
>>> 8208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C6362759497622
>>> 89898&sdata=Gxu0MBcOz7zuobBRVSpofWBDJTV36T60bLH3Wmn1v5Q%3D&reserved=0
>>> [2]
>>> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.a
>>> pache.org%2Fjira%2Fbrowse%2FOAK-6073&data=02%7C01%7C%7Cdcbd2fbcce2d43eed9
>>> 8208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C6362759497622
>>> 89898&sdata=Xmxhr0gndLBTf4UvWiM50j7Q71yt13wpENm5SAeTVwo%3D&reserved=0
>>> [3]
>>> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c
>>> om%2Fmreutegg%2Fjackrabbit-oak%2Ftree%2Fm12n&data=02%7C01%7C%7Cdcbd2fbcce
>>> 2d43eed98208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C63627
>>> 5949762289898&sdata=7gBCJs5NTQkAfvE6X0aAHSjaJsXlze9bPrTxBRFxXZE%3D&reserv
>>> ed=0.
>>>
>
Re: [m12] Effort to Improve Modularisation of Oak
Posted by Angela Schreiber <an...@adobe.com>.
Hi Michael
I try to describe the changes proposed by the PoC in
https://issues.apache.org/jira/browse/OAK-6073?focusedCommentId=15965623&pa
ge=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment
-15965623.
Additionally added some step-by-step instruction on how we proceeded.
What is still missing is additional extra summary of impact it has on
'oak-core' and the cleanup of the 'oak.util' package. will add that
tomorrow.
More comments and answers to your questions inline:
On 12/04/17 13:56, "Michael Dürig" <md...@apache.org> wrote:
>
>Thanks for driving this and coming up with a PoC. I like the direction
>this is taking. The module boundaries make sense to me and having
>certain dependencies de-tangled will certainly be helpful going forward.
>Could you share a bit of your experience doing this refactoring?
First of all it wasn't just me but a group of oak developers discussing
and working on this :-)
And I am sure they have more things to add here, looking at the whole
exercise from a different angle.
When we first looked at it on Tuesday last week, I thought that we would
end the exercise with a "tried hard but failed" summary. So, I am quite
pleased that actually ended up with a working PoC.
>What were the main difficulties?
Up to now we were looking for the hanging fruits. I guess the main
difficulties lay ahead of us with plugins.tree and some of the query/index
classes where exported stuff refers to private classes.
>
Looking back I thing the biggest issues are
- putting everything in oak-core was obviously convenient but it turned
out to be impossible to protect against boundary violations
- packages sometimes contain classes that not really belong together, e.g.
- org.apache.jackrabbit.oak.spi.lifecycle containing OakInitializer
- oak.apache.jackrabbit.oak.spi.whiteboard containing classes that
should be located with the corresponding feature (e.g. user mgt, index)
- impl specific methods that are not defined by an API contract such as
e.g. ValueImpl.getBlob, ValueImpl.getOakString... this was actually the
only place where we added a new interface and modified existing code
- somehow i got the impression that we didn't manage to make consistent
decision wrt package naming
- what should go into a 'spi' package?
- what should go into 'plugins'-something? and how is that different
from spi? (and what is e.g. the diff between spi.blob and plugins.blob?)
- when do we create a new package space oak.somethingnew and how are
those packages intended to be used.
Moving forward I think it would help a lot if we had a common
understanding here and come up with some description what is used for what.
Maybe we also need to take a closer look when adding new stuff to oak-core
while moving forward.
>Quick wins?
Well... for me the biggest win is the fact that 'oak-blob-azure',
'oak-blob-cloud' and 'oak-segment-tar' no longer would depend on oak-core.
Working on modularisation also is quite helpful to spot
'dumpster'-packages like e.g. oak.util
Also, working on the modularisation helped to identify unused imports, the
kind of things that come in naturally when trying to clean up.
>Is there anything that could be
>controversial?
We had a discussion in the group on whether it really makes sense to have
'oak-blob' (existing) and 'oak-blob-plugins' (the blob.plugins part of
oak-core)... I don't have a strong feeling here and would leave that
primarily to those working on the blob handling... at the end we kept it
separate to see the effective dependencies to and from those 2 modules
first and then decide on whether we should merge it. after all merging
should be fairly easy compared to the split.
>
>Looking at the list of modules, its size and the names, did you consider
>switching to a hierarchical module structure?
No, we didn't discuss that.
>Or could this make sense later on?
I don't have any strong preference here. We had some discussion on how we
should align the svn structure in general and what would be the best when
we want to start releasing modules individually.
>Otherwise can we come up with a naming scheme that implies
>grouping (e.g. node store implementations, blob store implementations,
>etc.)
Sure, makes a lot of sense to me. :-)
>
>Re. oak-base and oak-commons, these are probably separated to avoid
>circular dependencies. Is there anyway to otherwise clarify the
>difference between the two? I.e. if I implement a new class, which
>module it should go into? Would oak-base be something like oak-core-spi
>or even oak-spi? This would nicely dual the oak-store-spi module.
Exactly... and actually I like the name 'oak-core-spi' a lot better. In
OAK-6073 I stated that IMO that module might be a tmp solution as it
currently contains a somewhat loose collection of packages that were in
'oak-core' and didn't really fit into 'oak-commons' from my point of view.
After all I wanted to avoid converting 'oak-commons' into a second
'oak-core' :-).
That module is the one with the least consistency IMO. But things may
clarify if we move on... I definitely would love to move oak.spi.security
and oak.security.* out of oak-core... but that probably requires a second
round *wishful thinking*.
>
>Is there plans to move document/rdb stores to separate modules or is
>this beyond the current scope?
I guess that would be a natural step as we move on... but during the last
week we didn't look into this.
Kind regards
Angela
PS: will attach simplified picture to OAK-6073 to illustrated the big
picture.
>
>Michael
>
>On 12.04.17 11:21, Angela Schreiber wrote:
>> Hi
>>
>> As mentioned my Marcel this morning [0] we had some offline discussions
>> related to the oak-blob-azure module and how we could independently
>> release it. While we didn't see a satisfying solution for the 1.6
>>branch,
>> we concluded that we should pick up the modularisation discussion for
>> address this in the near future.
>>
>> Consequently a group of oak devs started to work on a PoC on how to
>> improve modularisation of Oak (in particular oak-core). As we managed to
>> get rid of the dependency of oak-blob-azure (and oak-segment-tar for
>>that
>> matter) from oak-core with a reasonable effort, we would like move
>>forward
>> with this in oak-trunk.
>>
>> For that matter I created a new epic "Modularisation of Oak" (OAK-6069
>>at
>> [1]) and added/linked a initial bunch of issues spotted during the
>> workshop and earlier. For the 'oak-blob-azure' topic I create a
>>dedicated
>> task OAK-6073 [2], where I will also add some detailed summary of the
>> initial effort. The latter can also be looked at on a github fork at
>>[3].
>>
>> Kind regards
>> Angela
>>
>> [0]
>>
>>https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmarkmail.
>>org%2Fmessage%2Fneoiyv5qsffo424e%3Fq%3Dazure%2Blist%3Aorg%252Eapache%252E
>>ja&data=02%7C01%7C%7Cdcbd2fbcce2d43eed98208d4819aeae0%7Cfa7b1b5a7b3443879
>>4aed2c178decee1%7C0%7C0%7C636275949762279890&sdata=sSEdgDegV4Sigh5%2B5R1y
>>uZQiMUY%2F7pOmvBhxojSpDV8%3D&reserved=0
>> ckrabbit%2Eoak-dev+from:%22Marcel+Reutegger%22&page=1
>> [1]
>>https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.a
>>pache.org%2Fjira%2Fbrowse%2FOAK-6069&data=02%7C01%7C%7Cdcbd2fbcce2d43eed9
>>8208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C6362759497622
>>89898&sdata=Gxu0MBcOz7zuobBRVSpofWBDJTV36T60bLH3Wmn1v5Q%3D&reserved=0
>> [2]
>>https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.a
>>pache.org%2Fjira%2Fbrowse%2FOAK-6073&data=02%7C01%7C%7Cdcbd2fbcce2d43eed9
>>8208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C6362759497622
>>89898&sdata=Xmxhr0gndLBTf4UvWiM50j7Q71yt13wpENm5SAeTVwo%3D&reserved=0
>> [3]
>>https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c
>>om%2Fmreutegg%2Fjackrabbit-oak%2Ftree%2Fm12n&data=02%7C01%7C%7Cdcbd2fbcce
>>2d43eed98208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C63627
>>5949762289898&sdata=7gBCJs5NTQkAfvE6X0aAHSjaJsXlze9bPrTxBRFxXZE%3D&reserv
>>ed=0.
>>
Re: [m12] Effort to Improve Modularisation of Oak
Posted by Michael Dürig <md...@apache.org>.
Hi Angela,
Thanks for driving this and coming up with a PoC. I like the direction
this is taking. The module boundaries make sense to me and having
certain dependencies de-tangled will certainly be helpful going forward.
Could you share a bit of your experience doing this refactoring? What
were the main difficulties? Quick wins? Is there anything that could be
controversial?
Looking at the list of modules, its size and the names, did you consider
switching to a hierarchical module structure? Or could this make sense
later on? Otherwise can we come up with a naming scheme that implies
grouping (e.g. node store implementations, blob store implementations,
etc.)
Re. oak-base and oak-commons, these are probably separated to avoid
circular dependencies. Is there anyway to otherwise clarify the
difference between the two? I.e. if I implement a new class, which
module it should go into? Would oak-base be something like oak-core-spi
or even oak-spi? This would nicely dual the oak-store-spi module.
Is there plans to move document/rdb stores to separate modules or is
this beyond the current scope?
Michael
On 12.04.17 11:21, Angela Schreiber wrote:
> Hi
>
> As mentioned my Marcel this morning [0] we had some offline discussions
> related to the oak-blob-azure module and how we could independently
> release it. While we didn't see a satisfying solution for the 1.6 branch,
> we concluded that we should pick up the modularisation discussion for
> address this in the near future.
>
> Consequently a group of oak devs started to work on a PoC on how to
> improve modularisation of Oak (in particular oak-core). As we managed to
> get rid of the dependency of oak-blob-azure (and oak-segment-tar for that
> matter) from oak-core with a reasonable effort, we would like move forward
> with this in oak-trunk.
>
> For that matter I created a new epic "Modularisation of Oak" (OAK-6069 at
> [1]) and added/linked a initial bunch of issues spotted during the
> workshop and earlier. For the 'oak-blob-azure' topic I create a dedicated
> task OAK-6073 [2], where I will also add some detailed summary of the
> initial effort. The latter can also be looked at on a github fork at [3].
>
> Kind regards
> Angela
>
> [0]
> http://markmail.org/message/neoiyv5qsffo424e?q=azure+list:org%2Eapache%2Eja
> ckrabbit%2Eoak-dev+from:%22Marcel+Reutegger%22&page=1
> [1] https://issues.apache.org/jira/browse/OAK-6069
> [2] https://issues.apache.org/jira/browse/OAK-6073
> [3] https://github.com/mreutegg/jackrabbit-oak/tree/m12n.
>