You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-dev@jackrabbit.apache.org by Francesco Mari <ma...@gmail.com> on 2017/09/04 11:55:49 UTC

OAK-6575 - A word of caution

The POC for OAK-6575 aims at introducing the AdapterManager and
AdapterFactory API in Oak. This is just a step away from introducing a
full-fledged implementation of Sling-like adapters in Oak. In fact,
the OakConversionService is just an Adapter for a single, very
specific conversion.

We had a very similar conversation for OAK-1963 in a thread [1] in
this mailing list. In that thread many people raised that concerns
about adapters. Many of us, me included, see adapters as a glorified
way of breaking data encapsulation and are not in favour of
introducing them.

After some quiet time in the thread and in OAK-1963, a new thread [2]
was created, a new issue [3] was opened and a POC [4] was written. The
content of the POC is little more than a permutation of what was
already discussed in OAK-1963. Why a new thread and issue were needed?
Why couldn't we discuss this implementation in the light of the
OAK-1963? Why the old concerns are not taken into consideration once
again? We already talked about all that.

If the majority of the people here is alright in going forward with
the aforementioned POC, so be it. But, at least, let's avoid closing
threads and issues and create new ones that are just carbon copies.
This might be misinterpreted as a way to clear concerns out of our
way.

[1]: http://markmail.org/thread/enmibgfwedypjnos
[2]: https://lists.apache.org/thread.html/2efb1da82b9b93a3a394c3abccbac960175ffc7a13facaac79c9181a@%3Coak-dev.jackrabbit.apache.org%3E
[3]: https://issues.apache.org/jira/browse/OAK-6575
[4]: https://github.com/apache/jackrabbit-oak/compare/trunk...ieb:OAK-6575?expand=1

Re: OAK-6575 - A word of caution

Posted by Francesco Mari <ma...@gmail.com>.
On Mon, Sep 4, 2017 at 4:41 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> If Oak committers feel that the contribution can't be included. Please feel
> free to close OAK-6575 and I will delete the GitHub branch.

Please don't! I'm not here to criticise your work, quite the contrary.
The idea is good, our users seem to need such a feature and we need to
provide an implementation. My personal opinions about the *need* for
this feature are, well, personal. With this thread I only wanted to
remind to the team that we had a similar conversation in the past
about a similar proposal and we didn't really like part of it. As you
stated, you couldn't know it. This is yet another reason why it's
often useful to keep older conversations going, instead of starting
new, seemingly fresh ones.

Re: OAK-6575 - A word of caution

Posted by Ian Boston <ie...@tfd.co.uk>.
Hi,
This is a review of the links shared, clarifying OAK-6575 relationship to
the discussion, where it is the same and where it is different. Apologies
for the long email, there is a lot of material to cover, and it would be
wrong to brush over the concerns of comitters with a few words.

---------------------------------------------------------------------

[1]
http://markmail.org/thread/enmibgfwedypjnos#query:+page:1+mid:5cf7dkcvfhen7c7x+state:results

I thread [1] a method to convert a binary to a File or URL which could
perform any action on the binary was presented.

The Value -> URI conversion in OAK-6575, performed by the blob-cloud bundle
does not do this.

It only allows CloudFront to perform a GET operation on the S3 bucket that
the instance of CloudFront is granted access to. The URI has a well defined
lifetime (60s by default, but could be 5s) and is signed with a shared
private key. It does not give an attacker the ability to guess or forge
other CloudFront URLs. It also does not allow any developer in the same JVM
to use the URI to access any other binary, store the URI or use it in any
other way. Obviously if the developer gets hold of the credentials to the
S3 bucket or the private key, or is prepared to use ASM, they can do almost
whatever they like.

This approach builds on the AWS recommended way of serving private content
via CloudFront.

The patch does introduce a conversion service, owned and controlled by Oak,
that will only allow Oak owned code to perform other conversions. This is
*not* the Sling AdapterManager and does *not* permit code that does not
have access to Oaks core-spi packages to add arbitrary conversions. It does
open the door for Oak committers to support object conversions without a
chain of API changes.

OAK-6575 may or may not address the concerns raised in the thread [1]. I
was a contributor to that thread, but had forgotten its outcome. I suspect
the conversation there informed the work done in OAK-6575. It would be
wrong to say OAK-6575 is a duplicate of [1]. [1] started the work of
documenting the binary use cases, which informed OAK-6575.

------------------------------

[2] https://issues.apache.org/jira/browse/OAK-1963

OAK-1963 considered exposing an generic unsigned S3 URL of File object to
give whatever access the holder of that URL could manage, either
because they had credentials to access to the bucket or because the bucket
was public to them. The URLs in OAK-1963 were long lived.

OAK-6575 does not do this. It signs a URL with a canned AWS access policy
that only allows GET and only by CloudFront. The S3 bucket is not public
and cant be accesses. The CloudFront instance must be configured and
granted access to the S3 bucket in the same way as any other AWS service.

------------------------

[3]
https://lists.apache.org/thread.html/2efb1da82b9b93a3a394c3abccbac960175ffc7a13facaac79c9181a@%3Coak-dev.jackrabbit.apache.org%3E

In particular
https://lists.apache.org/thread.html/acc9eeef966916791c6073e76d3baa232f48abece4ae6b31f264a8ba@%3Coak-dev.jackrabbit.apache.org%3E
posted by Tommaso, and I assume referenced in this thread in relation to
using JWT.

AFAIK, CloudFront does not support GET authorisations secured by JWT. It
only supports access to S3 private content using the recommended AWS signed
URLs approach. Although OAK-6575 provides an implementation emitting a
CloudFront Signed URL, the same approach could be used for a JWT URL.

------------------------------------------
[1]: https://lists.apache.org/thread.html/8b0021987b824b096ea9b470a4edd5
edf1a246ef10548a2343ad4668@1462438837@%3Coak-dev.jackrabbit.apache.org%3E
[2]: https://lists.apache.org/thread.html/4c352d247da81ca6ab05abfb7c5336
8fb88ed3c587fb09b42b87b6ae@1462439965@%3Coak-dev.jackrabbit.apache.org%3E
[3]: https://lists.apache.org/thread.html/fbf28d5e864adbebd677a425cc915f
89cbd7e0ef85a589fb4f948b51@1462784316@%3Coak-dev.jackrabbit.apache.org%3E
[4]: https://lists.apache.org/thread.html/77b05609b7e6f0deedbd3282f734f8
c606e6a7451db0698f29082d7b@1462891501@%3Coak-dev.jackrabbit.apache.org%3E
[5]: https://lists.apache.org/thread.html/d8da865c1f971ff4c84c9616d9f09e
a9369f3e0c6db20f98fbc6e4d3@%3Coak-dev.jackrabbit.apache.org%3E
[6]: https://lists.apache.org/thread.html/ab1d40572467eeee4ef5855af0a0a9
e87255d1144fe40762ff8e9d47@%3Coak-dev.jackrabbit.apache.org%3E
[7]: https://lists.apache.org/thread.html/acc9eeef966916791c6073e76d3baa
232f48abece4ae6b31f264a8ba@%3Coak-dev.jackrabbit.apache.org%3E

AFAICT, all of these references are references to specific messages in the
2 threads which were read from start to end and already commented on. I
dont think anyone would appreciate a comment on each link in turn. If there
is a concern that has been missed, please point it out.

--------------------------------------------------------------------------------------------------------------

In Summary:

I believe OAK-6575 to be different to what was proposed under the thread of
May 2016 and OAK-1963. It was informed by that discussion with the
intention of addressing the concerns, even though I did not remember
exactly which thread it was from the sha1 or the OAK issue number.

While it does introduce a pattern that would allow other conversions which
would, as Tommaso and Francesco (and others have pointed out) this can only
be utilized to create conversions by code owned by Oak, giving Oak complete
control over what, if anything can be converted. That assumes code outside
Oak is not allowed to implement Oak SPI interfaces and register those
implementations. This pattern was introduced at the request of other Oak
committers to remove the need for any Oak API changes.

It is fundamentally different from OAK-1963 because it provides a signed
URL that only allows the action that the current Oak session has been
granted and only for short period of time, just enough to perform a
redirect.

If Oak committers feel that the contribution can't be included. Please feel
free to close OAK-6575 and I will delete the GitHub branch.

(I am not a committer, so have nothing binding here, other than a desire to
improve Oak.)

Best Regards
Ian








On 4 September 2017 at 14:44, Ian Boston <ie...@tfd.co.uk> wrote:

> Hi Francesco and Tommaso,
> I was not aware of the previous discussions and will now read those
> threads and issues. I submitted the issue, patch and thread in good faith,
> not having a detailed knowledge of everything that has been discussed on
> oak-dev. I was not trying to ignore or circumvent any previous decisions or
> opinions. If OAK-6575 brings nothing new and does not address those
> concerns I would not be offended to have the patch and issue rejected and
> deleted and for Oak not to have this capability, although obviously I feel
> that Oak is weaker without the ability to offload bulk data streaming to
> infrastructure designed for that purpose.
>
> Let me read the links you have shared and get back to you.
> Best Regards
> Ian
>
> On 4 September 2017 at 14:20, Tommaso Teofili <to...@gmail.com>
> wrote:
>
>> I share Francesco's concerns, the same way I shared them when we first
>> discussed this way back; I tried to express my doubts on the current
>> proposal in the email thread for OAK-6575 (linked also in Francesco's
>> email), which got ignored; that's fine as long as the majority of us is
>> happy with the current solution, probably it's just me having this not so
>> good feeling here with me, that some of us want this feature to be in a
>> way
>> or another.
>>
>> Tommaso
>>
>>
>>
>> Il giorno lun 4 set 2017 alle ore 14:45 Francesco Mari <
>> mari.francesco@gmail.com> ha scritto:
>>
>> > On Mon, Sep 4, 2017 at 2:13 PM, Chetan Mehrotra
>> > <ch...@gmail.com> wrote:
>> > > Adaptable pattern in itself was not much discussed at that time.
>> >
>> > Concerns about the adaptable pattern and its implications in data
>> > encapsulation were expressed in the old thread at [1], [2], [3], [4],
>> > and in other messages in the same thread. In the new thread, it was
>> > pointed out [5] that solving the problem discussed in OAK-6575 is
>> > orthogonal to the introduction of the adaptable pattern. Moreover, in
>> > the new thread some concerns were expressed about the adaptable
>> > pattern as well at [6] and [7].
>> >
>> > [1]:
>> > https://lists.apache.org/thread.html/8b0021987b824b096ea9b47
>> 0a4edd5edf1a246ef10548a2343ad4668@1462438837@%3Coak-dev.jack
>> rabbit.apache.org%3E
>> > [2]:
>> > https://lists.apache.org/thread.html/4c352d247da81ca6ab05abf
>> b7c53368fb88ed3c587fb09b42b87b6ae@1462439965@%3Coak-dev.jack
>> rabbit.apache.org%3E
>> > [3]:
>> > https://lists.apache.org/thread.html/fbf28d5e864adbebd677a42
>> 5cc915f89cbd7e0ef85a589fb4f948b51@1462784316@%3Coak-dev.jack
>> rabbit.apache.org%3E
>> > [4]:
>> > https://lists.apache.org/thread.html/77b05609b7e6f0deedbd328
>> 2f734f8c606e6a7451db0698f29082d7b@1462891501@%3Coak-dev.jack
>> rabbit.apache.org%3E
>> > [5]:
>> > https://lists.apache.org/thread.html/d8da865c1f971ff4c84c961
>> 6d9f09ea9369f3e0c6db20f98fbc6e4d3@%3Coak-dev.jackrabbit.apache.org%3E
>> > [6]:
>> > https://lists.apache.org/thread.html/ab1d40572467eeee4ef5855
>> af0a0a9e87255d1144fe40762ff8e9d47@%3Coak-dev.jackrabbit.apache.org%3E
>> > [7]:
>> > https://lists.apache.org/thread.html/acc9eeef966916791c6073e
>> 76d3baa232f48abece4ae6b31f264a8ba@%3Coak-dev.jackrabbit.apache.org%3E
>> >
>>
>
>

Re: OAK-6575 - A word of caution

Posted by Michael Dürig <md...@apache.org>.

On 04.09.17 16:18, Bertrand Delacretaz wrote:
> On Mon, Sep 4, 2017 at 3:44 PM, Ian Boston <ie...@tfd.co.uk> wrote:
>> ...I feel
>> that Oak is weaker without the ability to offload bulk data streaming to
>> infrastructure designed for that purpose....
> 
> FWIW as an Oak user I share that feeling, IMO the use cases described
> at https://wiki.apache.org/jackrabbit/JCR%20Binary%20Usecase are
> becoming more and more important.
> 
> Not being a committer I don't really care about the internals, but
> please do not "throw the baby out with the bath water" if the
> internals need to change.
> 

I don't think we are doing that. There is progress on the matter but 
since there are many forces it is slow and non-linear. However, I 
actually prefer this over a rushed solution everybody regrets in a 
couple of months from now.

Michael

Re: OAK-6575 - A word of caution

Posted by Michael Dürig <md...@apache.org>.

>>> See https://github.com/mduerig/jackrabbit-oak/commit/2709c097b01
>>> a006784b7011135efcbbe3ce1ba88 for a *really* quickly hacked together and
>>> entirely untested POC. But it should get the idea across though.
>>
>>
>>
>> Thank you.
>> That makes sense.
>> I think it only needs the  java/org/apache/jackrabbit/
>> oak/blob/cloud/aws/s3/CloudFrontS3SignedUrlAdapterFactory.java and the
>> API to be inside Oak, everything else can be in Sling.
>> I'll update my patch and do a 2 options for Sling.
>>
> 
> 
> https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:OAK-6575-3?expand=1
> 
> and
> 
> https://github.com/apache/sling/compare/trunk...ieb:OAK-6575-3?expand=1
> 
> wdyt ?
> Obviously the second patch needs to be discussed with Sling dev, but is
> should not be too contentious.
> 

+1. I think this approach is lean and mean. Thanks for figuring out the 
things I left out in my initial cruft.

Michael

Oak Threat Model (was: Re: OAK-6575 - A word of caution)

Posted by Matt Ryan <os...@mvryan.org>.
Hi,

Moving the threat model discussion to a new mail thread.


On September 18, 2017 at 7:15:34 AM, Davide Giannella (davide@apache.org)
wrote:

On 18/09/2017 12:40, Ian Boston wrote:
> can be evaluated
> as part of the release process, ideally before the next stable release.

I mostly do releases here, let me drop a consideration. Our release
process is already long. Let's not add too much stuff even more,
specially if we have to do it at every release. Otherwise I don't think
we'll be able to cope with the bi-weekly release.

So let's be considerate about what adding to the release process.

A word of caution ;)

Cheers
Davide

Re: OAK-6575 - A word of caution

Posted by Angela Schreiber <an...@adobe.com.INVALID>.
Hi Davide

Oh... I guess there is a misunderstanding... don't think anybody wants to
make the release process more difficult. Taking our threat model into
consideration (once we have one) cannot be the responsibility of the
person that cuts the release; at that point it's too late anyway.

What we should do IMO is keep the threat model in mind when we develop new
stuff or back port them to a stable branch. We might make a team effort to
briefly review our changes from a threat model point of view before we cut
a new major release... but in an ideal world that would not be needed if
we take care of features as they get developed.

Kind regards
Angela

On 18/09/17 15:15, "Davide Giannella" <da...@apache.org> wrote:

>On 18/09/2017 12:40, Ian Boston wrote:
>> can be evaluated
>> as part of the release process, ideally before the next stable release.
>
>I mostly do releases here, let me drop a consideration. Our release
>process is already long. Let's not add too much stuff even more,
>specially if we have to do it at every release. Otherwise I don't think
>we'll be able to cope with the bi-weekly release.
>
>So let's be considerate about what adding to the release process.
>
>A word of caution ;)
>
>Cheers
>Davide
>
>


Re: OAK-6575 - A word of caution

Posted by Davide Giannella <da...@apache.org>.
On 18/09/2017 12:40, Ian Boston wrote:
> can be evaluated
> as part of the release process, ideally before the next stable release.

I mostly do releases here, let me drop a consideration. Our release
process is already long. Let's not add too much stuff even more,
specially if we have to do it at every release. Otherwise I don't think
we'll be able to cope with the bi-weekly release.

So let's be considerate about what adding to the release process.

A word of caution ;)

Cheers
Davide



Re: OAK-6575 - A word of caution

Posted by Tommaso Teofili <to...@gmail.com>.
thanks Ian for taking care of including all the provided feedback, I
finally managed to review the patches and it looks very good now as it
sounds simpler and more inline with the overall Oak design.

Thanks again,
Tommaso



Il giorno gio 7 set 2017 alle ore 16:39 Ian Boston <ie...@tfd.co.uk> ha
scritto:

> On 7 September 2017 at 14:41, Francesco Mari <ma...@gmail.com>
> wrote:
>
> > On Thu, Sep 7, 2017 at 11:05 AM, Ian Boston <ie...@tfd.co.uk> wrote:
> > > On 7 September 2017 at 07:22, Ian Boston <ie...@tfd.co.uk> wrote:
> > >
> > >> Hi,
> > >>
> > >> On 6 September 2017 at 22:43, Michael Dürig <md...@apache.org>
> wrote:
> > >>
> > >>>
> > >>>
> > >>> On 06.09.17 23:08, Michael Dürig wrote:
> > >>>
> > >>>>
> > >>>> Hi,
> > >>>>
> > >>>> On 05.09.17 14:09, Ian Boston wrote:
> > >>>>
> > >>>>> Repeating the comment to on OAK-6575 here for further discussion. 2
> > new
> > >>>>> Patches exploring both options.
> > >>>>>
> > >>>>
> > >>>> I would actually prefer the original patch (
> > >>>> https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:O
> > >>>> AK-6575?expand=1) in most parts. However I have concerns regarding
> the
> > >>>> generality of the new OakConversionService API as mentioned in my
> > previous
> > >>>> mail. I would be more comfortable if this could be restricted to
> > something
> > >>>> that resembles more like a "URIProvider", which given a blob returns
> > an URI.
> > >>>>
> > >>>> On the implementation side, why do we need to introduce the
> adaptable
> > >>>> machinery? Couldn't we re-use the Whiteboard and OSGiWhiteBoard
> > mechanisms
> > >>>> instead? I think these could be used to track URIProvider instances
> > >>>> registered by the various blob stores.
> > >>>>
> > >>>>
> > >>> See https://github.com/mduerig/jackrabbit-oak/commit/2709c097b01
> > >>> a006784b7011135efcbbe3ce1ba88 for a *really* quickly hacked together
> > and
> > >>> entirely untested POC. But it should get the idea across though.
> > >>
> > >>
> > >>
> > >> Thank you.
> > >> That makes sense.
> > >> I think it only needs the  java/org/apache/jackrabbit/
> > >> oak/blob/cloud/aws/s3/CloudFrontS3SignedUrlAdapterFactory.java and the
> > >> API to be inside Oak, everything else can be in Sling.
> > >> I'll update my patch and do a 2 options for Sling.
> > >>
> > >
> > >
> > >
> > >
> > > https://github.com/ieb/jackrabbit-oak/compare/trunk..
> > .ieb:OAK-6575-3?expand=1
> > >
> > > and
> > >
> > >
> https://github.com/apache/sling/compare/trunk...ieb:OAK-6575-3?expand=1
> > >
> > > wdyt ?
> >
> > I like this a lot. It keeps Oak's side simple and cleanly integrates
> > Oak's lower-level services in Sling.
> >
>
>
> Good news.
> I think we should hold off committing the patch until Monday or Tuesday to
> give those who may be offline this week a chance to comment. In particular
> I have not seen a comment from Angela who I would expect to have a view as
> this is acl/security related. That is assuming she is back online next
> week.
>
> Best Regards
> Ian
>
>
> >
> > > Obviously the second patch needs to be discussed with Sling dev, but is
> > > should not be too contentious.
> > >
> > > Best Regards
> > > Ian
> > >
> > >
> > >
> > >>
> > >> I think that should address others concerns since it drops all signs
> of
> > >> any generic object to object conversion from Oak (Francesco), and
> > doesn't
> > >> require wide scale fragile changes with implied requirements being
> > placed
> > >> on how intermediate classes are connected and behave (mine).
> > >>
> > >> Best Regards
> > >> Ian
> > >>
> > >>
> > >>>
> > >>> Michael
> > >>>
> > >>
> > >>
> >
>

Re: OAK-6575 - A word of caution

Posted by Ian Boston <ie...@tfd.co.uk>.
Hi Angela,
Thank you for the clarification.

To summarize, I think consensus has been reached.

1. Move forward with the patch in OAK-6575 discussed in this thread.
2. Develop a thread model so that this and future changes can be evaluated
as part of the release process, ideally before the next stable release.

If anyone feels consensus has not been reached, please continue this thread.

Best Regards
Ian

On 15 September 2017 at 07:52, Angela Schreiber <an...@adobe.com.invalid>
wrote:

> Hi Ian
>
> On 13/09/17 23:34, "Ian Boston" <ie...@tfd.co.uk> wrote:
>
> >Hi Angela,
> >
> >On 13 September 2017 at 06:50, Angela Schreiber
> ><an...@adobe.com.invalid>
> >wrote:
> >
> >> Hi Ian
> >>
> >> The new proposal looks a lot better to me.
> >>
> >> The only concern from a security perspective I could come up with is the
> >> one we expressed already with the very first proposal (see initial word
> >>of
> >> caution mail sent by Francesco): applications built on top of Oak can up
> >> to now be sure that all access to the repository content is subject to
> >>the
> >> same permission evaluation as configured with the repository setup. This
> >> is no longer guaranteed when we offer the ability to plug application
> >>code
> >> that may or may not by-pass the evaluation by allowing it to directly
> >> access binaries.
> >>
> >
> >I don't think this patch bypasses Oak security, and since the API can only
> >be implemented by Oak itself. I am sure any future patch would be subject
> >to the same scrutiny. If it can be implemented outside Oak, then Oak has
> >already been breached, something I can see no evidence of.
> >
> >In this case, the signed url is only issued after Oak security has granted
> >access to the binary, and only returned over the JCR API to the JCR
> >Session
> >that made the call, in the same way that an InputStream allows the bytes
> >of
> >the binary to be read by that session. The URL only allows read access.
> >
> >What the session does with that data, is outside the control of Oak.
> >Unlike the byte[] from the  that has no protection, the signed URL is
> >protected. It may only be used unmodified for the purpose it was intended
> >by Oak and only for a well defined period of time. In that respect,
> >arguably, its is more secure than the byte[] or InputStream.
>
> I am not worried about the security of the approach and maybe I am even
> mistaken if I think this changes our threat model. But even if it actually
> changes our threat model wouldn't mean that your proposal isn't secure.
> Those are two quite distinct things.
>
> That's why i would love us to finally reserve some time to create a threat
> model. It will help us with this discussion and with similar requests in
> the future. and even the threat model itself can evolve over time... but
> it allows us to review features from a different angle.
>
> >
> >>
> >> While I know that this is actually the goal of the whole exercise, we
> >>have
> >> to be aware that this also is a change in our Oak security model. As
> >>such
> >> this may look like a security breach and I have been told by my
> >>colleagues
> >> at Adobe that the 'single-way-to-access' is a relevant security question
> >> with a lot of potential customers.
> >>
> >> That doesn't mean that I am opposed to the patch in it's current form
> >>as I
> >> see the benefits from an Oak pov, I just want to highlight that we are
> >> going to make a fundamental change and we should treat and document it
> >> with the necessary care... maybe we should take this opportunity to
> >> finally create a threat model for Oak? Doing so at this stage would
> >>allow
> >> us to visualise the proposed change to all parties involved.
> >>
> >> wdyt?
> >>
> >
> >Agreed.
> >Having a fully developed threat model which clarified all the risks for
> >every aspect of Oak would, imho, be much better than not defining the
> >risks
> >that exist. Even the most secure application has risks, best exposed in a
> >threat model, however brief.
> >
> >Unfortunately Oak now exists in a world which is distributed, where
> >applications need to embrace the network. This is a fundamental change,
> >which Oak has to embrace. An Oak Threat model that recognises this will be
> >a great step forwards.
> >
> >On the other hand, if you are saying that the Oak Threat model has to be
> >developed and agreed, before this patch can be added, then I am concerned
> >that will take too long. Doing justice to an Oak Treat model will require
> >resource.
>
> No... that's not what I am saying :-)
> What I would strongly advocate though is that we start working on the
> threat model in parallel and make sure we have it completed before we
> create any stable release (1.8.x, 1.6.x, etc) that includes your patch.
> This will allow the whole Oak team to take a final look at it from a
> design of view. Since this has been subject to a lot of disputes in the
> past and actually got rejected then, I think having a threat model will be
> very valuable.
>
> Kind regards
> Angela
>
> >
> >Best `Regards
> >Ian
> >
> >
> >>
> >> Kind regards
> >> Angela
> >>
> >>
> >> On 07/09/17 16:39, "Ian Boston" <ie...@tfd.co.uk> wrote:
> >>
> >> >On 7 September 2017 at 14:41, Francesco Mari <mari.francesco@gmail.com
> >
> >> >wrote:
> >> >
> >> >> On Thu, Sep 7, 2017 at 11:05 AM, Ian Boston <ie...@tfd.co.uk> wrote:
> >> >> > On 7 September 2017 at 07:22, Ian Boston <ie...@tfd.co.uk> wrote:
> >> >> >
> >> >> >> Hi,
> >> >> >>
> >> >> >> On 6 September 2017 at 22:43, Michael Dürig <md...@apache.org>
> >> >>wrote:
> >> >> >>
> >> >> >>>
> >> >> >>>
> >> >> >>> On 06.09.17 23:08, Michael Dürig wrote:
> >> >> >>>
> >> >> >>>>
> >> >> >>>> Hi,
> >> >> >>>>
> >> >> >>>> On 05.09.17 14:09, Ian Boston wrote:
> >> >> >>>>
> >> >> >>>>> Repeating the comment to on OAK-6575 here for further
> >>discussion.
> >> >>2
> >> >> new
> >> >> >>>>> Patches exploring both options.
> >> >> >>>>>
> >> >> >>>>
> >> >> >>>> I would actually prefer the original patch (
> >> >> >>>> https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:O
> >> >> >>>> AK-6575?expand=1) in most parts. However I have concerns
> >>regarding
> >> >>the
> >> >> >>>> generality of the new OakConversionService API as mentioned in
> >>my
> >> >> previous
> >> >> >>>> mail. I would be more comfortable if this could be restricted to
> >> >> something
> >> >> >>>> that resembles more like a "URIProvider", which given a blob
> >> >>returns
> >> >> an URI.
> >> >> >>>>
> >> >> >>>> On the implementation side, why do we need to introduce the
> >> >>adaptable
> >> >> >>>> machinery? Couldn't we re-use the Whiteboard and OSGiWhiteBoard
> >> >> mechanisms
> >> >> >>>> instead? I think these could be used to track URIProvider
> >>instances
> >> >> >>>> registered by the various blob stores.
> >> >> >>>>
> >> >> >>>>
> >> >> >>> See https://github.com/mduerig/jackrabbit-oak/commit/2709c097b01
> >> >> >>> a006784b7011135efcbbe3ce1ba88 for a *really* quickly hacked
> >>together
> >> >> and
> >> >> >>> entirely untested POC. But it should get the idea across though.
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> Thank you.
> >> >> >> That makes sense.
> >> >> >> I think it only needs the  java/org/apache/jackrabbit/
> >> >> >> oak/blob/cloud/aws/s3/CloudFrontS3SignedUrlAdapterFactory.java
> and
> >> >>the
> >> >> >> API to be inside Oak, everything else can be in Sling.
> >> >> >> I'll update my patch and do a 2 options for Sling.
> >> >> >>
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> > https://github.com/ieb/jackrabbit-oak/compare/trunk..
> >> >> .ieb:OAK-6575-3?expand=1
> >> >> >
> >> >> > and
> >> >> >
> >> >> >
> >>
> >>>>https://github.com/apache/sling/compare/trunk...ieb:OAK-
> 6575-3?expand=1
> >> >> >
> >> >> > wdyt ?
> >> >>
> >> >> I like this a lot. It keeps Oak's side simple and cleanly integrates
> >> >> Oak's lower-level services in Sling.
> >> >>
> >> >
> >> >
> >> >Good news.
> >> >I think we should hold off committing the patch until Monday or
> >>Tuesday to
> >> >give those who may be offline this week a chance to comment. In
> >>particular
> >> >I have not seen a comment from Angela who I would expect to have a
> >>view as
> >> >this is acl/security related. That is assuming she is back online next
> >> >week.
> >> >
> >> >Best Regards
> >> >Ian
> >> >
> >> >
> >> >>
> >> >> > Obviously the second patch needs to be discussed with Sling dev,
> >>but
> >> >>is
> >> >> > should not be too contentious.
> >> >> >
> >> >> > Best Regards
> >> >> > Ian
> >> >> >
> >> >> >
> >> >> >
> >> >> >>
> >> >> >> I think that should address others concerns since it drops all
> >>signs
> >> >>of
> >> >> >> any generic object to object conversion from Oak (Francesco), and
> >> >> doesn't
> >> >> >> require wide scale fragile changes with implied requirements being
> >> >> placed
> >> >> >> on how intermediate classes are connected and behave (mine).
> >> >> >>
> >> >> >> Best Regards
> >> >> >> Ian
> >> >> >>
> >> >> >>
> >> >> >>>
> >> >> >>> Michael
> >> >> >>>
> >> >> >>
> >> >> >>
> >> >>
> >>
> >>
>
>

Re: OAK-6575 - A word of caution

Posted by Angela Schreiber <an...@adobe.com.INVALID>.
Hi Ian

On 13/09/17 23:34, "Ian Boston" <ie...@tfd.co.uk> wrote:

>Hi Angela,
>
>On 13 September 2017 at 06:50, Angela Schreiber
><an...@adobe.com.invalid>
>wrote:
>
>> Hi Ian
>>
>> The new proposal looks a lot better to me.
>>
>> The only concern from a security perspective I could come up with is the
>> one we expressed already with the very first proposal (see initial word
>>of
>> caution mail sent by Francesco): applications built on top of Oak can up
>> to now be sure that all access to the repository content is subject to
>>the
>> same permission evaluation as configured with the repository setup. This
>> is no longer guaranteed when we offer the ability to plug application
>>code
>> that may or may not by-pass the evaluation by allowing it to directly
>> access binaries.
>>
>
>I don't think this patch bypasses Oak security, and since the API can only
>be implemented by Oak itself. I am sure any future patch would be subject
>to the same scrutiny. If it can be implemented outside Oak, then Oak has
>already been breached, something I can see no evidence of.
>
>In this case, the signed url is only issued after Oak security has granted
>access to the binary, and only returned over the JCR API to the JCR
>Session
>that made the call, in the same way that an InputStream allows the bytes
>of
>the binary to be read by that session. The URL only allows read access.
>
>What the session does with that data, is outside the control of Oak.
>Unlike the byte[] from the  that has no protection, the signed URL is
>protected. It may only be used unmodified for the purpose it was intended
>by Oak and only for a well defined period of time. In that respect,
>arguably, its is more secure than the byte[] or InputStream.

I am not worried about the security of the approach and maybe I am even
mistaken if I think this changes our threat model. But even if it actually
changes our threat model wouldn't mean that your proposal isn't secure.
Those are two quite distinct things.

That's why i would love us to finally reserve some time to create a threat
model. It will help us with this discussion and with similar requests in
the future. and even the threat model itself can evolve over time... but
it allows us to review features from a different angle.

>
>>
>> While I know that this is actually the goal of the whole exercise, we
>>have
>> to be aware that this also is a change in our Oak security model. As
>>such
>> this may look like a security breach and I have been told by my
>>colleagues
>> at Adobe that the 'single-way-to-access' is a relevant security question
>> with a lot of potential customers.
>>
>> That doesn't mean that I am opposed to the patch in it's current form
>>as I
>> see the benefits from an Oak pov, I just want to highlight that we are
>> going to make a fundamental change and we should treat and document it
>> with the necessary care... maybe we should take this opportunity to
>> finally create a threat model for Oak? Doing so at this stage would
>>allow
>> us to visualise the proposed change to all parties involved.
>>
>> wdyt?
>>
>
>Agreed.
>Having a fully developed threat model which clarified all the risks for
>every aspect of Oak would, imho, be much better than not defining the
>risks
>that exist. Even the most secure application has risks, best exposed in a
>threat model, however brief.
>
>Unfortunately Oak now exists in a world which is distributed, where
>applications need to embrace the network. This is a fundamental change,
>which Oak has to embrace. An Oak Threat model that recognises this will be
>a great step forwards.
>
>On the other hand, if you are saying that the Oak Threat model has to be
>developed and agreed, before this patch can be added, then I am concerned
>that will take too long. Doing justice to an Oak Treat model will require
>resource.

No... that's not what I am saying :-)
What I would strongly advocate though is that we start working on the
threat model in parallel and make sure we have it completed before we
create any stable release (1.8.x, 1.6.x, etc) that includes your patch.
This will allow the whole Oak team to take a final look at it from a
design of view. Since this has been subject to a lot of disputes in the
past and actually got rejected then, I think having a threat model will be
very valuable.

Kind regards
Angela

>
>Best `Regards
>Ian
>
>
>>
>> Kind regards
>> Angela
>>
>>
>> On 07/09/17 16:39, "Ian Boston" <ie...@tfd.co.uk> wrote:
>>
>> >On 7 September 2017 at 14:41, Francesco Mari <ma...@gmail.com>
>> >wrote:
>> >
>> >> On Thu, Sep 7, 2017 at 11:05 AM, Ian Boston <ie...@tfd.co.uk> wrote:
>> >> > On 7 September 2017 at 07:22, Ian Boston <ie...@tfd.co.uk> wrote:
>> >> >
>> >> >> Hi,
>> >> >>
>> >> >> On 6 September 2017 at 22:43, Michael Dürig <md...@apache.org>
>> >>wrote:
>> >> >>
>> >> >>>
>> >> >>>
>> >> >>> On 06.09.17 23:08, Michael Dürig wrote:
>> >> >>>
>> >> >>>>
>> >> >>>> Hi,
>> >> >>>>
>> >> >>>> On 05.09.17 14:09, Ian Boston wrote:
>> >> >>>>
>> >> >>>>> Repeating the comment to on OAK-6575 here for further
>>discussion.
>> >>2
>> >> new
>> >> >>>>> Patches exploring both options.
>> >> >>>>>
>> >> >>>>
>> >> >>>> I would actually prefer the original patch (
>> >> >>>> https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:O
>> >> >>>> AK-6575?expand=1) in most parts. However I have concerns
>>regarding
>> >>the
>> >> >>>> generality of the new OakConversionService API as mentioned in
>>my
>> >> previous
>> >> >>>> mail. I would be more comfortable if this could be restricted to
>> >> something
>> >> >>>> that resembles more like a "URIProvider", which given a blob
>> >>returns
>> >> an URI.
>> >> >>>>
>> >> >>>> On the implementation side, why do we need to introduce the
>> >>adaptable
>> >> >>>> machinery? Couldn't we re-use the Whiteboard and OSGiWhiteBoard
>> >> mechanisms
>> >> >>>> instead? I think these could be used to track URIProvider
>>instances
>> >> >>>> registered by the various blob stores.
>> >> >>>>
>> >> >>>>
>> >> >>> See https://github.com/mduerig/jackrabbit-oak/commit/2709c097b01
>> >> >>> a006784b7011135efcbbe3ce1ba88 for a *really* quickly hacked
>>together
>> >> and
>> >> >>> entirely untested POC. But it should get the idea across though.
>> >> >>
>> >> >>
>> >> >>
>> >> >> Thank you.
>> >> >> That makes sense.
>> >> >> I think it only needs the  java/org/apache/jackrabbit/
>> >> >> oak/blob/cloud/aws/s3/CloudFrontS3SignedUrlAdapterFactory.java and
>> >>the
>> >> >> API to be inside Oak, everything else can be in Sling.
>> >> >> I'll update my patch and do a 2 options for Sling.
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > https://github.com/ieb/jackrabbit-oak/compare/trunk..
>> >> .ieb:OAK-6575-3?expand=1
>> >> >
>> >> > and
>> >> >
>> >> >
>> 
>>>>https://github.com/apache/sling/compare/trunk...ieb:OAK-6575-3?expand=1
>> >> >
>> >> > wdyt ?
>> >>
>> >> I like this a lot. It keeps Oak's side simple and cleanly integrates
>> >> Oak's lower-level services in Sling.
>> >>
>> >
>> >
>> >Good news.
>> >I think we should hold off committing the patch until Monday or
>>Tuesday to
>> >give those who may be offline this week a chance to comment. In
>>particular
>> >I have not seen a comment from Angela who I would expect to have a
>>view as
>> >this is acl/security related. That is assuming she is back online next
>> >week.
>> >
>> >Best Regards
>> >Ian
>> >
>> >
>> >>
>> >> > Obviously the second patch needs to be discussed with Sling dev,
>>but
>> >>is
>> >> > should not be too contentious.
>> >> >
>> >> > Best Regards
>> >> > Ian
>> >> >
>> >> >
>> >> >
>> >> >>
>> >> >> I think that should address others concerns since it drops all
>>signs
>> >>of
>> >> >> any generic object to object conversion from Oak (Francesco), and
>> >> doesn't
>> >> >> require wide scale fragile changes with implied requirements being
>> >> placed
>> >> >> on how intermediate classes are connected and behave (mine).
>> >> >>
>> >> >> Best Regards
>> >> >> Ian
>> >> >>
>> >> >>
>> >> >>>
>> >> >>> Michael
>> >> >>>
>> >> >>
>> >> >>
>> >>
>>
>>


Re: OAK-6575 - A word of caution

Posted by Francesco Mari <ma...@gmail.com>.
On Wed, Sep 13, 2017 at 11:34 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> On the other hand, if you are saying that the Oak Threat model has to be
> developed and agreed, before this patch can be added, then I am concerned
> that will take too long. Doing justice to an Oak Treat model will require
> resource.

I think it will not be necessary to wait for a threat model before
accepting this patch. In my opinion the patch already received a good
scrutiny. The threat model is something we need but should be an
effort independent from this patch.

Re: OAK-6575 - A word of caution

Posted by Ian Boston <ie...@tfd.co.uk>.
Hi,

On 14 September 2017 at 06:56, Alex Deparvu <al...@pfalabs.com> wrote:

> > Not to sound cold or uncaring, but IMO that’s a user configuration
> concern, it shouldn’t be Oak’s concern.  What if a user fails to put the
> proper ACLs on a resource in Oak and thus allows users to access it who
> shouldn’t?  Isn’t that basically the same concern?
>
> Except there's a subtle difference here. If I understand correctly (and
> feel free to point it out if that is not the case) security of the
> generated url relies on 2 things: a url that is hard to guess and a ttl
> that needs to be short enough to get the job done but not big enough to
> cause harm if the url is made public.


this is my understanding.


> So the concern I raised is about the
> ttl, this seems sensitive enough to warrant a discussion and probably a few
> log statements for traceability.
>

We can do that, 1 log line per signed url issued with enough of the url to
identify it later, but not enough to gather sufficient data to crack the
key from the logs (theoretically).

ie the last 10 chars of the signature, assume that changes frequently.

Best Regards
Ian

Re: OAK-6575 - A word of caution

Posted by Alex Deparvu <al...@pfalabs.com>.
> Not to sound cold or uncaring, but IMO that’s a user configuration
concern, it shouldn’t be Oak’s concern.  What if a user fails to put the
proper ACLs on a resource in Oak and thus allows users to access it who
shouldn’t?  Isn’t that basically the same concern?

Except there's a subtle difference here. If I understand correctly (and
feel free to point it out if that is not the case) security of the
generated url relies on 2 things: a url that is hard to guess and a ttl
that needs to be short enough to get the job done but not big enough to
cause harm if the url is made public. So the concern I raised is about the
ttl, this seems sensitive enough to warrant a discussion and probably a few
log statements for traceability.

Re: OAK-6575 - A word of caution

Posted by Matt Ryan <os...@mvryan.org>.
Hi,

On September 14, 2017 at 7:05:28 AM, Alex Deparvu (alex@pfalabs.com) wrote:

 how to make sure
this ttl will not get bumped up to a point where it becomes a real problem
(1 min/5mins is probably fine, what if someone sets it to a few hours).



Not to sound cold or uncaring, but IMO that’s a user configuration concern,
it shouldn’t be Oak’s concern.  What if a user fails to put the proper ACLs
on a resource in Oak and thus allows users to access it who shouldn’t?
Isn’t that basically the same concern?

-MR

Re: OAK-6575 - A word of caution

Posted by Ian Boston <ie...@tfd.co.uk>.
Hi,

On 14 September 2017 at 06:05, Alex Deparvu <al...@pfalabs.com> wrote:

> Hi,
>
> > I don't think this patch bypasses Oak security, and since the API can
> only
> > be implemented by Oak itself. I am sure any future patch would be subject
> > to the same scrutiny. If it can be implemented outside Oak, then Oak has
> > already been breached, something I can see no evidence of.
>
> I don't think I agree with this statement. Sure you start with a proper Oak
> session, but the patch facilitates any consumer code to bypass Oak security
> and generate stateless links to any resource, no session involved anymore
> but it's a signed url bound by a ttl. so your trust boundary goes from
> (oak-security) to (oak-security + any consumer code) and here I'm not
> talking about a possible breach, but simply bugs in the code, you now
> delegate the security concerns to code that might not be at the same
> quality level as oak. So it looks like yes, you are bypassing Oak security
> to achieve this goal.
>

Fair point,
Any data that passes over the JCR API boundary has the same
'oak-security+any consumer code' model, which is not the oak-security
model.

The biggest difference here is that Oak knows this bit of data provides
delegated access, whereas it has not knowledge of the sensitivity of other
bits of data that pass over the bounday.



>
>
> > I feel that Oak is weaker without the ability to offload bulk data
> streaming to infrastructure designed for that purpose.
>
> I pasted an older comment to come back to the reasoning behind this need.
> If you bind the urls to a ttl, how do you guarantee that the workflows (or
> whatever process does bulk data streaming) is completed within that time
> frame? Will there be a retry policy? And more specifically how to make sure
> this ttl will not get bumped up to a point where it becomes a real problem
> (1 min/5mins is probably fine, what if someone sets it to a few hours).
>

Workflows should make the request back to Sling/AEM as they do now with the
appropriate credentials. Those requests will get redirected, if appropriate
with a fresh signed URL. The signed URL should never be put into a message
that cant be acted on within the TTL time period.

It is quite possible, that the current CloudFront Singed URL implementation
will not be suitable for a Workflow to use, due to network topology,
although that was out so scope for the requirements in OAK-6575. There was
some suggestions r adding a getPrivateURI to the URIProvider interface.

btw, thanks for your comments in Jira, I have updated the patch.
Best Regards
Ian





>
>
>
> alex
>
>
>
> On Wed, Sep 13, 2017 at 11:34 PM, Ian Boston <ie...@tfd.co.uk> wrote:
>
> > Hi Angela,
> >
> > On 13 September 2017 at 06:50, Angela Schreiber
> <anchela@adobe.com.invalid
> > >
> > wrote:
> >
> > > Hi Ian
> > >
> > > The new proposal looks a lot better to me.
> > >
> > > The only concern from a security perspective I could come up with is
> the
> > > one we expressed already with the very first proposal (see initial word
> > of
> > > caution mail sent by Francesco): applications built on top of Oak can
> up
> > > to now be sure that all access to the repository content is subject to
> > the
> > > same permission evaluation as configured with the repository setup.
> This
> > > is no longer guaranteed when we offer the ability to plug application
> > code
> > > that may or may not by-pass the evaluation by allowing it to directly
> > > access binaries.
> > >
> >
> > I don't think this patch bypasses Oak security, and since the API can
> only
> > be implemented by Oak itself. I am sure any future patch would be subject
> > to the same scrutiny. If it can be implemented outside Oak, then Oak has
> > already been breached, something I can see no evidence of.
> >
> > In this case, the signed url is only issued after Oak security has
> granted
> > access to the binary, and only returned over the JCR API to the JCR
> Session
> > that made the call, in the same way that an InputStream allows the bytes
> of
> > the binary to be read by that session. The URL only allows read access.
> >
> > What the session does with that data, is outside the control of Oak.
> > Unlike the byte[] from the  that has no protection, the signed URL is
> > protected. It may only be used unmodified for the purpose it was intended
> > by Oak and only for a well defined period of time. In that respect,
> > arguably, its is more secure than the byte[] or InputStream.
> >
> >
> >
> > >
> > > While I know that this is actually the goal of the whole exercise, we
> > have
> > > to be aware that this also is a change in our Oak security model. As
> such
> > > this may look like a security breach and I have been told by my
> > colleagues
> > > at Adobe that the 'single-way-to-access' is a relevant security
> question
> > > with a lot of potential customers.
> > >
> > > That doesn't mean that I am opposed to the patch in it's current form
> as
> > I
> > > see the benefits from an Oak pov, I just want to highlight that we are
> > > going to make a fundamental change and we should treat and document it
> > > with the necessary care... maybe we should take this opportunity to
> > > finally create a threat model for Oak? Doing so at this stage would
> allow
> > > us to visualise the proposed change to all parties involved.
> > >
> > > wdyt?
> > >
> >
> > Agreed.
> > Having a fully developed threat model which clarified all the risks for
> > every aspect of Oak would, imho, be much better than not defining the
> risks
> > that exist. Even the most secure application has risks, best exposed in a
> > threat model, however brief.
> >
> > Unfortunately Oak now exists in a world which is distributed, where
> > applications need to embrace the network. This is a fundamental change,
> > which Oak has to embrace. An Oak Threat model that recognises this will
> be
> > a great step forwards.
> >
> > On the other hand, if you are saying that the Oak Threat model has to be
> > developed and agreed, before this patch can be added, then I am concerned
> > that will take too long. Doing justice to an Oak Treat model will require
> > resource.
> >
> > Best `Regards
> > Ian
> >
> >
> > >
> > > Kind regards
> > > Angela
> > >
> > >
> > > On 07/09/17 16:39, "Ian Boston" <ie...@tfd.co.uk> wrote:
> > >
> > > >On 7 September 2017 at 14:41, Francesco Mari <
> mari.francesco@gmail.com>
> > > >wrote:
> > > >
> > > >> On Thu, Sep 7, 2017 at 11:05 AM, Ian Boston <ie...@tfd.co.uk> wrote:
> > > >> > On 7 September 2017 at 07:22, Ian Boston <ie...@tfd.co.uk> wrote:
> > > >> >
> > > >> >> Hi,
> > > >> >>
> > > >> >> On 6 September 2017 at 22:43, Michael Dürig <md...@apache.org>
> > > >>wrote:
> > > >> >>
> > > >> >>>
> > > >> >>>
> > > >> >>> On 06.09.17 23:08, Michael Dürig wrote:
> > > >> >>>
> > > >> >>>>
> > > >> >>>> Hi,
> > > >> >>>>
> > > >> >>>> On 05.09.17 14:09, Ian Boston wrote:
> > > >> >>>>
> > > >> >>>>> Repeating the comment to on OAK-6575 here for further
> > discussion.
> > > >>2
> > > >> new
> > > >> >>>>> Patches exploring both options.
> > > >> >>>>>
> > > >> >>>>
> > > >> >>>> I would actually prefer the original patch (
> > > >> >>>> https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:O
> > > >> >>>> AK-6575?expand=1) in most parts. However I have concerns
> > regarding
> > > >>the
> > > >> >>>> generality of the new OakConversionService API as mentioned in
> my
> > > >> previous
> > > >> >>>> mail. I would be more comfortable if this could be restricted
> to
> > > >> something
> > > >> >>>> that resembles more like a "URIProvider", which given a blob
> > > >>returns
> > > >> an URI.
> > > >> >>>>
> > > >> >>>> On the implementation side, why do we need to introduce the
> > > >>adaptable
> > > >> >>>> machinery? Couldn't we re-use the Whiteboard and OSGiWhiteBoard
> > > >> mechanisms
> > > >> >>>> instead? I think these could be used to track URIProvider
> > instances
> > > >> >>>> registered by the various blob stores.
> > > >> >>>>
> > > >> >>>>
> > > >> >>> See https://github.com/mduerig/jackrabbit-oak/commit/
> 2709c097b01
> > > >> >>> a006784b7011135efcbbe3ce1ba88 for a *really* quickly hacked
> > together
> > > >> and
> > > >> >>> entirely untested POC. But it should get the idea across though.
> > > >> >>
> > > >> >>
> > > >> >>
> > > >> >> Thank you.
> > > >> >> That makes sense.
> > > >> >> I think it only needs the  java/org/apache/jackrabbit/
> > > >> >> oak/blob/cloud/aws/s3/CloudFrontS3SignedUrlAdapterFactory.java
> and
> > > >>the
> > > >> >> API to be inside Oak, everything else can be in Sling.
> > > >> >> I'll update my patch and do a 2 options for Sling.
> > > >> >>
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> > https://github.com/ieb/jackrabbit-oak/compare/trunk..
> > > >> .ieb:OAK-6575-3?expand=1
> > > >> >
> > > >> > and
> > > >> >
> > > >> >
> > > >>https://github.com/apache/sling/compare/trunk...ieb:OAK-
> > 6575-3?expand=1
> > > >> >
> > > >> > wdyt ?
> > > >>
> > > >> I like this a lot. It keeps Oak's side simple and cleanly integrates
> > > >> Oak's lower-level services in Sling.
> > > >>
> > > >
> > > >
> > > >Good news.
> > > >I think we should hold off committing the patch until Monday or
> Tuesday
> > to
> > > >give those who may be offline this week a chance to comment. In
> > particular
> > > >I have not seen a comment from Angela who I would expect to have a
> view
> > as
> > > >this is acl/security related. That is assuming she is back online next
> > > >week.
> > > >
> > > >Best Regards
> > > >Ian
> > > >
> > > >
> > > >>
> > > >> > Obviously the second patch needs to be discussed with Sling dev,
> but
> > > >>is
> > > >> > should not be too contentious.
> > > >> >
> > > >> > Best Regards
> > > >> > Ian
> > > >> >
> > > >> >
> > > >> >
> > > >> >>
> > > >> >> I think that should address others concerns since it drops all
> > signs
> > > >>of
> > > >> >> any generic object to object conversion from Oak (Francesco), and
> > > >> doesn't
> > > >> >> require wide scale fragile changes with implied requirements
> being
> > > >> placed
> > > >> >> on how intermediate classes are connected and behave (mine).
> > > >> >>
> > > >> >> Best Regards
> > > >> >> Ian
> > > >> >>
> > > >> >>
> > > >> >>>
> > > >> >>> Michael
> > > >> >>>
> > > >> >>
> > > >> >>
> > > >>
> > >
> > >
> >
>

Re: OAK-6575 - A word of caution

Posted by Alex Deparvu <al...@pfalabs.com>.
Hi,

> I don't think this patch bypasses Oak security, and since the API can only
> be implemented by Oak itself. I am sure any future patch would be subject
> to the same scrutiny. If it can be implemented outside Oak, then Oak has
> already been breached, something I can see no evidence of.

I don't think I agree with this statement. Sure you start with a proper Oak
session, but the patch facilitates any consumer code to bypass Oak security
and generate stateless links to any resource, no session involved anymore
but it's a signed url bound by a ttl. so your trust boundary goes from
(oak-security) to (oak-security + any consumer code) and here I'm not
talking about a possible breach, but simply bugs in the code, you now
delegate the security concerns to code that might not be at the same
quality level as oak. So it looks like yes, you are bypassing Oak security
to achieve this goal.


> I feel that Oak is weaker without the ability to offload bulk data
streaming to infrastructure designed for that purpose.

I pasted an older comment to come back to the reasoning behind this need.
If you bind the urls to a ttl, how do you guarantee that the workflows (or
whatever process does bulk data streaming) is completed within that time
frame? Will there be a retry policy? And more specifically how to make sure
this ttl will not get bumped up to a point where it becomes a real problem
(1 min/5mins is probably fine, what if someone sets it to a few hours).



alex



On Wed, Sep 13, 2017 at 11:34 PM, Ian Boston <ie...@tfd.co.uk> wrote:

> Hi Angela,
>
> On 13 September 2017 at 06:50, Angela Schreiber <anchela@adobe.com.invalid
> >
> wrote:
>
> > Hi Ian
> >
> > The new proposal looks a lot better to me.
> >
> > The only concern from a security perspective I could come up with is the
> > one we expressed already with the very first proposal (see initial word
> of
> > caution mail sent by Francesco): applications built on top of Oak can up
> > to now be sure that all access to the repository content is subject to
> the
> > same permission evaluation as configured with the repository setup. This
> > is no longer guaranteed when we offer the ability to plug application
> code
> > that may or may not by-pass the evaluation by allowing it to directly
> > access binaries.
> >
>
> I don't think this patch bypasses Oak security, and since the API can only
> be implemented by Oak itself. I am sure any future patch would be subject
> to the same scrutiny. If it can be implemented outside Oak, then Oak has
> already been breached, something I can see no evidence of.
>
> In this case, the signed url is only issued after Oak security has granted
> access to the binary, and only returned over the JCR API to the JCR Session
> that made the call, in the same way that an InputStream allows the bytes of
> the binary to be read by that session. The URL only allows read access.
>
> What the session does with that data, is outside the control of Oak.
> Unlike the byte[] from the  that has no protection, the signed URL is
> protected. It may only be used unmodified for the purpose it was intended
> by Oak and only for a well defined period of time. In that respect,
> arguably, its is more secure than the byte[] or InputStream.
>
>
>
> >
> > While I know that this is actually the goal of the whole exercise, we
> have
> > to be aware that this also is a change in our Oak security model. As such
> > this may look like a security breach and I have been told by my
> colleagues
> > at Adobe that the 'single-way-to-access' is a relevant security question
> > with a lot of potential customers.
> >
> > That doesn't mean that I am opposed to the patch in it's current form as
> I
> > see the benefits from an Oak pov, I just want to highlight that we are
> > going to make a fundamental change and we should treat and document it
> > with the necessary care... maybe we should take this opportunity to
> > finally create a threat model for Oak? Doing so at this stage would allow
> > us to visualise the proposed change to all parties involved.
> >
> > wdyt?
> >
>
> Agreed.
> Having a fully developed threat model which clarified all the risks for
> every aspect of Oak would, imho, be much better than not defining the risks
> that exist. Even the most secure application has risks, best exposed in a
> threat model, however brief.
>
> Unfortunately Oak now exists in a world which is distributed, where
> applications need to embrace the network. This is a fundamental change,
> which Oak has to embrace. An Oak Threat model that recognises this will be
> a great step forwards.
>
> On the other hand, if you are saying that the Oak Threat model has to be
> developed and agreed, before this patch can be added, then I am concerned
> that will take too long. Doing justice to an Oak Treat model will require
> resource.
>
> Best `Regards
> Ian
>
>
> >
> > Kind regards
> > Angela
> >
> >
> > On 07/09/17 16:39, "Ian Boston" <ie...@tfd.co.uk> wrote:
> >
> > >On 7 September 2017 at 14:41, Francesco Mari <ma...@gmail.com>
> > >wrote:
> > >
> > >> On Thu, Sep 7, 2017 at 11:05 AM, Ian Boston <ie...@tfd.co.uk> wrote:
> > >> > On 7 September 2017 at 07:22, Ian Boston <ie...@tfd.co.uk> wrote:
> > >> >
> > >> >> Hi,
> > >> >>
> > >> >> On 6 September 2017 at 22:43, Michael Dürig <md...@apache.org>
> > >>wrote:
> > >> >>
> > >> >>>
> > >> >>>
> > >> >>> On 06.09.17 23:08, Michael Dürig wrote:
> > >> >>>
> > >> >>>>
> > >> >>>> Hi,
> > >> >>>>
> > >> >>>> On 05.09.17 14:09, Ian Boston wrote:
> > >> >>>>
> > >> >>>>> Repeating the comment to on OAK-6575 here for further
> discussion.
> > >>2
> > >> new
> > >> >>>>> Patches exploring both options.
> > >> >>>>>
> > >> >>>>
> > >> >>>> I would actually prefer the original patch (
> > >> >>>> https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:O
> > >> >>>> AK-6575?expand=1) in most parts. However I have concerns
> regarding
> > >>the
> > >> >>>> generality of the new OakConversionService API as mentioned in my
> > >> previous
> > >> >>>> mail. I would be more comfortable if this could be restricted to
> > >> something
> > >> >>>> that resembles more like a "URIProvider", which given a blob
> > >>returns
> > >> an URI.
> > >> >>>>
> > >> >>>> On the implementation side, why do we need to introduce the
> > >>adaptable
> > >> >>>> machinery? Couldn't we re-use the Whiteboard and OSGiWhiteBoard
> > >> mechanisms
> > >> >>>> instead? I think these could be used to track URIProvider
> instances
> > >> >>>> registered by the various blob stores.
> > >> >>>>
> > >> >>>>
> > >> >>> See https://github.com/mduerig/jackrabbit-oak/commit/2709c097b01
> > >> >>> a006784b7011135efcbbe3ce1ba88 for a *really* quickly hacked
> together
> > >> and
> > >> >>> entirely untested POC. But it should get the idea across though.
> > >> >>
> > >> >>
> > >> >>
> > >> >> Thank you.
> > >> >> That makes sense.
> > >> >> I think it only needs the  java/org/apache/jackrabbit/
> > >> >> oak/blob/cloud/aws/s3/CloudFrontS3SignedUrlAdapterFactory.java and
> > >>the
> > >> >> API to be inside Oak, everything else can be in Sling.
> > >> >> I'll update my patch and do a 2 options for Sling.
> > >> >>
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > https://github.com/ieb/jackrabbit-oak/compare/trunk..
> > >> .ieb:OAK-6575-3?expand=1
> > >> >
> > >> > and
> > >> >
> > >> >
> > >>https://github.com/apache/sling/compare/trunk...ieb:OAK-
> 6575-3?expand=1
> > >> >
> > >> > wdyt ?
> > >>
> > >> I like this a lot. It keeps Oak's side simple and cleanly integrates
> > >> Oak's lower-level services in Sling.
> > >>
> > >
> > >
> > >Good news.
> > >I think we should hold off committing the patch until Monday or Tuesday
> to
> > >give those who may be offline this week a chance to comment. In
> particular
> > >I have not seen a comment from Angela who I would expect to have a view
> as
> > >this is acl/security related. That is assuming she is back online next
> > >week.
> > >
> > >Best Regards
> > >Ian
> > >
> > >
> > >>
> > >> > Obviously the second patch needs to be discussed with Sling dev, but
> > >>is
> > >> > should not be too contentious.
> > >> >
> > >> > Best Regards
> > >> > Ian
> > >> >
> > >> >
> > >> >
> > >> >>
> > >> >> I think that should address others concerns since it drops all
> signs
> > >>of
> > >> >> any generic object to object conversion from Oak (Francesco), and
> > >> doesn't
> > >> >> require wide scale fragile changes with implied requirements being
> > >> placed
> > >> >> on how intermediate classes are connected and behave (mine).
> > >> >>
> > >> >> Best Regards
> > >> >> Ian
> > >> >>
> > >> >>
> > >> >>>
> > >> >>> Michael
> > >> >>>
> > >> >>
> > >> >>
> > >>
> >
> >
>

Re: OAK-6575 - A word of caution

Posted by Ian Boston <ie...@tfd.co.uk>.
Hi Angela,

On 13 September 2017 at 06:50, Angela Schreiber <an...@adobe.com.invalid>
wrote:

> Hi Ian
>
> The new proposal looks a lot better to me.
>
> The only concern from a security perspective I could come up with is the
> one we expressed already with the very first proposal (see initial word of
> caution mail sent by Francesco): applications built on top of Oak can up
> to now be sure that all access to the repository content is subject to the
> same permission evaluation as configured with the repository setup. This
> is no longer guaranteed when we offer the ability to plug application code
> that may or may not by-pass the evaluation by allowing it to directly
> access binaries.
>

I don't think this patch bypasses Oak security, and since the API can only
be implemented by Oak itself. I am sure any future patch would be subject
to the same scrutiny. If it can be implemented outside Oak, then Oak has
already been breached, something I can see no evidence of.

In this case, the signed url is only issued after Oak security has granted
access to the binary, and only returned over the JCR API to the JCR Session
that made the call, in the same way that an InputStream allows the bytes of
the binary to be read by that session. The URL only allows read access.

What the session does with that data, is outside the control of Oak.
Unlike the byte[] from the  that has no protection, the signed URL is
protected. It may only be used unmodified for the purpose it was intended
by Oak and only for a well defined period of time. In that respect,
arguably, its is more secure than the byte[] or InputStream.



>
> While I know that this is actually the goal of the whole exercise, we have
> to be aware that this also is a change in our Oak security model. As such
> this may look like a security breach and I have been told by my colleagues
> at Adobe that the 'single-way-to-access' is a relevant security question
> with a lot of potential customers.
>
> That doesn't mean that I am opposed to the patch in it's current form as I
> see the benefits from an Oak pov, I just want to highlight that we are
> going to make a fundamental change and we should treat and document it
> with the necessary care... maybe we should take this opportunity to
> finally create a threat model for Oak? Doing so at this stage would allow
> us to visualise the proposed change to all parties involved.
>
> wdyt?
>

Agreed.
Having a fully developed threat model which clarified all the risks for
every aspect of Oak would, imho, be much better than not defining the risks
that exist. Even the most secure application has risks, best exposed in a
threat model, however brief.

Unfortunately Oak now exists in a world which is distributed, where
applications need to embrace the network. This is a fundamental change,
which Oak has to embrace. An Oak Threat model that recognises this will be
a great step forwards.

On the other hand, if you are saying that the Oak Threat model has to be
developed and agreed, before this patch can be added, then I am concerned
that will take too long. Doing justice to an Oak Treat model will require
resource.

Best `Regards
Ian


>
> Kind regards
> Angela
>
>
> On 07/09/17 16:39, "Ian Boston" <ie...@tfd.co.uk> wrote:
>
> >On 7 September 2017 at 14:41, Francesco Mari <ma...@gmail.com>
> >wrote:
> >
> >> On Thu, Sep 7, 2017 at 11:05 AM, Ian Boston <ie...@tfd.co.uk> wrote:
> >> > On 7 September 2017 at 07:22, Ian Boston <ie...@tfd.co.uk> wrote:
> >> >
> >> >> Hi,
> >> >>
> >> >> On 6 September 2017 at 22:43, Michael Dürig <md...@apache.org>
> >>wrote:
> >> >>
> >> >>>
> >> >>>
> >> >>> On 06.09.17 23:08, Michael Dürig wrote:
> >> >>>
> >> >>>>
> >> >>>> Hi,
> >> >>>>
> >> >>>> On 05.09.17 14:09, Ian Boston wrote:
> >> >>>>
> >> >>>>> Repeating the comment to on OAK-6575 here for further discussion.
> >>2
> >> new
> >> >>>>> Patches exploring both options.
> >> >>>>>
> >> >>>>
> >> >>>> I would actually prefer the original patch (
> >> >>>> https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:O
> >> >>>> AK-6575?expand=1) in most parts. However I have concerns regarding
> >>the
> >> >>>> generality of the new OakConversionService API as mentioned in my
> >> previous
> >> >>>> mail. I would be more comfortable if this could be restricted to
> >> something
> >> >>>> that resembles more like a "URIProvider", which given a blob
> >>returns
> >> an URI.
> >> >>>>
> >> >>>> On the implementation side, why do we need to introduce the
> >>adaptable
> >> >>>> machinery? Couldn't we re-use the Whiteboard and OSGiWhiteBoard
> >> mechanisms
> >> >>>> instead? I think these could be used to track URIProvider instances
> >> >>>> registered by the various blob stores.
> >> >>>>
> >> >>>>
> >> >>> See https://github.com/mduerig/jackrabbit-oak/commit/2709c097b01
> >> >>> a006784b7011135efcbbe3ce1ba88 for a *really* quickly hacked together
> >> and
> >> >>> entirely untested POC. But it should get the idea across though.
> >> >>
> >> >>
> >> >>
> >> >> Thank you.
> >> >> That makes sense.
> >> >> I think it only needs the  java/org/apache/jackrabbit/
> >> >> oak/blob/cloud/aws/s3/CloudFrontS3SignedUrlAdapterFactory.java and
> >>the
> >> >> API to be inside Oak, everything else can be in Sling.
> >> >> I'll update my patch and do a 2 options for Sling.
> >> >>
> >> >
> >> >
> >> >
> >> >
> >> > https://github.com/ieb/jackrabbit-oak/compare/trunk..
> >> .ieb:OAK-6575-3?expand=1
> >> >
> >> > and
> >> >
> >> >
> >>https://github.com/apache/sling/compare/trunk...ieb:OAK-6575-3?expand=1
> >> >
> >> > wdyt ?
> >>
> >> I like this a lot. It keeps Oak's side simple and cleanly integrates
> >> Oak's lower-level services in Sling.
> >>
> >
> >
> >Good news.
> >I think we should hold off committing the patch until Monday or Tuesday to
> >give those who may be offline this week a chance to comment. In particular
> >I have not seen a comment from Angela who I would expect to have a view as
> >this is acl/security related. That is assuming she is back online next
> >week.
> >
> >Best Regards
> >Ian
> >
> >
> >>
> >> > Obviously the second patch needs to be discussed with Sling dev, but
> >>is
> >> > should not be too contentious.
> >> >
> >> > Best Regards
> >> > Ian
> >> >
> >> >
> >> >
> >> >>
> >> >> I think that should address others concerns since it drops all signs
> >>of
> >> >> any generic object to object conversion from Oak (Francesco), and
> >> doesn't
> >> >> require wide scale fragile changes with implied requirements being
> >> placed
> >> >> on how intermediate classes are connected and behave (mine).
> >> >>
> >> >> Best Regards
> >> >> Ian
> >> >>
> >> >>
> >> >>>
> >> >>> Michael
> >> >>>
> >> >>
> >> >>
> >>
>
>

Re: OAK-6575 - A word of caution

Posted by Angela Schreiber <an...@adobe.com.INVALID>.
Hi Ian

The new proposal looks a lot better to me.

The only concern from a security perspective I could come up with is the
one we expressed already with the very first proposal (see initial word of
caution mail sent by Francesco): applications built on top of Oak can up
to now be sure that all access to the repository content is subject to the
same permission evaluation as configured with the repository setup. This
is no longer guaranteed when we offer the ability to plug application code
that may or may not by-pass the evaluation by allowing it to directly
access binaries.

While I know that this is actually the goal of the whole exercise, we have
to be aware that this also is a change in our Oak security model. As such
this may look like a security breach and I have been told by my colleagues
at Adobe that the 'single-way-to-access' is a relevant security question
with a lot of potential customers.

That doesn't mean that I am opposed to the patch in it's current form as I
see the benefits from an Oak pov, I just want to highlight that we are
going to make a fundamental change and we should treat and document it
with the necessary care... maybe we should take this opportunity to
finally create a threat model for Oak? Doing so at this stage would allow
us to visualise the proposed change to all parties involved.

wdyt?

Kind regards
Angela


On 07/09/17 16:39, "Ian Boston" <ie...@tfd.co.uk> wrote:

>On 7 September 2017 at 14:41, Francesco Mari <ma...@gmail.com>
>wrote:
>
>> On Thu, Sep 7, 2017 at 11:05 AM, Ian Boston <ie...@tfd.co.uk> wrote:
>> > On 7 September 2017 at 07:22, Ian Boston <ie...@tfd.co.uk> wrote:
>> >
>> >> Hi,
>> >>
>> >> On 6 September 2017 at 22:43, Michael Dürig <md...@apache.org>
>>wrote:
>> >>
>> >>>
>> >>>
>> >>> On 06.09.17 23:08, Michael Dürig wrote:
>> >>>
>> >>>>
>> >>>> Hi,
>> >>>>
>> >>>> On 05.09.17 14:09, Ian Boston wrote:
>> >>>>
>> >>>>> Repeating the comment to on OAK-6575 here for further discussion.
>>2
>> new
>> >>>>> Patches exploring both options.
>> >>>>>
>> >>>>
>> >>>> I would actually prefer the original patch (
>> >>>> https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:O
>> >>>> AK-6575?expand=1) in most parts. However I have concerns regarding
>>the
>> >>>> generality of the new OakConversionService API as mentioned in my
>> previous
>> >>>> mail. I would be more comfortable if this could be restricted to
>> something
>> >>>> that resembles more like a "URIProvider", which given a blob
>>returns
>> an URI.
>> >>>>
>> >>>> On the implementation side, why do we need to introduce the
>>adaptable
>> >>>> machinery? Couldn't we re-use the Whiteboard and OSGiWhiteBoard
>> mechanisms
>> >>>> instead? I think these could be used to track URIProvider instances
>> >>>> registered by the various blob stores.
>> >>>>
>> >>>>
>> >>> See https://github.com/mduerig/jackrabbit-oak/commit/2709c097b01
>> >>> a006784b7011135efcbbe3ce1ba88 for a *really* quickly hacked together
>> and
>> >>> entirely untested POC. But it should get the idea across though.
>> >>
>> >>
>> >>
>> >> Thank you.
>> >> That makes sense.
>> >> I think it only needs the  java/org/apache/jackrabbit/
>> >> oak/blob/cloud/aws/s3/CloudFrontS3SignedUrlAdapterFactory.java and
>>the
>> >> API to be inside Oak, everything else can be in Sling.
>> >> I'll update my patch and do a 2 options for Sling.
>> >>
>> >
>> >
>> >
>> >
>> > https://github.com/ieb/jackrabbit-oak/compare/trunk..
>> .ieb:OAK-6575-3?expand=1
>> >
>> > and
>> >
>> > 
>>https://github.com/apache/sling/compare/trunk...ieb:OAK-6575-3?expand=1
>> >
>> > wdyt ?
>>
>> I like this a lot. It keeps Oak's side simple and cleanly integrates
>> Oak's lower-level services in Sling.
>>
>
>
>Good news.
>I think we should hold off committing the patch until Monday or Tuesday to
>give those who may be offline this week a chance to comment. In particular
>I have not seen a comment from Angela who I would expect to have a view as
>this is acl/security related. That is assuming she is back online next
>week.
>
>Best Regards
>Ian
>
>
>>
>> > Obviously the second patch needs to be discussed with Sling dev, but
>>is
>> > should not be too contentious.
>> >
>> > Best Regards
>> > Ian
>> >
>> >
>> >
>> >>
>> >> I think that should address others concerns since it drops all signs
>>of
>> >> any generic object to object conversion from Oak (Francesco), and
>> doesn't
>> >> require wide scale fragile changes with implied requirements being
>> placed
>> >> on how intermediate classes are connected and behave (mine).
>> >>
>> >> Best Regards
>> >> Ian
>> >>
>> >>
>> >>>
>> >>> Michael
>> >>>
>> >>
>> >>
>>


Re: OAK-6575 - A word of caution

Posted by Ian Boston <ie...@tfd.co.uk>.
On 7 September 2017 at 14:41, Francesco Mari <ma...@gmail.com>
wrote:

> On Thu, Sep 7, 2017 at 11:05 AM, Ian Boston <ie...@tfd.co.uk> wrote:
> > On 7 September 2017 at 07:22, Ian Boston <ie...@tfd.co.uk> wrote:
> >
> >> Hi,
> >>
> >> On 6 September 2017 at 22:43, Michael Dürig <md...@apache.org> wrote:
> >>
> >>>
> >>>
> >>> On 06.09.17 23:08, Michael Dürig wrote:
> >>>
> >>>>
> >>>> Hi,
> >>>>
> >>>> On 05.09.17 14:09, Ian Boston wrote:
> >>>>
> >>>>> Repeating the comment to on OAK-6575 here for further discussion. 2
> new
> >>>>> Patches exploring both options.
> >>>>>
> >>>>
> >>>> I would actually prefer the original patch (
> >>>> https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:O
> >>>> AK-6575?expand=1) in most parts. However I have concerns regarding the
> >>>> generality of the new OakConversionService API as mentioned in my
> previous
> >>>> mail. I would be more comfortable if this could be restricted to
> something
> >>>> that resembles more like a "URIProvider", which given a blob returns
> an URI.
> >>>>
> >>>> On the implementation side, why do we need to introduce the adaptable
> >>>> machinery? Couldn't we re-use the Whiteboard and OSGiWhiteBoard
> mechanisms
> >>>> instead? I think these could be used to track URIProvider instances
> >>>> registered by the various blob stores.
> >>>>
> >>>>
> >>> See https://github.com/mduerig/jackrabbit-oak/commit/2709c097b01
> >>> a006784b7011135efcbbe3ce1ba88 for a *really* quickly hacked together
> and
> >>> entirely untested POC. But it should get the idea across though.
> >>
> >>
> >>
> >> Thank you.
> >> That makes sense.
> >> I think it only needs the  java/org/apache/jackrabbit/
> >> oak/blob/cloud/aws/s3/CloudFrontS3SignedUrlAdapterFactory.java and the
> >> API to be inside Oak, everything else can be in Sling.
> >> I'll update my patch and do a 2 options for Sling.
> >>
> >
> >
> >
> >
> > https://github.com/ieb/jackrabbit-oak/compare/trunk..
> .ieb:OAK-6575-3?expand=1
> >
> > and
> >
> > https://github.com/apache/sling/compare/trunk...ieb:OAK-6575-3?expand=1
> >
> > wdyt ?
>
> I like this a lot. It keeps Oak's side simple and cleanly integrates
> Oak's lower-level services in Sling.
>


Good news.
I think we should hold off committing the patch until Monday or Tuesday to
give those who may be offline this week a chance to comment. In particular
I have not seen a comment from Angela who I would expect to have a view as
this is acl/security related. That is assuming she is back online next week.

Best Regards
Ian


>
> > Obviously the second patch needs to be discussed with Sling dev, but is
> > should not be too contentious.
> >
> > Best Regards
> > Ian
> >
> >
> >
> >>
> >> I think that should address others concerns since it drops all signs of
> >> any generic object to object conversion from Oak (Francesco), and
> doesn't
> >> require wide scale fragile changes with implied requirements being
> placed
> >> on how intermediate classes are connected and behave (mine).
> >>
> >> Best Regards
> >> Ian
> >>
> >>
> >>>
> >>> Michael
> >>>
> >>
> >>
>

Re: OAK-6575 - A word of caution

Posted by Francesco Mari <ma...@gmail.com>.
On Thu, Sep 7, 2017 at 11:05 AM, Ian Boston <ie...@tfd.co.uk> wrote:
> On 7 September 2017 at 07:22, Ian Boston <ie...@tfd.co.uk> wrote:
>
>> Hi,
>>
>> On 6 September 2017 at 22:43, Michael Dürig <md...@apache.org> wrote:
>>
>>>
>>>
>>> On 06.09.17 23:08, Michael Dürig wrote:
>>>
>>>>
>>>> Hi,
>>>>
>>>> On 05.09.17 14:09, Ian Boston wrote:
>>>>
>>>>> Repeating the comment to on OAK-6575 here for further discussion. 2 new
>>>>> Patches exploring both options.
>>>>>
>>>>
>>>> I would actually prefer the original patch (
>>>> https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:O
>>>> AK-6575?expand=1) in most parts. However I have concerns regarding the
>>>> generality of the new OakConversionService API as mentioned in my previous
>>>> mail. I would be more comfortable if this could be restricted to something
>>>> that resembles more like a "URIProvider", which given a blob returns an URI.
>>>>
>>>> On the implementation side, why do we need to introduce the adaptable
>>>> machinery? Couldn't we re-use the Whiteboard and OSGiWhiteBoard mechanisms
>>>> instead? I think these could be used to track URIProvider instances
>>>> registered by the various blob stores.
>>>>
>>>>
>>> See https://github.com/mduerig/jackrabbit-oak/commit/2709c097b01
>>> a006784b7011135efcbbe3ce1ba88 for a *really* quickly hacked together and
>>> entirely untested POC. But it should get the idea across though.
>>
>>
>>
>> Thank you.
>> That makes sense.
>> I think it only needs the  java/org/apache/jackrabbit/
>> oak/blob/cloud/aws/s3/CloudFrontS3SignedUrlAdapterFactory.java and the
>> API to be inside Oak, everything else can be in Sling.
>> I'll update my patch and do a 2 options for Sling.
>>
>
>
>
>
> https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:OAK-6575-3?expand=1
>
> and
>
> https://github.com/apache/sling/compare/trunk...ieb:OAK-6575-3?expand=1
>
> wdyt ?

I like this a lot. It keeps Oak's side simple and cleanly integrates
Oak's lower-level services in Sling.

> Obviously the second patch needs to be discussed with Sling dev, but is
> should not be too contentious.
>
> Best Regards
> Ian
>
>
>
>>
>> I think that should address others concerns since it drops all signs of
>> any generic object to object conversion from Oak (Francesco), and doesn't
>> require wide scale fragile changes with implied requirements being placed
>> on how intermediate classes are connected and behave (mine).
>>
>> Best Regards
>> Ian
>>
>>
>>>
>>> Michael
>>>
>>
>>

Re: OAK-6575 - A word of caution

Posted by Ian Boston <ie...@tfd.co.uk>.
On 7 September 2017 at 07:22, Ian Boston <ie...@tfd.co.uk> wrote:

> Hi,
>
> On 6 September 2017 at 22:43, Michael Dürig <md...@apache.org> wrote:
>
>>
>>
>> On 06.09.17 23:08, Michael Dürig wrote:
>>
>>>
>>> Hi,
>>>
>>> On 05.09.17 14:09, Ian Boston wrote:
>>>
>>>> Repeating the comment to on OAK-6575 here for further discussion. 2 new
>>>> Patches exploring both options.
>>>>
>>>
>>> I would actually prefer the original patch (
>>> https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:O
>>> AK-6575?expand=1) in most parts. However I have concerns regarding the
>>> generality of the new OakConversionService API as mentioned in my previous
>>> mail. I would be more comfortable if this could be restricted to something
>>> that resembles more like a "URIProvider", which given a blob returns an URI.
>>>
>>> On the implementation side, why do we need to introduce the adaptable
>>> machinery? Couldn't we re-use the Whiteboard and OSGiWhiteBoard mechanisms
>>> instead? I think these could be used to track URIProvider instances
>>> registered by the various blob stores.
>>>
>>>
>> See https://github.com/mduerig/jackrabbit-oak/commit/2709c097b01
>> a006784b7011135efcbbe3ce1ba88 for a *really* quickly hacked together and
>> entirely untested POC. But it should get the idea across though.
>
>
>
> Thank you.
> That makes sense.
> I think it only needs the  java/org/apache/jackrabbit/
> oak/blob/cloud/aws/s3/CloudFrontS3SignedUrlAdapterFactory.java and the
> API to be inside Oak, everything else can be in Sling.
> I'll update my patch and do a 2 options for Sling.
>




https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:OAK-6575-3?expand=1

and

https://github.com/apache/sling/compare/trunk...ieb:OAK-6575-3?expand=1

wdyt ?
Obviously the second patch needs to be discussed with Sling dev, but is
should not be too contentious.

Best Regards
Ian



>
> I think that should address others concerns since it drops all signs of
> any generic object to object conversion from Oak (Francesco), and doesn't
> require wide scale fragile changes with implied requirements being placed
> on how intermediate classes are connected and behave (mine).
>
> Best Regards
> Ian
>
>
>>
>> Michael
>>
>
>

Re: OAK-6575 - A word of caution

Posted by Ian Boston <ie...@tfd.co.uk>.
Hi,

On 6 September 2017 at 22:43, Michael Dürig <md...@apache.org> wrote:

>
>
> On 06.09.17 23:08, Michael Dürig wrote:
>
>>
>> Hi,
>>
>> On 05.09.17 14:09, Ian Boston wrote:
>>
>>> Repeating the comment to on OAK-6575 here for further discussion. 2 new
>>> Patches exploring both options.
>>>
>>
>> I would actually prefer the original patch (https://github.com/ieb/jackra
>> bbit-oak/compare/trunk...ieb:OAK-6575?expand=1) in most parts. However I
>> have concerns regarding the generality of the new OakConversionService API
>> as mentioned in my previous mail. I would be more comfortable if this could
>> be restricted to something that resembles more like a "URIProvider", which
>> given a blob returns an URI.
>>
>> On the implementation side, why do we need to introduce the adaptable
>> machinery? Couldn't we re-use the Whiteboard and OSGiWhiteBoard mechanisms
>> instead? I think these could be used to track URIProvider instances
>> registered by the various blob stores.
>>
>>
> See https://github.com/mduerig/jackrabbit-oak/commit/2709c097b01
> a006784b7011135efcbbe3ce1ba88 for a *really* quickly hacked together and
> entirely untested POC. But it should get the idea across though.



Thank you.
That makes sense.
I think it only needs the
 java/org/apache/jackrabbit/oak/blob/cloud/aws/s3/CloudFrontS3SignedUrlAdapterFactory.java
and the API to be inside Oak, everything else can be in Sling.
I'll update my patch and do a 2 options for Sling.

I think that should address others concerns since it drops all signs of any
generic object to object conversion from Oak (Francesco), and doesn't
require wide scale fragile changes with implied requirements being placed
on how intermediate classes are connected and behave (mine).

Best Regards
Ian


>
> Michael
>

Re: OAK-6575 - A word of caution

Posted by Michael Dürig <md...@apache.org>.

On 06.09.17 23:08, Michael Dürig wrote:
> 
> Hi,
> 
> On 05.09.17 14:09, Ian Boston wrote:
>> Repeating the comment to on OAK-6575 here for further discussion. 2 new
>> Patches exploring both options.
> 
> I would actually prefer the original patch 
> (https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:OAK-6575?expand=1) 
> in most parts. However I have concerns regarding the generality of the 
> new OakConversionService API as mentioned in my previous mail. I would 
> be more comfortable if this could be restricted to something that 
> resembles more like a "URIProvider", which given a blob returns an URI.
> 
> On the implementation side, why do we need to introduce the adaptable 
> machinery? Couldn't we re-use the Whiteboard and OSGiWhiteBoard 
> mechanisms instead? I think these could be used to track URIProvider 
> instances registered by the various blob stores.
> 

See 
https://github.com/mduerig/jackrabbit-oak/commit/2709c097b01a006784b7011135efcbbe3ce1ba88 
for a *really* quickly hacked together and entirely untested POC. But it 
should get the idea across though.

Michael

Re: OAK-6575 - A word of caution

Posted by Michael Dürig <md...@apache.org>.
Hi,

On 05.09.17 14:09, Ian Boston wrote:
> Repeating the comment to on OAK-6575 here for further discussion. 2 new
> Patches exploring both options.

I would actually prefer the original patch 
(https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:OAK-6575?expand=1) 
in most parts. However I have concerns regarding the generality of the 
new OakConversionService API as mentioned in my previous mail. I would 
be more comfortable if this could be restricted to something that 
resembles more like a "URIProvider", which given a blob returns an URI.

On the implementation side, why do we need to introduce the adaptable 
machinery? Couldn't we re-use the Whiteboard and OSGiWhiteBoard 
mechanisms instead? I think these could be used to track URIProvider 
instances registered by the various blob stores.

Michael

Re: OAK-6575 - A word of caution

Posted by Francesco Mari <ma...@gmail.com>.
On Wed, Sep 6, 2017 at 1:59 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> Hi,
> Thanks for looking at them.
>
> On 6 September 2017 at 12:32, Francesco Mari <ma...@gmail.com>
> wrote:
>
>> I personally prefer the second approach.
>
>
> Is that OAK-6575-1 or OAK-6575-2 ?
> I assume OAK-6575-1 since OAK-6575 was my first approach ?

It is OAK-6575-2.

> If you mean OAK-6575-2, then I think someone with more knowledge of Oak
> will need to do the work as I am not at all confident I have covered the
> potential class/method navigation between a OakValue and a DataStore.... or
> if that navigation is even possible where the exposed datastore might
> actually be a composite datastore with the exposed part having no class
> based connection with the underlying S3 DataStore. (eg S3 DS cache). I am
> definitely not proud of OAK-6575-2, imho it's not elegant or efficient and
> would put up more barriers to future agility rather than remove them.

The rest of the team will surely give an opinion on the original
approach and the OAK-6575-1 and OAK-6575-2 variants. We will go with
whatever the majority prefers.

>> The only thing I'm not sure
>> about is if we want to define OakConversionService with such a
>> wildcard method. Assuming that OakConversionService will be called
>> from code running on top of the JCR API, we could provide instead more
>> specific conversion methods. For example,
>>
>> URI toURI(javax.jcr.Binary binary);
>>
>> What do you think about it? Is it too restrictive? Do we need a
>> wildcard method like currently defined in OakConversionsService?
>>
>
>
> Originally OakConversionsService would not have needed a new version of the
> package for each new conversion Oak supported, greatly simplifying
> dependencies downstream, especially where the source and target classes
> already exist.
>
> If a concrete method is used, the package will need to be versioned
> everytime. I suspect OSGi rules will require a minor version number
> increment each time, which is going to make a downstream developers life
> painful.
>
> In addition if an implementation bundle in Oak decides it wants to
> optionally support a conversion, it wont need to version the Oak API to
> achieve that. With concrete methods, ever change, wherever they are and
> however experimental will require a new version of the Oak API.
>
> This was the reason for going for a wildcard method. It allows extension
> without any downstream disruption, missing dependencies or out of band
> dependencies.

I understand the concerns. I wonder how many conversions we need to
implement, and how often we will introduce new conversions.

> I think the boils down to how much disruption Oak wants to inflict
> downstream to get new capabilities added, or inversely, how open Oak is to
> requests for API changes from downstream ?
>
>
>
>>
>> Moreover, I would leave PrivateURI out of the picture for the moment
>> since it's not clear from the patch how this is supposed to be used.
>> In fact, a comment in S3Backend explicitly states that is not
>> supported at this time.
>>
>
> PrivateURI was discussed on the OAK-6575 thread. It was added to the patch
> to illustrate how each patch would cope with extension of a new type. I
> propose to drop it from the final patch, however, in the second patch the
> disruption is quite large so it might be worth leaving it in there so that
> it can be implemented without more Oak API version changes.
>
> Best Regards
> Ian
>
>
>>
>> Finally, I suspect that in the second patch there was too much of an
>> aggressive rename refactoring. "types" was renamed to "customtypes" in
>> a lot of unrelated places. I would definitely double-check that.
>
>
>> On Tue, Sep 5, 2017 at 2:09 PM, Ian Boston <ie...@tfd.co.uk> wrote:
>> > Hi,
>> >
>> > Repeating the comment to on OAK-6575 here for further discussion. 2 new
>> > Patches exploring both options.
>> >
>> > https://github.com/ieb/jackrabbit-oak/compare/trunk..
>> .ieb:OAK-6575-1?expand=1
>> >
>> > This drops the OSGi AdapterManager/AdapterFactory in favour of a non OSGi
>> > static pattern. Implementations of the AdapterFactory self register
>> rather
>> > than rely on OSGi doing the wiring. This is probably an IoC anti pattern,
>> > but does avoid exposing the AdapterFactory/AdapterManager outside Oak.
>> >
>> > https://github.com/ieb/jackrabbit-oak/compare/trunk..
>> .ieb:OAK-6575-2?expand=1
>> >
>> > This drops the AdapterManager concept completely and attempts to get from
>> > Value to URI using mix in interfaces and instanceof. I cant be certain it
>> > manages to do this as there is a disconnect between Blob, Blobstore and
>> > DataStore implementations with no guarantee that a BlobStore as seen by
>> the
>> > Blob implementation actually implements DataStore, or the Blob that is
>> > exposed in the JCR Value (implemented by OakValue) actually connects to
>> the
>> > correct DataStore of it it connects to a FileDatastore cache on local
>> disk.
>> > I could only wire this as far as I did with API changes. I may have
>> broken
>> > some of the new multi node store and multi datastore code used for 0DT in
>> > the process. An Oak committer with global knowledge will probably be able
>> > to do better.
>> >
>> >
>> >
>> > On 5 September 2017 at 08:19, Ian Boston <ie...@tfd.co.uk> wrote:
>> >
>> >> Hi,
>> >>
>> >> On 5 September 2017 at 07:55, Francesco Mari <ma...@gmail.com>
>> >> wrote:
>> >>
>> >>> On Mon, Sep 4, 2017 at 6:18 PM, Ian Boston <ie...@tfd.co.uk> wrote:
>> >>> > Do you mean:
>> >>> >  keep the OakConversionService but put all the logic to convert from
>> a
>> >>> > Value to a URI inside that implementation using new Oak SPI/APIs if
>> >>> > necessary and drop the AdapterManager completely ?
>> >>>
>> >>> Yes. I think there is no need to provide a generic adapter-like
>> >>> implementation to solve this use case.
>> >>>
>> >>> > This would mean something the datastore implementation implements
>> which
>> >>> > oak-core can navigate to would have to implement a mix in interface
>> >>> with a
>> >>> > getURI() method. I am not certain what or how without trying to do
>> it.
>> >>> >
>> >>> > Would that address your concern here ?
>> >>>
>> >>> I think it's worth trying. Thanks for bringing the conversation
>> forward.
>> >>>
>> >>
>> >>
>> >> I will create 2 new branches.
>> >> 1 with no adapter manager relying on mixin interfaces and one with a non
>> >> OSGi adapter manager plugin pattern.
>> >>
>> >> Thanks for the input.
>> >> Best Regards
>> >> Ian
>> >>
>> >>
>>

Re: OAK-6575 - A word of caution

Posted by Michael Dürig <md...@apache.org>.

On 06.09.17 13:59, Ian Boston wrote:
> package for each new conversion Oak supported, greatly simplifying
> dependencies downstream, especially where the source and target classes
> already exist.
> 
> If a concrete method is used, the package will need to be versioned
> everytime. I suspect OSGi rules will require a minor version number
> increment each time, which is going to make a downstream developers life
> painful.
> 
> In addition if an implementation bundle in Oak decides it wants to
> optionally support a conversion, it wont need to version the Oak API to
> achieve that. With concrete methods, ever change, wherever they are and
> however experimental will require a new version of the Oak API.
> 
> This was the reason for going for a wildcard method. It allows extension
> without any downstream disruption, missing dependencies or out of band
> dependencies.
> 
> I think the boils down to how much disruption Oak wants to inflict
> downstream to get new capabilities added, or inversely, how open Oak is to
> requests for API changes from downstream ?

I agree with everything said here, but there is another way to look at 
it too: this is side stepping backward compatibility concerns by moving 
from a statically typed API to a dynamically typed one effectively 
evading semantic versioning. This eases development and deployment but 
might push problems further out to production.

For this reason (and as I said before) introducing such a general API is 
a significant change and would deserve a discussion on its own revolving 
around the pure API change, mostly decoupled from the "secure URI" 
issue, maybe mentioning it as a use case.




Re: OAK-6575 - A word of caution

Posted by Ian Boston <ie...@tfd.co.uk>.
Hi,
Thanks for looking at them.

On 6 September 2017 at 12:32, Francesco Mari <ma...@gmail.com>
wrote:

> I personally prefer the second approach.


Is that OAK-6575-1 or OAK-6575-2 ?
I assume OAK-6575-1 since OAK-6575 was my first approach ?

If you mean OAK-6575-2, then I think someone with more knowledge of Oak
will need to do the work as I am not at all confident I have covered the
potential class/method navigation between a OakValue and a DataStore.... or
if that navigation is even possible where the exposed datastore might
actually be a composite datastore with the exposed part having no class
based connection with the underlying S3 DataStore. (eg S3 DS cache). I am
definitely not proud of OAK-6575-2, imho it's not elegant or efficient and
would put up more barriers to future agility rather than remove them.


> The only thing I'm not sure
> about is if we want to define OakConversionService with such a
> wildcard method. Assuming that OakConversionService will be called
> from code running on top of the JCR API, we could provide instead more
> specific conversion methods. For example,
>
> URI toURI(javax.jcr.Binary binary);
>
> What do you think about it? Is it too restrictive? Do we need a
> wildcard method like currently defined in OakConversionsService?
>


Originally OakConversionsService would not have needed a new version of the
package for each new conversion Oak supported, greatly simplifying
dependencies downstream, especially where the source and target classes
already exist.

If a concrete method is used, the package will need to be versioned
everytime. I suspect OSGi rules will require a minor version number
increment each time, which is going to make a downstream developers life
painful.

In addition if an implementation bundle in Oak decides it wants to
optionally support a conversion, it wont need to version the Oak API to
achieve that. With concrete methods, ever change, wherever they are and
however experimental will require a new version of the Oak API.

This was the reason for going for a wildcard method. It allows extension
without any downstream disruption, missing dependencies or out of band
dependencies.

I think the boils down to how much disruption Oak wants to inflict
downstream to get new capabilities added, or inversely, how open Oak is to
requests for API changes from downstream ?



>
> Moreover, I would leave PrivateURI out of the picture for the moment
> since it's not clear from the patch how this is supposed to be used.
> In fact, a comment in S3Backend explicitly states that is not
> supported at this time.
>

PrivateURI was discussed on the OAK-6575 thread. It was added to the patch
to illustrate how each patch would cope with extension of a new type. I
propose to drop it from the final patch, however, in the second patch the
disruption is quite large so it might be worth leaving it in there so that
it can be implemented without more Oak API version changes.

Best Regards
Ian


>
> Finally, I suspect that in the second patch there was too much of an
> aggressive rename refactoring. "types" was renamed to "customtypes" in
> a lot of unrelated places. I would definitely double-check that.


> On Tue, Sep 5, 2017 at 2:09 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> > Hi,
> >
> > Repeating the comment to on OAK-6575 here for further discussion. 2 new
> > Patches exploring both options.
> >
> > https://github.com/ieb/jackrabbit-oak/compare/trunk..
> .ieb:OAK-6575-1?expand=1
> >
> > This drops the OSGi AdapterManager/AdapterFactory in favour of a non OSGi
> > static pattern. Implementations of the AdapterFactory self register
> rather
> > than rely on OSGi doing the wiring. This is probably an IoC anti pattern,
> > but does avoid exposing the AdapterFactory/AdapterManager outside Oak.
> >
> > https://github.com/ieb/jackrabbit-oak/compare/trunk..
> .ieb:OAK-6575-2?expand=1
> >
> > This drops the AdapterManager concept completely and attempts to get from
> > Value to URI using mix in interfaces and instanceof. I cant be certain it
> > manages to do this as there is a disconnect between Blob, Blobstore and
> > DataStore implementations with no guarantee that a BlobStore as seen by
> the
> > Blob implementation actually implements DataStore, or the Blob that is
> > exposed in the JCR Value (implemented by OakValue) actually connects to
> the
> > correct DataStore of it it connects to a FileDatastore cache on local
> disk.
> > I could only wire this as far as I did with API changes. I may have
> broken
> > some of the new multi node store and multi datastore code used for 0DT in
> > the process. An Oak committer with global knowledge will probably be able
> > to do better.
> >
> >
> >
> > On 5 September 2017 at 08:19, Ian Boston <ie...@tfd.co.uk> wrote:
> >
> >> Hi,
> >>
> >> On 5 September 2017 at 07:55, Francesco Mari <ma...@gmail.com>
> >> wrote:
> >>
> >>> On Mon, Sep 4, 2017 at 6:18 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> >>> > Do you mean:
> >>> >  keep the OakConversionService but put all the logic to convert from
> a
> >>> > Value to a URI inside that implementation using new Oak SPI/APIs if
> >>> > necessary and drop the AdapterManager completely ?
> >>>
> >>> Yes. I think there is no need to provide a generic adapter-like
> >>> implementation to solve this use case.
> >>>
> >>> > This would mean something the datastore implementation implements
> which
> >>> > oak-core can navigate to would have to implement a mix in interface
> >>> with a
> >>> > getURI() method. I am not certain what or how without trying to do
> it.
> >>> >
> >>> > Would that address your concern here ?
> >>>
> >>> I think it's worth trying. Thanks for bringing the conversation
> forward.
> >>>
> >>
> >>
> >> I will create 2 new branches.
> >> 1 with no adapter manager relying on mixin interfaces and one with a non
> >> OSGi adapter manager plugin pattern.
> >>
> >> Thanks for the input.
> >> Best Regards
> >> Ian
> >>
> >>
>

Re: OAK-6575 - A word of caution

Posted by Francesco Mari <ma...@gmail.com>.
I personally prefer the second approach. The only thing I'm not sure
about is if we want to define OakConversionService with such a
wildcard method. Assuming that OakConversionService will be called
from code running on top of the JCR API, we could provide instead more
specific conversion methods. For example,

URI toURI(javax.jcr.Binary binary);

What do you think about it? Is it too restrictive? Do we need a
wildcard method like currently defined in OakConversionsService?

Moreover, I would leave PrivateURI out of the picture for the moment
since it's not clear from the patch how this is supposed to be used.
In fact, a comment in S3Backend explicitly states that is not
supported at this time.

Finally, I suspect that in the second patch there was too much of an
aggressive rename refactoring. "types" was renamed to "customtypes" in
a lot of unrelated places. I would definitely double-check that.

On Tue, Sep 5, 2017 at 2:09 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> Hi,
>
> Repeating the comment to on OAK-6575 here for further discussion. 2 new
> Patches exploring both options.
>
> https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:OAK-6575-1?expand=1
>
> This drops the OSGi AdapterManager/AdapterFactory in favour of a non OSGi
> static pattern. Implementations of the AdapterFactory self register rather
> than rely on OSGi doing the wiring. This is probably an IoC anti pattern,
> but does avoid exposing the AdapterFactory/AdapterManager outside Oak.
>
> https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:OAK-6575-2?expand=1
>
> This drops the AdapterManager concept completely and attempts to get from
> Value to URI using mix in interfaces and instanceof. I cant be certain it
> manages to do this as there is a disconnect between Blob, Blobstore and
> DataStore implementations with no guarantee that a BlobStore as seen by the
> Blob implementation actually implements DataStore, or the Blob that is
> exposed in the JCR Value (implemented by OakValue) actually connects to the
> correct DataStore of it it connects to a FileDatastore cache on local disk.
> I could only wire this as far as I did with API changes. I may have broken
> some of the new multi node store and multi datastore code used for 0DT in
> the process. An Oak committer with global knowledge will probably be able
> to do better.
>
>
>
> On 5 September 2017 at 08:19, Ian Boston <ie...@tfd.co.uk> wrote:
>
>> Hi,
>>
>> On 5 September 2017 at 07:55, Francesco Mari <ma...@gmail.com>
>> wrote:
>>
>>> On Mon, Sep 4, 2017 at 6:18 PM, Ian Boston <ie...@tfd.co.uk> wrote:
>>> > Do you mean:
>>> >  keep the OakConversionService but put all the logic to convert from a
>>> > Value to a URI inside that implementation using new Oak SPI/APIs if
>>> > necessary and drop the AdapterManager completely ?
>>>
>>> Yes. I think there is no need to provide a generic adapter-like
>>> implementation to solve this use case.
>>>
>>> > This would mean something the datastore implementation implements which
>>> > oak-core can navigate to would have to implement a mix in interface
>>> with a
>>> > getURI() method. I am not certain what or how without trying to do it.
>>> >
>>> > Would that address your concern here ?
>>>
>>> I think it's worth trying. Thanks for bringing the conversation forward.
>>>
>>
>>
>> I will create 2 new branches.
>> 1 with no adapter manager relying on mixin interfaces and one with a non
>> OSGi adapter manager plugin pattern.
>>
>> Thanks for the input.
>> Best Regards
>> Ian
>>
>>

Re: OAK-6575 - A word of caution

Posted by Ian Boston <ie...@tfd.co.uk>.
Hi,

Repeating the comment to on OAK-6575 here for further discussion. 2 new
Patches exploring both options.

https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:OAK-6575-1?expand=1

This drops the OSGi AdapterManager/AdapterFactory in favour of a non OSGi
static pattern. Implementations of the AdapterFactory self register rather
than rely on OSGi doing the wiring. This is probably an IoC anti pattern,
but does avoid exposing the AdapterFactory/AdapterManager outside Oak.

https://github.com/ieb/jackrabbit-oak/compare/trunk...ieb:OAK-6575-2?expand=1

This drops the AdapterManager concept completely and attempts to get from
Value to URI using mix in interfaces and instanceof. I cant be certain it
manages to do this as there is a disconnect between Blob, Blobstore and
DataStore implementations with no guarantee that a BlobStore as seen by the
Blob implementation actually implements DataStore, or the Blob that is
exposed in the JCR Value (implemented by OakValue) actually connects to the
correct DataStore of it it connects to a FileDatastore cache on local disk.
I could only wire this as far as I did with API changes. I may have broken
some of the new multi node store and multi datastore code used for 0DT in
the process. An Oak committer with global knowledge will probably be able
to do better.



On 5 September 2017 at 08:19, Ian Boston <ie...@tfd.co.uk> wrote:

> Hi,
>
> On 5 September 2017 at 07:55, Francesco Mari <ma...@gmail.com>
> wrote:
>
>> On Mon, Sep 4, 2017 at 6:18 PM, Ian Boston <ie...@tfd.co.uk> wrote:
>> > Do you mean:
>> >  keep the OakConversionService but put all the logic to convert from a
>> > Value to a URI inside that implementation using new Oak SPI/APIs if
>> > necessary and drop the AdapterManager completely ?
>>
>> Yes. I think there is no need to provide a generic adapter-like
>> implementation to solve this use case.
>>
>> > This would mean something the datastore implementation implements which
>> > oak-core can navigate to would have to implement a mix in interface
>> with a
>> > getURI() method. I am not certain what or how without trying to do it.
>> >
>> > Would that address your concern here ?
>>
>> I think it's worth trying. Thanks for bringing the conversation forward.
>>
>
>
> I will create 2 new branches.
> 1 with no adapter manager relying on mixin interfaces and one with a non
> OSGi adapter manager plugin pattern.
>
> Thanks for the input.
> Best Regards
> Ian
>
>

Re: OAK-6575 - A word of caution

Posted by Ian Boston <ie...@tfd.co.uk>.
Hi,

On 5 September 2017 at 07:55, Francesco Mari <ma...@gmail.com>
wrote:

> On Mon, Sep 4, 2017 at 6:18 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> > Do you mean:
> >  keep the OakConversionService but put all the logic to convert from a
> > Value to a URI inside that implementation using new Oak SPI/APIs if
> > necessary and drop the AdapterManager completely ?
>
> Yes. I think there is no need to provide a generic adapter-like
> implementation to solve this use case.
>
> > This would mean something the datastore implementation implements which
> > oak-core can navigate to would have to implement a mix in interface with
> a
> > getURI() method. I am not certain what or how without trying to do it.
> >
> > Would that address your concern here ?
>
> I think it's worth trying. Thanks for bringing the conversation forward.
>


I will create 2 new branches.
1 with no adapter manager relying on mixin interfaces and one with a non
OSGi adapter manager plugin pattern.

Thanks for the input.
Best Regards
Ian

Re: OAK-6575 - A word of caution

Posted by Francesco Mari <ma...@gmail.com>.
On Mon, Sep 4, 2017 at 6:18 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> Do you mean:
>  keep the OakConversionService but put all the logic to convert from a
> Value to a URI inside that implementation using new Oak SPI/APIs if
> necessary and drop the AdapterManager completely ?

Yes. I think there is no need to provide a generic adapter-like
implementation to solve this use case.

> This would mean something the datastore implementation implements which
> oak-core can navigate to would have to implement a mix in interface with a
> getURI() method. I am not certain what or how without trying to do it.
>
> Would that address your concern here ?

I think it's worth trying. Thanks for bringing the conversation forward.

Re: OAK-6575 - A word of caution

Posted by Ian Boston <ie...@tfd.co.uk>.
Hi,

On 4 September 2017 at 16:43, Francesco Mari <ma...@gmail.com>
wrote:

> On Mon, Sep 4, 2017 at 4:57 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> > Hi,
> > IIUC There are 2 patterns:
> >
> > 1 Emitting a short lived signed URL as per the AWS CloudFront recommended
> > method of serving private content.
>
> I have nothing to object to choice of AWS CloudFront.  The past
> technological choices were very different. At the time, we were taking
> about converting a binary to an S3 bucket ID for users to manipulate.
> This had, as you could understand, a bigger impact on Oak from the
> point of view of security and data ownership.
>

agreed.


>
> I am still of the opinion, though, that if S3 and CloudFront are part
> of the technological stack chosen by a user, it's the user's
> responsibility to directly interact with it. Oak can transparently
> manage binaries in S3 for you, as long as it remains transparent. If
> as a user you need to manipulate buckets, or create CloudFront
> distributions, or use the AWS infrastructure to replicate S3 buckets
> in a different zone, then this is part of your business logic. Oak can
> still be used to store metadata and S3 bucket IDs, but the management
> of S3 and CloudFront it is up to you. This is just my view on the
> problem. If our users desperately want Oak to transparently store data
> in S3 but also to opt out when needed, we are going to provide this
> functionality. Still, my point of view is that it's wrong.
>

agreed.
Users can, to some extent already use CloudFront, but they have to make all
the content public to all users to do so. If anyone uses this patch, they
do so because they desperately want to have assets securely served as
private content by cloudfront. To do so, they will have to go some
deployment effort, generating the correct private keys and configuring the
various AWS components etc etc


>
> > 2 An Oak internal AdapterFactory/AdapterManager pattern to avoid Oak API.
> > changes.
>
> Sling adapters or a partial implementation of it is too dangerous in
> its flexibility. Today we are using this system to solve only one
> problem. Tomorrow we are going to end up like in Sling, where
> everything might be adapted to everything else and module boundaries
> will be more difficult to enforce. Sling adapters require too much
> discipline to get right and are too easy to misuse.
>
> Moreover, as soon as you register an AdapterManager in OSGi people are
> going to get curious. There is no real control in OSGi unless you use
> subsystems, and we are not using them. The most widely used commercial
> products based on Oak are not using OSGi subsystems either, and this
> is just going to exacerbate the problem.
>


The Oak AdapterManager was written to work in both OSGi and non OSGi
environments as guided by Chetan.

A previous version of the patch used a static factory pattern
(AdapterManager.getInstance()) and no OSGi at all. There are probably
several other patterns, all non OSGi that would stop anyone outside Oak
using it.

Would a non OSGi pattern address your concerns ?



>
> My take on this is that it's alright to have something like a
> OakConversionService, but please don't use a solution based on or
> inspired to Sling adapters to implement it. It's easier, safer and
> more controllable to implement this solution ad-hoc.
>

Do you mean:
 keep the OakConversionService but put all the logic to convert from a
Value to a URI inside that implementation using new Oak SPI/APIs if
necessary and drop the AdapterManager completely ?

This would mean something the datastore implementation implements which
oak-core can navigate to would have to implement a mix in interface with a
getURI() method. I am not certain what or how without trying to do it.

Would that address your concern here ?

Best Regards
ian



>
> > Would you be willing state your concerns for each one separately ?
> >
> > Best Regards
> > Ian
> >
> > On 4 September 2017 at 15:43, Francesco Mari <ma...@gmail.com>
> > wrote:
> >
> >> I'm in no position to veto the POC and I'm not willing to. I am well
> >> aware of the importance of this feature. I expressed my concerns and
> >> so did others. As the subject of this thread clearly stated, I only
> >> wanted to point out that I had the feeling that we had a "reboot" of
> >> the conversation for no good reason, with the unpleasant side effect
> >> of proposing once again a pattern that received a lot of criticism in
> >> the past.
> >>
> >> On Mon, Sep 4, 2017 at 4:18 PM, Bertrand Delacretaz
> >> <bd...@apache.org> wrote:
> >> > On Mon, Sep 4, 2017 at 3:44 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> >> >> ...I feel
> >> >> that Oak is weaker without the ability to offload bulk data
> streaming to
> >> >> infrastructure designed for that purpose....
> >> >
> >> > FWIW as an Oak user I share that feeling, IMO the use cases described
> >> > at https://wiki.apache.org/jackrabbit/JCR%20Binary%20Usecase are
> >> > becoming more and more important.
> >> >
> >> > Not being a committer I don't really care about the internals, but
> >> > please do not "throw the baby out with the bath water" if the
> >> > internals need to change.
> >> >
> >> > -Bertrand
> >>
>

Re: OAK-6575 - A word of caution

Posted by Francesco Mari <ma...@gmail.com>.
On Mon, Sep 4, 2017 at 4:57 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> Hi,
> IIUC There are 2 patterns:
>
> 1 Emitting a short lived signed URL as per the AWS CloudFront recommended
> method of serving private content.

I have nothing to object to choice of AWS CloudFront.  The past
technological choices were very different. At the time, we were taking
about converting a binary to an S3 bucket ID for users to manipulate.
This had, as you could understand, a bigger impact on Oak from the
point of view of security and data ownership.

I am still of the opinion, though, that if S3 and CloudFront are part
of the technological stack chosen by a user, it's the user's
responsibility to directly interact with it. Oak can transparently
manage binaries in S3 for you, as long as it remains transparent. If
as a user you need to manipulate buckets, or create CloudFront
distributions, or use the AWS infrastructure to replicate S3 buckets
in a different zone, then this is part of your business logic. Oak can
still be used to store metadata and S3 bucket IDs, but the management
of S3 and CloudFront it is up to you. This is just my view on the
problem. If our users desperately want Oak to transparently store data
in S3 but also to opt out when needed, we are going to provide this
functionality. Still, my point of view is that it's wrong.

> 2 An Oak internal AdapterFactory/AdapterManager pattern to avoid Oak API.
> changes.

Sling adapters or a partial implementation of it is too dangerous in
its flexibility. Today we are using this system to solve only one
problem. Tomorrow we are going to end up like in Sling, where
everything might be adapted to everything else and module boundaries
will be more difficult to enforce. Sling adapters require too much
discipline to get right and are too easy to misuse.

Moreover, as soon as you register an AdapterManager in OSGi people are
going to get curious. There is no real control in OSGi unless you use
subsystems, and we are not using them. The most widely used commercial
products based on Oak are not using OSGi subsystems either, and this
is just going to exacerbate the problem.

My take on this is that it's alright to have something like a
OakConversionService, but please don't use a solution based on or
inspired to Sling adapters to implement it. It's easier, safer and
more controllable to implement this solution ad-hoc.

> Would you be willing state your concerns for each one separately ?
>
> Best Regards
> Ian
>
> On 4 September 2017 at 15:43, Francesco Mari <ma...@gmail.com>
> wrote:
>
>> I'm in no position to veto the POC and I'm not willing to. I am well
>> aware of the importance of this feature. I expressed my concerns and
>> so did others. As the subject of this thread clearly stated, I only
>> wanted to point out that I had the feeling that we had a "reboot" of
>> the conversation for no good reason, with the unpleasant side effect
>> of proposing once again a pattern that received a lot of criticism in
>> the past.
>>
>> On Mon, Sep 4, 2017 at 4:18 PM, Bertrand Delacretaz
>> <bd...@apache.org> wrote:
>> > On Mon, Sep 4, 2017 at 3:44 PM, Ian Boston <ie...@tfd.co.uk> wrote:
>> >> ...I feel
>> >> that Oak is weaker without the ability to offload bulk data streaming to
>> >> infrastructure designed for that purpose....
>> >
>> > FWIW as an Oak user I share that feeling, IMO the use cases described
>> > at https://wiki.apache.org/jackrabbit/JCR%20Binary%20Usecase are
>> > becoming more and more important.
>> >
>> > Not being a committer I don't really care about the internals, but
>> > please do not "throw the baby out with the bath water" if the
>> > internals need to change.
>> >
>> > -Bertrand
>>

Re: OAK-6575 - A word of caution

Posted by Michael Dürig <md...@apache.org>.

On 04.09.17 16:57, Ian Boston wrote:
> Hi,
> IIUC There are 2 patterns:
> 
> 1 Emitting a short lived signed URL as per the AWS CloudFront recommended
> method of serving private content.

I think this is an area where your patch made a lot of progress. From 
your description my initial concerns in this area have been mostly 
addressed (e.g. no more leakage of implementation details). A missing 
bit would be clarification how this feature interacts with other 
features like active blob deletion / garbage collection. However I think 
it is fine to address those via documentation once we have a full 
understanding.

> 
> 2 An Oak internal AdapterFactory/AdapterManager pattern to avoid Oak API
> changes.

This is the area where some of us were taken by surprise. I'm fine with 
the current solution ("the conversion service") even though I would have 
preferred "the OakValueFactory" approach, which IMO is more consistent 
with the rest of the Oak API. My main concern with this kind of APIs is 
that it in a way circumvents statical type safety and along with it 
statical code analysis (e.g. re. backwards compatibility). OTOH Sling is 
using similar patterns successfully a lot and moving closer to our main 
API consumer is not a bad thing. Also IIUC the conversion service might 
in future line up nicely (and maybe even be replaced by) similar OSGi 
functionality!?

Michael

> 
> Would you be willing state your concerns for each one separately ?
> 
> Best Regards
> Ian
> 
> On 4 September 2017 at 15:43, Francesco Mari <ma...@gmail.com>
> wrote:
> 
>> I'm in no position to veto the POC and I'm not willing to. I am well
>> aware of the importance of this feature. I expressed my concerns and
>> so did others. As the subject of this thread clearly stated, I only
>> wanted to point out that I had the feeling that we had a "reboot" of
>> the conversation for no good reason, with the unpleasant side effect
>> of proposing once again a pattern that received a lot of criticism in
>> the past.
>>
>> On Mon, Sep 4, 2017 at 4:18 PM, Bertrand Delacretaz
>> <bd...@apache.org> wrote:
>>> On Mon, Sep 4, 2017 at 3:44 PM, Ian Boston <ie...@tfd.co.uk> wrote:
>>>> ...I feel
>>>> that Oak is weaker without the ability to offload bulk data streaming to
>>>> infrastructure designed for that purpose....
>>>
>>> FWIW as an Oak user I share that feeling, IMO the use cases described
>>> at https://wiki.apache.org/jackrabbit/JCR%20Binary%20Usecase are
>>> becoming more and more important.
>>>
>>> Not being a committer I don't really care about the internals, but
>>> please do not "throw the baby out with the bath water" if the
>>> internals need to change.
>>>
>>> -Bertrand
>>
> 

Re: OAK-6575 - A word of caution

Posted by Ian Boston <ie...@tfd.co.uk>.
Hi,
IIUC There are 2 patterns:

1 Emitting a short lived signed URL as per the AWS CloudFront recommended
method of serving private content.


2 An Oak internal AdapterFactory/AdapterManager pattern to avoid Oak API
changes.

Would you be willing state your concerns for each one separately ?

Best Regards
Ian

On 4 September 2017 at 15:43, Francesco Mari <ma...@gmail.com>
wrote:

> I'm in no position to veto the POC and I'm not willing to. I am well
> aware of the importance of this feature. I expressed my concerns and
> so did others. As the subject of this thread clearly stated, I only
> wanted to point out that I had the feeling that we had a "reboot" of
> the conversation for no good reason, with the unpleasant side effect
> of proposing once again a pattern that received a lot of criticism in
> the past.
>
> On Mon, Sep 4, 2017 at 4:18 PM, Bertrand Delacretaz
> <bd...@apache.org> wrote:
> > On Mon, Sep 4, 2017 at 3:44 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> >> ...I feel
> >> that Oak is weaker without the ability to offload bulk data streaming to
> >> infrastructure designed for that purpose....
> >
> > FWIW as an Oak user I share that feeling, IMO the use cases described
> > at https://wiki.apache.org/jackrabbit/JCR%20Binary%20Usecase are
> > becoming more and more important.
> >
> > Not being a committer I don't really care about the internals, but
> > please do not "throw the baby out with the bath water" if the
> > internals need to change.
> >
> > -Bertrand
>

Re: OAK-6575 - A word of caution

Posted by Michael Dürig <md...@apache.org>.
Hi,

I think the discussion did move forward between the various issues but 
this might have been obfuscated because several topics where discussed 
at the same time. To me the two main topics touched exposure of an URI 
to binaries and an API to expose this from Oak.

In the meanwhile I just noted that Ian replied along the same lines 
already. I will follow up there re. these separate concerns.

Michael




On 04.09.17 16:43, Francesco Mari wrote:
> I'm in no position to veto the POC and I'm not willing to. I am well
> aware of the importance of this feature. I expressed my concerns and
> so did others. As the subject of this thread clearly stated, I only
> wanted to point out that I had the feeling that we had a "reboot" of
> the conversation for no good reason, with the unpleasant side effect
> of proposing once again a pattern that received a lot of criticism in
> the past.
> 
> On Mon, Sep 4, 2017 at 4:18 PM, Bertrand Delacretaz
> <bd...@apache.org> wrote:
>> On Mon, Sep 4, 2017 at 3:44 PM, Ian Boston <ie...@tfd.co.uk> wrote:
>>> ...I feel
>>> that Oak is weaker without the ability to offload bulk data streaming to
>>> infrastructure designed for that purpose....
>>
>> FWIW as an Oak user I share that feeling, IMO the use cases described
>> at https://wiki.apache.org/jackrabbit/JCR%20Binary%20Usecase are
>> becoming more and more important.
>>
>> Not being a committer I don't really care about the internals, but
>> please do not "throw the baby out with the bath water" if the
>> internals need to change.
>>
>> -Bertrand

Re: OAK-6575 - A word of caution

Posted by Francesco Mari <ma...@gmail.com>.
I'm in no position to veto the POC and I'm not willing to. I am well
aware of the importance of this feature. I expressed my concerns and
so did others. As the subject of this thread clearly stated, I only
wanted to point out that I had the feeling that we had a "reboot" of
the conversation for no good reason, with the unpleasant side effect
of proposing once again a pattern that received a lot of criticism in
the past.

On Mon, Sep 4, 2017 at 4:18 PM, Bertrand Delacretaz
<bd...@apache.org> wrote:
> On Mon, Sep 4, 2017 at 3:44 PM, Ian Boston <ie...@tfd.co.uk> wrote:
>> ...I feel
>> that Oak is weaker without the ability to offload bulk data streaming to
>> infrastructure designed for that purpose....
>
> FWIW as an Oak user I share that feeling, IMO the use cases described
> at https://wiki.apache.org/jackrabbit/JCR%20Binary%20Usecase are
> becoming more and more important.
>
> Not being a committer I don't really care about the internals, but
> please do not "throw the baby out with the bath water" if the
> internals need to change.
>
> -Bertrand

Re: OAK-6575 - A word of caution

Posted by Bertrand Delacretaz <bd...@apache.org>.
On Mon, Sep 4, 2017 at 3:44 PM, Ian Boston <ie...@tfd.co.uk> wrote:
> ...I feel
> that Oak is weaker without the ability to offload bulk data streaming to
> infrastructure designed for that purpose....

FWIW as an Oak user I share that feeling, IMO the use cases described
at https://wiki.apache.org/jackrabbit/JCR%20Binary%20Usecase are
becoming more and more important.

Not being a committer I don't really care about the internals, but
please do not "throw the baby out with the bath water" if the
internals need to change.

-Bertrand

Re: OAK-6575 - A word of caution

Posted by Ian Boston <ie...@tfd.co.uk>.
Hi Francesco and Tommaso,
I was not aware of the previous discussions and will now read those threads
and issues. I submitted the issue, patch and thread in good faith, not
having a detailed knowledge of everything that has been discussed on
oak-dev. I was not trying to ignore or circumvent any previous decisions or
opinions. If OAK-6575 brings nothing new and does not address those
concerns I would not be offended to have the patch and issue rejected and
deleted and for Oak not to have this capability, although obviously I feel
that Oak is weaker without the ability to offload bulk data streaming to
infrastructure designed for that purpose.

Let me read the links you have shared and get back to you.
Best Regards
Ian

On 4 September 2017 at 14:20, Tommaso Teofili <to...@gmail.com>
wrote:

> I share Francesco's concerns, the same way I shared them when we first
> discussed this way back; I tried to express my doubts on the current
> proposal in the email thread for OAK-6575 (linked also in Francesco's
> email), which got ignored; that's fine as long as the majority of us is
> happy with the current solution, probably it's just me having this not so
> good feeling here with me, that some of us want this feature to be in a way
> or another.
>
> Tommaso
>
>
>
> Il giorno lun 4 set 2017 alle ore 14:45 Francesco Mari <
> mari.francesco@gmail.com> ha scritto:
>
> > On Mon, Sep 4, 2017 at 2:13 PM, Chetan Mehrotra
> > <ch...@gmail.com> wrote:
> > > Adaptable pattern in itself was not much discussed at that time.
> >
> > Concerns about the adaptable pattern and its implications in data
> > encapsulation were expressed in the old thread at [1], [2], [3], [4],
> > and in other messages in the same thread. In the new thread, it was
> > pointed out [5] that solving the problem discussed in OAK-6575 is
> > orthogonal to the introduction of the adaptable pattern. Moreover, in
> > the new thread some concerns were expressed about the adaptable
> > pattern as well at [6] and [7].
> >
> > [1]:
> > https://lists.apache.org/thread.html/8b0021987b824b096ea9b470a4edd5
> edf1a246ef10548a2343ad4668@1462438837@%3Coak-dev.jackrabbit.apache.org%3E
> > [2]:
> > https://lists.apache.org/thread.html/4c352d247da81ca6ab05abfb7c5336
> 8fb88ed3c587fb09b42b87b6ae@1462439965@%3Coak-dev.jackrabbit.apache.org%3E
> > [3]:
> > https://lists.apache.org/thread.html/fbf28d5e864adbebd677a425cc915f
> 89cbd7e0ef85a589fb4f948b51@1462784316@%3Coak-dev.jackrabbit.apache.org%3E
> > [4]:
> > https://lists.apache.org/thread.html/77b05609b7e6f0deedbd3282f734f8
> c606e6a7451db0698f29082d7b@1462891501@%3Coak-dev.jackrabbit.apache.org%3E
> > [5]:
> > https://lists.apache.org/thread.html/d8da865c1f971ff4c84c9616d9f09e
> a9369f3e0c6db20f98fbc6e4d3@%3Coak-dev.jackrabbit.apache.org%3E
> > [6]:
> > https://lists.apache.org/thread.html/ab1d40572467eeee4ef5855af0a0a9
> e87255d1144fe40762ff8e9d47@%3Coak-dev.jackrabbit.apache.org%3E
> > [7]:
> > https://lists.apache.org/thread.html/acc9eeef966916791c6073e76d3baa
> 232f48abece4ae6b31f264a8ba@%3Coak-dev.jackrabbit.apache.org%3E
> >
>

Re: OAK-6575 - A word of caution

Posted by Tommaso Teofili <to...@gmail.com>.
I share Francesco's concerns, the same way I shared them when we first
discussed this way back; I tried to express my doubts on the current
proposal in the email thread for OAK-6575 (linked also in Francesco's
email), which got ignored; that's fine as long as the majority of us is
happy with the current solution, probably it's just me having this not so
good feeling here with me, that some of us want this feature to be in a way
or another.

Tommaso



Il giorno lun 4 set 2017 alle ore 14:45 Francesco Mari <
mari.francesco@gmail.com> ha scritto:

> On Mon, Sep 4, 2017 at 2:13 PM, Chetan Mehrotra
> <ch...@gmail.com> wrote:
> > Adaptable pattern in itself was not much discussed at that time.
>
> Concerns about the adaptable pattern and its implications in data
> encapsulation were expressed in the old thread at [1], [2], [3], [4],
> and in other messages in the same thread. In the new thread, it was
> pointed out [5] that solving the problem discussed in OAK-6575 is
> orthogonal to the introduction of the adaptable pattern. Moreover, in
> the new thread some concerns were expressed about the adaptable
> pattern as well at [6] and [7].
>
> [1]:
> https://lists.apache.org/thread.html/8b0021987b824b096ea9b470a4edd5edf1a246ef10548a2343ad4668@1462438837@%3Coak-dev.jackrabbit.apache.org%3E
> [2]:
> https://lists.apache.org/thread.html/4c352d247da81ca6ab05abfb7c53368fb88ed3c587fb09b42b87b6ae@1462439965@%3Coak-dev.jackrabbit.apache.org%3E
> [3]:
> https://lists.apache.org/thread.html/fbf28d5e864adbebd677a425cc915f89cbd7e0ef85a589fb4f948b51@1462784316@%3Coak-dev.jackrabbit.apache.org%3E
> [4]:
> https://lists.apache.org/thread.html/77b05609b7e6f0deedbd3282f734f8c606e6a7451db0698f29082d7b@1462891501@%3Coak-dev.jackrabbit.apache.org%3E
> [5]:
> https://lists.apache.org/thread.html/d8da865c1f971ff4c84c9616d9f09ea9369f3e0c6db20f98fbc6e4d3@%3Coak-dev.jackrabbit.apache.org%3E
> [6]:
> https://lists.apache.org/thread.html/ab1d40572467eeee4ef5855af0a0a9e87255d1144fe40762ff8e9d47@%3Coak-dev.jackrabbit.apache.org%3E
> [7]:
> https://lists.apache.org/thread.html/acc9eeef966916791c6073e76d3baa232f48abece4ae6b31f264a8ba@%3Coak-dev.jackrabbit.apache.org%3E
>

Re: OAK-6575 - A word of caution

Posted by Francesco Mari <ma...@gmail.com>.
On Mon, Sep 4, 2017 at 2:13 PM, Chetan Mehrotra
<ch...@gmail.com> wrote:
> Adaptable pattern in itself was not much discussed at that time.

Concerns about the adaptable pattern and its implications in data
encapsulation were expressed in the old thread at [1], [2], [3], [4],
and in other messages in the same thread. In the new thread, it was
pointed out [5] that solving the problem discussed in OAK-6575 is
orthogonal to the introduction of the adaptable pattern. Moreover, in
the new thread some concerns were expressed about the adaptable
pattern as well at [6] and [7].

[1]: https://lists.apache.org/thread.html/8b0021987b824b096ea9b470a4edd5edf1a246ef10548a2343ad4668@1462438837@%3Coak-dev.jackrabbit.apache.org%3E
[2]: https://lists.apache.org/thread.html/4c352d247da81ca6ab05abfb7c53368fb88ed3c587fb09b42b87b6ae@1462439965@%3Coak-dev.jackrabbit.apache.org%3E
[3]: https://lists.apache.org/thread.html/fbf28d5e864adbebd677a425cc915f89cbd7e0ef85a589fb4f948b51@1462784316@%3Coak-dev.jackrabbit.apache.org%3E
[4]: https://lists.apache.org/thread.html/77b05609b7e6f0deedbd3282f734f8c606e6a7451db0698f29082d7b@1462891501@%3Coak-dev.jackrabbit.apache.org%3E
[5]: https://lists.apache.org/thread.html/d8da865c1f971ff4c84c9616d9f09ea9369f3e0c6db20f98fbc6e4d3@%3Coak-dev.jackrabbit.apache.org%3E
[6]: https://lists.apache.org/thread.html/ab1d40572467eeee4ef5855af0a0a9e87255d1144fe40762ff8e9d47@%3Coak-dev.jackrabbit.apache.org%3E
[7]: https://lists.apache.org/thread.html/acc9eeef966916791c6073e76d3baa232f48abece4ae6b31f264a8ba@%3Coak-dev.jackrabbit.apache.org%3E

Re: OAK-6575 - A word of caution

Posted by Chetan Mehrotra <ch...@gmail.com>.
> Why couldn't we discuss this implementation in the light of the OAK-1963?
> Why the old concerns are not taken in to consideration once again? We already talked about all that.

I closed OAK-1963 because it was specially related to exposing the
File instance url while objective of OAK-6575 is to provide a secure
url. My reading of discussion done in OAK-1963 that people had concern
around exposing the File instance location managed by Oak. Adaptable
pattern in itself was not much discussed at that time. Hence it made
sense to close OAK-1963 and have a fresh discussion on OAK-6575

As for OAK-6575 the approach taken there has been discussed in detail
at [1] and in the end there was consensus on the proposed approach by
Ian.

Chetan Mehrotra
[1] http://markmail.org/thread/lhljp2ksegw7dcod

On Mon, Sep 4, 2017 at 5:25 PM, Francesco Mari <ma...@gmail.com> wrote:
> The POC for OAK-6575 aims at introducing the AdapterManager and
> AdapterFactory API in Oak. This is just a step away from introducing a
> full-fledged implementation of Sling-like adapters in Oak. In fact,
> the OakConversionService is just an Adapter for a single, very
> specific conversion.
>
> We had a very similar conversation for OAK-1963 in a thread [1] in
> this mailing list. In that thread many people raised that concerns
> about adapters. Many of us, me included, see adapters as a glorified
> way of breaking data encapsulation and are not in favour of
> introducing them.
>
> After some quiet time in the thread and in OAK-1963, a new thread [2]
> was created, a new issue [3] was opened and a POC [4] was written. The
> content of the POC is little more than a permutation of what was
> already discussed in OAK-1963. Why a new thread and issue were needed?
> Why couldn't we discuss this implementation in the light of the
> OAK-1963? Why the old concerns are not taken into consideration once
> again? We already talked about all that.
>
> If the majority of the people here is alright in going forward with
> the aforementioned POC, so be it. But, at least, let's avoid closing
> threads and issues and create new ones that are just carbon copies.
> This might be misinterpreted as a way to clear concerns out of our
> way.
>
> [1]: http://markmail.org/thread/enmibgfwedypjnos
> [2]: https://lists.apache.org/thread.html/2efb1da82b9b93a3a394c3abccbac960175ffc7a13facaac79c9181a@%3Coak-dev.jackrabbit.apache.org%3E
> [3]: https://issues.apache.org/jira/browse/OAK-6575
> [4]: https://github.com/apache/jackrabbit-oak/compare/trunk...ieb:OAK-6575?expand=1