Initial work for the specification of a remote API

[Francesco Mari <ma...@gmail.com>]

Hi all,

since I don't have access to the wiki, I started to write down a draft
for the remote API in a public GitHub repository [1].

I didn't write much so far, but I invite every interested party to
take a look at it for suggestions and improvements.

Thanks,

Francesco

[1]: https://github.com/francescomari/oak-remote

Re: Initial work for the specification of a remote API

[Stefan Guggisberg <st...@gmail.com>]

On Tue, Jan 27, 2015 at 2:23 PM, Francesco Mari
<ma...@gmail.com> wrote:
> My personal point of view is that the remote API should make Oak
> interoperable between different software stacks. Some of these
> software stacks may want to use their own authentication strategies
> which may (or may not) be the authentication strategies provided by
> Oak.
>
> In a typical web application, in example, every time a user A wants to
> do something, the following happens:
>
> - The user U sends a request to the web application and provides his
> personal credentials.
>
> - The web application connects to a database. This connection usually
> uses a set of credentials D that are necessary to establish a
> conversation with the database.
>
> - The web application extracts the credentials for U from the request
> and authenticates them. Information about U are usually saved in the
> database.
>
> - The web application reads and writes data on behalf of U, on top of
> a database connection that was previously authenticated with the
> credentials D.
>
> One of the goals of the remote interface, in my opinion, should be to
> enable this scenario. It should be possible to replace the word
> "database" with "Oak" in every point in the list above. If an
> application uses Oak as a storage system, that application should not
> be limited to the authentication strategies supported by Oak.

IIUC we're talking here about the remoting layer of the repository
(oak), not about hypothetical applications.

an oak client should IMO be able to use the same credentials
regardless of the deployment (local vs remote), i.e. it should be
transparent. requiring a remote client to provide oak 'admin'
credentials just feels totally wrong.

cheers
stefan

>
> On the other end, we still want to leverage the authorization
> mechanism built into Oak. The way authorization works in Oak is
> strongly decoupled from authentication. As long as a user exists,
> regardless of how it was authenticated, authorization will still work.
> This is one of the strongest point of authorization in Oak, in my
> opinion.
>
> 2015-01-27 12:11 GMT+01:00 Stefan Guggisberg <st...@gmail.com>:
>> On Tue, Jan 27, 2015 at 11:31 AM, Francesco Mari
>> <ma...@gmail.com> wrote:
>>> Should the API expose every authentication strategy implemented in
>>> Oak?
>>
>> which strategy should be supported by the remoting layer is something
>> that needs TBD.
>> i'd say we start with the simple case (password credentials).
>>
>> cheers
>> stefan
>>
>>> How is this API should look like?
>>>
>>> 2015-01-27 10:39 GMT+01:00 Stefan Guggisberg <st...@gmail.com>:
>>>> On Tue, Jan 27, 2015 at 10:16 AM, Francesco Mari
>>>> <ma...@gmail.com> wrote:
>>>>> Which other alternatives do we have regarding authentication?
>>>>
>>>> remote and local users should IMO use the same credentials. it should
>>>> be transparent to the user whether he's accessing a local or a remote
>>>> repo.
>>>>
>>>> cheers
>>>> stefan
>>>>
>>>>>
>>>>> 2015-01-27 9:16 GMT+01:00 Stefan Guggisberg <st...@gmail.com>:
>>>>>> hi francesco,
>>>>>>
>>>>>> On Fri, Jan 23, 2015 at 6:09 PM, Francesco Mari
>>>>>> <ma...@gmail.com> wrote:
>>>>>>> Hi all,
>>>>>>>
>>>>>>> since I don't have access to the wiki, I started to write down a draft
>>>>>>> for the remote API in a public GitHub repository [1].
>>>>>>>
>>>>>>> I didn't write much so far, but I invite every interested party to
>>>>>>> take a look at it for suggestions and improvements.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Francesco
>>>>>>>
>>>>>>> [1]: https://github.com/francescomari/oak-remote
>>>>>>
>>>>>> "To access the Oak Remote Interface a client needs to have the
>>>>>> credentials of the master user."
>>>>>>
>>>>>> are you saying that a client needs master user credentials to access a
>>>>>> remote oak instance? that would IMO be wrong.
>>>>>>
>>>>>> cheers
>>>>>> stefan

Re: Initial work for the specification of a remote API

[Francesco Mari <ma...@gmail.com>]

My personal point of view is that the remote API should make Oak
interoperable between different software stacks. Some of these
software stacks may want to use their own authentication strategies
which may (or may not) be the authentication strategies provided by
Oak.

In a typical web application, in example, every time a user A wants to
do something, the following happens:

- The user U sends a request to the web application and provides his
personal credentials.

- The web application connects to a database. This connection usually
uses a set of credentials D that are necessary to establish a
conversation with the database.

- The web application extracts the credentials for U from the request
and authenticates them. Information about U are usually saved in the
database.

- The web application reads and writes data on behalf of U, on top of
a database connection that was previously authenticated with the
credentials D.

One of the goals of the remote interface, in my opinion, should be to
enable this scenario. It should be possible to replace the word
"database" with "Oak" in every point in the list above. If an
application uses Oak as a storage system, that application should not
be limited to the authentication strategies supported by Oak.

On the other end, we still want to leverage the authorization
mechanism built into Oak. The way authorization works in Oak is
strongly decoupled from authentication. As long as a user exists,
regardless of how it was authenticated, authorization will still work.
This is one of the strongest point of authorization in Oak, in my
opinion.

2015-01-27 12:11 GMT+01:00 Stefan Guggisberg <st...@gmail.com>:
> On Tue, Jan 27, 2015 at 11:31 AM, Francesco Mari
> <ma...@gmail.com> wrote:
>> Should the API expose every authentication strategy implemented in
>> Oak?
>
> which strategy should be supported by the remoting layer is something
> that needs TBD.
> i'd say we start with the simple case (password credentials).
>
> cheers
> stefan
>
>> How is this API should look like?
>>
>> 2015-01-27 10:39 GMT+01:00 Stefan Guggisberg <st...@gmail.com>:
>>> On Tue, Jan 27, 2015 at 10:16 AM, Francesco Mari
>>> <ma...@gmail.com> wrote:
>>>> Which other alternatives do we have regarding authentication?
>>>
>>> remote and local users should IMO use the same credentials. it should
>>> be transparent to the user whether he's accessing a local or a remote
>>> repo.
>>>
>>> cheers
>>> stefan
>>>
>>>>
>>>> 2015-01-27 9:16 GMT+01:00 Stefan Guggisberg <st...@gmail.com>:
>>>>> hi francesco,
>>>>>
>>>>> On Fri, Jan 23, 2015 at 6:09 PM, Francesco Mari
>>>>> <ma...@gmail.com> wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> since I don't have access to the wiki, I started to write down a draft
>>>>>> for the remote API in a public GitHub repository [1].
>>>>>>
>>>>>> I didn't write much so far, but I invite every interested party to
>>>>>> take a look at it for suggestions and improvements.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Francesco
>>>>>>
>>>>>> [1]: https://github.com/francescomari/oak-remote
>>>>>
>>>>> "To access the Oak Remote Interface a client needs to have the
>>>>> credentials of the master user."
>>>>>
>>>>> are you saying that a client needs master user credentials to access a
>>>>> remote oak instance? that would IMO be wrong.
>>>>>
>>>>> cheers
>>>>> stefan

Re: Initial work for the specification of a remote API

[Stefan Guggisberg <st...@gmail.com>]

On Tue, Jan 27, 2015 at 11:31 AM, Francesco Mari
<ma...@gmail.com> wrote:
> Should the API expose every authentication strategy implemented in
> Oak?

which strategy should be supported by the remoting layer is something
that needs TBD.
i'd say we start with the simple case (password credentials).

cheers
stefan

> How is this API should look like?
>
> 2015-01-27 10:39 GMT+01:00 Stefan Guggisberg <st...@gmail.com>:
>> On Tue, Jan 27, 2015 at 10:16 AM, Francesco Mari
>> <ma...@gmail.com> wrote:
>>> Which other alternatives do we have regarding authentication?
>>
>> remote and local users should IMO use the same credentials. it should
>> be transparent to the user whether he's accessing a local or a remote
>> repo.
>>
>> cheers
>> stefan
>>
>>>
>>> 2015-01-27 9:16 GMT+01:00 Stefan Guggisberg <st...@gmail.com>:
>>>> hi francesco,
>>>>
>>>> On Fri, Jan 23, 2015 at 6:09 PM, Francesco Mari
>>>> <ma...@gmail.com> wrote:
>>>>> Hi all,
>>>>>
>>>>> since I don't have access to the wiki, I started to write down a draft
>>>>> for the remote API in a public GitHub repository [1].
>>>>>
>>>>> I didn't write much so far, but I invite every interested party to
>>>>> take a look at it for suggestions and improvements.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Francesco
>>>>>
>>>>> [1]: https://github.com/francescomari/oak-remote
>>>>
>>>> "To access the Oak Remote Interface a client needs to have the
>>>> credentials of the master user."
>>>>
>>>> are you saying that a client needs master user credentials to access a
>>>> remote oak instance? that would IMO be wrong.
>>>>
>>>> cheers
>>>> stefan

Re: Initial work for the specification of a remote API

[Francesco Mari <ma...@gmail.com>]

Should the API expose every authentication strategy implemented in
Oak? How is this API should look like?

2015-01-27 10:39 GMT+01:00 Stefan Guggisberg <st...@gmail.com>:
> On Tue, Jan 27, 2015 at 10:16 AM, Francesco Mari
> <ma...@gmail.com> wrote:
>> Which other alternatives do we have regarding authentication?
>
> remote and local users should IMO use the same credentials. it should
> be transparent to the user whether he's accessing a local or a remote
> repo.
>
> cheers
> stefan
>
>>
>> 2015-01-27 9:16 GMT+01:00 Stefan Guggisberg <st...@gmail.com>:
>>> hi francesco,
>>>
>>> On Fri, Jan 23, 2015 at 6:09 PM, Francesco Mari
>>> <ma...@gmail.com> wrote:
>>>> Hi all,
>>>>
>>>> since I don't have access to the wiki, I started to write down a draft
>>>> for the remote API in a public GitHub repository [1].
>>>>
>>>> I didn't write much so far, but I invite every interested party to
>>>> take a look at it for suggestions and improvements.
>>>>
>>>> Thanks,
>>>>
>>>> Francesco
>>>>
>>>> [1]: https://github.com/francescomari/oak-remote
>>>
>>> "To access the Oak Remote Interface a client needs to have the
>>> credentials of the master user."
>>>
>>> are you saying that a client needs master user credentials to access a
>>> remote oak instance? that would IMO be wrong.
>>>
>>> cheers
>>> stefan

Re: Initial work for the specification of a remote API

[Stefan Guggisberg <st...@gmail.com>]

On Tue, Jan 27, 2015 at 10:16 AM, Francesco Mari
<ma...@gmail.com> wrote:
> Which other alternatives do we have regarding authentication?

remote and local users should IMO use the same credentials. it should
be transparent to the user whether he's accessing a local or a remote
repo.

cheers
stefan

>
> 2015-01-27 9:16 GMT+01:00 Stefan Guggisberg <st...@gmail.com>:
>> hi francesco,
>>
>> On Fri, Jan 23, 2015 at 6:09 PM, Francesco Mari
>> <ma...@gmail.com> wrote:
>>> Hi all,
>>>
>>> since I don't have access to the wiki, I started to write down a draft
>>> for the remote API in a public GitHub repository [1].
>>>
>>> I didn't write much so far, but I invite every interested party to
>>> take a look at it for suggestions and improvements.
>>>
>>> Thanks,
>>>
>>> Francesco
>>>
>>> [1]: https://github.com/francescomari/oak-remote
>>
>> "To access the Oak Remote Interface a client needs to have the
>> credentials of the master user."
>>
>> are you saying that a client needs master user credentials to access a
>> remote oak instance? that would IMO be wrong.
>>
>> cheers
>> stefan

Re: Initial work for the specification of a remote API

[Francesco Mari <ma...@gmail.com>]

Which other alternatives do we have regarding authentication?

2015-01-27 9:16 GMT+01:00 Stefan Guggisberg <st...@gmail.com>:
> hi francesco,
>
> On Fri, Jan 23, 2015 at 6:09 PM, Francesco Mari
> <ma...@gmail.com> wrote:
>> Hi all,
>>
>> since I don't have access to the wiki, I started to write down a draft
>> for the remote API in a public GitHub repository [1].
>>
>> I didn't write much so far, but I invite every interested party to
>> take a look at it for suggestions and improvements.
>>
>> Thanks,
>>
>> Francesco
>>
>> [1]: https://github.com/francescomari/oak-remote
>
> "To access the Oak Remote Interface a client needs to have the
> credentials of the master user."
>
> are you saying that a client needs master user credentials to access a
> remote oak instance? that would IMO be wrong.
>
> cheers
> stefan

Re: Initial work for the specification of a remote API

[Stefan Guggisberg <st...@gmail.com>]

hi francesco,

On Fri, Jan 23, 2015 at 6:09 PM, Francesco Mari
<ma...@gmail.com> wrote:
> Hi all,
>
> since I don't have access to the wiki, I started to write down a draft
> for the remote API in a public GitHub repository [1].
>
> I didn't write much so far, but I invite every interested party to
> take a look at it for suggestions and improvements.
>
> Thanks,
>
> Francesco
>
> [1]: https://github.com/francescomari/oak-remote

"To access the Oak Remote Interface a client needs to have the
credentials of the master user."

are you saying that a client needs master user credentials to access a
remote oak instance? that would IMO be wrong.

cheers
stefan

Re: Initial work for the specification of a remote API

[Francesco Mari <ma...@gmail.com>]

I agree that a mechanism to cope with children is needed, other than
filtering by name.

I partially agree when you say that the end result of a single
operation can be either successful, skipped or failed. I am not really
sure if we need the skipped result, because a set of operations is
supposed to be built on top of an immutable state of the repository. A
user reads revision X, applies some operations on the tree at revision
X, and submits the operations to the server. In this case, a delete
operation on a non-existing node is not tolerable from the client.

On the other hand, a skipped result makes sense if we expect clients
that want to build a set of operations on a revision X and apply that
set of operations on a revision Y, with X older than Y. In this case,
we have to respect the "intent" of the operation: the removal of a
non-existent node should succeed.

In this last case, though, I better prefer the branch/commit/merge
approach of the Microkernel, which works with non-skippable, exact
operations and makes the difference clear between errors caused by
malformed operations and errors caused by conflicting state.

2015-02-02 14:36 GMT+01:00 Michael Dürig <md...@apache.org>:
>
> Hi,
>
> I finally found some time to read your draft:
>
> * what about large number of child nodes? I think we need some kind of
> support for paging/chunking to avoid reading millions of children in one go.
>
> * the write operations all fail if the operation "has been done already":
> E.g. the remove operation fails if there isn't any node at the given path.
> However, in many cases this is not interesting because the intention is to
> make sure the node is gone. To prevent extra round trips, I suggest to
> provide a way for callers to distinguish a failed operation from this case.
> Same goes for adding items where the same items has already been added. See
> http://markmail.org/message/flst4eiqvbp4gi3z for a related discussion where
> this has come up within Oak's user management.
>
> Michael
>
> On 23.1.15 6:09 , Francesco Mari wrote:
>>
>> Hi all,
>>
>> since I don't have access to the wiki, I started to write down a draft
>> for the remote API in a public GitHub repository [1].
>>
>> I didn't write much so far, but I invite every interested party to
>> take a look at it for suggestions and improvements.
>>
>> Thanks,
>>
>> Francesco
>>
>> [1]: https://github.com/francescomari/oak-remote
>>
>

Re: Initial work for the specification of a remote API

[Michael Dürig <md...@apache.org>]

Hi,

I finally found some time to read your draft:

* what about large number of child nodes? I think we need some kind of 
support for paging/chunking to avoid reading millions of children in one go.

* the write operations all fail if the operation "has been done 
already": E.g. the remove operation fails if there isn't any node at the 
given path. However, in many cases this is not interesting because the 
intention is to make sure the node is gone. To prevent extra round 
trips, I suggest to provide a way for callers to distinguish a failed 
operation from this case. Same goes for adding items where the same 
items has already been added. See 
http://markmail.org/message/flst4eiqvbp4gi3z for a related discussion 
where this has come up within Oak's user management.

Michael

On 23.1.15 6:09 , Francesco Mari wrote:
> Hi all,
>
> since I don't have access to the wiki, I started to write down a draft
> for the remote API in a public GitHub repository [1].
>
> I didn't write much so far, but I invite every interested party to
> take a look at it for suggestions and improvements.
>
> Thanks,
>
> Francesco
>
> [1]: https://github.com/francescomari/oak-remote
>

Re: Initial work for the specification of a remote API

[Michael Dürig <md...@apache.org>]

On 23.1.15 6:09 , Francesco Mari wrote:
> since I don't have access to the wiki, I started to write down a draft
> for the remote API in a public GitHub repository [1].

Let me know the desired user name and I'll grant you access.

Michael