[ApacheDS] Comments on Stored Procedure Implementation (was Re: [Roadmap]Apache Directory Server 2.0 Roadmap proposal)

[Alex Karasulu <ak...@apache.org>]

On 9/28/07, Stefan Zoerner <st...@labeo.de> wrote:
>
> Alex Karasulu wrote:
> >     * Support for Scripting languages for Stored Procedures (e.g.
> Groovy)
> >
> >
> > This also would be really cool to have.  We just need more participation
> > on getting stored procedures
> > implemented properly.  It would be great if you could lend a hand if it
> > interests you of course.
>
> It seems that it is not much code we talk about. I am interested.


This is great Stefan.

<note>
Stefan you and others insterested in this stuff may not be interested in
these details below
or how they will be managed by the stored procedure subsystem in the
server.  Wanting to
do it with groovy should not have to force you to have to deal with these
details.  So please
feel free to skip the material below.  However at a minimum understanding
these problems
and their impact on you may help you a bit.
</note>

Incidentally I want to just list some of the issues we presently have on the
table regarding our
stored procedure implementation:

1). Security: Rights and Identity (owner verses invoker)

Stored procedures can be invoked by anyone if access to the SP is allowed.
The visibility of SP entries to
invoker principals can be controlled with ACI.  However if the invoker is
different from the owner then with
who's identity and authorization permissions do we execute the procedure?
This is a big question we've been
discussing for some time.  We can continue to discuss this on this thread if
people like but IMO the SP should
execute with the authorization permissions of the owner with the identity of
the invoker for auditing purposes.

How to do this is entirely another question that our implementation must
answer.

2). Security: Sandboxing SP Code

SP code is code which needs to be sandboxed so code with lesser rights does
not have the ability to
execute internal code with administrator like rights.  Today SPs in the
server can do whatever they
want no matter who invokes them.  This is why they are turned off by default
with the server as an
insecure experimental feature.  To take this to the next level we need to
sandbox SP code and allow
access to internal methods based on the rights and identity of the user.
This is a really difficult problem
to solve elegantly in ApacheDS as it stands today but it is a must.

3). SP Invocation: Procedure Qualification

Right now we have an ExtendedOperation used to invoke stored procedures
using qualified names based
on the namespace of the procedure language.  This is a questionable
mechanism since it requires the user
to know what the language of the implementation of the procedure is and how
to qualify the procedure in that
language.  If a universal call specification is devised it can link language
independent invocation requests to
the proper language specific method.  It would then be the job of the writer
to add this call spec and not the
job of the user to track which language the stored procedure was implemented
in.

4). SP Invocation: Parameter Bindings

This is connected to 3.  The same extended operation allows invokers to
provide parameters to the stored
procedure.  It would not be fun to have SPs without parameters.  This opens
up a slew of problems for us.
First we need to define now a language independent means to specify
parameters.  ASN.1 does this nicely
so we can leverage ASN.1 primitive types at first.  ApacheDS can easily
transform these types into the types
expected by the implementation language of the SP.  This however requires
some primitive type bindings and
the code to translate them.

Right now the extended operation works with primitive Java types and
serialized objects (I think but Ersin can
confirm).  This makes it so a C client cannot really invoke a SP implemented
in Java unless it knows how to
serialize java objects into a byte buffer.  This is not really feasible so
we need to specify some type bindings
and use that instead of serializing objects.  We cannot expect invokers from
python, groovy, php, c, perl etc
to have to know anything about Java.  Invocation should not have be language
dependent to allow for maximum
flexibility.

I'm adding these details btw to a the first draft (version 00) of an IETF
draft submission.

Alex

Re: [ApacheDS] Comments on Stored Procedure Implementation (was Re: [Roadmap]Apache Directory Server 2.0 Roadmap proposal)

[Alex Karasulu <ak...@apache.org>]

Incidentally I just got a simple idea on how (in the implementation) one
principal can execute operations with the rights of another.  This may help
with both sp/triggers and implementing the authorization proxy control.  We
simply need to track the authorization principal with the LdapPrincipal.
The authorization principal is the one used by the Authz subsystem's access
control decision function (ACDF).  The authentication principal is then used
as the identity while running with the permissions of the authorization
principal.  Based on controls and stored procedure ownership the AuthZ
principal can be changed at any time and reverted back to the AuthN
principal by the server.

This would not be too hard to implement given the architecture of the server
and of the AuthZ service.

Alex

Re: [ApacheDS] Comments on Stored Procedure Implementation (was Re: [Roadmap]Apache Directory Server 2.0 Roadmap proposal)

[Alex Karasulu <ak...@apache.org>]

On 9/28/07, Ersin Er <er...@gmail.com> wrote:
>
>
>
> On 9/28/07, Alex Karasulu <ak...@apache.org> wrote:
>

SNIP ...

3). SP Invocation: Procedure Qualification
> >
> > Right now we have an ExtendedOperation used to invoke stored procedures
> > using qualified names based
> > on the namespace of the procedure language.  This is a questionable
> > mechanism since it requires the user
> > to know what the language of the implementation of the procedure is and
> > how to qualify the procedure in that
> > language.  If a universal call specification is devised it can link
> > language independent invocation requests to
> > the proper language specific method.  It would then be the job of the
> > writer to add this call spec and not the
> > job of the user to track which language the stored procedure was
> > implemented in.
> >
>
> This leads to us towards implemeting something like web services stack for
> LDAP. We need to be careful about making this really simple. And I think we
> should not focus on the external invocation specification during the initial
> standardization effort.
>

I agree and disagree :).  First I disagree that this is like a web services
stack for LDAP because we're keeping it
simple with primitive static bindings for parameters.  I would not propose
that we go all the way with devising dynamic
bindings for complex types or structures.  If we did then yes you would be
right.

I agree with keeping it simple by managing the scope and I think it's
inherent in the original proposal for bindings
to primitive types and things like Dates etc.

LDAP stored procedures are useless unless they treat all invokers alike
while removing the need for clients to be aware of
the implementation language of the stored procedure doing the work.  So this
absolutely must be addressed and I have
already started on this and from my observations this is not a monumental
task to achieve.

4). SP Invocation: Parameter Bindings
> >
> > This is connected to 3.  The same extended operation allows invokers to
> > provide parameters to the stored
> > procedure.  It would not be fun to have SPs without parameters.  This
> > opens up a slew of problems for us.
> > First we need to define now a language independent means to specify
> > parameters.  ASN.1 does this nicely
> > so we can leverage ASN.1 primitive types at first.  ApacheDS can easily
> > transform these types into the types
> > expected by the implementation language of the SP.  This however
> > requires some primitive type bindings and
> > the code to translate them.
> >
> > Right now the extended operation works with primitive Java types and
> > serialized objects (I think but Ersin can
> > confirm).  This makes it so a C client cannot really invoke a SP
> > implemented in Java unless it knows how to
> > serialize java objects into a byte buffer.  This is not really feasible
> > so we need to specify some type bindings
> > and use that instead of serializing objects.  We cannot expect invokers
> > from python, groovy, php, c, perl etc
> > to have to know anything about Java.  Invocation should not have be
> > language dependent to allow for maximum
> > flexibility.
> >
>
> As this is not an easy to solve problem quickly and as stored procedures
> are more important from the triggers point, we can focus on SP-Trigger
> integration more. Trigger Specifications allow you to inject operation
> specific parameters to stored procedures and determining the Java types for
> these parameters should be handled by an RFC alone.
>

Yes you are correct. These are not simple parameters however they are static
and will not change over time.  They simply
add to the number of static bindings to be supported.

I'm adding these details btw to a the first draft (version 00) of an IETF
> > draft submission.
> >
> > Alex
> >
>
>
> I think about the following drafts initially:
>
> LDAP Triggers: Representation and Abstract Model of Execution
> LDAP Stored Procedures - Representation
>

I have started on these two as one draft since the abstract model is the
representation IMO.

LDAP Stored Procedures - Abstract Model of Invocation by LDAP Triggers
> Java Language Bindings for Java LDAP Stored Procedures Invoked by LDAP
> Triggers
>

These may be part of the draft for triggers.

Alex

Re: [ApacheDS] Comments on Stored Procedure Implementation (was Re: [Roadmap]Apache Directory Server 2.0 Roadmap proposal)

[Ersin Er <er...@gmail.com>]

On 9/28/07, Alex Karasulu <ak...@apache.org> wrote:
>
>
>
> On 9/28/07, Stefan Zoerner <stefan@labeo.de > wrote:
> >
> > Alex Karasulu wrote:
> > >     * Support for Scripting languages for Stored Procedures (e.g.
> > Groovy)
> > >
> > >
> > > This also would be really cool to have.  We just need more
> > participation
> > > on getting stored procedures
> > > implemented properly.  It would be great if you could lend a hand if
> > it
> > > interests you of course.
> >
> > It seems that it is not much code we talk about. I am interested.
>
>
> This is great Stefan.
>
> <note>
> Stefan you and others insterested in this stuff may not be interested in
> these details below
> or how they will be managed by the stored procedure subsystem in the
> server.  Wanting to
> do it with groovy should not have to force you to have to deal with these
> details.  So please
> feel free to skip the material below.  However at a minimum understanding
> these problems
> and their impact on you may help you a bit.
> </note>
>
> Incidentally I want to just list some of the issues we presently have on
> the table regarding our
> stored procedure implementation:
>
> 1). Security: Rights and Identity (owner verses invoker)
>
> Stored procedures can be invoked by anyone if access to the SP is
> allowed.  The visibility of SP entries to
> invoker principals can be controlled with ACI.  However if the invoker is
> different from the owner then with
> who's identity and authorization permissions do we execute the procedure?
> This is a big question we've been
> discussing for some time.  We can continue to discuss this on this thread
> if people like but IMO the SP should
> execute with the authorization permissions of the owner with the identity
> of the invoker for auditing purposes.
>
> How to do this is entirely another question that our implementation must
> answer.
>

2). Security: Sandboxing SP Code
>
> SP code is code which needs to be sandboxed so code with lesser rights
> does not have the ability to
> execute internal code with administrator like rights.  Today SPs in the
> server can do whatever they
> want no matter who invokes them.  This is why they are turned off by
> default with the server as an
> insecure experimental feature.  To take this to the next level we need to
> sandbox SP code and allow
> access to internal methods based on the rights and identity of the user.
> This is a really difficult problem
> to solve elegantly in ApacheDS as it stands today but it is a must.
>
> 3). SP Invocation: Procedure Qualification
>
> Right now we have an ExtendedOperation used to invoke stored procedures
> using qualified names based
> on the namespace of the procedure language.  This is a questionable
> mechanism since it requires the user
> to know what the language of the implementation of the procedure is and
> how to qualify the procedure in that
> language.  If a universal call specification is devised it can link
> language independent invocation requests to
> the proper language specific method.  It would then be the job of the
> writer to add this call spec and not the
> job of the user to track which language the stored procedure was
> implemented in.
>

This leads to us towards implemeting something like web services stack for
LDAP. We need to be careful about making this really simple. And I think we
should not focus on the external invocation specification during the initial
standardization effort.

4). SP Invocation: Parameter Bindings
>
> This is connected to 3.  The same extended operation allows invokers to
> provide parameters to the stored
> procedure.  It would not be fun to have SPs without parameters.  This
> opens up a slew of problems for us.
> First we need to define now a language independent means to specify
> parameters.  ASN.1 does this nicely
> so we can leverage ASN.1 primitive types at first.  ApacheDS can easily
> transform these types into the types
> expected by the implementation language of the SP.  This however requires
> some primitive type bindings and
> the code to translate them.
>
> Right now the extended operation works with primitive Java types and
> serialized objects (I think but Ersin can
> confirm).  This makes it so a C client cannot really invoke a SP
> implemented in Java unless it knows how to
> serialize java objects into a byte buffer.  This is not really feasible so
> we need to specify some type bindings
> and use that instead of serializing objects.  We cannot expect invokers
> from python, groovy, php, c, perl etc
> to have to know anything about Java.  Invocation should not have be
> language dependent to allow for maximum
> flexibility.
>

As this is not an easy to solve problem quickly and as stored procedures are
more important from the triggers point, we can focus on SP-Trigger
integration more. Trigger Specifications allow you to inject operation
specific parameters to stored procedures and determining the Java types for
these parameters should be handled by an RFC alone.

I'm adding these details btw to a the first draft (version 00) of an IETF
> draft submission.
>
> Alex
>


I think about the following drafts initially:

LDAP Triggers: Representation and Abstract Model of Execution
LDAP Stored Procedures - Representation
LDAP Stored Procedures - Abstract Model of Invocation by LDAP Triggers
Java Language Bindings for Java LDAP Stored Procedures Invoked by LDAP
Triggers

I am not sure about the namings but I hope I reflected my intent.

-- 
Ersin Er
http://www.ersin-er.name

Re: [ApacheDS] Comments on Stored Procedure Implementation (was Re: [Roadmap]Apache Directory Server 2.0 Roadmap proposal)

[David Jencks <da...@yahoo.com>]

On Sep 28, 2007, at 12:19 PM, Alex Karasulu wrote:

>
>
> On 9/28/07, Stefan Zoerner <stefan@labeo.de > wrote:
> Alex Karasulu wrote:
> >     * Support for Scripting languages for Stored Procedures (e.g.  
> Groovy)
> >
> >
> > This also would be really cool to have.  We just need more  
> participation
> > on getting stored procedures
> > implemented properly.  It would be great if you could lend a hand  
> if it
> > interests you of course.
>
> It seems that it is not much code we talk about. I am interested.
>
> This is great Stefan.
>
> <note>
> Stefan you and others insterested in this stuff may not be  
> interested in these details below
> or how they will be managed by the stored procedure subsystem in  
> the server.  Wanting to
> do it with groovy should not have to force you to have to deal with  
> these details.  So please
> feel free to skip the material below.  However at a minimum  
> understanding these problems
> and their impact on you may help you a bit.
> </note>
>
> Incidentally I want to just list some of the issues we presently  
> have on the table regarding our
> stored procedure implementation:
>
> 1). Security: Rights and Identity (owner verses invoker)
>
> Stored procedures can be invoked by anyone if access to the SP is  
> allowed.  The visibility of SP entries to
> invoker principals can be controlled with ACI.  However if the  
> invoker is different from the owner then with
> who's identity and authorization permissions do we execute the  
> procedure? This is a big question we've been
> discussing for some time.  We can continue to discuss this on this  
> thread if people like but IMO the SP should
> execute with the authorization permissions of the owner with the  
> identity of the invoker for auditing purposes.
>
> How to do this is entirely another question that our implementation  
> must answer.
>
> 2). Security: Sandboxing SP Code
>
> SP code is code which needs to be sandboxed so code with lesser  
> rights does not have the ability to
> execute internal code with administrator like rights.  Today SPs in  
> the server can do whatever they
> want no matter who invokes them.  This is why they are turned off  
> by default with the server as an
> insecure experimental feature.  To take this to the next level we  
> need to sandbox SP code and allow
> access to internal methods based on the rights and identity of the  
> user.  This is a really difficult problem
> to solve elegantly in ApacheDS as it stands today but it is a must.

I'm not sure exactly how 1 and 2 differ except in emphasis.  One  
possibility is to use something like JACC.... define java permissions  
for the actions and check them against a policy provider for an  
application context, using the users identity along with the info in  
the permission (such as sp name etc).  This might get circular seeing  
as how I'm thinking of implementing jacc through ldap.... but it  
still might be a useful approach.

david jencks

>
> 3). SP Invocation: Procedure Qualification
>
> Right now we have an ExtendedOperation used to invoke stored  
> procedures using qualified names based
> on the namespace of the procedure language.  This is a questionable  
> mechanism since it requires the user
> to know what the language of the implementation of the procedure is  
> and how to qualify the procedure in that
> language.  If a universal call specification is devised it can link  
> language independent invocation requests to
> the proper language specific method.  It would then be the job of  
> the writer to add this call spec and not the
> job of the user to track which language the stored procedure was  
> implemented in.
>
> 4). SP Invocation: Parameter Bindings
>
> This is connected to 3.  The same extended operation allows  
> invokers to provide parameters to the stored
> procedure.  It would not be fun to have SPs without parameters.   
> This opens up a slew of problems for us.
> First we need to define now a language independent means to specify  
> parameters.  ASN.1 does this nicely
> so we can leverage ASN.1 primitive types at first.  ApacheDS can  
> easily transform these types into the types
> expected by the implementation language of the SP.  This however  
> requires some primitive type bindings and
> the code to translate them.
>
> Right now the extended operation works with primitive Java types  
> and serialized objects (I think but Ersin can
> confirm).  This makes it so a C client cannot really invoke a SP  
> implemented in Java unless it knows how to
> serialize java objects into a byte buffer.  This is not really  
> feasible so we need to specify some type bindings
> and use that instead of serializing objects.  We cannot expect  
> invokers from python, groovy, php, c, perl etc
> to have to know anything about Java.  Invocation should not have be  
> language dependent to allow for maximum
> flexibility.
>
> I'm adding these details btw to a the first draft (version 00) of  
> an IETF draft submission.
>
> Alex
>
>
>
>
>
>
>