You are viewing a plain text version of this content. The canonical link for it is here.
Posted to slide-dev@jakarta.apache.org by Jean-Philippe Courson <co...@noos.fr> on 2001/12/18 18:10:03 UTC
Quota soon supported !
Hi
I am currently implementing quota for Slide.
In the mailling list, I read that slide developpers were interrested in
including this feature into slide's kernel.
So I'm searching feedback before going further in my implementation to
be sure it feets slide's developpers needs and way of coding.
I began the implementation following advices Dirk made on an email dated
from august 2001 in which he was saying :
- the more generic way to integrate quota is to do it into
ContentInterceptor class
- user's quota and bytes used can be stored into protected properties
from user's node
- the algorithm could be resumed as this : in preStoreContent method
check if user has enougth space and put a lock on the user, in the
postStoreContent method update bytes used and unlock user's node.
- to deal with content removing, add a postRemoveContent method to the
ContentInterceptor class.
- what to do if clients don't send a contentLength ?
- to allow webdav methods to indicate webdav clients the reason why the
operation failed, ContentInterceptor class must throw a special
exception that has to be catched by webdav methods to send webdav
clients an insufficient storage error code (507)
So here is the begin of an implementation that takes in account (almost)
all these advices.
I have added to package content a new Exception called
InsufficientStorageException
I think that the best place to add a parameter to enable quota is in the
<configuration> section of the domain.xml file.
This one is read by the NamespaceConfig class that instanciates the
ContentInterceptors.
I've added to the ContentInterceptor class a new contructor that has to
be called by NamespaceConfig class if Domain.xml file tells to enable quota.
Some questions I am asking myself :
Why is there an array of contentInterceptor instances into
NamespaceConfig class rather than a unique instance ?
Some ContentInterceptor methods need a contentHelper and/or a lockHelper
to do their job. What would be the best : to give them to constructor to
initialize private properties or to give them as method argument ?
Is postStoreContent method called if an exception has been raised by
preContentStore method or someone else ?
Which subject, type and expiration use to lock the user's node ? I think
that with this token we can only use the user as subject but if the lock
is put by the user, it can be kill by the user, isn't it ?
With a lock on the user's node, user won't be able to perform any other
operation ?
What could be done with webdav clients that don't send a contentLength
header ?
What could be done better than it is in my code ?
All remarks are welcome
JP
Re: Quota soon supported !
Posted by Remy Maucherat <re...@apache.org>.
> Hi JP
>
> sorry for the lare reply
> see comments inline
>
>
> Jean-Philippe Courson wrote:
> >
> > We first have to establish a road map : I think that we could first
> > integrate
> > user's quota into branch 1.0 and to deal with versioning and properties
move
> > to the structure later.
> >
> > Others issues are : plug quota correctly, transactions, forbid users
> > to remove their node locks and deal without contentLengths.
> >
> > 1) Plug quota correctly :
> > Ok to make ContentInterceptor abstract and for
UserQuotaContentInterceptor,
> > but what are you meaning by pluggable Interceptors ?
> > Which is the way they should be plugged ?
>
> Today the interceptors can be configured in the domain.xml file
> (NamespaceConfig.java:834)
> UserQuotaContentInterceptor should follow the same principle.
I plan to commit the interceptor initial code ASAP, son that we can work on
further improvements. The name you suggest looks good :)
Remy
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Quota soon supported !
Posted by Dirk Verbeeck <di...@pandora.be>.
Hi JP
sorry for the lare reply
see comments inline
Jean-Philippe Courson wrote:
>
> We first have to establish a road map : I think that we could first
> integrate
> user's quota into branch 1.0 and to deal with versioning and properties move
> to the structure later.
>
> Others issues are : plug quota correctly, transactions, forbid users
> to remove their node locks and deal without contentLengths.
>
> 1) Plug quota correctly :
> Ok to make ContentInterceptor abstract and for UserQuotaContentInterceptor,
> but what are you meaning by pluggable Interceptors ?
> Which is the way they should be plugged ?
Today the interceptors can be configured in the domain.xml file
(NamespaceConfig.java:834)
UserQuotaContentInterceptor should follow the same principle.
> 2) Transactions :
> I first need to have a look at transactional code.
> Basicaly, how rollbacks are performed ?
> Transactions scheme in the doc indicates that helpers can act with stores
> without using the transaction manager. Is it rigth ? If yes in which cases ?
For read only operations you don't have to start a transaction.
You can do write operations as well, you get only a warning not an error
but you better don't.
The behaviour is undefined, the store can revert your changes at any
time, store all you changes or silently ignore everything.
> 3) forbid users to remove their node locks :
> This can be done now with permissions.
> But how would internal locks be placed ? Using which credentials and
> sort of token ?
(from the second mail)
> Sorry for the mistake : forbid users to remove their node locks can't
> be done with permissions. Some internal locks placed using root
> credential or something like that need to be done.
>
> But performing all actions in code using a token with user's credential
> is a very good concept for security.
>
> What can we do ?
The lock on the user node is not really an absolute requirement.
In the postStoreContent, you can check the quota again and if the user
has gone over his limit reject and rollback. Then there is only some
waste of CPU and bandwith.
> 4) Null contentLength headers :
> Are there webdav clients that don't send them ?
> Allowing such requests processing is a big DOS attack risk.
Make it a configurable option, there will always be clients that
misbehave
allow content with unknown size = true/false
>
> JP
Cheers
Dirk
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Quota soon supported !
Posted by Jean-Philippe Courson <co...@noos.fr>.
> 3) forbid users to remove their node locks :
> This can be done now with permissions.
> But how would internal locks be placed ? Using which credentials and
> sort of token ?
Sorry for the mistake : forbid users to remove their node locks can't
be done with permissions. Some internal locks placed using root
credential or something like that need to be done.
But performing all actions in code using a token with user's credential
is a very good concept for security.
What can we do ?
JP
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Quota soon supported !
Posted by Jean-Philippe Courson <co...@noos.fr>.
We first have to establish a road map : I think that we could first
integrate
user's quota into branch 1.0 and to deal with versioning and properties move
to the structure later.
Others issues are : plug quota correctly, transactions, forbid users
to remove their node locks and deal without contentLengths.
1) Plug quota correctly :
Ok to make ContentInterceptor abstract and for UserQuotaContentInterceptor,
but what are you meaning by pluggable Interceptors ?
Which is the way they should be plugged ?
2) Transactions :
I first need to have a look at transactional code.
Basicaly, how rollbacks are performed ?
Transactions scheme in the doc indicates that helpers can act with stores
without using the transaction manager. Is it rigth ? If yes in which cases ?
3) forbid users to remove their node locks :
This can be done now with permissions.
But how would internal locks be placed ? Using which credentials and
sort of token ?
4) Null contentLength headers :
Are there webdav clients that don't send them ?
Allowing such requests processing is a big DOS attack risk.
JP
> > Hi
> >
> > I am currently implementing quota for Slide.
>
> That's something which got a lot of feature requests.
>
> > In the mailling list, I read that slide developpers were interrested in
> > including this feature into slide's kernel.
> >
> > So I'm searching feedback before going further in my implementation to
> > be sure it feets slide's developpers needs and way of coding.
> >
> > I began the implementation following advices Dirk made on an email dated
> > from august 2001 in which he was saying :
> > - the more generic way to integrate quota is to do it into
> > ContentInterceptor class
>
> We can do that at first. The best would be to then move it to a separate
> ContentInterceptor (which is supposed to be a base class) after the code
> needed to add the interceptors through the config files are done. Maybe we
> could name it UserQuotaContentInterceptor.
>
> > - user's quota and bytes used can be stored into protected properties
> > from user's node
>
> That should protect it from modifying it thought a PROPPATCH.
> In the future (Slide 2.0), I think the quota should be treated like
> ownership, and move to the structure, but it's not urgent.
>
> > - the algorithm could be resumed as this : in preStoreContent method
> > check if user has enougth space and put a lock on the user, in the
> > postStoreContent method update bytes used and unlock user's node.
>
> Yes. The only problems are with transactions (if it gets rolledback).
Also,
> how is it supposed to behave when versioning is used ?
>
> > - to deal with content removing, add a postRemoveContent method to the
> > ContentInterceptor class.
>
> +1.
>
> > - what to do if clients don't send a contentLength ?
>
> After returning from the store, the content length in the descriptors
should
> have been updated if it was unknown (more below).
>
> > - to allow webdav methods to indicate webdav clients the reason why the
> > operation failed, ContentInterceptor class must throw a special
> > exception that has to be catched by webdav methods to send webdav
> > clients an insufficient storage error code (507)
>
> We can probably subclass ContentException, as you apparently did, and add
> some special handling in the WebDAV layer.
>
> > So here is the begin of an implementation that takes in account (almost)
> > all these advices.
> >
> > I have added to package content a new Exception called
> > InsufficientStorageException
> >
> > I think that the best place to add a parameter to enable quota is in the
> > <configuration> section of the domain.xml file.
> > This one is read by the NamespaceConfig class that instanciates the
> > ContentInterceptors.
>
> Yes, except that it should be a separate implementation of
> ContentInterceptor (which could be made abstract to make that obvious).
>
> > I've added to the ContentInterceptor class a new contructor that has to
> > be called by NamespaceConfig class if Domain.xml file tells to enable
> quota.
> >
> > Some questions I am asking myself :
> >
> > Why is there an array of contentInterceptor instances into
> > NamespaceConfig class rather than a unique instance ?
>
> I thought it could be useful to associate multiple interceptors (it's
> supposed to be pluggable) ?
>
> > Some ContentInterceptor methods need a contentHelper and/or a lockHelper
> > to do their job. What would be the best : to give them to constructor to
> > initialize private properties or to give them as method argument ?
> >
> > Is postStoreContent method called if an exception has been raised by
> > preContentStore method or someone else ?
>
> I don't think so. The code for calling the interceptors is in the content
> helper.
>
> > Which subject, type and expiration use to lock the user's node ? I think
> > that with this token we can only use the user as subject but if the lock
> > is put by the user, it can be kill by the user, isn't it ?
> >
> > With a lock on the user's node, user won't be able to perform any other
> > operation ?
>
> That's likely. The Slide locks aren't fine grained locks, they're designed
> to be user side locks; we need internal locks here (which was one of the
> major new ideas I was considering in the new core proposal I talked about
in
> the past).
>
> > What could be done with webdav clients that don't send a contentLength
> > header ?
>
> In the preStore, you don't know the contentLength, so you either have to
> reject the request or allow it and see what happens. In the postStore, the
> content length will be set, but unfortunately, that is after the fact.
> The only solution I can think of is to buffer the content to a temp file
if
> the content length is unknown :-(
>
> Remy
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Quota soon supported !
Posted by Remy Maucherat <re...@apache.org>.
> Hi
>
> I am currently implementing quota for Slide.
That's something which got a lot of feature requests.
> In the mailling list, I read that slide developpers were interrested in
> including this feature into slide's kernel.
>
> So I'm searching feedback before going further in my implementation to
> be sure it feets slide's developpers needs and way of coding.
>
> I began the implementation following advices Dirk made on an email dated
> from august 2001 in which he was saying :
> - the more generic way to integrate quota is to do it into
> ContentInterceptor class
We can do that at first. The best would be to then move it to a separate
ContentInterceptor (which is supposed to be a base class) after the code
needed to add the interceptors through the config files are done. Maybe we
could name it UserQuotaContentInterceptor.
> - user's quota and bytes used can be stored into protected properties
> from user's node
That should protect it from modifying it thought a PROPPATCH.
In the future (Slide 2.0), I think the quota should be treated like
ownership, and move to the structure, but it's not urgent.
> - the algorithm could be resumed as this : in preStoreContent method
> check if user has enougth space and put a lock on the user, in the
> postStoreContent method update bytes used and unlock user's node.
Yes. The only problems are with transactions (if it gets rolledback). Also,
how is it supposed to behave when versioning is used ?
> - to deal with content removing, add a postRemoveContent method to the
> ContentInterceptor class.
+1.
> - what to do if clients don't send a contentLength ?
After returning from the store, the content length in the descriptors should
have been updated if it was unknown (more below).
> - to allow webdav methods to indicate webdav clients the reason why the
> operation failed, ContentInterceptor class must throw a special
> exception that has to be catched by webdav methods to send webdav
> clients an insufficient storage error code (507)
We can probably subclass ContentException, as you apparently did, and add
some special handling in the WebDAV layer.
> So here is the begin of an implementation that takes in account (almost)
> all these advices.
>
> I have added to package content a new Exception called
> InsufficientStorageException
>
> I think that the best place to add a parameter to enable quota is in the
> <configuration> section of the domain.xml file.
> This one is read by the NamespaceConfig class that instanciates the
> ContentInterceptors.
Yes, except that it should be a separate implementation of
ContentInterceptor (which could be made abstract to make that obvious).
> I've added to the ContentInterceptor class a new contructor that has to
> be called by NamespaceConfig class if Domain.xml file tells to enable
quota.
>
> Some questions I am asking myself :
>
> Why is there an array of contentInterceptor instances into
> NamespaceConfig class rather than a unique instance ?
I thought it could be useful to associate multiple interceptors (it's
supposed to be pluggable) ?
> Some ContentInterceptor methods need a contentHelper and/or a lockHelper
> to do their job. What would be the best : to give them to constructor to
> initialize private properties or to give them as method argument ?
>
> Is postStoreContent method called if an exception has been raised by
> preContentStore method or someone else ?
I don't think so. The code for calling the interceptors is in the content
helper.
> Which subject, type and expiration use to lock the user's node ? I think
> that with this token we can only use the user as subject but if the lock
> is put by the user, it can be kill by the user, isn't it ?
>
> With a lock on the user's node, user won't be able to perform any other
> operation ?
That's likely. The Slide locks aren't fine grained locks, they're designed
to be user side locks; we need internal locks here (which was one of the
major new ideas I was considering in the new core proposal I talked about in
the past).
> What could be done with webdav clients that don't send a contentLength
> header ?
In the preStore, you don't know the contentLength, so you either have to
reject the request or allow it and see what happens. In the postStore, the
content length will be set, but unfortunately, that is after the fact.
The only solution I can think of is to buffer the content to a temp file if
the content length is unknown :-(
Remy
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>