You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Rares Vernica <rv...@gmail.com> on 2008/04/10 00:32:27 UTC
set maximum file size for commits
Hello,
Is there a way to set the maximum file size for commits? That is, if I
set the maximum file size to 4Mb and a user tries to commit a file which
has 4.1Mb, the commit will fail.
Thanks,
Ray
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: set maximum file size for commits
Posted by km...@rockwellcollins.com.
eg <eg...@gmail.com> wrote on 04/10/2008 11:53:08 AM:
> kmradke@rockwellcollins.com wrote:
> > Is there an easy way to use the python or perl api to see a file sizes
> > in a transaction? That would at least remove the
> > "svnlook cat | wc -c" overhead...
> >
> > Maybe transaction/file size could be added to svnlook info?
> >
>
>
> I sent this link yesterday but it seems to be lost in the void
somewhere...
>
>
> See the following for a python pre-commit hook for an example:
>
> http://www.davidgrant.ca/limit_size_of_subversion_commits_with_this_hook
Well, that is interesting. Just check the transactions directory size.
Probably
wouldn't have thought of that. This limits to FSFS, but that is ok for
me...
Kevin R.
Re: set maximum file size for commits
Posted by eg <eg...@gmail.com>.
kmradke@rockwellcollins.com wrote:
>
> Is there an easy way to use the python or perl api to see a file sizes
> in a transaction? That would at least remove the
> "svnlook cat | wc -c" overhead...
>
> Maybe transaction/file size could be added to svnlook info?
>
I sent this link yesterday but it seems to be lost in the void somewhere...
See the following for a python pre-commit hook for an example:
http://www.davidgrant.ca/limit_size_of_subversion_commits_with_this_hook
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: set maximum file size for commits
Posted by "Mark E. Hamilton" <mh...@sandia.gov>.
Andy Levy wrote:
>> I was hoping someone had experimented with some Apache options to limit
>> transaction size. (Yet another thing on my todo list that I hope someone
>> else has more experience with...) Possibly something like
>> LimitRequestBody 4194304 ????
>
> In that scenario, consider that Subversion only transfers differences
> across the wire. So how would you handle a situation where a user
> commits at 2MB file, then appends 3MB to it and commits that? Their
> individual commits were both under 4MB, but the resulting file is now
> 5MB.
Also, not everyone uses http/https; some of us use svn+ssh. As I
understand it Apache isn't involved in those at all.
--
----------------
Mark E. Hamilton
Orion International Technologies, Inc.
Sandia National Laboratory, NM.
505-844-7666
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: set maximum file size for commits
Posted by Erik Huelsmann <eh...@gmail.com>.
> > > I was hoping someone had experimented with some Apache options to limit
> > > transaction size. (Yet another thing on my todo list that I hope
> someone
> > > else has more experience with...) Possibly something like
> > > LimitRequestBody 4194304 ????
> > >
> >
> > In that scenario, consider that Subversion only transfers differences
> > across the wire. So how would you handle a situation where a user
> > commits at 2MB file, then appends 3MB to it and commits that? Their
> > individual commits were both under 4MB, but the resulting file is now
> > 5MB.
> >
>
> I am using https and dav module and I am OK with the drawbacks of this
> solution. Limiting only the amount an user commits at one time is OK.
>
> I added
> LimitRequestBody 4194304
> in various places in my apache2.conf (server config, virtual host,
> location), but none of them seem to make a difference when I upload a file
> of 5Mb.
The request body is likely smaller than 4 MB when you commit a
compressable file of 5 MB. Try committing a file of 10, 15 or even
20MB when the request body is set to 4MB: Subversion compresses files
it sends to the repository.
> I also tried the use the XML version of the argument, that is
> LimitXMLRequestBody
> but the results are the same.
>
> If I lower the limit, things start to happen. I get a "413 Request Entity
> Too Large" error when I use:
> LimitRequestBody 437
> LimitXMLRequestBody 437
That would mean an extreme compression rate. Did you try to commit
something which compresses badly, like a TIFF or JPG file?
> Anything higher that that will work just fine. One thing to notice is that
> I use https.
>
> I appreciate any suggestions on how to track this down.
HTH,
Erik.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: set maximum file size for commits
Posted by Rares Vernica <rv...@gmail.com>.
Andy Levy wrote:
> On Thu, Apr 10, 2008 at 10:25 AM, <km...@rockwellcollins.com> wrote:
>> Blair Zajac <bl...@orcaware.com> wrote on 04/09/2008 10:39:46 PM:
>>
>>> Rares Vernica wrote:
>> > > Hello,
>> > >
>> > > Is there a way to set the maximum file size for commits? That is, if I
>>
>> > > set the maximum file size to 4Mb and a user tries to commit a file
>> which
>> > > has 4.1Mb, the commit will fail.
>> > >
>> > > Thanks,
>> > > Ray
>> >
>> > You can check this in a pre-commit script. You'll need to get a list of
>> all
>> > added or modified files using svnlook and then cat each one.
>>
>> Any other way? This seems a bit late in the game, and fairly inefficient.
>> For
>> example, we have a number of groups who have committed multiple DVD .iso
>> files
>> in one transaction.
>>
>> While I don't mind punishing them by making them wait to transfer 4G+ and
>> then
>> wait some more while the server cats the file to count the bytes, both of
>> those actions are needlessly resource wasteful...
>>
>> I was hoping someone had experimented with some Apache options to limit
>> transaction size. (Yet another thing on my todo list that I hope someone
>> else has more experience with...) Possibly something like
>> LimitRequestBody 4194304 ????
>
> In that scenario, consider that Subversion only transfers differences
> across the wire. So how would you handle a situation where a user
> commits at 2MB file, then appends 3MB to it and commits that? Their
> individual commits were both under 4MB, but the resulting file is now
> 5MB.
I am using https and dav module and I am OK with the drawbacks of this
solution. Limiting only the amount an user commits at one time is OK.
I added
LimitRequestBody 4194304
in various places in my apache2.conf (server config, virtual host,
location), but none of them seem to make a difference when I upload a
file of 5Mb.
I also tried the use the XML version of the argument, that is
LimitXMLRequestBody
but the results are the same.
If I lower the limit, things start to happen. I get a "413 Request
Entity Too Large" error when I use:
LimitRequestBody 437
LimitXMLRequestBody 437
Anything higher that that will work just fine. One thing to notice is
that I use https.
I appreciate any suggestions on how to track this down.
Thanks,
Ray
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: set maximum file size for commits
Posted by km...@rockwellcollins.com.
"Andy Levy" <an...@gmail.com> wrote on 04/10/2008 09:50:51 AM:
> On Thu, Apr 10, 2008 at 10:25 AM, <km...@rockwellcollins.com> wrote:
> > Blair Zajac <bl...@orcaware.com> wrote on 04/09/2008 10:39:46 PM:
> >
> > > Rares Vernica wrote:
> > > > Hello,
> > > >
> > > > Is there a way to set the maximum file size for commits? That is,
if I
> >
> > > > set the maximum file size to 4Mb and a user tries to commit a
file
> > which
> > > > has 4.1Mb, the commit will fail.
> > > >
> > > > Thanks,
> > > > Ray
> > >
> > > You can check this in a pre-commit script. You'll need to get a
list of
> > all
> > > added or modified files using svnlook and then cat each one.
> >
> > Any other way? This seems a bit late in the game, and fairly
inefficient.
> > For
> > example, we have a number of groups who have committed multiple DVD
.iso
> > files
> > in one transaction.
> >
> > While I don't mind punishing them by making them wait to transfer 4G+
and
> > then
> > wait some more while the server cats the file to count the bytes,
both of
> > those actions are needlessly resource wasteful...
> >
> > I was hoping someone had experimented with some Apache options to
limit
> > transaction size. (Yet another thing on my todo list that I hope
someone
> > else has more experience with...) Possibly something like
> > LimitRequestBody 4194304 ????
>
> In that scenario, consider that Subversion only transfers differences
> across the wire. So how would you handle a situation where a user
> commits at 2MB file, then appends 3MB to it and commits that? Their
> individual commits were both under 4MB, but the resulting file is now
> 5MB.
Yes, that is a flaw in using something like LimitRequestBody.
For my needs (which are different than the OP), I'm not so interesting
in limiting actual total file sizes, just wanting to remind users
that they may not be using Subversion in the most optimum
way. I.E. DVDs are best archived in other ways...
Is there an easy way to use the python or perl api to see a file sizes
in a transaction? That would at least remove the
"svnlook cat | wc -c" overhead...
Maybe transaction/file size could be added to svnlook info?
I'm just rambling thoughts...
Kevin R.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: set maximum file size for commits
Posted by Andy Levy <an...@gmail.com>.
On Thu, Apr 10, 2008 at 10:25 AM, <km...@rockwellcollins.com> wrote:
> Blair Zajac <bl...@orcaware.com> wrote on 04/09/2008 10:39:46 PM:
>
> > Rares Vernica wrote:
> > > Hello,
> > >
> > > Is there a way to set the maximum file size for commits? That is, if I
>
> > > set the maximum file size to 4Mb and a user tries to commit a file
> which
> > > has 4.1Mb, the commit will fail.
> > >
> > > Thanks,
> > > Ray
> >
> > You can check this in a pre-commit script. You'll need to get a list of
> all
> > added or modified files using svnlook and then cat each one.
>
> Any other way? This seems a bit late in the game, and fairly inefficient.
> For
> example, we have a number of groups who have committed multiple DVD .iso
> files
> in one transaction.
>
> While I don't mind punishing them by making them wait to transfer 4G+ and
> then
> wait some more while the server cats the file to count the bytes, both of
> those actions are needlessly resource wasteful...
>
> I was hoping someone had experimented with some Apache options to limit
> transaction size. (Yet another thing on my todo list that I hope someone
> else has more experience with...) Possibly something like
> LimitRequestBody 4194304 ????
In that scenario, consider that Subversion only transfers differences
across the wire. So how would you handle a situation where a user
commits at 2MB file, then appends 3MB to it and commits that? Their
individual commits were both under 4MB, but the resulting file is now
5MB.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: set maximum file size for commits
Posted by km...@rockwellcollins.com.
Blair Zajac <bl...@orcaware.com> wrote on 04/09/2008 10:39:46 PM:
> Rares Vernica wrote:
> > Hello,
> >
> > Is there a way to set the maximum file size for commits? That is, if I
> > set the maximum file size to 4Mb and a user tries to commit a file
which
> > has 4.1Mb, the commit will fail.
> >
> > Thanks,
> > Ray
>
> You can check this in a pre-commit script. You'll need to get a list of
all
> added or modified files using svnlook and then cat each one.
Any other way? This seems a bit late in the game, and fairly inefficient.
For
example, we have a number of groups who have committed multiple DVD .iso
files
in one transaction.
While I don't mind punishing them by making them wait to transfer 4G+ and
then
wait some more while the server cats the file to count the bytes, both of
those actions are needlessly resource wasteful...
I was hoping someone had experimented with some Apache options to limit
transaction size. (Yet another thing on my todo list that I hope someone
else has more experience with...) Possibly something like
LimitRequestBody 4194304 ????
Kevin R.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: set maximum file size for commits
Posted by Blair Zajac <bl...@orcaware.com>.
Rares Vernica wrote:
> Hello,
>
> Is there a way to set the maximum file size for commits? That is, if I
> set the maximum file size to 4Mb and a user tries to commit a file which
> has 4.1Mb, the commit will fail.
>
> Thanks,
> Ray
You can check this in a pre-commit script. You'll need to get a list of all
added or modified files using svnlook and then cat each one.
Regards,
Blair
--
Blair Zajac, Ph.D.
CTO, OrcaWare Technologies
<bl...@orcaware.com>
Subversion training, consulting and support
http://www.orcaware.com/svn/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org