You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Justin Johnson <ju...@gmail.com> on 2007/11/06 20:44:29 UTC
Limit to commit size?
Hi,
I have an AIX 5.3 server running Apache 2.2 as the server for
Subversion 1.4.0. The other day one of my users performed an import
of a > 7 GB directory into the repository. It completed in a couple
hours with no error, but only added a little over 1.5 GB to the
repository, and didn't provide any error message indicating any
problems. I'm still doing a comparison to see what wasn't added.
Has anyone else seen this sort of behavior? Do I need to change some
configuration setting in Apache?
Thanks,
Justin
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Limit to commit size?
Posted by Jeremy Pereira <je...@jeremyp.net>.
On 6 Nov 2007, at 21:16, Justin Johnson wrote:
>
> My export completed. TortoiseSVN reported 1886.60 MB transferred and
> 1022 files/directories added. Windows reports a directory size of
> 7.04 GB.
It could be quite simply that the files in your export are mostly very
small. Even a one byte file needs 4Kb disk space when stored on an
NTFS partition because that is the size of a block on disk.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Limit to commit size?
Posted by Justin Johnson <ju...@gmail.com>.
On 11/6/07, Justin Johnson <ju...@gmail.com> wrote:
> On 11/6/07, Mark Phippard <ma...@gmail.com> wrote:
> > On 11/6/07, Justin Johnson <ju...@gmail.com> wrote:
> > > On 11/6/07, Andy Levy <an...@gmail.com> wrote:
> > > > On Nov 6, 2007 3:55 PM, Justin Johnson <ju...@gmail.com> wrote:
> > > > > On 11/6/07, Mark Phippard <ma...@gmail.com> wrote:
> > > > > > On 11/6/07, Justin Johnson <ju...@gmail.com> wrote:
> > > > > > > I have an AIX 5.3 server running Apache 2.2 as the server for
> > > > > > > Subversion 1.4.0. The other day one of my users performed an import
> > > > > > > of a > 7 GB directory into the repository. It completed in a couple
> > > > > > > hours with no error, but only added a little over 1.5 GB to the
> > > > > > > repository, and didn't provide any error message indicating any
> > > > > > > problems. I'm still doing a comparison to see what wasn't added.
> > > > > > >
> > > > > > > Has anyone else seen this sort of behavior? Do I need to change some
> > > > > > > configuration setting in Apache?
> > > > > >
> > > > > > If it did not commit everything you should have got an error.
> > > > > > Subversion uses a delta algorithm that can shrink the size of what was
> > > > > > committed. What does this directory gzip to?
> > > > > >
> > > > > > I guess my point is that it is possible, perhaps even likely, that
> > > > > > nothing was lost or missing.
> > > > > >
> > > > > > svn log -v should tell you the paths that were committed.
> > > > >
> > > > > The user also checked out a working copy of the repository and the
> > > > > size reported by TortoiseSVN after checkout was 1751.35 MB. I'm
> > > > > running an export right now and will check the size when done. I
> > > > > don't have access to the original directory structure to compare,
> > > > > though I confirmed via a remote control session that the size was > 7
> > > > > GB and that the top level directory structure matches what was
> > > > > imported. I'll have the user send me a recursive listing of files
> > > > > from the original directory so I can compare with svn log output.
> > > >
> > > > Is it possible that portions of what's contained in the directory he
> > > > imported would have been caught by svn:ignore or his global-ignores
> > > > settings, resulting in large amounts of data not being imported?
> > >
> > > No, this doesn't have anything to do with svn:ignore.
> > >
> > > TortoiseSVN is reporting the amount of data transported, which happens
> > > to be significantly less than the size of the data once sitting
> > > locally on the user's computer (apparently due to Subversion's
> > > efficient storage?). I'm doing an export at the moment and
> > > TortoiseSVN is reporting 1662 MB transferred, but the directory is
> > > already > 6 GB. Does that sound right to everyone? I assumed the
> > > efficient storage only came into play when there were changes in the
> > > repository (i.e. when the efficient binary *delta* was being
> > > utilized).
> >
> > TortoiseSVN reports the size of the data that comes over the wire.
> > The data is in svndiff1 format plus might also have gzip compression
> > if you are using Apache and mod_deflate. Likewise, if you are using
> > SSL then the encryption factors in (probably increases the size).
>
> We are not using mod_deflate or SSL. Would svndiff1 format be
> compressed in this case?
My export completed. TortoiseSVN reported 1886.60 MB transferred and
1022 files/directories added. Windows reports a directory size of
7.04 GB.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Limit to commit size?
Posted by Justin Johnson <ju...@gmail.com>.
On 11/6/07, Mark Phippard <ma...@gmail.com> wrote:
> On 11/6/07, Justin Johnson <ju...@gmail.com> wrote:
> > On 11/6/07, Andy Levy <an...@gmail.com> wrote:
> > > On Nov 6, 2007 3:55 PM, Justin Johnson <ju...@gmail.com> wrote:
> > > > On 11/6/07, Mark Phippard <ma...@gmail.com> wrote:
> > > > > On 11/6/07, Justin Johnson <ju...@gmail.com> wrote:
> > > > > > I have an AIX 5.3 server running Apache 2.2 as the server for
> > > > > > Subversion 1.4.0. The other day one of my users performed an import
> > > > > > of a > 7 GB directory into the repository. It completed in a couple
> > > > > > hours with no error, but only added a little over 1.5 GB to the
> > > > > > repository, and didn't provide any error message indicating any
> > > > > > problems. I'm still doing a comparison to see what wasn't added.
> > > > > >
> > > > > > Has anyone else seen this sort of behavior? Do I need to change some
> > > > > > configuration setting in Apache?
> > > > >
> > > > > If it did not commit everything you should have got an error.
> > > > > Subversion uses a delta algorithm that can shrink the size of what was
> > > > > committed. What does this directory gzip to?
> > > > >
> > > > > I guess my point is that it is possible, perhaps even likely, that
> > > > > nothing was lost or missing.
> > > > >
> > > > > svn log -v should tell you the paths that were committed.
> > > >
> > > > The user also checked out a working copy of the repository and the
> > > > size reported by TortoiseSVN after checkout was 1751.35 MB. I'm
> > > > running an export right now and will check the size when done. I
> > > > don't have access to the original directory structure to compare,
> > > > though I confirmed via a remote control session that the size was > 7
> > > > GB and that the top level directory structure matches what was
> > > > imported. I'll have the user send me a recursive listing of files
> > > > from the original directory so I can compare with svn log output.
> > >
> > > Is it possible that portions of what's contained in the directory he
> > > imported would have been caught by svn:ignore or his global-ignores
> > > settings, resulting in large amounts of data not being imported?
> >
> > No, this doesn't have anything to do with svn:ignore.
> >
> > TortoiseSVN is reporting the amount of data transported, which happens
> > to be significantly less than the size of the data once sitting
> > locally on the user's computer (apparently due to Subversion's
> > efficient storage?). I'm doing an export at the moment and
> > TortoiseSVN is reporting 1662 MB transferred, but the directory is
> > already > 6 GB. Does that sound right to everyone? I assumed the
> > efficient storage only came into play when there were changes in the
> > repository (i.e. when the efficient binary *delta* was being
> > utilized).
>
> TortoiseSVN reports the size of the data that comes over the wire.
> The data is in svndiff1 format plus might also have gzip compression
> if you are using Apache and mod_deflate. Likewise, if you are using
> SSL then the encryption factors in (probably increases the size).
We are not using mod_deflate or SSL. Would svndiff1 format be
compressed in this case?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Limit to commit size?
Posted by Mark Phippard <ma...@gmail.com>.
On 11/6/07, Justin Johnson <ju...@gmail.com> wrote:
> On 11/6/07, Andy Levy <an...@gmail.com> wrote:
> > On Nov 6, 2007 3:55 PM, Justin Johnson <ju...@gmail.com> wrote:
> > > On 11/6/07, Mark Phippard <ma...@gmail.com> wrote:
> > > > On 11/6/07, Justin Johnson <ju...@gmail.com> wrote:
> > > > > I have an AIX 5.3 server running Apache 2.2 as the server for
> > > > > Subversion 1.4.0. The other day one of my users performed an import
> > > > > of a > 7 GB directory into the repository. It completed in a couple
> > > > > hours with no error, but only added a little over 1.5 GB to the
> > > > > repository, and didn't provide any error message indicating any
> > > > > problems. I'm still doing a comparison to see what wasn't added.
> > > > >
> > > > > Has anyone else seen this sort of behavior? Do I need to change some
> > > > > configuration setting in Apache?
> > > >
> > > > If it did not commit everything you should have got an error.
> > > > Subversion uses a delta algorithm that can shrink the size of what was
> > > > committed. What does this directory gzip to?
> > > >
> > > > I guess my point is that it is possible, perhaps even likely, that
> > > > nothing was lost or missing.
> > > >
> > > > svn log -v should tell you the paths that were committed.
> > >
> > > The user also checked out a working copy of the repository and the
> > > size reported by TortoiseSVN after checkout was 1751.35 MB. I'm
> > > running an export right now and will check the size when done. I
> > > don't have access to the original directory structure to compare,
> > > though I confirmed via a remote control session that the size was > 7
> > > GB and that the top level directory structure matches what was
> > > imported. I'll have the user send me a recursive listing of files
> > > from the original directory so I can compare with svn log output.
> >
> > Is it possible that portions of what's contained in the directory he
> > imported would have been caught by svn:ignore or his global-ignores
> > settings, resulting in large amounts of data not being imported?
>
> No, this doesn't have anything to do with svn:ignore.
>
> TortoiseSVN is reporting the amount of data transported, which happens
> to be significantly less than the size of the data once sitting
> locally on the user's computer (apparently due to Subversion's
> efficient storage?). I'm doing an export at the moment and
> TortoiseSVN is reporting 1662 MB transferred, but the directory is
> already > 6 GB. Does that sound right to everyone? I assumed the
> efficient storage only came into play when there were changes in the
> repository (i.e. when the efficient binary *delta* was being
> utilized).
TortoiseSVN reports the size of the data that comes over the wire.
The data is in svndiff1 format plus might also have gzip compression
if you are using Apache and mod_deflate. Likewise, if you are using
SSL then the encryption factors in (probably increases the size).
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Limit to commit size?
Posted by Justin Johnson <ju...@gmail.com>.
On 11/6/07, Andy Levy <an...@gmail.com> wrote:
> On Nov 6, 2007 3:55 PM, Justin Johnson <ju...@gmail.com> wrote:
> > On 11/6/07, Mark Phippard <ma...@gmail.com> wrote:
> > > On 11/6/07, Justin Johnson <ju...@gmail.com> wrote:
> > > > I have an AIX 5.3 server running Apache 2.2 as the server for
> > > > Subversion 1.4.0. The other day one of my users performed an import
> > > > of a > 7 GB directory into the repository. It completed in a couple
> > > > hours with no error, but only added a little over 1.5 GB to the
> > > > repository, and didn't provide any error message indicating any
> > > > problems. I'm still doing a comparison to see what wasn't added.
> > > >
> > > > Has anyone else seen this sort of behavior? Do I need to change some
> > > > configuration setting in Apache?
> > >
> > > If it did not commit everything you should have got an error.
> > > Subversion uses a delta algorithm that can shrink the size of what was
> > > committed. What does this directory gzip to?
> > >
> > > I guess my point is that it is possible, perhaps even likely, that
> > > nothing was lost or missing.
> > >
> > > svn log -v should tell you the paths that were committed.
> >
> > The user also checked out a working copy of the repository and the
> > size reported by TortoiseSVN after checkout was 1751.35 MB. I'm
> > running an export right now and will check the size when done. I
> > don't have access to the original directory structure to compare,
> > though I confirmed via a remote control session that the size was > 7
> > GB and that the top level directory structure matches what was
> > imported. I'll have the user send me a recursive listing of files
> > from the original directory so I can compare with svn log output.
>
> Is it possible that portions of what's contained in the directory he
> imported would have been caught by svn:ignore or his global-ignores
> settings, resulting in large amounts of data not being imported?
No, this doesn't have anything to do with svn:ignore.
TortoiseSVN is reporting the amount of data transported, which happens
to be significantly less than the size of the data once sitting
locally on the user's computer (apparently due to Subversion's
efficient storage?). I'm doing an export at the moment and
TortoiseSVN is reporting 1662 MB transferred, but the directory is
already > 6 GB. Does that sound right to everyone? I assumed the
efficient storage only came into play when there were changes in the
repository (i.e. when the efficient binary *delta* was being
utilized).
Thanks,
Justin
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Limit to commit size?
Posted by Andy Levy <an...@gmail.com>.
On Nov 6, 2007 3:55 PM, Justin Johnson <ju...@gmail.com> wrote:
> On 11/6/07, Mark Phippard <ma...@gmail.com> wrote:
> > On 11/6/07, Justin Johnson <ju...@gmail.com> wrote:
> > > I have an AIX 5.3 server running Apache 2.2 as the server for
> > > Subversion 1.4.0. The other day one of my users performed an import
> > > of a > 7 GB directory into the repository. It completed in a couple
> > > hours with no error, but only added a little over 1.5 GB to the
> > > repository, and didn't provide any error message indicating any
> > > problems. I'm still doing a comparison to see what wasn't added.
> > >
> > > Has anyone else seen this sort of behavior? Do I need to change some
> > > configuration setting in Apache?
> >
> > If it did not commit everything you should have got an error.
> > Subversion uses a delta algorithm that can shrink the size of what was
> > committed. What does this directory gzip to?
> >
> > I guess my point is that it is possible, perhaps even likely, that
> > nothing was lost or missing.
> >
> > svn log -v should tell you the paths that were committed.
>
> The user also checked out a working copy of the repository and the
> size reported by TortoiseSVN after checkout was 1751.35 MB. I'm
> running an export right now and will check the size when done. I
> don't have access to the original directory structure to compare,
> though I confirmed via a remote control session that the size was > 7
> GB and that the top level directory structure matches what was
> imported. I'll have the user send me a recursive listing of files
> from the original directory so I can compare with svn log output.
Is it possible that portions of what's contained in the directory he
imported would have been caught by svn:ignore or his global-ignores
settings, resulting in large amounts of data not being imported?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Limit to commit size?
Posted by Justin Johnson <ju...@gmail.com>.
On 11/6/07, Mark Phippard <ma...@gmail.com> wrote:
> On 11/6/07, Justin Johnson <ju...@gmail.com> wrote:
> > I have an AIX 5.3 server running Apache 2.2 as the server for
> > Subversion 1.4.0. The other day one of my users performed an import
> > of a > 7 GB directory into the repository. It completed in a couple
> > hours with no error, but only added a little over 1.5 GB to the
> > repository, and didn't provide any error message indicating any
> > problems. I'm still doing a comparison to see what wasn't added.
> >
> > Has anyone else seen this sort of behavior? Do I need to change some
> > configuration setting in Apache?
>
> If it did not commit everything you should have got an error.
> Subversion uses a delta algorithm that can shrink the size of what was
> committed. What does this directory gzip to?
>
> I guess my point is that it is possible, perhaps even likely, that
> nothing was lost or missing.
>
> svn log -v should tell you the paths that were committed.
The user also checked out a working copy of the repository and the
size reported by TortoiseSVN after checkout was 1751.35 MB. I'm
running an export right now and will check the size when done. I
don't have access to the original directory structure to compare,
though I confirmed via a remote control session that the size was > 7
GB and that the top level directory structure matches what was
imported. I'll have the user send me a recursive listing of files
from the original directory so I can compare with svn log output.
Justin
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Limit to commit size?
Posted by Mark Phippard <ma...@gmail.com>.
On 11/6/07, Justin Johnson <ju...@gmail.com> wrote:
> I have an AIX 5.3 server running Apache 2.2 as the server for
> Subversion 1.4.0. The other day one of my users performed an import
> of a > 7 GB directory into the repository. It completed in a couple
> hours with no error, but only added a little over 1.5 GB to the
> repository, and didn't provide any error message indicating any
> problems. I'm still doing a comparison to see what wasn't added.
>
> Has anyone else seen this sort of behavior? Do I need to change some
> configuration setting in Apache?
If it did not commit everything you should have got an error.
Subversion uses a delta algorithm that can shrink the size of what was
committed. What does this directory gzip to?
I guess my point is that it is possible, perhaps even likely, that
nothing was lost or missing.
svn log -v should tell you the paths that were committed.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org