You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Daniel Shahaf <d....@daniel.shahaf.name> on 2012/05/01 18:43:32 UTC
Re: svn ci performance issue with 1.7.x and nfs mounted working
copies
Moe, Mark wrote on Tue, May 01, 2012 at 10:33:36 -0500:
> Yes, I tried it and it was very effective! See below:
>
> 1.6.17 1.7.4 1.7.4-patched
> NFS benchmark* 1534s 4074s 572s
> Local benchmark* 365s 162s 118s
> NFS svn co 29s 163s 18s
> Local svn co 4s 4s 3s
>
> Are there any downsides to this patch? It sounds like the only downside is a delay to simultaneous read operations for the same working-copy. Those operations are very rare for compared to the huge gains in NFS working-copy performance.
>
> Thanks for figuring out this patch!
What patch are you talking about? I don't recall seeing a patch on list
Re: svn ci performance issue with 1.7.x and nfs mounted working copies
Posted by Johan Corveleyn <jc...@gmail.com>.
On Tue, May 15, 2012 at 7:55 PM, Moe, Mark <ma...@medtronic.com> wrote:
>> I rewrote some parts of the commit processing on trunk last week, which
>> should have a positive effect on the use cases reported in this thread.
>
>> Is it possible for somebody to see what the performance difference on NFS
>> is?
>
> Does this include Phillip's patch (http://subversion.tigris.org/issues/show_bug.cgi?id=4176) or is this a similar performance improvement not related to that patch?
>
> Please excuse my question; I'm new to testing the source: if I compile the trunk (http://svn.apache.org/repos/asf/subversion/trunk/), then am I getting a lot of changes compared with the release v1.7.4? I'm asking because I wonder if it would be better to test Bert's changes directly applied to v1.7.4 (with or without Philip's patch for bug 4176) for comparison sake.
>
I don't know the extent of Bert's changes in this area, but it's
likely that it will not be easy to backport that to the 1.7.x
codebase. But if it is backportable, sure, that would be a good
comparison ...
Another possible useful comparison could be to compare "trunk" with
"trunk-without-Bert's-changes".
> [CONFIDENTIALITY AND PRIVACY NOTICE]
[snip]
Please do not send such legal boilerplate, it doesn't make any sense
on a public (and archived) mailinglist. If you're forced to use it by
your company, you should probably get and use another email address
for these mailinglists ...
--
Johan
RE: svn ci performance issue with 1.7.x and nfs mounted working
copies
Posted by "Moe, Mark" <ma...@medtronic.com>.
> I rewrote some parts of the commit processing on trunk last week, which
> should have a positive effect on the use cases reported in this thread.
> Is it possible for somebody to see what the performance difference on NFS
> is?
Does this include Phillip's patch (http://subversion.tigris.org/issues/show_bug.cgi?id=4176) or is this a similar performance improvement not related to that patch?
Please excuse my question; I'm new to testing the source: if I compile the trunk (http://svn.apache.org/repos/asf/subversion/trunk/), then am I getting a lot of changes compared with the release v1.7.4? I'm asking because I wonder if it would be better to test Bert's changes directly applied to v1.7.4 (with or without Philip's patch for bug 4176) for comparison sake.
Thanks,
- Mark
[CONFIDENTIALITY AND PRIVACY NOTICE]
Information transmitted by this email is proprietary to Medtronic and is intended for use only by the individual or entity to which it is addressed, and may contain information that is private, privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly prohibited. In such cases, please delete this mail from your records.
To view this notice in other languages you can either select the following link or manually copy and paste the link into the address bar of a web browser: http://emaildisclaimer.medtronic.com
RE: svn ci performance issue with 1.7.x and nfs mounted working copies
Posted by Bert Huijben <be...@qqmail.nl>.
> -----Original Message-----
> From: Greg Stein [mailto:gstein@gmail.com]
> Sent: vrijdag 4 mei 2012 21:15
> To: Moe, Mark
> Cc: Philip Martin; dev@subversion.apache.org; Braun, Eric
> Subject: Re: svn ci performance issue with 1.7.x and nfs mounted working
> copies
>
> Well... your interest has already been noted :-) Maybe you're asking
> how you can help get the bug fixed? Volunteering, of course (as we all
> are).
>
> Philip noted a couple problems in the bug report. Focusing on how to
> resolve those is likely the start of the work. I imagine at least two
> approaches: ensure the same underlying wc_db instance is used in both
> cases, or that we have a way to open non-exclusive in limited cases.
>
> For example, with the latter approach: maybe when client context is
> created, it can use the default exclusive case, and certain clients
> can specify non-exclusive access (as a GUI may choose to do). Of
> course, then we get into problems with that one client that defaults
> to exclusive and tries to manipulate a working copy being
> non-exclusively accessed by that GUI. ... etc :-)
>
> If you're not talking about digging into the code yourself, then... I
> dunno. Maybe one of the Subversion vendors is willing to do spot fixes
> for a fee? Outside of that, it is tough to know when somebody will
> want to dig into the problem :-/
I rewrote some parts of the commit processing on trunk last week, which
should have a positive effect on the use cases reported in this thread.
Is it possible for somebody to see what the performance difference on NFS
is?
I'm using Windows for most of my development and testing, so I can't measure
the impact on NFS. I do see a huge improvement on Windows: The commit
harvesting time is now in the same region as the svn status time for me.
Bert
RE: svn ci performance issue with 1.7.x and nfs mounted working
copies
Posted by "Moe, Mark" <ma...@medtronic.com>.
> Well... your interest has already been noted :-) Maybe you're asking
> how you can help get the bug fixed? Volunteering, of course (as we all
> are).
...
> If you're not talking about digging into the code yourself, then... I
> dunno. Maybe one of the Subversion vendors is willing to do spot fixes
> for a fee? Outside of that, it is tough to know when somebody will
> want to dig into the problem :-/
Ok, thanks Greg. Make sense.
- Mark
[CONFIDENTIALITY AND PRIVACY NOTICE]
Information transmitted by this email is proprietary to Medtronic and is intended for use only by the individual or entity to which it is addressed, and may contain information that is private, privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly prohibited. In such cases, please delete this mail from your records.
To view this notice in other languages you can either select the following link or manually copy and paste the link into the address bar of a web browser: http://emaildisclaimer.medtronic.com
Re: svn ci performance issue with 1.7.x and nfs mounted working copies
Posted by Greg Stein <gs...@gmail.com>.
Well... your interest has already been noted :-) Maybe you're asking
how you can help get the bug fixed? Volunteering, of course (as we all
are).
Philip noted a couple problems in the bug report. Focusing on how to
resolve those is likely the start of the work. I imagine at least two
approaches: ensure the same underlying wc_db instance is used in both
cases, or that we have a way to open non-exclusive in limited cases.
For example, with the latter approach: maybe when client context is
created, it can use the default exclusive case, and certain clients
can specify non-exclusive access (as a GUI may choose to do). Of
course, then we get into problems with that one client that defaults
to exclusive and tries to manipulate a working copy being
non-exclusively accessed by that GUI. ... etc :-)
If you're not talking about digging into the code yourself, then... I
dunno. Maybe one of the Subversion vendors is willing to do spot fixes
for a fee? Outside of that, it is tough to know when somebody will
want to dig into the problem :-/
Cheers,
-g
On Fri, May 4, 2012 at 12:46 PM, Moe, Mark <ma...@medtronic.com> wrote:
> Thanks Philip!
>
> Please excuse my ignorance (again), but how does the priority work? Or, how can I best show my interest in having this resolved?
>
> - Mark
>
> -----Original Message-----
> From: MARTIN PHILIP [mailto:codematters@ntlworld.com] On Behalf Of Philip Martin
> Sent: Friday, May 04, 2012 4:08 AM
> To: Moe, Mark
> Cc: Daniel Shahaf; dev@subversion.apache.org; Braun, Eric
> Subject: Re: svn ci performance issue with 1.7.x and nfs mounted working copies
>
> "Moe, Mark" <ma...@medtronic.com> writes:
>
>> Can this patch be marked as an issue or enhancement idea?
>
> I've raised http://subversion.tigris.org/issues/show_bug.cgi?id=4176
>
> --
> uberSVN: Apache Subversion Made Easy
> http://www.uberSVN.com
>
>
> [CONFIDENTIALITY AND PRIVACY NOTICE]
>
> Information transmitted by this email is proprietary to Medtronic and is intended for use only by the individual or entity to which it is addressed, and may contain information that is private, privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly prohibited. In such cases, please delete this mail from your records.
>
> To view this notice in other languages you can either select the following link or manually copy and paste the link into the address bar of a web browser: http://emaildisclaimer.medtronic.com
>
RE: svn ci performance issue with 1.7.x and nfs mounted working
copies
Posted by "Moe, Mark" <ma...@medtronic.com>.
Thanks Philip!
Please excuse my ignorance (again), but how does the priority work? Or, how can I best show my interest in having this resolved?
- Mark
-----Original Message-----
From: MARTIN PHILIP [mailto:codematters@ntlworld.com] On Behalf Of Philip Martin
Sent: Friday, May 04, 2012 4:08 AM
To: Moe, Mark
Cc: Daniel Shahaf; dev@subversion.apache.org; Braun, Eric
Subject: Re: svn ci performance issue with 1.7.x and nfs mounted working copies
"Moe, Mark" <ma...@medtronic.com> writes:
> Can this patch be marked as an issue or enhancement idea?
I've raised http://subversion.tigris.org/issues/show_bug.cgi?id=4176
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com
[CONFIDENTIALITY AND PRIVACY NOTICE]
Information transmitted by this email is proprietary to Medtronic and is intended for use only by the individual or entity to which it is addressed, and may contain information that is private, privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly prohibited. In such cases, please delete this mail from your records.
To view this notice in other languages you can either select the following link or manually copy and paste the link into the address bar of a web browser: http://emaildisclaimer.medtronic.com
Re: svn ci performance issue with 1.7.x and nfs mounted working copies
Posted by Philip Martin <ph...@wandisco.com>.
"Moe, Mark" <ma...@medtronic.com> writes:
> Can this patch be marked as an issue or enhancement idea?
I've raised http://subversion.tigris.org/issues/show_bug.cgi?id=4176
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com
RE: svn ci performance issue with 1.7.x and nfs mounted working
copies
Posted by "Moe, Mark" <ma...@medtronic.com>.
Can this patch be marked as an issue or enhancement idea?
[I'm not sure of the process but would like to help get this implemented if possible.]
Thanks,
- Mark
-----Original Message-----
From: MARTIN PHILIP [mailto:codematters@ntlworld.com] On Behalf Of Philip Martin
Sent: Tuesday, May 01, 2012 2:34 PM
To: Daniel Shahaf
Cc: Moe, Mark; dev@subversion.apache.org; Braun, Eric
Subject: Re: svn ci performance issue with 1.7.x and nfs mounted working copies
Philip Martin <ph...@wandisco.com> writes:
> Daniel Shahaf <d....@daniel.shahaf.name> writes:
>
>> What patch are you talking about? I don't recall seeing a patch on list
>
> http://svn.haxx.se/dev/archive-2012-02/0413.shtml
>
> It executes the SQLite statement "pragma locking_mode = exclusive" after
> opening wc.db.
We can't simply commit that patch. We are a bit careless about leaving
open databases around and this causes some of the regression tests to
fail. It also affects the command line client directly, for example
"svn cp URL wc" fails when svn_wc_create_tmp_file2 opens the database.
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com
[CONFIDENTIALITY AND PRIVACY NOTICE]
Information transmitted by this email is proprietary to Medtronic and is intended for use only by the individual or entity to which it is addressed, and may contain information that is private, privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly prohibited. In such cases, please delete this mail from your records.
To view this notice in other languages you can either select the following link or manually copy and paste the link into the address bar of a web browser: http://emaildisclaimer.medtronic.com
Re: svn ci performance issue with 1.7.x and nfs mounted working copies
Posted by Philip Martin <ph...@wandisco.com>.
Philip Martin <ph...@wandisco.com> writes:
> Daniel Shahaf <d....@daniel.shahaf.name> writes:
>
>> What patch are you talking about? I don't recall seeing a patch on list
>
> http://svn.haxx.se/dev/archive-2012-02/0413.shtml
>
> It executes the SQLite statement "pragma locking_mode = exclusive" after
> opening wc.db.
We can't simply commit that patch. We are a bit careless about leaving
open databases around and this causes some of the regression tests to
fail. It also affects the command line client directly, for example
"svn cp URL wc" fails when svn_wc_create_tmp_file2 opens the database.
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com
Re: svn ci performance issue with 1.7.x and nfs mounted working copies
Posted by Philip Martin <ph...@wandisco.com>.
Daniel Shahaf <d....@daniel.shahaf.name> writes:
> What patch are you talking about? I don't recall seeing a patch on list
http://svn.haxx.se/dev/archive-2012-02/0413.shtml
It executes the SQLite statement "pragma locking_mode = exclusive" after
opening wc.db.
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com