You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Alfred Perlstein <br...@mu.org> on 2008/10/09 00:13:27 UTC
request: Subversion internals consultant.
The company I'm working at is evaluating subversion and we have a
number of successful pilots in place.
There are some issues we would like to address and I'm looking to
talk to some people with Subversion internals experience that can
help us optimize the Subversion experience for our users.
I have some tentative budget allocated and we are a large company
so the opportunity for a good long term contract or employment
(if interested) may be in the cards.
Here are some of the issues we would like resolved:
1) Reduce the ".lock" operation when doing updates.
When svn does most "update" or "stat" operations, it traverses the
tree making "lock files" this causes the speed of the tool be to
much slower than it could be. We'd like to explore options for a
custom build that would reduce that.
2) Reduce "entries.log" operation when doing updates/checkout:
> For instance, when doing a "svn checkout" I'll see ".svn/tmp/entries"
> created many times, and then renamed over ".svn/entries".
>
> Is there any way to reduce these items when you _know_ you're going
> to be working in isolation?
>
> Could this be made any faster by just updating the files in place?
3) Reduce the size of the checkout, we'd like to explore opportunities for not having copies of the files in the "svn-base" directory, either storing this on the server (as an option) or maybe just compressing it.
There will be more work, and this potentially could be a long
term contract.
I would like to talk someone with some internals experience that
_potentially_ get these sort of changes pushed upstream into the
client.
Work would very likely be open sourced.
Our location is Sunnyvale CA, and the job at minimum require some
travel to our campus to interact with developers and assist with
deployment of features. Someone willing to relocate to Sunnyvale
would probably work best.
Please respond with a resume, cover letter and ball-park figure for
your hourly rate or salary requirements if looking for full-time.
Finally, yes, I am aware of "collab.net", however I've had trouble
spinning up interest from them on these issues.
--
- Alfred Perlstein
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: request: Subversion internals consultant.
Posted by Mark Phippard <ma...@gmail.com>.
On Wed, Oct 8, 2008 at 8:13 PM, Alfred Perlstein <br...@mu.org> wrote:
> The company I'm working at is evaluating subversion and we have a
> number of successful pilots in place.
>
> There are some issues we would like to address and I'm looking to
> talk to some people with Subversion internals experience that can
> help us optimize the Subversion experience for our users.
>
> I have some tentative budget allocated and we are a large company
> so the opportunity for a good long term contract or employment
> (if interested) may be in the cards.
>
> Here are some of the issues we would like resolved:
>
> 1) Reduce the ".lock" operation when doing updates.
> When svn does most "update" or "stat" operations, it traverses the
> tree making "lock files" this causes the speed of the tool be to
> much slower than it could be. We'd like to explore options for a
> custom build that would reduce that.
>
> 2) Reduce "entries.log" operation when doing updates/checkout:
>> For instance, when doing a "svn checkout" I'll see ".svn/tmp/entries"
>> created many times, and then renamed over ".svn/entries".
>>
>> Is there any way to reduce these items when you _know_ you're going
>> to be working in isolation?
>>
>> Could this be made any faster by just updating the files in place?
>
> 3) Reduce the size of the checkout, we'd like to explore opportunities for not having copies of the files in the "svn-base" directory, either storing this on the server (as an option) or maybe just compressing it.
>
> There will be more work, and this potentially could be a long
> term contract.
>
> I would like to talk someone with some internals experience that
> _potentially_ get these sort of changes pushed upstream into the
> client.
>
> Work would very likely be open sourced.
>
> Our location is Sunnyvale CA, and the job at minimum require some
> travel to our campus to interact with developers and assist with
> deployment of features. Someone willing to relocate to Sunnyvale
> would probably work best.
>
> Please respond with a resume, cover letter and ball-park figure for
> your hourly rate or salary requirements if looking for full-time.
>
> Finally, yes, I am aware of "collab.net", however I've had trouble
> spinning up interest from them on these issues.
FWIW, your questions made it to me via your CollabNet contact last
week and I replied with answers similar to what Greg gave along with
some names of people that might be interested. That said, I suspect
they would not be interested in working from Sunnyvale. If you want
core Subversion work performed, would that really be needed? That
requirement was not communicated to me last week else I'd have not
suggested them.
I suspect that the people I suggested are being contacted about their
availability and rates so that an answer could be provided back to
you. I forwarded your email to your CollabNet contact to let them
know you were not happy about not receiving a reply back.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: request: Subversion internals consultant.
Posted by Alfred Perlstein <br...@mu.org>.
* Chris Frost <ch...@frostnet.net> [081010 16:00] wrote:
> On Wed, Oct 08, 2008 at 05:13:27PM -0700, Alfred Perlstein wrote:
> > 3) Reduce the size of the checkout, we'd like to explore opportunities for not having copies of the files in the "svn-base" directory, either storing this on the server (as an option) or maybe just compressing it.
> >
> > There will be more work, and this potentially could be a long
> > term contract.
> >
> > I would like to talk someone with some internals experience that
> > _potentially_ get these sort of changes pushed upstream into the
> > client.
>
> Though not part of subversion itself, scord may be useful in reducing
> checkout sizes for your deployments: http://scord.sourceforge.net/
This is cool, but we use FreeBSD, so I'm not sure about support
for this issue, I have to look into it.
thank you very much,
-Alfred
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: request: Subversion internals consultant.
Posted by Chris Frost <ch...@frostnet.net>.
On Wed, Oct 08, 2008 at 05:13:27PM -0700, Alfred Perlstein wrote:
> 3) Reduce the size of the checkout, we'd like to explore opportunities for not having copies of the files in the "svn-base" directory, either storing this on the server (as an option) or maybe just compressing it.
>
> There will be more work, and this potentially could be a long
> term contract.
>
> I would like to talk someone with some internals experience that
> _potentially_ get these sort of changes pushed upstream into the
> client.
Though not part of subversion itself, scord may be useful in reducing
checkout sizes for your deployments: http://scord.sourceforge.net/
--
Chris Frost | http://www.frostnet.net/chris/
-------------+--------------------------------
PGP: http://www.frostnet.net/chris/pgpkey.txt
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: request: Subversion internals consultant.
Posted by David Glasser <gl...@davidglasser.net>.
On Thu, Oct 9, 2008 at 1:10 AM, Greg Stein <gs...@gmail.com> wrote:
> On Wed, Oct 8, 2008 at 11:51 PM, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
>> Greg Stein wrote on Wed, 8 Oct 2008 at 18:43 -0700:
>>...
>>> In 1.7, there will be one sqlite database replacing all of a working
>>> copy's "entries" files and no locks whatsoever. The two operations
>>> will do a quick traversal of the working copy (update will look for
>>> missing items, status will look for text modifications).
>>
>> I thought you and jszakmeister agreed on IRC that 'update' doesn't care
>> about missing items, only about missing text-bases?
>
> svn update needs to *restore* missing items. That is part of its
> normal operation. So it needs to locate the problem and repopulate
> ACTUAL.
Of course, this is really just "update" doing work that ought to be
done by "revert", except that when it's easy to wipe out wc metadata,
you need to talk to the server to fix the problem. Perhaps in 1.7,
this task will go back to "revert" where it belongs...
--dave
> It doesn't need to scan the working copy to generate the report to the
> server (this was the specific result from IRC). It can look at
> WORKING, and send that to the server. When the response comes back,
> then it can start fetching new content, bumping revisions, and
> rebuilding missing files/dirs.
>
> Cheers,
> -g
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
--
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: request: Subversion internals consultant.
Posted by Greg Stein <gs...@gmail.com>.
On Wed, Oct 8, 2008 at 11:51 PM, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> Greg Stein wrote on Wed, 8 Oct 2008 at 18:43 -0700:
>...
>> In 1.7, there will be one sqlite database replacing all of a working
>> copy's "entries" files and no locks whatsoever. The two operations
>> will do a quick traversal of the working copy (update will look for
>> missing items, status will look for text modifications).
>
> I thought you and jszakmeister agreed on IRC that 'update' doesn't care
> about missing items, only about missing text-bases?
svn update needs to *restore* missing items. That is part of its
normal operation. So it needs to locate the problem and repopulate
ACTUAL.
It doesn't need to scan the working copy to generate the report to the
server (this was the specific result from IRC). It can look at
WORKING, and send that to the server. When the response comes back,
then it can start fetching new content, bumping revisions, and
rebuilding missing files/dirs.
Cheers,
-g
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: request: Subversion internals consultant.
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Greg Stein wrote on Wed, 8 Oct 2008 at 18:43 -0700:
> > 1) Reduce the ".lock" operation when doing updates.
> > When svn does most "update" or "stat" operations, it traverses the
> > tree making "lock files" this causes the speed of the tool be to
> > much slower than it could be. We'd like to explore options for a
> > custom build that would reduce that.
>
> Yeah... it probably shouldn't take out locks for update or stat, but
> I'm assuming it does to ensure read-consistency.
>
> In 1.7, there will be one sqlite database replacing all of a working
> copy's "entries" files and no locks whatsoever. The two operations
> will do a quick traversal of the working copy (update will look for
> missing items, status will look for text modifications).
I thought you and jszakmeister agreed on IRC that 'update' doesn't care
about missing items, only about missing text-bases?
IOW, 'update' might have to traverse the .svn/pristine/ dir, but not the
wc itself.
> We may have shortcut versions that skip the traversal entirely.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: request: Subversion internals consultant.
Posted by Greg Stein <gs...@gmail.com>.
[-users list]
Hi Alfred,
In short, the answer to your issues is "wait for Subversion 1.7, which
will have a brand new working copy library". Unfortunately for you,
the expected date for that is June-ish. But it does completely address
your three issues.
More details in-line below:
On Wed, Oct 8, 2008 at 5:13 PM, Alfred Perlstein <br...@mu.org> wrote:
>...
> 1) Reduce the ".lock" operation when doing updates.
> When svn does most "update" or "stat" operations, it traverses the
> tree making "lock files" this causes the speed of the tool be to
> much slower than it could be. We'd like to explore options for a
> custom build that would reduce that.
Yeah... it probably shouldn't take out locks for update or stat, but
I'm assuming it does to ensure read-consistency.
In 1.7, there will be one sqlite database replacing all of a working
copy's "entries" files and no locks whatsoever. The two operations
will do a quick traversal of the working copy (update will look for
missing items, status will look for text modifications). We may have
shortcut versions that skip the traversal entirely.
> 2) Reduce "entries.log" operation when doing updates/checkout:
>> For instance, when doing a "svn checkout" I'll see ".svn/tmp/entries"
>> created many times, and then renamed over ".svn/entries".
>>
>> Is there any way to reduce these items when you _know_ you're going
>> to be working in isolation?
>>
>> Could this be made any faster by just updating the files in place?
Subversion does this to ensure that a half-written/corrupted copy of
entries is not left in place if its operation is interrupted for some
reason, rather than for multiple-user operation.
Reducing the number of times entries is rewritten would probably be
difficult (we already try to avoid it), but is mooted by the new
strategy in 1.7 (no entries files at all).
> 3) Reduce the size of the checkout, we'd like to explore opportunities for not having copies of the files in the "svn-base" directory, either storing this on the server (as an option) or maybe just compressing it.
We've explored this in the past. The svn-base files are expected to be
present by too many things. There is a lot of coupling going on in the
working copy library. This is one of the basic reasons that the
library is being rewritten entirely for 1.7, and that version should
support omitting and/or compressing the text base files.
Happy to answer any further questions you may have!
Cheers,
-g
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: request: Subversion internals consultant.
Posted by Mark Phippard <ma...@gmail.com>.
On Wed, Oct 8, 2008 at 8:13 PM, Alfred Perlstein <br...@mu.org> wrote:
> The company I'm working at is evaluating subversion and we have a
> number of successful pilots in place.
>
> There are some issues we would like to address and I'm looking to
> talk to some people with Subversion internals experience that can
> help us optimize the Subversion experience for our users.
>
> I have some tentative budget allocated and we are a large company
> so the opportunity for a good long term contract or employment
> (if interested) may be in the cards.
>
> Here are some of the issues we would like resolved:
>
> 1) Reduce the ".lock" operation when doing updates.
> When svn does most "update" or "stat" operations, it traverses the
> tree making "lock files" this causes the speed of the tool be to
> much slower than it could be. We'd like to explore options for a
> custom build that would reduce that.
>
> 2) Reduce "entries.log" operation when doing updates/checkout:
>> For instance, when doing a "svn checkout" I'll see ".svn/tmp/entries"
>> created many times, and then renamed over ".svn/entries".
>>
>> Is there any way to reduce these items when you _know_ you're going
>> to be working in isolation?
>>
>> Could this be made any faster by just updating the files in place?
>
> 3) Reduce the size of the checkout, we'd like to explore opportunities for not having copies of the files in the "svn-base" directory, either storing this on the server (as an option) or maybe just compressing it.
>
> There will be more work, and this potentially could be a long
> term contract.
>
> I would like to talk someone with some internals experience that
> _potentially_ get these sort of changes pushed upstream into the
> client.
>
> Work would very likely be open sourced.
>
> Our location is Sunnyvale CA, and the job at minimum require some
> travel to our campus to interact with developers and assist with
> deployment of features. Someone willing to relocate to Sunnyvale
> would probably work best.
>
> Please respond with a resume, cover letter and ball-park figure for
> your hourly rate or salary requirements if looking for full-time.
>
> Finally, yes, I am aware of "collab.net", however I've had trouble
> spinning up interest from them on these issues.
FWIW, your questions made it to me via your CollabNet contact last
week and I replied with answers similar to what Greg gave along with
some names of people that might be interested. That said, I suspect
they would not be interested in working from Sunnyvale. If you want
core Subversion work performed, would that really be needed? That
requirement was not communicated to me last week else I'd have not
suggested them.
I suspect that the people I suggested are being contacted about their
availability and rates so that an answer could be provided back to
you. I forwarded your email to your CollabNet contact to let them
know you were not happy about not receiving a reply back.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org