You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Daniel Näslund <da...@longitudo.com> on 2010/06/24 07:28:34 UTC
How does the workqueues provide atomic updates, or do they?
Hi!
Some questions about how the workqueues... work.
Have I got it right?
How are files installed during an update?
----------------------------------------
We acquire a write lock
We crawl the wc
We tell the repos about our state
The repo runs an editor
open_root(): We set status to _status_incomplete
open_directory(): We set status t _status_incomplete.
close_file(): We run the wq
close_directory(): We run the wq
close_edit(): Remove status_incomplete
release write_lock
What if we get interrupted?
----------------------------
We have this incoming change:
M A/B/lambda
M A/B/beta
But we only process lambda and get cancelled before we start processing
beta. Then there's no wq started for beta. How does the wq know that the
update isn't complete? Or is the status that tells us that the update is
not complete? E.g. the workqueues provide write atomicity but it's the
status that tells us if the whole operation has succeded?
I think it works like this:
------------------------------
* If the next operation tries to acquire a write lock it will fail and
tell the user to run 'svn cleanup'
* svn_wc_cleanup() will run all remaining work items in the different
workqueues (will be one queue when we have one db).
* Something detects that the update is incomplete and resumes it *or*
something detects that the update is incomplete and reverts what has
been done so far.
I assume that the best thing would have been if all items could have
been placed in one workqueue and then be run but that we don't do that
since we don't want to increase the memory usage or slow down the system
by writing a lot of tempfiles holding the incoming changes.
My problem is that I haven't found the code parts that deals with
status_incomplete. I only see:
If write locks already held -> run the items left in the workqueues.
Thanks,
Daniel
Re: How does the workqueues provide atomic updates, or do they?
Posted by Daniel Näslund <da...@longitudo.com>.
On Thu, Jun 24, 2010 at 09:20:22AM +0100, Philip Martin wrote:
> Daniel Näslund <da...@longitudo.com> writes:
>
> > How are files installed during an update?
> > ----------------------------------------
> > We acquire a write lock
> > We crawl the wc
> > We tell the repos about our state
> > The repo runs an editor
> > open_root(): We set status to _status_incomplete
> > open_directory(): We set status t _status_incomplete.
> > close_file(): We run the wq
> > close_directory(): We run the wq
>
> close_directory will remove status_incomplete for the directory, see
> maybe_bump_dir_info. At present it's a direct database operation but
> it may become part of the wq for the directory.
>
> > close_edit(): Remove status_incomplete
> > release write_lock
> >
> > What if we get interrupted?
> > ----------------------------
> > We have this incoming change:
> > M A/B/lambda
> > M A/B/beta
> >
> > But we only process lambda and get cancelled before we start processing
> > beta. Then there's no wq started for beta. How does the wq know that the
> > update isn't complete?
>
> The wq doesn't know or care. All the wq knows is whether there are
> outstanding items.
>
> > Or is the status that tells us that the update is
> > not complete? E.g. the workqueues provide write atomicity but it's the
> > status that tells us if the whole operation has succeded?
>
> The status_incomplete remains on the directory. It will remain until
> the user runs update successfully.
>
> > I think it works like this:
> > ------------------------------
> > * If the next operation tries to acquire a write lock it will fail and
> > tell the user to run 'svn cleanup'
> > * svn_wc_cleanup() will run all remaining work items in the different
> > workqueues (will be one queue when we have one db).
>
> Yes. Although note that pool cleanups attempt to run any outstanding
> wq so controlled cancel will often run any wq and remove locks, just
> leaving the incomplete status.
Ok. For update that would be update_editor.c::cleanup_dir_baton() that
gets registered in cleanup_dir_baton_child() with a call to
apr_pool_cleanup_kill(). (Just putting it here for reference).
> > * Something detects that the update is incomplete and resumes it *or*
> > something detects that the update is incomplete and reverts what has
> > been done so far.
>
> No. In future it may change but at present the user needs to invoke
> update to finish the operation.
Thanks for your answers!
Daniel
Re: How does the workqueues provide atomic updates, or do they?
Posted by Daniel Näslund <da...@longitudo.com>.
On Thu, Jun 24, 2010 at 09:20:22AM +0100, Philip Martin wrote:
> Daniel Näslund <da...@longitudo.com> writes:
>
> > How are files installed during an update?
> > ----------------------------------------
> > We acquire a write lock
> > We crawl the wc
> > We tell the repos about our state
> > The repo runs an editor
> > open_root(): We set status to _status_incomplete
> > open_directory(): We set status t _status_incomplete.
> > close_file(): We run the wq
> > close_directory(): We run the wq
>
> close_directory will remove status_incomplete for the directory, see
> maybe_bump_dir_info. At present it's a direct database operation but
> it may become part of the wq for the directory.
>
> > close_edit(): Remove status_incomplete
> > release write_lock
> >
> > What if we get interrupted?
> > ----------------------------
> > We have this incoming change:
> > M A/B/lambda
> > M A/B/beta
> >
> > But we only process lambda and get cancelled before we start processing
> > beta. Then there's no wq started for beta. How does the wq know that the
> > update isn't complete?
>
> The wq doesn't know or care. All the wq knows is whether there are
> outstanding items.
>
> > Or is the status that tells us that the update is
> > not complete? E.g. the workqueues provide write atomicity but it's the
> > status that tells us if the whole operation has succeded?
>
> The status_incomplete remains on the directory. It will remain until
> the user runs update successfully.
>
> > I think it works like this:
> > ------------------------------
> > * If the next operation tries to acquire a write lock it will fail and
> > tell the user to run 'svn cleanup'
> > * svn_wc_cleanup() will run all remaining work items in the different
> > workqueues (will be one queue when we have one db).
>
> Yes. Although note that pool cleanups attempt to run any outstanding
> wq so controlled cancel will often run any wq and remove locks, just
> leaving the incomplete status.
Ok. For update that would be update_editor.c::cleanup_dir_baton() that
gets registered in cleanup_dir_baton_child() with a call to
apr_pool_cleanup_kill(). (Just putting it here for reference).
> > * Something detects that the update is incomplete and resumes it *or*
> > something detects that the update is incomplete and reverts what has
> > been done so far.
>
> No. In future it may change but at present the user needs to invoke
> update to finish the operation.
Thanks for your answers!
Daniel
Re: How does the workqueues provide atomic updates, or do they?
Posted by Philip Martin <ph...@wandisco.com>.
Daniel Näslund <da...@longitudo.com> writes:
> How are files installed during an update?
> ----------------------------------------
> We acquire a write lock
> We crawl the wc
> We tell the repos about our state
> The repo runs an editor
> open_root(): We set status to _status_incomplete
> open_directory(): We set status t _status_incomplete.
> close_file(): We run the wq
> close_directory(): We run the wq
close_directory will remove status_incomplete for the directory, see
maybe_bump_dir_info. At present it's a direct database operation but
it may become part of the wq for the directory.
> close_edit(): Remove status_incomplete
> release write_lock
>
> What if we get interrupted?
> ----------------------------
> We have this incoming change:
> M A/B/lambda
> M A/B/beta
>
> But we only process lambda and get cancelled before we start processing
> beta. Then there's no wq started for beta. How does the wq know that the
> update isn't complete?
The wq doesn't know or care. All the wq knows is whether there are
outstanding items.
> Or is the status that tells us that the update is
> not complete? E.g. the workqueues provide write atomicity but it's the
> status that tells us if the whole operation has succeded?
The status_incomplete remains on the directory. It will remain until
the user runs update successfully.
> I think it works like this:
> ------------------------------
> * If the next operation tries to acquire a write lock it will fail and
> tell the user to run 'svn cleanup'
> * svn_wc_cleanup() will run all remaining work items in the different
> workqueues (will be one queue when we have one db).
Yes. Although note that pool cleanups attempt to run any outstanding
wq so controlled cancel will often run any wq and remove locks, just
leaving the incomplete status.
> * Something detects that the update is incomplete and resumes it *or*
> something detects that the update is incomplete and reverts what has
> been done so far.
No. In future it may change but at present the user needs to invoke
update to finish the operation.
--
Philip