[RFC] Obliterate - patch for making BDB dup a txn

[Julian Foad <ju...@wandisco.com>]

I don't expect anyone to review the whole of this, but any comments
would be welcome. It's about time I posted what I'm up to. It's not a
"finished" patch, by any stretch of the imagination, so please don't
commit it.

[[[
For Obliterate, teach the BDB FS implementation to duplicate an existing
revision into a new transaction, with a new txn-id but otherwise
identical. The purpose is to be able to modify the new txn (delete
one or more nodes out of it) and then replace the old one with it.

* subversion/libsvn_fs_base/revs-txns.c
  (rep_dup, copy_dup, node_rev_dup): New functions to make a deep copy
    of a rep, a copy or a node-rev, while also changing any references
    to the old txn-id.
  (begin_txn_args): Rename the 'rev' member to 'based_on_rev' for
    clarity.
  (txn_body_begin_txn, svn_fs_base__begin_txn): Adjust accordingly.
  (txn_body_begin_obliteration_txn): Add code to make a recursive
    duplicate of all the contents of the txn. Add documentation.
  (svn_fs_base__begin_obliteration_txn): Return success, because it is
    now implemented. Rename 'rev' to 'replacing_rev'. Add documentation.

* subversion/libsvn_fs_base/revs-txns.h
  (svn_fs_base__get_txn_ids): Fix a typo in the doc string.
  (svn_fs_base__begin_obliteration_txn): Rename the 'rev' argument to
    'replacing_rev' for clarity.

* subversion/libsvn_fs_base/tree.c
  (svn_fs_base__commit_obliteration_txn):
    ### Just return success, for testing, even though not implemented yet.

* subversion/tests/cmdline/obliterate_tests.py
  New file, with a skeleton of a test for obliterate. So far, this just
  exercises the obliterate operation and doesn't actually check the
  result.

* subversion/tests/cmdline/svntest/objects.py
  New file, with classes for easily performing common testing steps on a
  repository and on a WC.
  ### The repos class "dump" function calls an external script in my home dir.

=====

Concerns/unfinished business:

  - Should I be using the DAG layer to do these manipulations? Are there
    DAG functions that already do such duplication, or part of it? I
    found some that were useful, but don't understand what the
    significant difference is between the DAG layer and other levels.
    (I did read the description in <> section "".)

  - All the "dup" functions could take the old-txn-id as a parameter, and
    only change txn-ids that match it. If there are any txn-id fields that
    refer to the current txn in some cases but not in other cases, then
    we need this, and need to compare with the old-txn-id to decide whether
    to change it. Are there any such cases?

  - I don't know whether I should change the copy-ids. I don't think I
    should, because future revs refer to them, but I can't dup the copies
    table rows in the same way unless I give them new copy-ids. If
    necessary, I could possibly dup them to new ids and then translate
    the new ids back to the old ones at "commit-obliteration" time.

  - In a normal txn, are the "changes" rows used and updated during txn
    building, or only filled in at commit time? If the former, I need to
    dup them at begin-obliteration-txn time; if the latter, at
    commit-obliteration-txn time.

  - In "rep_dup()", when I dup a child rep in a "rep window" (or "delta
    chunk"), I need to set an "offset" field. I expect there's a
    function I should be using to ensure that gets set correctly.
]]]

- Julian

Re: [RFC] Obliterate - patch for making BDB dup a txn

[Julian Foad <ju...@wandisco.com>]

On Fri, 2009-12-04, Julian Foad wrote:
> I don't expect anyone to review the whole of this, but any comments
> would be welcome. It's about time I posted what I'm up to. It's not a
> "finished" patch, by any stretch of the imagination, so please don't
> commit it.

I fixed up a load of this and committed it in r889320.

> Concerns/unfinished business:
> 
>   - Should I be using the DAG layer to do these manipulations? Are there
>     DAG functions that already do such duplication, or part of it? I
>     found some that were useful, but don't understand what the
>     significant difference is between the DAG layer and other levels.
>     (I did read the description in <> section "".)

Not resolved, but I think the right direction is to call dag-layer
functions from revs-txns, and for the dag-layer functions to call down
further.

For now, I have put my helper functions in a separate file
subversion/libsvn_fs_base/obliterate.[ch].

>   - All the "dup" functions could take the old-txn-id as a parameter, and
>     only change txn-ids that match it. If there are any txn-id fields that
>     refer to the current txn in some cases but not in other cases, then
>     we need this, and need to compare with the old-txn-id to decide whether
>     to change it. Are there any such cases?

I think it's correct as is.

>   - I don't know whether I should change the copy-ids. I don't think I
>     should, because future revs refer to them, but I can't dup the copies
>     table rows in the same way unless I give them new copy-ids. If
>     necessary, I could possibly dup them to new ids and then translate
>     the new ids back to the old ones at "commit-obliteration" time.

Decided I should not change any copy-ids, and should just update the
existing entries in the copies table at oblit-commit time. Not done yet.

>   - In a normal txn, are the "changes" rows used and updated during txn
>     building, or only filled in at commit time? If the former, I need to
>     dup them at begin-obliteration-txn time; if the latter, at
>     commit-obliteration-txn time.

I think the latter, and added comments to that effect.

>   - In "rep_dup()", when I dup a child rep in a "rep window" (or "delta
>     chunk"), I need to set an "offset" field. I expect there's a
>     function I should be using to ensure that gets set correctly.

Not resolved.

- Julian