You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Karl Fogel <kf...@newton.ch.collab.net> on 2002/06/24 15:28:34 UTC
Re: svn commit: rev 2312 - trunk/tools/client-side
Hmmm. Blair, I wonder, does this have any bearing on issue #599
("Need a feature like CVS's vendor branches")? It sounds like this
external script is starting have the same functionality that CVS
implements internally for vendor branches...
-K
blair@tigris.org writes:
> Log:
> Make it easier to follow file and directory renames by presenting to
> the user those files and directories that have been added to the
> import directory and deleted from the working copy so that the user
> can match them up. Perform svn mv's on the user specified files and
> directories.
>
> * tools/client-side/svn_load_dirs.pl
> (read_from_process): New subroutine that calls safe_read_from_pipe
> and checks for the success or failure of the command. It quits
> the script if the child process fails for any reason.
> (filter_renamed_files): New subroutine to remove from a list of
> files and directories a list of files and directories that have
> been renamed. This is used since svn does not delete renamed
> directories until committed and old pre-renamed directories should
> not be deleted until committed.
> (recusurive_ls_and_hash): New subroutine that given a directory
> name, returns a hash keyed by filename whose value is a hash
> containing the type of file and its digest hash if it is a file.
> (digest_hash_file): Check for the correct number of arguments.
> (main):
> Use Text::Wrap to wrap all log messages to 72 characters.
> Replace all the calls to safe_read_from_pipe that require a
> successful result with calls to read_from_process.
> Let the user specify files and directories to rename in the
> working copy and use svn mv to perform the renames.
> Move the code that performs the finds of files and directories in
> the source and working copy directories into the rename loop.
> Change the temporary working directory template from
> svn_log_dirs_XXXXXXXXXX to svn_load_dirs_XXXXXXXXXX.
>
>
> Modified: trunk/tools/client-side/svn_load_dirs.pl
> ==============================================================================
> --- trunk/tools/client-side/svn_load_dirs.pl (original)
> +++ trunk/tools/client-side/svn_load_dirs.pl Sun Jun 23 02:00:50 2002
> @@ -9,8 +9,11 @@
> use File::Path 1.0404;
> use File::Temp 0.12 qw(tempdir);
> use Getopt::Long 2.25;
> +use Text::Wrap;
> use URI 1.17;
>
> +$Text::Wrap::columns = 72;
> +
> my $VERSION = 0.02;
>
> # Process the command line options.
> @@ -150,13 +153,7 @@
> }
>
> # Check that the svn base URL works by running svn log on it.
> -{
> - my @command = ('svn', 'log', $repos_base_uri);
> - my ($status, @output) = safe_read_from_pipe(@command);
> - if ($status) {
> - die join("\n", "$0: @command failed with this output:", @output), "\n";
> - }
> -}
> +read_from_process('svn', 'log', $repos_base_uri);
>
> my $orig_cwd = cwd;
>
> @@ -195,7 +192,7 @@
> }
>
> # Create a temporary directory for svn to work in.
> -my $temp_template = '/tmp/svn_log_dirs_XXXXXXXXXX';
> +my $temp_template = '/tmp/svn_load_dirs_XXXXXXXXXX';
> my $temp_dir = tempdir($temp_template);
>
> # Create an object that when DESTROY'ed will delete the temporary
> @@ -211,13 +208,7 @@
> # fixed directory name.
> my $checkout_dir_name = 'ZZZ';
> print "Checking out $repos_base_uri into $temp_dir/$checkout_dir_name\n";
> -{
> - my @command = ('svn', 'co', $repos_base_uri, '-d', $checkout_dir_name);
> - my ($status, @output) = safe_read_from_pipe(@command);
> - if ($status) {
> - die join("\n", "$0: @command failed with this output:", @output), "\n";
> - }
> -}
> +read_from_process('svn', 'co', $repos_base_uri, '-d', $checkout_dir_name);
>
> # Change into the top level directory of the repository and record the
> # absolute path to this location because the script will come back
> @@ -275,22 +266,26 @@
> $message .= "* $dir: New directory.\n";
> }
> }
> - my @command = ('svn', 'add', @dirs_to_create);
> - my ($status, @output) = safe_read_from_pipe(@command);
> - if ($status) {
> - die join("\n", "$0: @command failed with this output:", @output), "\n";
> - }
> + $message = wrap('', '', $message);
>
> - @command = ('svn', 'commit', '-m', $message);
> - ($status, @output) = safe_read_from_pipe(@command);
> - if ($status) {
> - die join("\n", "$0: @command failed with this output:", @output), "\n";
> - }
> + read_from_process('svn', 'add', @dirs_to_create);
> + read_from_process('svn', 'commit', '-m', $message);
> } else {
> print "No directories need to be created for loading.\n";
> }
> }
>
> +# Set up the names for the path to the import and tag directories.
> +my $repos_load_abs_path;
> +if ($repos_load_rel_path eq '.') {
> + $repos_load_abs_path = length($repos_base_path_segment) ?
> + $repos_base_path_segment : "/";
> +} else {
> + $repos_load_abs_path = length($repos_base_path_segment) ?
> + "$repos_base_path_segment/$repos_load_rel_path" :
> + $repos_load_rel_path;
> +}
> +
> # Now go through each source directory and copy each file from the
> # source directory to the target directory. For new target files, add
> # them to svn. For files that no longer exist, delete them.
> @@ -307,45 +302,259 @@
> print "Loading $load_dir.\n";
> }
>
> - my @add_files;
> - my @del_files;
> + # This hash is keyed by the old name in a rename.
> + my %rename_files;
>
> - # First get a list of all the existing files and directories in the
> - # repository. This is used to see what files should be deleted from
> - # the repository.
> - chdir($wc_import_dir_cwd) or
> - die "$0: cannot chdir `$wc_import_dir_cwd': $!\n";
> + # Loop here until the user has performed all the file and directory
> + # renames.
> + my $repeat_loop;
> + do {
> + $repeat_loop = 0;
>
> - my %existing_files;
> - my $wanted = sub {
> - s#^\./##;
> - return if $_ eq '.';
> - my $file_type = &file_type($_);
> - my $file_digest;
> - if ($file_type eq 'f' or (stat($_) and -f _)) {
> - $file_digest = &digest_hash_file($_);
> + my %add_files;
> + my %del_files;
> +
> + # Get the list of files and directories in the repository working
> + # copy. This hash is called %del_files because each file or
> + # directory will be deleted from the hash using the list of files
> + # and directories in the source directory, leaving the files and
> + # directories that need to be deleted.
> + %del_files = &recusurive_ls_and_hash($wc_import_dir_cwd);
> +
> + # This anonymous subroutine finds all the files and directories in
> + # the directory to load. It notes the file type and for each file
> + # found, it deletes it from %del_files.
> + my $wanted = sub {
> + s#^\./##;
> + return if $_ eq '.';
> +
> + my $source_path = $_;
> + my $dest_path = "$wc_import_dir_cwd/$_";
> +
> + my $source_type = &file_type($source_path);
> + my $dest_type = &file_type($dest_path);
> +
> + # Fail if the destination type exists but is of a different type
> + # of file than the source type.
> + if ($dest_type ne '0' and $source_type ne $dest_type) {
> + die "$0: does not handle changing source and destination type for ",
> + "$source_path.\n";
> + }
> +
> + if ($source_type ne 'f' and $source_type ne 'd') {
> + die "$0: does not handle copying files of type `$source_type'.\n";
> + }
> +
> + unless (defined delete $del_files{$source_path}) {
> + $add_files{$source_path}{type} = $source_type;
> + }
> + };
> +
> + # Now change into the directory containing the files to load.
> + # First change to the original directory where this script was run
> + # so that if the specified directory is a relative directory path,
> + # then the script can change into it.
> + chdir($orig_cwd) or
> + die "$0: cannot chdir `$orig_cwd': $!\n";
> + chdir($load_dir) or
> + die "$0: cannot chdir `$load_dir': $!\n";
> +
> + find({no_chdir => 1,
> + preprocess => sub { sort { $b cmp $a } @_ },
> + wanted => $wanted
> + }, '.');
> +
> + # At this point %add_files contains the list of new files and
> + # directories to be created in the working copy tree and
> + # %del_files contains the files and directories that need to be
> + # deleted. Because there may be renames that have taken place,
> + # give the user the opportunity to rename any deleted files and
> + # directories to ones being added.
> + my @add_files = sort keys %add_files;
> + my @del_files = sort keys %del_files;
> +
> + # Because the source code management system may keep the original
> + # renamed file or directory in the working copy until a commit,
> + # remove them from the list of deleted files or directories.
> + &filter_renamed_files(\@del_files, \%rename_files);
> +
> + # Now change into the working copy directory in case any renames
> + # need to be performed.
> + chdir($wc_import_dir_cwd) or
> + die "$0: cannot chdir `$wc_import_dir_cwd': $!\n";
> +
> + # Only do renames if there are both added and deleted files and
> + # directories.
> + if (@add_files and @del_files) {
> + my $max = @add_files > @del_files ? @add_files : @del_files;
> +
> + # Print the files that have been added and deleted. Find the
> + # deleted file with the longest name and use that for the width
> + # of the filename column. Add one to the filename width to let
> + # the directory / character be appended to a directory name.
> + my $line_number_width = 4;
> + my $filename_width = 0;
> + foreach my $f (@del_files) {
> + my $l = length($f);
> + $filename_width = $l if $l > $filename_width;
> + }
> + ++$filename_width;
> + my $printf_format = "%${line_number_width}d";
> +
> + print "\n",
> + "The following are deleted files and directories on the left\n",
> + "added files and directories on the right that you have the\n",
> + "opportunity to match up as renames instead of deletes and\n",
> + "adds. This is a Good Thing. Match up a deleted item from\n",
> + "the left column with an added item from the right column.\n",
> + "Note the line numbers on the left which you type in to this\n",
> + "script to have a rename performed.\n",
> + "\n",
> + " " x $line_number_width,
> + " ",
> + "Deleted", " " x ($filename_width - length("Deleted")),
> + " ",
> + "Added\n";
> + for (my $i=0; $i<$max; ++$i) {
> + my $add_filename = '';
> + my $del_filename = '';
> + if ($i < @add_files) {
> + $add_filename = $add_files[$i];
> + if ($add_files{$add_filename}{type} eq 'd') {
> + $add_filename .= '/';
> + }
> + }
> + if ($i < @del_files) {
> + $del_filename = $del_files[$i];
> + if ($del_files{$del_filename}{type} eq 'd') {
> + $del_filename .= '/';
> + }
> + }
> +
> + printf $printf_format, $i;
> + print " ", $del_filename,
> + " " x ($filename_width - length($del_filename)),
> + " ", $add_filename, "\n";
> +
> + if ($i and $i % 23 == 0) {
> + print "Continue printing (y/n)? ";
> + last unless &get_yes_or_no;
> + }
> + }
> +
> + # Get the feedback from the user.
> + my $line;
> + my $add_filename;
> + my $add_index;
> + my $del_filename;
> + my $del_index;
> + my $got_line = 0;
> + do {
> + print "Enter two indexes for each column to rename or n to continue: ";
> + $line = <>;
> + $line ||= '';
> + if ($line =~ /^n$/i) {
> + $got_line = 1;
> + } elsif ($line =~ /^(\d+)\s+(\d+)$/) {
> + $del_index = $1;
> + $add_index = $2;
> + if ($del_index >= @del_files) {
> + print "Delete index $del_index is larger than maximum index of ",
> + scalar @del_files - 1, ".\n";
> + $del_index = undef;
> + }
> + if ($add_index > @add_files) {
> + print "Add index $add_index is larger than maximum index of ",
> + scalar @add_files - 1, ".\n";
> + $add_index = undef;
> + }
> + $got_line = defined $del_index && defined $add_index;
> +
> + # Check that the file or directory to be renamed has the
> + # same file type.
> + if ($got_line) {
> + $add_filename = $add_files[$add_index];
> + $del_filename = $del_files[$del_index];
> + if ($add_files{$add_filename}{type} ne
> + $del_files{$del_filename}{type}) {
> + print "File types for $del_filename and $add_filename differ.\n";
> + $got_line = undef;
> + }
> + }
> + }
> + } until ($got_line);
> +
> + if ($line !~ /^n$/i) {
> + print "Renaming $del_filename to $add_filename.\n";
> +
> + # Check that any required directories to do the rename exist.
> + my @add_segments = split('/', $add_filename);
> + pop(@add_segments);
> + my $add_dir = '';
> + my @add_dirs;
> + foreach my $segment (@add_segments) {
> + $add_dir = length($add_dir) ? "$add_dir/$segment" : $segment;
> + unless (-d $add_dir) {
> + push(@add_dirs, $add_dir);
> + }
> + }
> +
> + if (@add_dirs) {
> + read_from_process('svn', 'mkdir', @add_dirs);
> + }
> +
> + $repeat_loop = 1;
> +
> + $rename_files{$del_filename} = $del_files{$del_filename};
> +
> + read_from_process('svn', 'mv', $del_filename, $add_filename);
> +
> + # Because subversion cannot rename the same file or directory
> + # twice, a commit has to be performed before any other
> + # operations.
> + my $full_add_filename = "$repos_load_abs_path/$add_filename";
> + my $full_del_filename = "$repos_load_abs_path/$del_filename";
> + my $message = "To prepare to load $load_dir into " .
> + "$repos_load_abs_path, rename " .
> + "$full_del_filename to $full_add_filename.\n";
> + $message = wrap('', '', $message);
> +
> + # Change to the top of the working copy so that any
> + # directories will also be updated.
> + my $cwd = cwd;
> + chdir($wc_top_dir_cwd) or
> + die "$0: cannot chdir `$wc_top_dir_cwd': $!\n";
> + read_from_process('svn', 'commit', '-m', $message);
> + read_from_process('svn', 'update');
> + chdir($cwd) or
> + die "$0: cannot chdir `$cwd': $!\n";
> +
> + # Some versions of subversion have a bug where renamed files
> + # or directories are not deleted after a commit, so do that
> + # here.
> + my @del_files = sort {length($b) <=> length($a) || $a cmp $b }
> + keys %rename_files;
> + rmtree(\@del_files, 1, 0);
> + foreach my $f (@del_files) {
> + delete $rename_files{$f};
> + }
> + }
> }
> - $existing_files{$_} = {type => $file_type,
> - digest => $file_digest};
> - };
> - find({no_chdir => 1,
> - preprocess => sub { grep { $_ !~ /\.svn$/ } @_ },
> - wanted => $wanted
> - }, '.');
> + } while ($repeat_loop);
>
> - # Now change into the directory containing the files to load. First
> - # change to the original directory where this script was run so that
> - # if the specified directory is a relative directory path, then the
> - # script can change into it.
> - chdir($orig_cwd) or
> - die "$0: cannot chdir `$orig_cwd': $!\n";
> - chdir($load_dir) or
> - die "$0: cannot chdir `$load_dir': $!\n";
> + # At this point all renames have been performed. Now get the final
> + # list of files and directories in the working copy directory. The
> + # %add_files hash will contain the list of files and directories to
> + # add to the working copy and %del_files starts with all the files
> + # already in the working copy and gets files removed that are in the
> + # imported directory, which results in a list of files that should
> + # be deleted.
> + my %add_files;
> + my %del_files = &recusurive_ls_and_hash($wc_import_dir_cwd);
>
> - # This anonymous subroutine finds all the files and directories in
> - # the directory to load. It notes the file type and records file
> - # hash digests.
> - $wanted = sub {
> + # This anonymous subroutine copies files from the source directory
> + # to the working copy directory.
> + my $wanted = sub {
> s#^\./##;
> return if $_ eq '.';
>
> @@ -359,22 +568,22 @@
> # of file than the source type.
> if ($dest_type ne '0' and $source_type ne $dest_type) {
> die "$0: does not handle changing source and destination type for ",
> - "$source_path.\n";
> + "$source_path.\n";
> }
>
> # Determine if the file is being added or is an update to an
> # already existing file using the file's digest.
> - my $existing_info = delete $existing_files{$source_path};
> - if (defined $existing_info) {
> - if (defined (my $existing_digest = $existing_info->{digest})) {
> + my $del_info = delete $del_files{$source_path};
> + if (defined $del_info) {
> + if (defined (my $del_digest = $del_info->{digest})) {
> my $new_digest = &digest_hash_file($source_path);
> - if ($new_digest ne $existing_digest) {
> + if ($new_digest ne $del_digest) {
> print "U $source_path\n";
> }
> }
> } else {
> print "A $source_path\n";
> - push(@add_files, $source_path);
> + $add_files{$source_path}{type} = $source_type;
> }
>
> # Now make sure the file or directory in the source directory
> @@ -392,34 +601,43 @@
> }
> };
>
> + # Now change into the directory containing the files to load.
> + # First change to the original directory where this script was run
> + # so that if the specified directory is a relative directory path,
> + # then the script can change into it.
> + chdir($orig_cwd) or
> + die "$0: cannot chdir `$orig_cwd': $!\n";
> + chdir($load_dir) or
> + die "$0: cannot chdir `$load_dir': $!\n";
> +
> find({no_chdir => 1,
> preprocess => sub { sort { $b cmp $a } @_ },
> wanted => $wanted
> }, '.');
>
> - # The files and directories that are left in %existing_files are the
> - # files and directories that need to be deleted. Because svn will
> - # return an error if a file or directory is deleted in a directory
> - # that will then be deleted, first find all directories and delete
> - # any files and directories inside those directories from this list.
> - # Work through the list repeatedly working from short to long names
> - # so that directories containing other files and directories will be
> - # deleted first.
> - my $repeat_loop;
> + # The files and directories that are in %del_files are the files and
> + # directories that need to be deleted. Because svn will return an
> + # error if a file or directory is deleted in a directory that
> + # subsequently is deleted, first find all directories and remove
> + # from the list any files and directories inside those directories
> + # from this list. Work through the list repeatedly working from
> + # short to long names so that directories containing other files and
> + # directories will be deleted first.
> do {
> $repeat_loop = 0;
> - my @list = sort {length($a) <=> length($b) || $a cmp $b}
> - keys %existing_files;
> - foreach my $file (@list) {
> - if ($existing_files{$file}{type} eq 'd') {
> + my @del_files = sort {length($a) <=> length($b) || $a cmp $b}
> + keys %del_files;
> + &filter_renamed_files(\@del_files, \%rename_files);
> + foreach my $file (@del_files) {
> + if ($del_files{$file}{type} eq 'd') {
> my $dir = "$file/";
> my $dir_length = length($dir);
> - foreach my $f (@list) {
> + foreach my $f (@del_files) {
> next if $file eq $f;
> if (length($f) >= $dir_length and
> substr($f, 0, $dir_length) eq $dir) {
> print "d $f\n";
> - delete $existing_files{$f};
> + delete $del_files{$f};
> $repeat_loop = 1;
> }
> }
> @@ -427,10 +645,10 @@
> # If there were any deletions of files and/or directories
> # inside a directory that will be deleted, then restart the
> # entire loop again, because one or more keys have been
> - # deleted from %existing_files. Equally important is not to
> - # stop this loop if no deletions have been done, otherwise
> - # later directories that may contain files and directories to
> - # be deleted will not be deleted.
> + # deleted from %del_files. Equally important is not to stop
> + # this loop if no deletions have been done, otherwise later
> + # directories that may contain files and directories to be
> + # deleted will not be deleted.
> last if $repeat_loop;
> }
> }
> @@ -443,8 +661,9 @@
> # deleted, instead of trying to figure out which directories and
> # files are contained in other directories, just reverse sort by the
> # path length and then alphabetically.
> - @del_files = sort {length($b) <=> length($a) || $a cmp $b }
> - keys %existing_files;
> + my @del_files = sort {length($b) <=> length($a) || $a cmp $b }
> + keys %del_files;
> + &filter_renamed_files(\@del_files, \%rename_files);
> foreach my $file (@del_files) {
> print "D $file\n";
> }
> @@ -453,37 +672,17 @@
> chdir($wc_import_dir_cwd) or
> die "$0: cannot chdir `$wc_import_dir_cwd': $!\n";
>
> - if (@add_files) {
> - my @command = ('svn', 'add', @add_files);
> - my ($status, @output) = safe_read_from_pipe(@command);
> - if ($status) {
> - die join("\n", "$0: @command failed with this output:", @output), "\n";
> - }
> + if (keys %add_files) {
> + my @add_files = sort {length($a) <=> length($b) || $a cmp $b}
> + keys %add_files;
> + read_from_process('svn', 'add', @add_files);
> }
> if (@del_files) {
> - my @command = ('svn', 'rm', @del_files);
> - my ($status, @output) = safe_read_from_pipe(@command);
> - if ($status) {
> - die join("\n", "$0: @command failed with this output:", @output), "\n";
> - }
> + read_from_process('svn', 'rm', @del_files);
> }
>
> - # Set up the names for the path to the import and tag directories.
> - my $repos_load_abs_path;
> - if ($repos_load_rel_path eq '.') {
> - $repos_load_abs_path = length($repos_base_path_segment) ?
> - $repos_base_path_segment : "/";
> - } else {
> - $repos_load_abs_path = length($repos_base_path_segment) ?
> - "$repos_base_path_segment/$repos_load_rel_path" :
> - $repos_load_rel_path;
> - }
> - my @command = ('svn', 'commit',
> - '-m', "Load $load_dir into $repos_load_abs_path.\n");
> - my ($status, @output) = safe_read_from_pipe(@command);
> - if ($status) {
> - die join("\n", "$0: @command failed with this output:", @output), "\n";
> - }
> + my $message = wrap('', '', "Load $load_dir into $repos_load_abs_path.\n");
> + read_from_process('svn', 'commit', '-m', $message);
>
> # Now remove any files and directories to be deleted in the repository.
> if (@del_files) {
> @@ -500,13 +699,10 @@
> $repos_load_rel_path :
> "$repos_base_url/$repos_load_rel_path";
> my $to_url = "$repos_base_url/$load_tag";
> - @command = ('svn', 'cp',
> - '-m', "Tag $repos_load_abs_path as $repos_tag_abs_path.\n",
> - $from_url, $to_url);
> - ($status, @output) = safe_read_from_pipe(@command);
> - if ($status) {
> - die join("\n", "$0: @command failed with this output:", @output), "\n";
> - }
> +
> + $message = wrap("", "",
> + "Tag $repos_load_abs_path as $repos_tag_abs_path.\n");
> + read_from_process('svn', 'cp', '-m', $message, $from_url, $to_url);
> }
>
> # Finally, go to the top of the svn checked out tree and do an
> @@ -514,24 +710,16 @@
> chdir($wc_top_dir_cwd) or
> die "$0: cannot chdir `$wc_top_dir_cwd': $!\n";
>
> - @command = qw(svn update);
> - ($status, @output) = safe_read_from_pipe(@command);
> - if ($status) {
> - die join("\n", "$0: @command failed with this output:", @output), "\n";
> - }
> + read_from_process(qw(svn update));
>
> # Now run a recursive diff between the original source directory and
> # the tag for a consistency check.
> if (defined $load_tag) {
> chdir($orig_cwd) or
> die "$0: cannot chdir `$orig_cwd': $!\n";
> - @command = ('diff',
> - '-x', '.svn',
> - '-r', $load_dir, $wc_tag_dir_cwd);
> - ($status, @output) = safe_read_from_pipe(@command);
> - if ($status) {
> - die join("\n", "$0: @command failed with this output:", @output), "\n";
> - }
> + read_from_process('diff',
> + '-x', '.svn',
> + '-r', $load_dir, $wc_tag_dir_cwd);
> }
> }
>
> @@ -554,6 +742,7 @@
> return '?';
> }
>
> +# Start a child process safely without using /bin/sh.
> sub safe_read_from_pipe {
> unless (@_) {
> croak "$0: safe_read_from_pipe passed no arguments.\n";
> @@ -589,7 +778,111 @@
> }
> }
>
> +# Use safe_read_from_pipe to start a child process safely and exit the
> +# script if the child failed for whatever reason.
> +sub read_from_process {
> + unless (@_) {
> + croak "$0: read_from_process passed no arguments.\n";
> + }
> + my ($status, @output) = &safe_read_from_pipe(@_);
> + if ($status) {
> + print STDERR "$0: @_ failed with this output:\n", join("\n", @output),
> + "\n",
> + "Press return to quit and clean up svn working directory: ";
> + <STDIN>;
> + exit 1;
> + } else {
> + return @output;
> + }
> +}
> +
> +# Get a list of all the files and directories in the specified
> +# directory, the type of file and a digest hash of file types.
> +sub recusurive_ls_and_hash {
> + unless (@_ == 1) {
> + croak "$0: recusurive_ls_and_hash passed incorrect number of ",
> + "arguments.\n";
> + }
> +
> + # This is the directory to change into.
> + my $dir = shift;
> +
> + # Get the current directory so that the script can change into the
> + # current working directory after changing into the specified
> + # directory.
> + my $return_cwd = cwd;
> +
> + chdir($dir) or
> + die "$0: cannot chdir `$dir': $!\n";
> +
> + my %files;
> +
> + my $wanted = sub {
> + s#^\./##;
> + return if $_ eq '.';
> + my $file_type = &file_type($_);
> + my $file_digest;
> + if ($file_type eq 'f' or (stat($_) and -f _)) {
> + $file_digest = &digest_hash_file($_);
> + }
> + $files{$_} = {type => $file_type,
> + digest => $file_digest};
> + };
> + find({no_chdir => 1,
> + preprocess => sub { grep { $_ !~ /\.svn$/ } @_ },
> + wanted => $wanted
> + }, '.');
> +
> + chdir($return_cwd) or
> + die "$0: cannot chdir `$return_cwd': $!\n";
> +
> + %files;
> +}
> +
> +# Take an array reference containing a list of files and directories
> +# and take a hash reference and remove from the array reference any
> +# files and directories and the files the directory contains listed in
> +# the hash.
> +sub filter_renamed_files {
> + unless (@_ == 2) {
> + croak "$0: filter_renamed_files passed incorrect number of arguments.\n";
> + }
> +
> + my $array_ref = shift;
> + my $hash_ref = shift;
> +
> + foreach my $remove_filename (keys %$hash_ref) {
> + my $remove_file_type = $hash_ref->{$remove_filename}{type};
> + for (my $i=0; $i<@$array_ref;) {
> + my $array_filename = $array_ref->[$i];
> + if ($array_filename eq $remove_filename) {
> + splice(@$array_ref, $i, 1);
> + next;
> + }
> +
> + # If the file being removed is a directory, append the directory
> + # separator and see if there is a string submatch.
> + if ($remove_file_type eq 'd') {
> + my $f = "$remove_filename/";
> + my $lf = length($f);
> + if ($lf <= length($array_filename) and
> + $f eq substr($array_filename, 0, $lf)) {
> + splice(@$array_ref, $i, 1);
> + next;
> + }
> + }
> +
> + ++$i;
> + }
> + }
> +}
> +
> +# Get a digest hash of the specified filename.
> sub digest_hash_file {
> + unless (@_ == 1) {
> + croak "$0: digest_hash_file passed incorrect number of arguments.\n";
> + }
> +
> my $filename = shift;
>
> my $ctx = Digest::MD5->new;
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: svn-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: svn-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 2312 - trunk/tools/client-side
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
This issue is not going to be settled by list discussion; it's going
to be settled by users doing imports and re-imports and seeing how
things work for them. As Blair says, if the script helps with that
process, then the script can be incrementally improved over time just
like any other part of Subversion. If it doesn't, people won't use
it.
So while I can't really call this thread off-topic, it's not
particularly helpful to anyone either. I humbly suggest it end, at
least until someone has a *concrete* suggestion for either making the
script better, or for otherwise addressing issue #599.
-K
Blair Zajac <bl...@orcaware.com> writes:
> Eric Gillespie wrote:
> >
> > Blair Zajac <bz...@demografx.com> writes:
> >
> > > If you have little projects and only a few renames and deletes and
> > > feel like taking the time to make sure that the new vendor branch
> > > exactly matches the old one, then sure, typing in those commands is
> >
> > Some of the projects are small, some are large; renames and deleted
> > and added files are rarely painful, but i would not trust it to
> > automation.
>
> What is the concern about trust?
>
> One of my concerns was that the script wouldn't do the job correctly and
> the checked in vendor branch wouldn't match exactly the directory being
> imported.
>
> To handle this and to ensure that the script did its job correctly, it
> does the following:
>
> 1) Run all the svn add and svn rm's it needs to and copy all the
> files from the imported directory to the working copy.
> 2) svn commit
> 3) Make a cheap copy tag via "svn cp URL URL" on the repos itself.
> 4) "svn update" its working copy to include the tag.
> 5) "diff -r -x .svn" on the imported directory and the working copy
> tag to verify that everything is exactly correct.
>
> One outstanding issue is that the script runs automatically and
> doesn't ask for user confirmation on anything except for renames,
> so if it does something unwanted, then it can leave the repository
> in an undesirable state.
>
> Currently the work around for this is that the user make a backup
> of the repository itself before running the script. This is a
> problem, because it may require an admin to make the snapshot.
> One long-term solution is to show all the steps that the script
> does (which it does already) and ask for confirmation before doing
> any commits on the repository (which is easy).
>
> > What i don't want to see is this script become blessed as the way
> > to manage vendor branches, when it does more than CVS did, and
> > arguably too much. I suspect it will become a source of complaints,
> > as it will not be flexible enough to do what people want.
>
> What does it do more than CVS did? Since svn does more than CVS
> does, why shouldn't the vendor support be more powerful? How is it
> not flexible enough? What's missing to make it more flexible?
>
> There are no specific issues with the script here, just generalities
> that it automates too much, may "become a source of complaints" and
> isn't "flexible enough."
>
> Here's my request. Run the script to import a bunch of stuff and see
> what doesn't work about it and let us know those things. Those things
> that don't work can be fixed.
>
> >
> > > In the mean time, keep your code quality comments to yourself.
> >
> > I think Karl already said anything i would say in my defense here,
> > but i'd better say it too. Don't take offense; i didn't mean
> > anything about the code quality of your script, per se. It's just
> > that so often when a tool has a problem, the answer is "Oh, that's
> > easy, just write a perl script." As i said above, i just think
> > this script is being over-sold.
>
> So what a solution here then? Is it porting the functionality in the
> script to be in svn itself?
>
> Best,
> Blair
>
> --
> Blair Zajac <bl...@orcaware.com>
> Web and OS performance plots - http://www.orcaware.com/orca/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 2312 - trunk/tools/client-side
Posted by Blair Zajac <bl...@orcaware.com>.
Eric Gillespie wrote:
>
> Blair Zajac <bz...@demografx.com> writes:
>
> > If you have little projects and only a few renames and deletes and
> > feel like taking the time to make sure that the new vendor branch
> > exactly matches the old one, then sure, typing in those commands is
>
> Some of the projects are small, some are large; renames and deleted
> and added files are rarely painful, but i would not trust it to
> automation.
What is the concern about trust?
One of my concerns was that the script wouldn't do the job correctly and
the checked in vendor branch wouldn't match exactly the directory being
imported.
To handle this and to ensure that the script did its job correctly, it
does the following:
1) Run all the svn add and svn rm's it needs to and copy all the
files from the imported directory to the working copy.
2) svn commit
3) Make a cheap copy tag via "svn cp URL URL" on the repos itself.
4) "svn update" its working copy to include the tag.
5) "diff -r -x .svn" on the imported directory and the working copy
tag to verify that everything is exactly correct.
One outstanding issue is that the script runs automatically and
doesn't ask for user confirmation on anything except for renames,
so if it does something unwanted, then it can leave the repository
in an undesirable state.
Currently the work around for this is that the user make a backup
of the repository itself before running the script. This is a
problem, because it may require an admin to make the snapshot.
One long-term solution is to show all the steps that the script
does (which it does already) and ask for confirmation before doing
any commits on the repository (which is easy).
> What i don't want to see is this script become blessed as the way
> to manage vendor branches, when it does more than CVS did, and
> arguably too much. I suspect it will become a source of complaints,
> as it will not be flexible enough to do what people want.
What does it do more than CVS did? Since svn does more than CVS
does, why shouldn't the vendor support be more powerful? How is it
not flexible enough? What's missing to make it more flexible?
There are no specific issues with the script here, just generalities
that it automates too much, may "become a source of complaints" and
isn't "flexible enough."
Here's my request. Run the script to import a bunch of stuff and see
what doesn't work about it and let us know those things. Those things
that don't work can be fixed.
>
> > In the mean time, keep your code quality comments to yourself.
>
> I think Karl already said anything i would say in my defense here,
> but i'd better say it too. Don't take offense; i didn't mean
> anything about the code quality of your script, per se. It's just
> that so often when a tool has a problem, the answer is "Oh, that's
> easy, just write a perl script." As i said above, i just think
> this script is being over-sold.
So what a solution here then? Is it porting the functionality in the
script to be in svn itself?
Best,
Blair
--
Blair Zajac <bl...@orcaware.com>
Web and OS performance plots - http://www.orcaware.com/orca/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 2312 - trunk/tools/client-side
Posted by Eric Gillespie <ep...@pretzelnet.org>.
Blair Zajac <bz...@demografx.com> writes:
> If you have little projects and only a few renames and deletes and
> feel like taking the time to make sure that the new vendor branch
> exactly matches the old one, then sure, typing in those commands is
Some of the projects are small, some are large; renames and deleted
and added files are rarely painful, but i would not trust it to
automation. I have only had to deal with a complete reorganization
once, and cvs could not handle it. I just went into the repository
and moved the directory to a different name and imported fresh.
Thankfully, this will not be necessary with svn. Handling a big
reorganization would still be a bitch, but again, i don't trust
this to automation.
I contend that merges like this are rare.
> fine. If you're like me and have 6 different projects using 60
> different directories to import, then it's very useful.
Well, good for you.
This script solves *your* problem, and it may solve the problems
of others. It may not. I think it is being over-sold. What i
don't want to see is this script become blessed as the way to manage
vendor branches, when it does more than CVS did, and arguably too
much. I suspect it will become a source of complaints, as it will
not be flexible enough to do what people want.
> In the mean time, keep your code quality comments to yourself.
I think Karl already said anything i would say in my defense here,
but i'd better say it too. Don't take offense; i didn't mean
anything about the code quality of your script, per se. It's just
that so often when a tool has a problem, the answer is "Oh, that's
easy, just write a perl script." As i said above, i just think
this script is being over-sold.
--
Eric Gillespie <*> epg@pretzelnet.org
Build a fire for a man, and he'll be warm for a day. Set a man on
fire, and he'll be warm for the rest of his life. -Terry Pratchett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 2312 - trunk/tools/client-side
Posted by Blair Zajac <bl...@orcaware.com>.
Karl Fogel wrote:
>
> Blair Zajac <bz...@demografx.com> writes:
> > > For what it's worth, i have adopted the procedure documented at
> > > <http://hackers.progeny.com/~epg/svn-3rdparty.html>. It's simple,
> > > straight-forward, and doesn't rely on any hacked-up perl scripts
> > > and their bogus automation. I'm happy with it.
> >
> > In the mean time, keep your code quality comments to yourself.
>
> Blair, Eric wasn't commenting on your coding. He was merely trying to
> say that any attempt to automate this inherently complex problem is
> going to be bogus, at least to some degree. (I suspect he's on to
> something, too; even CVS's longstanding vendor branch support requires
> human intervention.)
My sense was that by judging what Eric had to say about the automation
and the language it is written in, he didn't take a good look at what
the script does, how it runs and didn't provide any positive criticism on
how it could be improved, instead of just dismissing it.
That said, there's no automation in the script beyond what you would
expect the script to accomplish. The rename support just prints the
files and directories that exist in one but not both the svn repository
directory or the directory to be imported and let's the user match
them up so that the script can run svn mv. The script doesn't do any
renames unless the user tells it to.
Aside from the rename support, it does the deletes, copies and adds
that you would normally have to do by hand to bring a vendor branch
in the svn repository up to date with a new version, so you don't
have to worry about making any mistakes or forget to add or delete
files from svn. This looks like a pretty easy place to miss files
and have to do repeated commits to fix the tree. The script leads
to a smaller and cleaner repository with fewer commits.
>
> And there are many programmers for whom *any* Perl script
> automatically qualifies as "hacked-up"; when they apply that adjective
> to a Perl script, they're really just repeating the assertion that
> it's a Perl script. :-)
I think the problem is that if the functionality were written in C
code and in svn itself or written in Python, then it would have
gotten a different reaction.
Blair
--
Blair Zajac <bl...@orcaware.com>
Web and OS performance plots - http://www.orcaware.com/orca/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 2312 - trunk/tools/client-side
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Blair Zajac <bz...@demografx.com> writes:
> > For what it's worth, i have adopted the procedure documented at
> > <http://hackers.progeny.com/~epg/svn-3rdparty.html>. It's simple,
> > straight-forward, and doesn't rely on any hacked-up perl scripts
> > and their bogus automation. I'm happy with it.
>
> In the mean time, keep your code quality comments to yourself.
Blair, Eric wasn't commenting on your coding. He was merely trying to
say that any attempt to automate this inherently complex problem is
going to be bogus, at least to some degree. (I suspect he's on to
something, too; even CVS's longstanding vendor branch support requires
human intervention.)
And there are many programmers for whom *any* Perl script
automatically qualifies as "hacked-up"; when they apply that adjective
to a Perl script, they're really just repeating the assertion that
it's a Perl script. :-)
Don't take it so personally.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 2312 - trunk/tools/client-side
Posted by Blair Zajac <bz...@demografx.com>.
Eric Gillespie wrote:
>
> Karl Fogel <kf...@newton.ch.collab.net> writes:
>
> > I don't feel a need to get that functionality in the client itself --
> > anyone who's maintaining a vendor branch can jolly well run a script
> > to help them do it :-). But if others feel differently, please sound
> > off.
>
> For what it's worth, i have adopted the procedure documented at
> <http://hackers.progeny.com/~epg/svn-3rdparty.html>. It's simple,
> straight-forward, and doesn't rely on any hacked-up perl scripts
> and their bogus automation. I'm happy with it.
Well, good for you.
If you have little projects and only a few renames and deletes and
feel like taking the time to make sure that the new vendor branch
exactly matches the old one, then sure, typing in those commands is
fine. If you're like me and have 6 different projects using 60
different directories to import, then it's very useful.
In the mean time, keep your code quality comments to yourself.
Blair
--
Blair Zajac <bl...@orcaware.com>
Web and OS performance plots - http://www.orcaware.com/orca/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 2312 - trunk/tools/client-side
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Eric Gillespie <ep...@pretzelnet.org> writes:
> For what it's worth, i have adopted the procedure documented at
> <http://hackers.progeny.com/~epg/svn-3rdparty.html>. It's simple,
> straight-forward, and doesn't rely on any hacked-up perl scripts
> and their bogus automation. I'm happy with it.
Yeah, I saw that link in the issue, and visited the page. It's a
great writeup!
Having no firsthand experience with the procedure, I can't confidently
say that more automation is called for; but it does seem rather more
complex than the automation CVS offers for vendor branches. Not
saying we absolutely have to match CVS's automation, just keeping an
eye open for opportunities, that's all...
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 2312 - trunk/tools/client-side
Posted by Eric Gillespie <ep...@pretzelnet.org>.
Karl Fogel <kf...@newton.ch.collab.net> writes:
> I don't feel a need to get that functionality in the client itself --
> anyone who's maintaining a vendor branch can jolly well run a script
> to help them do it :-). But if others feel differently, please sound
> off.
For what it's worth, i have adopted the procedure documented at
<http://hackers.progeny.com/~epg/svn-3rdparty.html>. It's simple,
straight-forward, and doesn't rely on any hacked-up perl scripts
and their bogus automation. I'm happy with it.
--
Eric Gillespie <*> epg@pretzelnet.org
Build a fire for a man, and he'll be warm for a day. Set a man on
fire, and he'll be warm for the rest of his life. -Terry Pratchett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 2312 - trunk/tools/client-side
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Blair Zajac <bl...@orcaware.com> writes:
> If we keep it separate, then maybe we should rename the script to make
> the name clearer to svn users. Something like svn_import_dirs and have
> it installed automatically into $prefix/bin. Using the word "load" in
> the name doesn't have the semantic meaning people would look for and
> svn_import_dirs may be better.
>
> Comments.
Sounds good to me; I'm thinking something like "svn-reimport", but
whatever. I dumped your mail into the issue, so we don't lose the
thoughts, and will leave it at that for now (unless some volunteer
comes along all fierce on vendor branches).
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 2312 - trunk/tools/client-side
Posted by Blair Zajac <bl...@orcaware.com>.
Karl Fogel wrote:
>
> Blair Zajac <bl...@orcaware.com> writes:
> > Hmm, sure sounds like it does have a bearing. I didn't read #599
> > before writing the script, but it looks like it comes pretty close
> > to the feature set requested.
> >
> > Anything we want to do with the script?
> >
> > It sounds like may people want the functionality of this script in
> > svn itself.
>
> I'm just curious if you can make the script essentially duplicate (or
> exceed) the functionality of CVS's vendor branches. If so, then we
> can close that issue, and simply point people to an explanation of how
> to get the desired effect from svn_load_dirs.
Not having used CVS vendor branches but having read
http://www.cvshome.org/docs/manual/cvs_13.html
it looks like the script has all the functionality that CVS vendor
branches has, but somebody who's used it should verify it. If there's
something to add, I can add it to the script, because at this point, I
don't think it would be that hard to do.
>
> I don't feel a need to get that functionality in the client itself --
> anyone who's maintaining a vendor branch can jolly well run a script
> to help them do it :-). But if others feel differently, please sound
> off.
If we keep it separate, then maybe we should rename the script to make
the name clearer to svn users. Something like svn_import_dirs and have
it installed automatically into $prefix/bin. Using the word "load" in
the name doesn't have the semantic meaning people would look for and
svn_import_dirs may be better.
Comments.
Blair
--
Blair Zajac <bl...@orcaware.com>
Web and OS performance plots - http://www.orcaware.com/orca/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 2312 - trunk/tools/client-side
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Blair Zajac <bl...@orcaware.com> writes:
> Hmm, sure sounds like it does have a bearing. I didn't read #599
> before writing the script, but it looks like it comes pretty close
> to the feature set requested.
>
> Anything we want to do with the script?
>
> It sounds like may people want the functionality of this script in
> svn itself.
I'm just curious if you can make the script essentially duplicate (or
exceed) the functionality of CVS's vendor branches. If so, then we
can close that issue, and simply point people to an explanation of how
to get the desired effect from svn_load_dirs.
I don't feel a need to get that functionality in the client itself --
anyone who's maintaining a vendor branch can jolly well run a script
to help them do it :-). But if others feel differently, please sound
off.
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: rev 2312 - trunk/tools/client-side
Posted by Blair Zajac <bl...@orcaware.com>.
Karl,
Hmm, sure sounds like it does have a bearing. I didn't read #599
before writing the script, but it looks like it comes pretty close
to the feature set requested.
Anything we want to do with the script?
It sounds like may people want the functionality of this script in
svn itself.
Blair
Karl Fogel wrote:
>
> Hmmm. Blair, I wonder, does this have any bearing on issue #599
> ("Need a feature like CVS's vendor branches")? It sounds like this
> external script is starting have the same functionality that CVS
> implements internally for vendor branches...
>
> -K
>
> blair@tigris.org writes:
> > Log:
> > Make it easier to follow file and directory renames by presenting to
> > the user those files and directories that have been added to the
> > import directory and deleted from the working copy so that the user
> > can match them up. Perform svn mv's on the user specified files and
> > directories.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org