svnmucc --revision 0 no longer works when creating a file

[sebb <se...@gmail.com>]

When I first started using svnmucc, it used to be the case that
svnmucc 'put' --revision 0 would fail if the target file already
existed. This no longer happens.

The previous behaviour was very useful, so are there any plans to reinstate it?

I don't think there is a straightforward way to guarantee the same
behaviour now.

The closest I could get is:

1) get current parent directory revision
2) check if target file does not exist. This is not as easy as it
sounds, as the target directory may have too many files to list
efficiently, and any other file-based command may fail for a reason
other than a missing file.
3) Put the file using the revision obtained in step 1.
AFAICT this is guaranteed not to replace an existing file.

However it may fail to create the file if the target directory has
been updated in the meantime.

It's only safe to repeat the attempted create if the command failed
due to an out of date revision.
So the failure reason will have to be analysed.

Re: svnmucc --revision 0 no longer works when creating a file

[Daniel Shahaf <d....@daniel.shahaf.name>]

sebb wrote on Thu, 23 Jul 2020 11:18 +0100:
> On Thu, 23 Jul 2020 at 01:17, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> > You can either parse stderr despite this complication, or use the
> > API directly, in which case you'll sidestep this complication entirely
> > (you'll get just one integer, rather than two).  
> 
> What API are you referring to here?

The C API or the various language bindings, bypassing the cmdline client.

Re: svnmucc --revision 0 no longer works when creating a file

[sebb <se...@gmail.com>]

On Thu, 23 Jul 2020 at 01:17, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
>
> sebb wrote on Wed, 22 Jul 2020 22:34 +0100:
> > On Wed, 22 Jul 2020 at 17:46, Nathan Hartman <ha...@gmail.com> wrote:
> > >
> > > On Wed, Jul 22, 2020 at 12:10 PM sebb <se...@gmail.com> wrote:
> > > > > Use the machine-parseable E000042 error codes.  That's exactly what
> > > > > they're for.  (which-error.py and svn_error_symbolic_name() can be used
> > > > > to convert numbers to symbolic names.)
> > > >
> > > > Where are these error codes defined?
> > > > I could not find any reference to them in the documentation.
> > >
> > > If you mean where in the source code:
> > >
> > > subversion/include/svn_error_codes.h
> > > which is included by subversion/include/svn_error.h, which is further
> > > included by subversion/svnmucc/svnmucc.c.
> > >
> > > Hope that helps,
> >
> > Thanks, but not really.
> >
> > If the error codes are intended to be machine-parseable then the
> > programmer needs to have documentation of the values and their
> > meanings.
> > There needs to be at least a mention in the documentation that such
> > codes exist and where to find the values. If there is such a mention,
> > I could not find it.
>
> Doxygen docs of svn_error_t::apr_err.

But how does one find that information?

> > Also, I had a look at the file and it does not show the numeric
> > values,
>
> I refer you again to which-error.py.

Again, how is one supposed to know about this script?
Also, I assume this requires Python.

> > and whilst there is a text string associated with each one it
> > does not detail when it might be used.
> > Nor indeed do the strings agree with the actual messages generated by
> > SVN as far as I can tell.
>
> No, they don't.  The strings in that file are defaults, only used when
> the line of code that raised an error didn't provide a more specific
> string.
>
> > e.g. svn list reports the following:
> >
> > svn: warning: W160013: Path '/x' not found
> > svn: E200009: Could not list all targets because some targets don't exist
>
> Great.  Run which-error.py on this and you'll find that the former code
> is SVN_ERR_FS_NOT_FOUND.  That's basically the svn API equivalent of
> ENOENT, i.e., "There's no node by that name in the versioned
> filesystem".  Therefore, you'll want to write your script to handle
> E160013 errors by ignoring them and continuing.

I would need to know whether E200009 can only be produced when a
target does not exist.
Otherwise it might look like the file does not exist when it does.

> The complication here is that `svn list` supports multiple target
> arguments, and has to handle the case that some but not all of them are
> invalid.  It does so by reporting the errors from individual targets as
> W* codes rather than E* codes and adding a generic E200009 error at the
> end.  You can either parse stderr despite this complication, or use the
> API directly, in which case you'll sidestep this complication entirely
> (you'll get just one integer, rather than two).

What API are you referring to here?

> Daniel

Re: svnmucc --revision 0 no longer works when creating a file

[Daniel Shahaf <d....@daniel.shahaf.name>]

sebb wrote on Wed, 22 Jul 2020 22:34 +0100:
> On Wed, 22 Jul 2020 at 17:46, Nathan Hartman <ha...@gmail.com> wrote:
> >
> > On Wed, Jul 22, 2020 at 12:10 PM sebb <se...@gmail.com> wrote:  
> > > > Use the machine-parseable E000042 error codes.  That's exactly what
> > > > they're for.  (which-error.py and svn_error_symbolic_name() can be used
> > > > to convert numbers to symbolic names.)  
> > >
> > > Where are these error codes defined?
> > > I could not find any reference to them in the documentation.  
> >
> > If you mean where in the source code:
> >
> > subversion/include/svn_error_codes.h
> > which is included by subversion/include/svn_error.h, which is further
> > included by subversion/svnmucc/svnmucc.c.
> >
> > Hope that helps,  
> 
> Thanks, but not really.
> 
> If the error codes are intended to be machine-parseable then the
> programmer needs to have documentation of the values and their
> meanings.
> There needs to be at least a mention in the documentation that such
> codes exist and where to find the values. If there is such a mention,
> I could not find it.

Doxygen docs of svn_error_t::apr_err.

> Also, I had a look at the file and it does not show the numeric
> values,

I refer you again to which-error.py.

> and whilst there is a text string associated with each one it
> does not detail when it might be used.
> Nor indeed do the strings agree with the actual messages generated by
> SVN as far as I can tell.

No, they don't.  The strings in that file are defaults, only used when
the line of code that raised an error didn't provide a more specific
string.

> e.g. svn list reports the following:
> 
> svn: warning: W160013: Path '/x' not found
> svn: E200009: Could not list all targets because some targets don't exist

Great.  Run which-error.py on this and you'll find that the former code
is SVN_ERR_FS_NOT_FOUND.  That's basically the svn API equivalent of
ENOENT, i.e., "There's no node by that name in the versioned
filesystem".  Therefore, you'll want to write your script to handle
E160013 errors by ignoring them and continuing.

The complication here is that `svn list` supports multiple target
arguments, and has to handle the case that some but not all of them are
invalid.  It does so by reporting the errors from individual targets as
W* codes rather than E* codes and adding a generic E200009 error at the
end.  You can either parse stderr despite this complication, or use the
API directly, in which case you'll sidestep this complication entirely
(you'll get just one integer, rather than two).

Daniel

Re: svnmucc --revision 0 no longer works when creating a file

[sebb <se...@gmail.com>]

On Wed, 22 Jul 2020 at 17:46, Nathan Hartman <ha...@gmail.com> wrote:
>
> On Wed, Jul 22, 2020 at 12:10 PM sebb <se...@gmail.com> wrote:
> > > Use the machine-parseable E000042 error codes.  That's exactly what
> > > they're for.  (which-error.py and svn_error_symbolic_name() can be used
> > > to convert numbers to symbolic names.)
> >
> > Where are these error codes defined?
> > I could not find any reference to them in the documentation.
>
> If you mean where in the source code:
>
> subversion/include/svn_error_codes.h
> which is included by subversion/include/svn_error.h, which is further
> included by subversion/svnmucc/svnmucc.c.
>
> Hope that helps,

Thanks, but not really.

If the error codes are intended to be machine-parseable then the
programmer needs to have documentation of the values and their
meanings.
There needs to be at least a mention in the documentation that such
codes exist and where to find the values. If there is such a mention,
I could not find it.

Also, I had a look at the file and it does not show the numeric
values, and whilst there is a text string associated with each one it
does not detail when it might be used.
Nor indeed do the strings agree with the actual messages generated by
SVN as far as I can tell.
e.g. svn list reports the following:

svn: warning: W160013: Path '/x' not found
svn: E200009: Could not list all targets because some targets don't exist

But I could not find similar strings in the header file; e.g. the only
string containing targets is:

"Duplicate targets in svn:externals property"

So I don't think the header file is suitable for writing software to
analyse the error codes.



> Nathan

Re: svnmucc --revision 0 no longer works when creating a file

[Nathan Hartman <ha...@gmail.com>]

On Wed, Jul 22, 2020 at 12:10 PM sebb <se...@gmail.com> wrote:
> > Use the machine-parseable E000042 error codes.  That's exactly what
> > they're for.  (which-error.py and svn_error_symbolic_name() can be used
> > to convert numbers to symbolic names.)
>
> Where are these error codes defined?
> I could not find any reference to them in the documentation.

If you mean where in the source code:

subversion/include/svn_error_codes.h

which is included by subversion/include/svn_error.h, which is further
included by subversion/svnmucc/svnmucc.c.

Hope that helps,
Nathan

Re: svnmucc --revision 0 no longer works when creating a file

[sebb <se...@gmail.com>]

On Sun, 12 Jul 2020 at 18:13, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
>
> sebb wrote on Sun, 12 Jul 2020 16:55 +0100:
> > On Sun, 12 Jul 2020 at 15:26, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> > >
> > > sebb wrote on Tue, 07 Jul 2020 20:43 +0100:
> > > > When I first started using svnmucc, it used to be the case that
> > > > svnmucc 'put' --revision 0 would fail if the target file already
> > > > existed. This no longer happens.
> > > >
> > >
> > > Is the file-to-be's parent directory the root directory?
> >
> > No, it's not.
> >
> > > If that isn't the case, then the new behaviour is correct.
> >
> > Why is that?
> >
>
> Because the target of the 'put' operation didn't exist at r0, and the
> base revision is specified to be a revision in which the target of the
> operation existed.  (See svn_delta_editor_t::open_root()'s docstring.)
>
> Moreover, even that syntax it did work, it should arguably fail if the
> file had been created and subsequently deleted, which isn't the same
> semantics as the algorithm you posted.
>
> > > You might wish to post the error message.
> >
> > Just tried with a local SVN repo:
> >
> > $ svnmucc -mBug --revision 0 -- put /dev/null
> > file:///var/tools/svnrep/asf/x/b.tmp
> > svnmucc: E160016: Can't commit to 'file:///var/tools/svnrep/asf/x'
> > because it is not a directory
> >
> > That message is wrong, because /x/ *is* a directory.
>
> Runnable reproduction recipe, please.
>
> I wonder if the error is reported because /x isn't a directory _at r0_,
> per the above explanation.  What happens if you try to put a file into
> the root directory?  If you keep the target as-is but change the value
> of the --revision argument to the revision in which ^/x was created?  To
> the revision just before that?
>
> > The same error occurs regardless of whether b.tmp is present.
> >
> > > > The previous behaviour was very useful, so are there any plans to reinstate it?
> > > >
> > >
> > > Patches welcome.  (You'll have to propose a new syntax, of course.)
> >
> > --revision -1
>
> -1 actually already has a meaning (SVN_INVALID_REVNUM).  More
> importantly, this approach makes it impossible to specify a base
> revision if any single operation is a "create exclusively" operation.
> Shouldn't the new syntax be per-operation, so people could combine
> "create exclusively" operations, "create or update" operations, and
> other kinds of operations in the same command line, _and_ have the
> option of specifying a base revision as well?
>
> > > > I don't think there is a straightforward way to guarantee the same
> > > > behaviour now.
> > > >
> > >
> > > Try:
> > >
> > > svn checkout --depth=empty $URL wc
> > > cd wc
> > > svn up --set-depth=infinite iota
> > > touch iota
> > > svn add iota
> > > svn commit -mm
> > > svn up --set-depth=empty iota
> > > svn cleanup                        # prune .svn/pristine
> >
> > Not exactly straightforward, but it does fail if the file has been
> > created meanwhile
> > However the error response still has to be analysed
> >
> > Also the script can fail in at least two places, depending on when the
> > file is created.
> >
>
> And why is that a problem?
>
> In the future, please provide all the needed information (reproduction
> recipes, error messages, the answer to "Why is it a problem?") up front.
> I don't intend to guess the missing parts and I don't have the
> brainwidth to prompt you every time.
>
> > > > The closest I could get is:
> > > >
> > > > 1) get current parent directory revision
> > > > 2) check if target file does not exist. This is not as easy as it
> > > > sounds, as the target directory may have too many files to list
> > > > efficiently, and any other file-based command may fail for a reason
> > > > other than a missing file.
> > >
> > > How is «svn info $URL/to/file@$REV» not sufficient?  You can use
> > > $URL/to{,/file}@$REV if you want, too.
> >
> > Same issue: svn info only returns success if the file exists.
> > An error may mean the file did not exist or something else, so the
> > error text has to be analysed.
>
> Use the machine-parseable E000042 error codes.  That's exactly what
> they're for.  (which-error.py and svn_error_symbolic_name() can be used
> to convert numbers to symbolic names.)

Where are these error codes defined?
I could not find any reference to them in the documentation.

> > > > 3) Put the file using the revision obtained in step 1.
> > > > AFAICT this is guaranteed not to replace an existing file.
> > > >
> > > > However it may fail to create the file if the target directory has
> > > > been updated in the meantime.
> > > >
> > > > It's only safe to repeat the attempted create if the command failed
> > > > due to an out of date revision.
> > > > So the failure reason will have to be analysed.
> > >
> > > What part of the above is a problem, and why?
> >
> > It requires analysing the error response, which is likely to be fragile.
>
> See above.

Re: svnmucc --revision 0 no longer works when creating a file

[Daniel Shahaf <d....@daniel.shahaf.name>]

sebb wrote on Sun, 12 Jul 2020 16:55 +0100:
> On Sun, 12 Jul 2020 at 15:26, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> >
> > sebb wrote on Tue, 07 Jul 2020 20:43 +0100:  
> > > When I first started using svnmucc, it used to be the case that
> > > svnmucc 'put' --revision 0 would fail if the target file already
> > > existed. This no longer happens.
> > >  
> >
> > Is the file-to-be's parent directory the root directory?  
> 
> No, it's not.
> 
> > If that isn't the case, then the new behaviour is correct.  
> 
> Why is that?
> 

Because the target of the 'put' operation didn't exist at r0, and the
base revision is specified to be a revision in which the target of the
operation existed.  (See svn_delta_editor_t::open_root()'s docstring.)

Moreover, even that syntax it did work, it should arguably fail if the
file had been created and subsequently deleted, which isn't the same
semantics as the algorithm you posted.

> > You might wish to post the error message.  
> 
> Just tried with a local SVN repo:
> 
> $ svnmucc -mBug --revision 0 -- put /dev/null
> file:///var/tools/svnrep/asf/x/b.tmp
> svnmucc: E160016: Can't commit to 'file:///var/tools/svnrep/asf/x'
> because it is not a directory
> 
> That message is wrong, because /x/ *is* a directory.

Runnable reproduction recipe, please.

I wonder if the error is reported because /x isn't a directory _at r0_,
per the above explanation.  What happens if you try to put a file into
the root directory?  If you keep the target as-is but change the value
of the --revision argument to the revision in which ^/x was created?  To
the revision just before that?

> The same error occurs regardless of whether b.tmp is present.
> 
> > > The previous behaviour was very useful, so are there any plans to reinstate it?
> > >  
> >
> > Patches welcome.  (You'll have to propose a new syntax, of course.)  
> 
> --revision -1

-1 actually already has a meaning (SVN_INVALID_REVNUM).  More
importantly, this approach makes it impossible to specify a base
revision if any single operation is a "create exclusively" operation.
Shouldn't the new syntax be per-operation, so people could combine
"create exclusively" operations, "create or update" operations, and
other kinds of operations in the same command line, _and_ have the
option of specifying a base revision as well?

> > > I don't think there is a straightforward way to guarantee the same
> > > behaviour now.
> > >  
> >
> > Try:
> >
> > svn checkout --depth=empty $URL wc
> > cd wc
> > svn up --set-depth=infinite iota
> > touch iota
> > svn add iota
> > svn commit -mm
> > svn up --set-depth=empty iota
> > svn cleanup                        # prune .svn/pristine  
> 
> Not exactly straightforward, but it does fail if the file has been
> created meanwhile
> However the error response still has to be analysed
> 
> Also the script can fail in at least two places, depending on when the
> file is created.
> 

And why is that a problem?

In the future, please provide all the needed information (reproduction
recipes, error messages, the answer to "Why is it a problem?") up front.
I don't intend to guess the missing parts and I don't have the
brainwidth to prompt you every time.

> > > The closest I could get is:
> > >
> > > 1) get current parent directory revision
> > > 2) check if target file does not exist. This is not as easy as it
> > > sounds, as the target directory may have too many files to list
> > > efficiently, and any other file-based command may fail for a reason
> > > other than a missing file.  
> >
> > How is «svn info $URL/to/file@$REV» not sufficient?  You can use  
> > $URL/to{,/file}@$REV if you want, too.  
> 
> Same issue: svn info only returns success if the file exists.
> An error may mean the file did not exist or something else, so the
> error text has to be analysed.

Use the machine-parseable E000042 error codes.  That's exactly what
they're for.  (which-error.py and svn_error_symbolic_name() can be used
to convert numbers to symbolic names.)

> > > 3) Put the file using the revision obtained in step 1.
> > > AFAICT this is guaranteed not to replace an existing file.
> > >
> > > However it may fail to create the file if the target directory has
> > > been updated in the meantime.
> > >
> > > It's only safe to repeat the attempted create if the command failed
> > > due to an out of date revision.
> > > So the failure reason will have to be analysed.  
> >
> > What part of the above is a problem, and why?  
> 
> It requires analysing the error response, which is likely to be fragile.

See above.

Re: svnmucc --revision 0 no longer works when creating a file

[sebb <se...@gmail.com>]

On Sun, 12 Jul 2020 at 15:26, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
>
> sebb wrote on Tue, 07 Jul 2020 20:43 +0100:
> > When I first started using svnmucc, it used to be the case that
> > svnmucc 'put' --revision 0 would fail if the target file already
> > existed. This no longer happens.
> >
>
> Is the file-to-be's parent directory the root directory?

No, it's not.

> If that isn't the case, then the new behaviour is correct.

Why is that?

> You might wish to post the error message.

Just tried with a local SVN repo:

$ svnmucc -mBug --revision 0 -- put /dev/null
file:///var/tools/svnrep/asf/x/b.tmp
svnmucc: E160016: Can't commit to 'file:///var/tools/svnrep/asf/x'
because it is not a directory

That message is wrong, because /x/ *is* a directory.

The same error occurs regardless of whether b.tmp is present.

> > The previous behaviour was very useful, so are there any plans to reinstate it?
> >
>
> Patches welcome.  (You'll have to propose a new syntax, of course.)

--revision -1

> > I don't think there is a straightforward way to guarantee the same
> > behaviour now.
> >
>
> Try:
>
> svn checkout --depth=empty $URL wc
> cd wc
> svn up --set-depth=infinite iota
> touch iota
> svn add iota
> svn commit -mm
> svn up --set-depth=empty iota
> svn cleanup                        # prune .svn/pristine

Not exactly straightforward, but it does fail if the file has been
created meanwhile
However the error response still has to be analysed

Also the script can fail in at least two places, depending on when the
file is created.

> > The closest I could get is:
> >
> > 1) get current parent directory revision
> > 2) check if target file does not exist. This is not as easy as it
> > sounds, as the target directory may have too many files to list
> > efficiently, and any other file-based command may fail for a reason
> > other than a missing file.
>
> How is «svn info $URL/to/file@$REV» not sufficient?  You can use
> $URL/to{,/file}@$REV if you want, too.

Same issue: svn info only returns success if the file exists.
An error may mean the file did not exist or something else, so the
error text has to be analysed.

> > 3) Put the file using the revision obtained in step 1.
> > AFAICT this is guaranteed not to replace an existing file.
> >
> > However it may fail to create the file if the target directory has
> > been updated in the meantime.
> >
> > It's only safe to repeat the attempted create if the command failed
> > due to an out of date revision.
> > So the failure reason will have to be analysed.
>
> What part of the above is a problem, and why?

It requires analysing the error response, which is likely to be fragile.

Re: svnmucc --revision 0 no longer works when creating a file

[Daniel Shahaf <d....@daniel.shahaf.name>]

sebb wrote on Tue, 07 Jul 2020 20:43 +0100:
> When I first started using svnmucc, it used to be the case that
> svnmucc 'put' --revision 0 would fail if the target file already
> existed. This no longer happens.
> 

Is the file-to-be's parent directory the root directory?  If that isn't
the case, then the new behaviour is correct.

You might wish to post the error message.

> The previous behaviour was very useful, so are there any plans to reinstate it?
> 

Patches welcome.  (You'll have to propose a new syntax, of course.)

> I don't think there is a straightforward way to guarantee the same
> behaviour now.
> 

Try:

svn checkout --depth=empty $URL wc
cd wc
svn up --set-depth=infinite iota
touch iota
svn add iota
svn commit -mm
svn up --set-depth=empty iota
svn cleanup                        # prune .svn/pristine

> The closest I could get is:
> 
> 1) get current parent directory revision
> 2) check if target file does not exist. This is not as easy as it
> sounds, as the target directory may have too many files to list
> efficiently, and any other file-based command may fail for a reason
> other than a missing file.

How is «svn info $URL/to/file@$REV» not sufficient?  You can use
$URL/to{,/file}@$REV if you want, too.

> 3) Put the file using the revision obtained in step 1.
> AFAICT this is guaranteed not to replace an existing file.
> 
> However it may fail to create the file if the target directory has
> been updated in the meantime.
> 
> It's only safe to repeat the attempted create if the command failed
> due to an out of date revision.
> So the failure reason will have to be analysed.

What part of the above is a problem, and why?