You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Lathan Bidwell <la...@andrews.edu> on 2015/01/27 16:38:31 UTC
Perl Bindings: Assertion svn_uri_is_canonical failed
I am running SVN 1.6 (will be upgrading to 1.8.8) through apache 2.4 /
mod_perl.
I'm trying to get email notifications of errors, but whenever libsvn fails
one of its assertions, it dies, and causes the whole apache process to core
dump / seg fault.
Is there some way to either check for these assertions in a non-lethal way
before hand, or make it possible to surivve the error so my error reporting
can run?
Usually these assertions are problems with my url handling: double slash,
or slash at end of directory. But I'd still like those errors to be
reported, so I know that my users are hitting problems.
Thanks,
Lathan
Re: Perl Bindings: Assertion svn_uri_is_canonical failed
Posted by Branko Čibej <br...@wandisco.com>.
On 27.01.2015 17:41, Stefan Sperling wrote:
> On Tue, Jan 27, 2015 at 11:30:55AM -0500, Lathan Bidwell wrote:
>> I fully agree, the error is correct and deserved.
>> It is happening on both my 1.6 prod servers and my 1.8.8 test servers.
>>
>> And I fully understand, and will do more to sanitize my inputs.
>>
>> But the fact that warnings cause a seg fault / core fault of the whole
>> apache process makes my error reporting very difficult.
>>
>> All I am asking is how / why does that error completely kill my execution
>> chain.
> Well, these crashes are not supposed to happen at all.
>
> As an API user you're supposed to run your URL input through
> svn_uri_canonicalize(). Else you'll get undefined behaviour,
> which allows SVN to abort() or do anything else it wants.
> Apart from forcing API users to provide sane input we also use
> this to find bugs in SVN's own code.
I think he's asking whether he can intercept the assertions before they
abort the process.
We do have that option in C, where you can register a malfunction
handler. JavaHL uses it to magically turn an abort into a segfault,
which lets the JVM produce a crash dump for easier debugging.
I don't know if we expose the malfunction handler API in the Perl bindings.
-- Brane
Re: Perl Bindings: Assertion svn_uri_is_canonical failed
Posted by Lathan Bidwell <la...@andrews.edu>.
Hi Ben,
Thanks, you've answered all my questions before I could even ask them.
I wrote a perl function to find those calls, and pull usage out of the
runtime errors.
But now, perldoc ftw.
By the way, these aren't too bad for docs:
http://search.cpan.org/~mschwern/Alien-SVN-v1.8.11.0/
On Tue, Jan 27, 2015 at 4:12 PM, Ben Reser <be...@reser.org> wrote:
> On 1/27/15 9:07 AM, Lathan Bidwell wrote:
> > By the way, the SWIG definitions are different enough that they could
> use their
> > own documentation page.
>
> They actually do have a fair amount of documentation. It's just not on a
> web
> page on the website.
>
> Use the perldoc command against the modules e.g.
> perldoc SVN::Client
> perldoc SVN::Core
> and so on...
>
> > However, even fixing that, I am still worried about errors that just
> crash the
> > apache process.
>
> All the SWIG bindings require the user to do their own path
> canonicalizations.
> You probably want to read this bit of documentation from the C API:
> http://subversion.apache.org/docs/api/1.8/svn__dirent__uri_8h.html#details
>
> As mentioned by Stefan Sperling failure to canonicalize paths will result
> in
> undefined behavior, generally aborts(). This is a design decision of the C
> APIs. Unfortunately, nobody decided to make the SWIG bindings
> automatically
> canonicalize for you like JavaHL does.
>
> I believe our APIs always provide canonical paths. So any place you're
> generating your own paths (and yes URIs are paths) you need to do your own
> canonicalization before passing it to the SVN libraries.
>
> I updated the documentation to reflect this last year on trunk, and it'll
> show
> up in the 1.9.0 release.
>
> For instance the following definitions of $path and $target help point you
> the
> right way:
> $path
> This is a path to a file or directory on the local file system.
> Paths need to be canonicalized before being passed into the
> Subversion APIs. Paths on the local file system are called
> dirents
> and can be canonicalized by calling
> "SVN::Core::dirent_canonicalize".
>
> $target
> This is a path to a file or directory in a working copy or a
> URL to
> a file or directory in a subversion repository. Both paths and
> URLs need to be canonicalized before being passed into the
> Subversion APIs. Paths on the local file system are called
> dirents
> and can be canonicalized by calling
> "SVN::Core::dirent_canonicalize". URLs can be canonicalized by
> calling "SVN::Core::uri_canonicalize".
>
> The example in the SVN::Client has also been updated to show that you
> should be
> doing this:
>
> use SVN::Client;
> my $client = new SVN::Client();
>
> # setup to handle authentication the same as the command line
> client
> my $config_dir = undef; # use default location
> my $config = SVN:Core::config_get_config($config_dir);
> my $config_category = $cfg->{SVN::Core::CONFIG_CATEGORY_CONFIG};
> $client->auth(
> SVN::Core::cmdline_create_auth_baton(0,
> #non_interactive
> undef, #username
> undef, #password
> $config_dir,
> 0,
> #no_auth_cache
> 0,
> #trust_server_cert
> $config_category,
> undef)
> #cancel_callback
> );
>
> # Use first argument as target and canonicalize it before using
> my $target;
> if (SVN::Core::path_is_url($ARGV[0])) {
> $target = SVN::Core::uri_canonicalize($ARGV[0]);
> } else {
> $target = SVN::Core::dirent_canonicalize($ARGV[0]);
> }
>
> # fetch the head revision of the target
> $client->cat(\*STDOUT, $target, 'HEAD');
>
> > Yes, I'd like to intercept the errors, notify the development email
> address of
> > the error, and give my user a nicer error message.
> >
> > I'm not that great going through SWIG source, what would it look like?
>
> You'd need to use svn_error_set_malfunction_handler():
>
> http://subversion.apache.org/docs/api/1.8/group__svn__error__malfunction__assertion.html#ga399ab33aa7fcea5cc776a62b56b7ad06
>
> But I don't believe we've got a thunk for that in any SWIG bindings so I
> don't
> think you can actually pass in a Perl function for it to callback when a
> malfunction happens. I'll look at wrapping this but it'll take a while to
> show
> up in a released version (either a 1.8.x backport or 1.9.0).
>
> So your more immediate solution is to canonicalize before calling our APIs.
>
>
>
> --
> BEGIN-ANTISPAM-VOTING-LINKS
> ------------------------------------------------------
>
> Teach CanIt if this mail (ID 0aNJ9cUOn) is spam:
> Spam:
> http://www.andrews.edu/spam/b.php?i=0aNJ9cUOn&m=8af47455e497&c=s
> Not spam:
> http://www.andrews.edu/spam/b.php?i=0aNJ9cUOn&m=8af47455e497&c=n
> Forget vote:
> http://www.andrews.edu/spam/b.php?i=0aNJ9cUOn&m=8af47455e497&c=f
> ------------------------------------------------------
> END-ANTISPAM-VOTING-LINKS
>
>
Re: Perl Bindings: Assertion svn_uri_is_canonical failed
Posted by Ben Reser <be...@reser.org>.
On 1/27/15 9:07 AM, Lathan Bidwell wrote:
> By the way, the SWIG definitions are different enough that they could use their
> own documentation page.
They actually do have a fair amount of documentation. It's just not on a web
page on the website.
Use the perldoc command against the modules e.g.
perldoc SVN::Client
perldoc SVN::Core
and so on...
> However, even fixing that, I am still worried about errors that just crash the
> apache process.
All the SWIG bindings require the user to do their own path canonicalizations.
You probably want to read this bit of documentation from the C API:
http://subversion.apache.org/docs/api/1.8/svn__dirent__uri_8h.html#details
As mentioned by Stefan Sperling failure to canonicalize paths will result in
undefined behavior, generally aborts(). This is a design decision of the C
APIs. Unfortunately, nobody decided to make the SWIG bindings automatically
canonicalize for you like JavaHL does.
I believe our APIs always provide canonical paths. So any place you're
generating your own paths (and yes URIs are paths) you need to do your own
canonicalization before passing it to the SVN libraries.
I updated the documentation to reflect this last year on trunk, and it'll show
up in the 1.9.0 release.
For instance the following definitions of $path and $target help point you the
right way:
$path
This is a path to a file or directory on the local file system.
Paths need to be canonicalized before being passed into the
Subversion APIs. Paths on the local file system are called dirents
and can be canonicalized by calling
"SVN::Core::dirent_canonicalize".
$target
This is a path to a file or directory in a working copy or a URL to
a file or directory in a subversion repository. Both paths and
URLs need to be canonicalized before being passed into the
Subversion APIs. Paths on the local file system are called dirents
and can be canonicalized by calling
"SVN::Core::dirent_canonicalize". URLs can be canonicalized by
calling "SVN::Core::uri_canonicalize".
The example in the SVN::Client has also been updated to show that you should be
doing this:
use SVN::Client;
my $client = new SVN::Client();
# setup to handle authentication the same as the command line client
my $config_dir = undef; # use default location
my $config = SVN:Core::config_get_config($config_dir);
my $config_category = $cfg->{SVN::Core::CONFIG_CATEGORY_CONFIG};
$client->auth(
SVN::Core::cmdline_create_auth_baton(0, #non_interactive
undef, #username
undef, #password
$config_dir,
0, #no_auth_cache
0, #trust_server_cert
$config_category,
undef) #cancel_callback
);
# Use first argument as target and canonicalize it before using
my $target;
if (SVN::Core::path_is_url($ARGV[0])) {
$target = SVN::Core::uri_canonicalize($ARGV[0]);
} else {
$target = SVN::Core::dirent_canonicalize($ARGV[0]);
}
# fetch the head revision of the target
$client->cat(\*STDOUT, $target, 'HEAD');
> Yes, I'd like to intercept the errors, notify the development email address of
> the error, and give my user a nicer error message.
>
> I'm not that great going through SWIG source, what would it look like?
You'd need to use svn_error_set_malfunction_handler():
http://subversion.apache.org/docs/api/1.8/group__svn__error__malfunction__assertion.html#ga399ab33aa7fcea5cc776a62b56b7ad06
But I don't believe we've got a thunk for that in any SWIG bindings so I don't
think you can actually pass in a Perl function for it to callback when a
malfunction happens. I'll look at wrapping this but it'll take a while to show
up in a released version (either a 1.8.x backport or 1.9.0).
So your more immediate solution is to canonicalize before calling our APIs.
Re: Perl Bindings: Assertion svn_uri_is_canonical failed
Posted by Ben Reser <be...@reser.org>.
On 1/27/15 9:24 AM, Lathan Bidwell wrote:
> In the Perl SWIG bindings:
>
> $ctx->url_from_path
>
> I believe calls the svn_uri_canonicalize in 1.8.8, but I believe that its a
> secondary effect (in 1.6, it doesn't cleanse the url)
$ctx->url_from_path is calling svn_client_url_from_path():
http://subversion.apache.org/docs/api/1.8/group__clnt__sessions.html#ga32b1f7f851452ec8ecc8c2d899de2b0e
svn_client is dropped since $ctx is the client library "object".
The purpose of that function is to provide a entry URL for a node in the
repository. You can abuse it to do canonicalizations (because it was poorly
designed) but that's not the purpose of the API. So you should use
SVN::Core::uri_canonicalize().
Re: Perl Bindings: Assertion svn_uri_is_canonical failed
Posted by Branko Čibej <br...@wandisco.com>.
On 27.01.2015 20:30, Stefan Sperling wrote:
> On Tue, Jan 27, 2015 at 07:52:49PM +0100, Stefan Sperling wrote:
>> prints "http:///svn.example.com/foo/bar"
>> (Why 3 slashes after "http:" instead of 2? I don't know. It seems our test
>> suite doesn't cover this case. I'm not sure what's expected.)
> This turns out to be expected. Whether or not such a URL works depends
> on the scheme. An extra slash is specified for "file://" but not "http://".
> svn_uri_canonicalize() only cares about syntax, not semantics.
> http://colabti.org/irclogger/irclogger_log/svn-dev?date=2015-01-27#l408
Let's be precise here: It's not "an extra slash" but "an empty authority
section in the URL". As I said on IRC, it's not svn_uri_canonicalize's
job to make sure the URL is valid, only that it's in a canonical form.
-- Brane
Re: Perl Bindings: Assertion svn_uri_is_canonical failed
Posted by Stefan Sperling <st...@elego.de>.
On Tue, Jan 27, 2015 at 07:52:49PM +0100, Stefan Sperling wrote:
> prints "http:///svn.example.com/foo/bar"
> (Why 3 slashes after "http:" instead of 2? I don't know. It seems our test
> suite doesn't cover this case. I'm not sure what's expected.)
This turns out to be expected. Whether or not such a URL works depends
on the scheme. An extra slash is specified for "file://" but not "http://".
svn_uri_canonicalize() only cares about syntax, not semantics.
http://colabti.org/irclogger/irclogger_log/svn-dev?date=2015-01-27#l408
Re: Perl Bindings: Assertion svn_uri_is_canonical failed
Posted by Ben Reser <be...@reser.org>.
On 1/27/15 10:52 AM, Stefan Sperling wrote:
> I suppose _Core (with leading underscore) is a private namespace.
> I don't know why this isn't part of the public SVN::Core:: namespace.
> It probably should be.
It is. SVN:Core::uri_canonicalize();
_Core is the raw SWIG wrapped interfaces.
Core cuts out extraneous bits of the symbols and in many cases provide a lot of
help to make things behave in a more Perl like manner (in this case it's just a
thin wrapper).
In general if it's a in libsvn_subr then it's in SVN:Core with the svn_ prefix
dropped.
> Unfortunately the bindings are badly maintained and documentation is
> almost non-existent :(
> See http://svn.haxx.se/dev/archive-2011-02/0544.shtml
While your statement isn't entirely untrue, the link is about the Python
bindings. The Perl bindings have some documentation, far more than the Python
bindings. But could certainly use more and of course patches would be welcome.
Re: Perl Bindings: Assertion svn_uri_is_canonical failed
Posted by Stefan Sperling <st...@elego.de>.
On Tue, Jan 27, 2015 at 12:24:47PM -0500, Lathan Bidwell wrote:
> In the Perl SWIG bindings:
>
> $ctx->url_from_path
>
> I believe calls the svn_uri_canonicalize in 1.8.8, but I believe that its a
> secondary effect (in 1.6, it doesn't cleanse the url)
>
> Is there a perl SWIG binding for svn_uri_canonicalize?
I'm only familiar with the C API, not much with the bindings.
There is this: SVN::_Core::svn_uri_canonicalize
The script:
use SVN::Core;
my $uri = SVN::_Core::svn_uri_canonicalize("http:////svn.example.com//foo/bar///");
print $uri."\n"
prints "http:///svn.example.com/foo/bar"
(Why 3 slashes after "http:" instead of 2? I don't know. It seems our test
suite doesn't cover this case. I'm not sure what's expected.)
I suppose _Core (with leading underscore) is a private namespace.
I don't know why this isn't part of the public SVN::Core:: namespace.
It probably should be.
Unfortunately the bindings are badly maintained and documentation is
almost non-existent :(
See http://svn.haxx.se/dev/archive-2011-02/0544.shtml
and also "Improve bindings to other programming languages"
on this page: http://subversion.apache.org/ideas.html#ideas
Re: Perl Bindings: Assertion svn_uri_is_canonical failed
Posted by Lathan Bidwell <la...@andrews.edu>.
In the Perl SWIG bindings:
$ctx->url_from_path
I believe calls the svn_uri_canonicalize in 1.8.8, but I believe that its a
secondary effect (in 1.6, it doesn't cleanse the url)
Is there a perl SWIG binding for svn_uri_canonicalize?
Lathan
On Tue, Jan 27, 2015 at 12:07 PM, Lathan Bidwell <la...@andrews.edu> wrote:
>
> svn_uri_canonicalize(). Else you'll get undefined behaviour,
>>
>> I think I have found the SWIG perl binding for one of these cleansing
> functions, I will test that out and put it in.
>
> By the way, the SWIG definitions are different enough that they could use
> their own documentation page. svn_uri_canonicalize isn't passed, but I
> think it might be url_from_path
>
> However, even fixing that, I am still worried about errors that just crash
> the apache process.
>
> Branko Čibej says:
> >I think he's asking whether he can intercept the assertions before they
> abort the process.
>
> Yes, I'd like to intercept the errors, notify the development email
> address of the error, and give my user a nicer error message.
>
> I'm not that great going through SWIG source, what would it look like?
>
> Lathan
>
Re: Perl Bindings: Assertion svn_uri_is_canonical failed
Posted by Lathan Bidwell <la...@andrews.edu>.
> svn_uri_canonicalize(). Else you'll get undefined behaviour,
>
> I think I have found the SWIG perl binding for one of these cleansing
functions, I will test that out and put it in.
By the way, the SWIG definitions are different enough that they could use
their own documentation page. svn_uri_canonicalize isn't passed, but I
think it might be url_from_path
However, even fixing that, I am still worried about errors that just crash
the apache process.
Branko Čibej says:
>I think he's asking whether he can intercept the assertions before they
abort the process.
Yes, I'd like to intercept the errors, notify the development email address
of the error, and give my user a nicer error message.
I'm not that great going through SWIG source, what would it look like?
Lathan
Re: Perl Bindings: Assertion svn_uri_is_canonical failed
Posted by Stefan Sperling <st...@elego.de>.
On Tue, Jan 27, 2015 at 11:30:55AM -0500, Lathan Bidwell wrote:
> I fully agree, the error is correct and deserved.
> It is happening on both my 1.6 prod servers and my 1.8.8 test servers.
>
> And I fully understand, and will do more to sanitize my inputs.
>
> But the fact that warnings cause a seg fault / core fault of the whole
> apache process makes my error reporting very difficult.
>
> All I am asking is how / why does that error completely kill my execution
> chain.
Well, these crashes are not supposed to happen at all.
As an API user you're supposed to run your URL input through
svn_uri_canonicalize(). Else you'll get undefined behaviour,
which allows SVN to abort() or do anything else it wants.
Apart from forcing API users to provide sane input we also use
this to find bugs in SVN's own code.
Re: Perl Bindings: Assertion svn_uri_is_canonical failed
Posted by Lathan Bidwell <la...@andrews.edu>.
I fully agree, the error is correct and deserved.
It is happening on both my 1.6 prod servers and my 1.8.8 test servers.
And I fully understand, and will do more to sanitize my inputs.
But the fact that warnings cause a seg fault / core fault of the whole
apache process makes my error reporting very difficult.
All I am asking is how / why does that error completely kill my execution
chain.
Perl test case example below:
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
use SVN::Client;
sub SVN_CMS_BASE { return 'http://websvn.cc.domain.com/svn/www/cms' }
sub get_svn_ctx
{
return new SVN::Client(
auth => [
SVN::Client::get_ssl_server_trust_prompt_provider(\&_trust_callback),
# SVN::Client::get_ssl_server_trust_file_provider(),
SVN::Client::get_simple_provider(),
SVN::Client::get_simple_prompt_provider(\&simple_prompt, 2),
SVN::Client::get_username_provider()
]
) || warn "Unable to get connection...?!?";
}
sub simple_prompt
{
my $cred = shift;
my $realm = shift;
my $default_username = shift;
my $may_save = shift;
my $pool = shift;
$cred->username('svnuser');
$cred->password('svnpassword');
}
my $ctx = get_svn_ctx();
eval{
$ctx->ls(SVN_CMS_BASE . '/','HEAD',1);
};
if($@){
print "Error Occured!!! Let's report it\n";
die $@;
}
print "We survived!!\n";
On Tue, Jan 27, 2015 at 11:05 AM, Stefan Sperling <st...@elego.de> wrote:
> On Tue, Jan 27, 2015 at 10:38:31AM -0500, Lathan Bidwell wrote:
> > I am running SVN 1.6 (will be upgrading to 1.8.8) through apache 2.4 /
> > mod_perl.
> >
> > I'm trying to get email notifications of errors, but whenever libsvn
> fails
> > one of its assertions, it dies, and causes the whole apache process to
> core
> > dump / seg fault.
> >
> > Is there some way to either check for these assertions in a non-lethal
> way
> > before hand, or make it possible to surivve the error so my error
> reporting
> > can run?
> >
> > Usually these assertions are problems with my url handling: double slash,
> > or slash at end of directory. But I'd still like those errors to be
> > reported, so I know that my users are hitting problems.
> >
> > Thanks,
> > Lathan
>
> A failure in svn_uri_is_canonical() sounds like you are running
> something (a Perl script since you mention mod_perl?) against the
> SVN APIs that provides bad input to SVN. Is this the case? If so,
> whatever is running will need to be fixed to provide valid input.
> To find out what SVN considers canonical please refer to the comment
> on top of this file:
>
> https://svn.apache.org/repos/asf/subversion/trunk/subversion/include/svn_dirent_uri.h
> as well as the comment on top of the function svn_uri_canonicalize()
> in the same file.
>
> 1.6 is very old so it's possible that your problems will disappear after
> an update to 1.8. If that's not the case then please invest some time
> into writing a problem report that includes detailed information about
> your setup (actual error messages extracted from log files, config files,
> hook scripts you are running, etc.)
>
>
> --
> BEGIN-ANTISPAM-VOTING-LINKS
> ------------------------------------------------------
>
> NOTE: This message was trained as non-spam. If this is wrong,
> please correct the training as soon as possible.
>
> Teach CanIt if this mail (ID 0aNJ45U2o) is spam:
> Spam:
> http://www.andrews.edu/spam/b.php?i=0aNJ45U2o&m=2a430390b425&c=s
> Not spam:
> http://www.andrews.edu/spam/b.php?i=0aNJ45U2o&m=2a430390b425&c=n
> Forget vote:
> http://www.andrews.edu/spam/b.php?i=0aNJ45U2o&m=2a430390b425&c=f
> ------------------------------------------------------
> END-ANTISPAM-VOTING-LINKS
>
>
Re: Perl Bindings: Assertion svn_uri_is_canonical failed
Posted by Stefan Sperling <st...@elego.de>.
On Tue, Jan 27, 2015 at 10:38:31AM -0500, Lathan Bidwell wrote:
> I am running SVN 1.6 (will be upgrading to 1.8.8) through apache 2.4 /
> mod_perl.
>
> I'm trying to get email notifications of errors, but whenever libsvn fails
> one of its assertions, it dies, and causes the whole apache process to core
> dump / seg fault.
>
> Is there some way to either check for these assertions in a non-lethal way
> before hand, or make it possible to surivve the error so my error reporting
> can run?
>
> Usually these assertions are problems with my url handling: double slash,
> or slash at end of directory. But I'd still like those errors to be
> reported, so I know that my users are hitting problems.
>
> Thanks,
> Lathan
A failure in svn_uri_is_canonical() sounds like you are running
something (a Perl script since you mention mod_perl?) against the
SVN APIs that provides bad input to SVN. Is this the case? If so,
whatever is running will need to be fixed to provide valid input.
To find out what SVN considers canonical please refer to the comment
on top of this file:
https://svn.apache.org/repos/asf/subversion/trunk/subversion/include/svn_dirent_uri.h
as well as the comment on top of the function svn_uri_canonicalize()
in the same file.
1.6 is very old so it's possible that your problems will disappear after
an update to 1.8. If that's not the case then please invest some time
into writing a problem report that includes detailed information about
your setup (actual error messages extracted from log files, config files,
hook scripts you are running, etc.)