You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Stefan Küng <to...@gmail.com> on 2013/06/20 20:19:43 UTC
Another crash in ra_serf in 1.8.0
Hi,
Another crash that's climbing up in the crash report statistics for TSVN.
Seems to be related to the previously discussed problem with checkouts
in TSVN.
The stack trace:
BowPad
libsvn_tsvn.dll!svn_ra_serf__credentials_callback(char * * username=0xffffffffffffffff, char * * password=0x0000000002ba0210, serf_request_t * request=0x0000000002ba0280, void * baton=0x0000000002c12588, int code=407, const char * authn_type=0x000007fee0bf0b58, const char * realm=0x0000000002c12b60, apr_pool_t * pool=0x0000000002bb8258) Line 1789 C
libsvn_tsvn.dll!serf__handle_basic_auth(int code=-524813808, serf_request_t * request=0x0000000002ba8368, serf_bucket_t * response=0x0000000000000001, const char * auth_hdr=0x0000000002bb30be, const char * auth_attr=0x0000000002bb30be, void * baton=0x0000000002bac270, apr_pool_t * pool=0x0000000002ba0198) Line 89 C
libsvn_tsvn.dll!handle_auth_header(void * baton=0x0000000002bb2fb8, const char * key=0x0000000002bac270, const char * header=0x0000000002ba8368) Line 223 C
libsvn_tsvn.dll!dispatch_auth(int code=45834840, serf_request_t * request=0x0000000002bac270, serf_bucket_t * response=0x0000000000000197, void * baton=0x0000000002bb24b8, apr_pool_t * pool=0x0000000002bb4238) Line 289 C
libsvn_tsvn.dll!serf__handle_auth_response(int * consumed_response=0x0000000000000000, serf_request_t * request=0x0000000062f73368, serf_bucket_t * response=0x0000000002bae419, void * baton=0x0000000000000002, apr_pool_t * pool=0x0000000002bb4238) Line 353 C
libsvn_tsvn.dll!handle_response(serf_request_t * request=0x0000000000000001, apr_pool_t * pool=0x0000000000000000) Line 859 C
libsvn_tsvn.dll!read_from_connection(serf_connection_t * conn=0x0000000002bb4238) Line 995 C
libsvn_tsvn.dll!serf__process_connection(serf_connection_t * conn=0x0000000002c127f8, short events=0) Line 1108 C
libsvn_tsvn.dll!serf_event_trigger(serf_context_t * s=0x0000000002c127f8, void * serf_baton=0x0000000002ba0210, const apr_pollfd_t * desc=0x0000000002baa1e8) Line 240 C
libsvn_tsvn.dll!serf_context_run(serf_context_t * ctx=0x0000000002b9e270, int duration=45744656, apr_pool_t * pool=0x0000000002ba0210) Line 308 C
libsvn_tsvn.dll!svn_ra_serf__context_run_wait(int * done=0x0000000002baa1e8, svn_ra_serf__session_t * sess=0x0000000002b9e188, apr_pool_t * scratch_pool=0x0000000002b9e620) Line 743 C
libsvn_tsvn.dll!svn_ra_serf__exchange_capabilities(svn_ra_serf__session_t * serf_sess=0x0000000002b9e518, const char * * corrected_url=0x000000000449efd1, apr_pool_t * pool=0x0000000002c12558) Line 488 C
libsvn_tsvn.dll!svn_ra_serf__open(svn_ra_session_t * session=0x0000000002b9e248, const char * * corrected_url=0x0000000002b9e4b8, const char * session_URL=0x0000000002b9e248, const svn_ra_callbacks2_t * callbacks=0x0000000002ba21a8, void * callback_baton=0x0000000002b99ac8, apr_hash_t * config=0x0000000002b982a0, apr_pool_t * pool=0x0000000002b9e188) Line 483 C
libsvn_tsvn.dll!svn_ra_open4(svn_ra_session_t * * session_p=0x000000000449f2c0, const char * * corrected_url_p=0x000000000449f200, const char * repos_URL=0x0000000000000002, const char * uuid=0x0000000000000000, const svn_ra_callbacks2_t * callbacks=0x0000000002b99af0, void * callback_baton=0x0000000002b99ac8, apr_hash_t * config=0x0000000002b982a0, apr_pool_t * pool=0x0000000002b98158) Line 482 C
libsvn_tsvn.dll!svn_client__open_ra_session_internal(svn_ra_session_t * * ra_session=0x000000000449f2c0, const char * * corrected_url=0x000000000449f2b8, const char * base_url=0x000000000449f590, const char * base_dir_abspath=0x0000000100000001, const apr_array_header_t * commit_items=0x0000000000000000, int write_dav_props=0, int read_dav_props=0, svn_client_ctx_t * ctx=0x0000000002b98df0, apr_pool_t * result_pool=0x0000000002b98158, apr_pool_t * scratch_pool=0x0000000002b98158) Line 393 C
libsvn_tsvn.dll!svn_client_info3(const char * abspath_or_url=0x000000000449f670, const svn_opt_revision_t * peg_revision=0x00000000001d6168, const svn_opt_revision_t * revision=0x000000000449f590, svn_depth_t depth=71956080, int fetch_excluded=1, int fetch_actual_only=1, const apr_array_header_t * changelists=0x0000000000000000, svn_error_t * (void *, const char *, const svn_client_info2_t *, apr_pool_t *) * receiver=0x000000013f4d09e0, void * receiver_baton=0x000000000449f670, svn_client_ctx_t * ctx=0x0000000002b98df0, apr_pool_t * pool=0x0000000002b98158) Line 300 C
TortoiseProc.exe!SVNInfo::GetFirstFileInfo(const CTSVNPath & path={...}, SVNRev pegrev={...}, SVNRev revision={...}, svn_depth_t depth=svn_depth_empty, bool fetchExcluded=112, bool fetchActualOnly=112) Line 136 C++
The crash is in libsvn_ra_serf\util.c, Snippet
svn_ra_serf__credentials_callback, line 1789:
BowPad *username = apr_pstrdup(pool, session->proxy_username);
*password = apr_pstrdup(pool, session->proxy_password);
The code here is 407, not 401.
Crashdumps are here:
https://www.crash-server.com/Problem.aspx?ClientID=tsvn&ProblemID=26622
Stefan
--
___
oo // \\ "De Chelonian Mobile"
(_,\/ \_/ \ TortoiseSVN
\ \_/_\_/> The coolest interface to (Sub)version control
/_/ \_\ http://tortoisesvn.net
Re: Another crash in ra_serf in 1.8.0
Posted by Greg Stein <gs...@gmail.com>.
On Fri, Jun 21, 2013 at 3:26 PM, Stefan Küng <to...@gmail.com> wrote:
>...
> I'm not sure, but another crash report seems related to this:
> https://www.crash-server.com/Problem.aspx?ClientID=tsvn&ProblemID=26624
>
> The stacktrace of this one:
>...
> libsvn_tsvn.dll!handle_response(serf_request_t * request=0x000000df49d27f18,
> serf_bucket_t * response=0x000000df49d2a678, svn_ra_serf__handler_t *
> handler=0x000000df49d2a5d8, int * serf_status=0x0000000000000000, apr_pool_t
> * scratch_pool=0x000000df49d2dbb8) Line 2060 C
> libsvn_tsvn.dll!handle_response_cb(serf_request_t *
> request=0x000000df49d27f18, serf_bucket_t * response=0x000000df49d2a5d8,
> void * baton=0x000007fe00000000, apr_pool_t *
> scratch_pool=0x0000000000000000) Line 2096 C
>...
I don't think this is the same. You appear to have a valid
request/response there.
At this point, I'd say it is unrelated. Needs to be fixed, sure, but
not related to proxy auth and SSL tunnels.
> The problem here is that in svn_wc__conflict_invoke_resolver(), the call
> if (cancel_func)
> SVN_ERR(cancel_func(cancel_baton));
>
> calls into invalid memory. The exception is actually "Access violation
> executing code" so it seems that here too a wrong/invalid ctx is used.
No idea right now... (focused on the fixes Lieven just made)
Cheers,
-g
Re: Another crash in ra_serf in 1.8.0
Posted by Stefan Küng <to...@gmail.com>.
On 20.06.2013 23:05, Lieven Govaerts wrote:
> Looks like it's the authentication handling when setting up a SSL
> tunnel that's at fault here, at least I can easily reproduce it with
> an apache http proxy connetion to a https repo. The ssl tunnel is
> started by a CONNECT request created by serf. When the proxy requests
> credentials, serf will call back to the application. As the
> application doesn't know about this request, it doesn't get a valid
> baton either, so can't get baton->session ... That baton it gets is
> the ctx used by the ssltunnel code. Hm, have to think about how we can
> solve this. Not sure it can be done with the existing API.
I'm not sure, but another crash report seems related to this:
https://www.crash-server.com/Problem.aspx?ClientID=tsvn&ProblemID=26624
The stacktrace of this one:
BowPad
TortoiseProc.exe!SVN::cancel(void * baton=0x000000df49062108) Line 1782 C++
libsvn_tsvn.dll!svn_wc__conflict_invoke_resolver(svn_wc__db_t * db=0x000000df49d1eac0, const char * local_abspath=0x000000df49063598, const svn_skel_t * conflict_skel=0x000000df49063598, const apr_array_header_t * merge_options=0x0000000000000000, svn_error_t * (svn_wc_conflict_result_t * *, const svn_wc_conflict_description2_t *, void *, apr_pool_t *, apr_pool_t *) * resolver_func=0x000007fe5cf570b0, void * resolver_baton=0x000000df48e10238, svn_error_t * (void *) * cancel_func=0x000007f74b709220, void * cancel_baton=0x000000df48e10238, apr_pool_t * scratch_pool=0x000000df49062108) Line 1937 C
libsvn_tsvn.dll!close_directory(void * dir_baton=0x000000df48e4c480, apr_pool_t * pool=0x000000df48e34218) Line 2911 C
libsvn_tsvn.dll!close_directory(void * dir_baton=0x000000df48e30298, apr_pool_t * pool=0x000000df48e304c8) Line 262 C
libsvn_tsvn.dll!close_dir(report_dir_t * dir=0x000000df00000259) Line 807 C
libsvn_tsvn.dll!maybe_close_dir_chain(report_dir_t * dir=0x0000000000000000) Line 1394 C
libsvn_tsvn.dll!end_report(svn_ra_serf__xml_parser_t * parser=0x000000df00000259, svn_ra_serf__dav_props_t name, apr_pool_t * scratch_pool=0x000000df49d9c880) Line 2238 C
libsvn_tsvn.dll!end_xml(void * userData=0x000000df49dd5610, const char * raw_name=0x000000df48e42c18) Line 1314 C
libaprutil_tsvn.dll!doContent(XML_ParserStruct * parser=0x0000000000000000, int startTagLevel=0, const encoding * enc=0x0000000057663180, const char * s=0x000000df4909d39b, const char * end=0x000000df4909ddc7, const char * * nextPtr=0x000000df49dd5640) Line 2236 C
libaprutil_tsvn.dll!contentProcessor(XML_ParserStruct * parser=0x000000df48e442a8, const char * start=0x0000000000000003, const char * end=0x000000df48e42418, const char * * endPtr=0x0000000000000000) Line 1817 C
libaprutil_tsvn.dll!XML_ParseBuffer(XML_ParserStruct * parser=0x000000df49dd5610, int len=0, int isFinal=0) Line 1482 C
libaprutil_tsvn.dll!XML_Parse(XML_ParserStruct * parser=0x0000000000000000, const char * s=0x00000000000000c8, int len=1238531864, int isFinal=1238541944) Line 1472 C
libsvn_tsvn.dll!svn_ra_serf__handle_xml_parser(serf_request_t * request=0x000000df49d27f18, serf_bucket_t * response=0x000000df49d31c19, void * baton=0x0000000000000ad7, apr_pool_t * pool=0x0000000000000000) Line 1681 C
libsvn_tsvn.dll!handle_response(serf_request_t * request=0x000000df49d27f18, serf_bucket_t * response=0x000000df49d2a678, svn_ra_serf__handler_t * handler=0x000000df49d2a5d8, int * serf_status=0x0000000000000000, apr_pool_t * scratch_pool=0x000000df49d2dbb8) Line 2060 C
libsvn_tsvn.dll!handle_response_cb(serf_request_t * request=0x000000df49d27f18, serf_bucket_t * response=0x000000df49d2a5d8, void * baton=0x000007fe00000000, apr_pool_t * scratch_pool=0x0000000000000000) Line 2096 C
libsvn_tsvn.dll!handle_response(serf_request_t * request=0x0000000000000000, apr_pool_t * pool=0x0000000000000000) Line 872 C
libsvn_tsvn.dll!read_from_connection(serf_connection_t * conn=0x000000df49d2dbb8) Line 995 C
libsvn_tsvn.dll!serf__process_connection(serf_connection_t * conn=0x000000df4668a318, short events=-22920) Line 1108 C
libsvn_tsvn.dll!serf_event_trigger(serf_context_t * s=0x000000df4668a318, void * serf_baton=0x000000df49d2fc40, const apr_pollfd_t * desc=0x000000df48e16128) Line 240 C
libsvn_tsvn.dll!serf_context_run(serf_context_t * ctx=0x0000000000000000, int duration=1238563904, apr_pool_t * pool=0x000000df49d2fc40) Line 308 C
libsvn_tsvn.dll!finish_report(void * report_baton=0x000000df48e16128, apr_pool_t * pool=0x0000000000000000) Line 2854 C
libsvn_tsvn.dll!svn_wc_crawl_revisions5(svn_wc_context_t * wc_ctx=0x000000df49d1eaa8, const char * local_abspath=0x000000df48e10208, const svn_ra_reporter3_t * reporter=0x000007fe5d239ca8, void * report_baton=0x000000df48e3a248, int restore_files=1, svn_depth_t depth=svn_depth_unknown, int honor_depth_exclude=1, int depth_compatibility_trick=1, int use_commit_times=0, svn_error_t * (void *) * cancel_func=0x000007f74b709220, void * cancel_baton=0x000000df4641e030, void (void *, const svn_wc_notify_t *, apr_pool_t *) * notify_func=0x000007f74b708bf0, void * notify_baton=0x000000df4641e030, apr_pool_t * scratch_pool=0x000000df48e100f8) Line 845 C
libsvn_tsvn.dll!update_internal(long * result_rev=0x000000df4b20f280, apr_hash_t * conflicted_paths=0x000000df48e10238, const char * local_abspath=0x000000df48e10208, const char * anchor_abspath=0x000000df48e103b0, const svn_opt_revision_t * revision=0x000000df4b20f1b8, svn_depth_t depth=svn_depth_unknown, int depth_is_sticky=0, int ignore_externals=0, int allow_unver_obstructions=1, int adds_as_modification=1, int * timestamp_sleep=0x000000df4b20f284, int notify_summary=1, svn_client_ctx_t * ctx=0x0000000000000000, apr_pool_t * pool=0x000000df48e100f8) Line 466 C
libsvn_tsvn.dll!svn_client__update_internal(long * result_rev=0x000000df49d7aab8, const char * local_abspath=0x000000df48e10208, const svn_opt_revision_t * revision=0x000000df48e10238, svn_depth_t depth=svn_depth_unknown, int depth_is_sticky=0, int ignore_externals=0, int allow_unver_obstructions=1, int adds_as_modification=1, int make_parents=1, int innerupdate=0, int * timestamp_sleep=0x000000df4b20f284, svn_client_ctx_t * ctx=0x000000df48e103b0, apr_pool_t * pool=0x000000df48e100f8) Line 600 C
libsvn_tsvn.dll!svn_client_update4(apr_array_header_t * * result_revs=0x0000000000000000, const apr_array_header_t * paths=0x000000df49d21bf8, const svn_opt_revision_t * revision=0x000000df4641e750, svn_depth_t depth=svn_depth_unknown, int depth_is_sticky=0, int ignore_externals=0, int allow_unver_obstructions=1, int adds_as_modification=1, int make_parents=1, svn_client_ctx_t * ctx=0x000000df49d1e9f0, apr_pool_t * pool=0x000000df49d37c08) Line 675 C
The problem here is that in Snippet svn_wc__conflict_invoke_resolver(),
the call
if (cancel_func)
SVN_ERR(cancel_func(cancel_baton));
calls into invalid memory. The exception is actually "Access violation
*executing *code" so it seems that here too a wrong/invalid ctx is used.
(sorry for the html mail, but with text the long lines of the stack
trace just get mangled to an unreadable state)
Stefan
--
___
oo // \\ "De Chelonian Mobile"
(_,\/ \_/ \ TortoiseSVN
\ \_/_\_/> The coolest interface to (Sub)version control
/_/ \_\ http://tortoisesvn.net
Re: Another crash in ra_serf in 1.8.0
Posted by Lieven Govaerts <sv...@mobsol.be>.
On Thu, Jun 20, 2013 at 10:30 PM, Greg Stein <gs...@gmail.com> wrote:
> On Thu, Jun 20, 2013 at 2:19 PM, Stefan Küng <to...@gmail.com> wrote:
>> Hi,
>>
>> Another crash that's climbing up in the crash report statistics for TSVN.
>> Seems to be related to the previously discussed problem with checkouts in
>> TSVN.
Thanks Stefan.
>> The stack trace:
>>
>> BowPad
>>
>> libsvn_tsvn.dll!svn_ra_serf__credentials_callback(char * *
>> username=0xffffffffffffffff, char * * password=0x0000000002ba0210,
>> serf_request_t * request=0x0000000002ba0280, void *
>> baton=0x0000000002c12588, int code=407, const char *
>> authn_type=0x000007fee0bf0b58, const char * realm=0x0000000002c12b60,
>> apr_pool_t * pool=0x0000000002bb8258) Line 1789 C
>
> The 407 code means a proxy authorization is needed.
>
> The problem in this stack frame is the username address. It clearly
> has a problematic value. It should only be about four bytes off from
> the password value.
>
>> libsvn_tsvn.dll!serf__handle_basic_auth(int code=-524813808, serf_request_t
>> * request=0x0000000002ba8368, serf_bucket_t * response=0x0000000000000001,
>> const char * auth_hdr=0x0000000002bb30be, const char *
>> auth_attr=0x0000000002bb30be, void * baton=0x0000000002bac270, apr_pool_t *
>> pool=0x0000000002ba0198) Line 89 C
>
> The code parameter in this traceback is clearly incorrect. It should
> be 407, just like what got passed to the callback.
>
> Either these tracebacks are wonky, or there may be some stack smashing
> going on. Stefan?
I guess it's optimized code.
>>...
>> The crash is in libsvn_ra_serf\util.c, Snippet
>> svn_ra_serf__credentials_callback, line 1789:
>> BowPad *username = apr_pstrdup(pool, session->proxy_username);
>> *password = apr_pstrdup(pool, session->proxy_password);
>
> Was session bad? Or was the proxy_username bad?
>
Looks like it's the authentication handling when setting up a SSL
tunnel that's at fault here, at least I can easily reproduce it with
an apache http proxy connetion to a https repo.
The ssl tunnel is started by a CONNECT request created by serf. When
the proxy requests credentials, serf will call back to the
application. As the application doesn't know about this request, it
doesn't get a valid baton either, so can't get baton->session ...
That baton it gets is the ctx used by the ssltunnel code.
Hm, have to think about how we can solve this. Not sure it can be done
with the existing API.
> Thx,
> -g
Lieven
Re: Another crash in ra_serf in 1.8.0
Posted by Ivan Zhakov <iv...@visualsvn.com>.
On Fri, Jun 21, 2013 at 12:30 AM, Greg Stein <gs...@gmail.com> wrote:
> On Thu, Jun 20, 2013 at 2:19 PM, Stefan Küng <to...@gmail.com> wrote:
>> Hi,
>>
>> Another crash that's climbing up in the crash report statistics for TSVN.
>> Seems to be related to the previously discussed problem with checkouts in
>> TSVN.
>>
>> The stack trace:
>>
>> BowPad
>>
>> libsvn_tsvn.dll!svn_ra_serf__credentials_callback(char * *
>> username=0xffffffffffffffff, char * * password=0x0000000002ba0210,
>> serf_request_t * request=0x0000000002ba0280, void *
>> baton=0x0000000002c12588, int code=407, const char *
>> authn_type=0x000007fee0bf0b58, const char * realm=0x0000000002c12b60,
>> apr_pool_t * pool=0x0000000002bb8258) Line 1789 C
>
> The 407 code means a proxy authorization is needed.
>
> The problem in this stack frame is the username address. It clearly
> has a problematic value. It should only be about four bytes off from
> the password value.
>
The compiler may optimize the code and dump files doesn't have enough
information to find parameters values.
>> libsvn_tsvn.dll!serf__handle_basic_auth(int code=-524813808, serf_request_t
>> * request=0x0000000002ba8368, serf_bucket_t * response=0x0000000000000001,
>> const char * auth_hdr=0x0000000002bb30be, const char *
>> auth_attr=0x0000000002bb30be, void * baton=0x0000000002bac270, apr_pool_t *
>> pool=0x0000000002ba0198) Line 89 C
>
> The code parameter in this traceback is clearly incorrect. It should
> be 407, just like what got passed to the callback.
>
> Either these tracebacks are wonky, or there may be some stack smashing
> going on. Stefan?
>
>>...
>> The crash is in libsvn_ra_serf\util.c, Snippet
>> svn_ra_serf__credentials_callback, line 1789:
>> BowPad *username = apr_pstrdup(pool, session->proxy_username);
>> *password = apr_pstrdup(pool, session->proxy_password);
>
> Was session bad? Or was the proxy_username bad?
>
I've checked disassembly and it crashes on attempt to read
proxy_password from session. Because session pointer is zero, most
likely because 'handler' points to freed memory.
--
Ivan Zhakov
CTO | VisualSVN | http://www.visualsvn.com
Re: Another crash in ra_serf in 1.8.0
Posted by Greg Stein <gs...@gmail.com>.
On Thu, Jun 20, 2013 at 2:19 PM, Stefan Küng <to...@gmail.com> wrote:
> Hi,
>
> Another crash that's climbing up in the crash report statistics for TSVN.
> Seems to be related to the previously discussed problem with checkouts in
> TSVN.
>
> The stack trace:
>
> BowPad
>
> libsvn_tsvn.dll!svn_ra_serf__credentials_callback(char * *
> username=0xffffffffffffffff, char * * password=0x0000000002ba0210,
> serf_request_t * request=0x0000000002ba0280, void *
> baton=0x0000000002c12588, int code=407, const char *
> authn_type=0x000007fee0bf0b58, const char * realm=0x0000000002c12b60,
> apr_pool_t * pool=0x0000000002bb8258) Line 1789 C
The 407 code means a proxy authorization is needed.
The problem in this stack frame is the username address. It clearly
has a problematic value. It should only be about four bytes off from
the password value.
> libsvn_tsvn.dll!serf__handle_basic_auth(int code=-524813808, serf_request_t
> * request=0x0000000002ba8368, serf_bucket_t * response=0x0000000000000001,
> const char * auth_hdr=0x0000000002bb30be, const char *
> auth_attr=0x0000000002bb30be, void * baton=0x0000000002bac270, apr_pool_t *
> pool=0x0000000002ba0198) Line 89 C
The code parameter in this traceback is clearly incorrect. It should
be 407, just like what got passed to the callback.
Either these tracebacks are wonky, or there may be some stack smashing
going on. Stefan?
>...
> The crash is in libsvn_ra_serf\util.c, Snippet
> svn_ra_serf__credentials_callback, line 1789:
> BowPad *username = apr_pstrdup(pool, session->proxy_username);
> *password = apr_pstrdup(pool, session->proxy_password);
Was session bad? Or was the proxy_username bad?
Thx,
-g