You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modules-dev@httpd.apache.org by oh...@cox.net on 2012/06/22 01:49:52 UTC
SOLVED was Re: How to compiling/link/use Apache module that uses
shared library?
---- ohaya@cox.net wrote:
>
> ---- Sorin Manolache <so...@gmail.com> wrote:
> > And I forgot to say: run gdb in some sort of environment where you see
> > your current source code line and a couple of surrounding lines. You
> > could achieve this with the "list" command, but I prefer running gdb in
> > emacs and let emacs do the nice listing of source code in a different panel.
> >
> > S
>
>
> Here's the function from my source. It's the original from mod_headers.c, plus my printf:
>
>
> static int header_post_config(apr_pool_t *pconf, apr_pool_t *plog,
> apr_pool_t *ptemp, server_rec *s)
> {
> printf("In header_post_config\n");
> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> return OK;
> }
>
> Jim
Hi,
I was able to get the segfault to go away. Here's what I had to do:
- Created /etc/ld.so.conf.d/my.conf, and added the directory where my libobaccess.so was
- Run 'ldconfig' to activate.
- In the apxs command, DON'T include the -L and -l arguments
After that, Apache appears to start ok, without segfault :)!!
Thanks for all of the great help, esp. the suggestion about checking "ldconfig -p". I still don't understand why, but I'm just glad that I can get past this piece so now I can debug my module :)...
Later,
Jim
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use Apache
module that uses shared library?
Posted by Ben Noordhuis <in...@bnoordhuis.nl>.
On Fri, Jun 22, 2012 at 3:32 AM, <oh...@cox.net> wrote:
> Program received signal SIGSEGV, Segmentation fault.
> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>
> So, it's actually blowing up in "BN_num_bits()" in /lib64/libcrypto.so.4?
Type `bt full` and you'll get a backtrace + locals. That should tell
you where the error originates from and maybe why it happens.
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use
Apache module that uses shared library?
Posted by oh...@cox.net.
---- Rainer Jung <ra...@kippdata.de> wrote:
> On 22.06.2012 14:16, ohaya@cox.net wrote:
> >
> > ---- ohaya@cox.net wrote:
> >>
> >> ---- Rainer Jung <ra...@kippdata.de> wrote:
> >>> On 22.06.2012 06:10, Joe Lewis wrote:
> >>>> On 6/21/12 10:02 PM, ohaya@cox.net wrote:
> >>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >>>>>> On 6/21/12 9:39 PM, ohaya@cox.net wrote:
> >>>>>>> ---- ohaya@cox.net wrote:
> >>>>>>>> ---- ohaya@cox.net wrote:
> >>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >>>>>>>>>> On 6/21/12 7:32 PM, ohaya@cox.net wrote:
> >>>>>>>>>>> ---- ohaya@cox.net wrote:
> >>>>>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >>>>>>>>>>>>> On 6/21/12 6:46 PM, ohaya@cox.net wrote:
> >>>>>>>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >>>>>>>>>>>>>>> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
> >>>>>>>>>>>>>>>> ---- ohaya@cox.net wrote:
> >>>>>>>>>>>>>>>>> ---- Sorin Manolache<so...@gmail.com> wrote:
> >>>>>>>>>>>>>>>>>> And I forgot to say: run gdb in some sort of environment
> >>>>>>>>>>>>>>>>>> where you see
> >>>>>>>>>>>>>>>>>> your current source code line and a couple of surrounding
> >>>>>>>>>>>>>>>>>> lines. You
> >>>>>>>>>>>>>>>>>> could achieve this with the "list" command, but I prefer
> >>>>>>>>>>>>>>>>>> running gdb in
> >>>>>>>>>>>>>>>>>> emacs and let emacs do the nice listing of source code in
> >>>>>>>>>>>>>>>>>> a different panel.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> S
> >>>>>>>>>>>>>>>>> Here's the function from my source. It's the original
> >>>>>>>>>>>>>>>>> from mod_headers.c, plus my printf:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> static int header_post_config(apr_pool_t *pconf,
> >>>>>>>>>>>>>>>>> apr_pool_t *plog,
> >>>>>>>>>>>>>>>>> apr_pool_t *ptemp,
> >>>>>>>>>>>>>>>>> server_rec *s)
> >>>>>>>>>>>>>>>>> {
> >>>>>>>>>>>>>>>>> printf("In header_post_config\n");
> >>>>>>>>>>>>>>>>> header_ssl_lookup =
> >>>>>>>>>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>>>>>>>>>>> return OK;
> >>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Jim
> >>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I was able to get the segfault to go away. Here's what I
> >>>>>>>>>>>>>>>> had to do:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> - Created /etc/ld.so.conf.d/my.conf, and added the
> >>>>>>>>>>>>>>>> directory where my libobaccess.so was
> >>>>>>>>>>>>>>>> - Run 'ldconfig' to activate.
> >>>>>>>>>>>>>>>> - In the apxs command, DON'T include the -L and -l arguments
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> After that, Apache appears to start ok, without segfault :)!!
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks for all of the great help, esp. the suggestion about
> >>>>>>>>>>>>>>>> checking "ldconfig -p". I still don't understand why, but
> >>>>>>>>>>>>>>>> I'm just glad that I can get past this piece so now I can
> >>>>>>>>>>>>>>>> debug my module :)...
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Later,
> >>>>>>>>>>>>>>>> Jim
> >>>>>>>>>>>>>>> I'm just glad this list is as good as it is!
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> FYI, the ldconfig is the dynamic linker control, and those
> >>>>>>>>>>>>>>> /etc/ld.so.conf.d files provide additional search
> >>>>>>>>>>>>>>> directories for the
> >>>>>>>>>>>>>>> linker to check in when loading a library.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Joe
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> http://www.silverhawk.net/
> >>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Sorry to report, but my earlier report was a "false positive"
> >>>>>>>>>>>>>> :)...
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I forgot that the mod_headers.c that I was doing the earlier
> >>>>>>>>>>>>>> testing with had all references to the libobaccess.so removed
> >>>>>>>>>>>>>> :(!!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> So, I'm still stuck with basically the same problem, now,
> >>>>>>>>>>>>>> working with my "full" code, with the calls in it:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> - If I compile with -L and -l, Apache segfaults when it starts
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> - If I compile without -L and -l, then I get "undefined
> >>>>>>>>>>>>>> symbol" errors when I try to start Apache, e.g.:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> [root@apachemodule build-mod_headers]#
> >>>>>>>>>>>>>> /apps/httpd2222/bin/apachectl -k start -X
> >>>>>>>>>>>>>> httpd: Syntax error on line 84 of
> >>>>>>>>>>>>>> /apps/httpd2222/conf/httpd.conf: Cannot load
> >>>>>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so into server:
> >>>>>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so: undefined symbol:
> >>>>>>>>>>>>>> ObResource_isProtected
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> That "ObResource_isProtected" should be a symbol in
> >>>>>>>>>>>>>> libobaccess.so, and in fact, if I do "nm --dynamic", I get:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> [root@apachemodule build-mod_headers]# nm --dynamic
> >>>>>>>>>>>>>> /apps/netpoint/lib64/libobaccess.so | grep
> >>>>>>>>>>>>>> "ObResource_isProtected"
> >>>>>>>>>>>>>> 00000000000a6d80 T ObResource_isProtected
> >>>>>>>>>>>>>> [root@apachemodule build-mod_headers]#
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I'm *assuming* that the reason for the "undefined symbol"
> >>>>>>>>>>>>>> error is that libobaccess.so is actually not being loaded,
> >>>>>>>>>>>>>> but then when I try to load libobaccess.so, either via -L and
> >>>>>>>>>>>>>> -l in the apxs, or using LoadFile in httpd.conf, I get the
> >>>>>>>>>>>>>> segfault (same gdb info, BTW).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Catch-22?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Sorry for the false alarm :(!!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Jim
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> Not a catch-22. The -L and -l specify linker options when
> >>>>>>>>>>>>> assembling
> >>>>>>>>>>>>> the code. The ldconfig is a run-time thing. If you are
> >>>>>>>>>>>>> getting the
> >>>>>>>>>>>>> stderr messages, you are making it all the way into your
> >>>>>>>>>>>>> library. I'd
> >>>>>>>>>>>>> suggest commenting out the following line and see if you get
> >>>>>>>>>>>>> farther :
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> That should tell you if the problem is the ssl_var_lookup.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Joe
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> http://www.silverhawk.net/
> >>>>>>>>>>>> Hi,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks for the suggestion. I just tried what you suggested,
> >>>>>>>>>>>> and got a segfault when I started Apache with the modified module.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Jim
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>> Hi,
> >>>>>>>>>>>
> >>>>>>>>>>> As a reminder, here's the gdb with the library loaded:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> (gdb) b header_post_config
> >>>>>>>>>>> Function "header_post_config" not defined.
> >>>>>>>>>>> Make breakpoint pending on future shared library load? (y or [n]) y
> >>>>>>>>>>> Breakpoint 1 (header_post_config) pending.
> >>>>>>>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> >>>>>>>>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/
> >>>>>>>>>>> -f /apps/httpd2222/conf/httpd.conf
> >>>>>>>>>>> [Thread debugging using libthread_db enabled]
> >>>>>>>>>>> [New Thread 182897610272 (LWP 11317)]
> >>>>>>>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> >>>>>>>>>>> Pending breakpoint "header_post_config" resolved
> >>>>>>>>>>> mod_headers-jl V0.09 - start calling OAM API
> >>>>>>>>>>> In register_hooks
> >>>>>>>>>>> In create_headers_dir_config
> >>>>>>>>>>> In create_headers_dir_config
> >>>>>>>>>>> In header_cmd
> >>>>>>>>>>> In header_inout_cmd
> >>>>>>>>>>> In parse_format_tag
> >>>>>>>>>>> In parse_misc_string
> >>>>>>>>>>> In create_headers_dir_config
> >>>>>>>>>>> In header_cmd
> >>>>>>>>>>> In header_inout_cmd
> >>>>>>>>>>> In parse_format_tag
> >>>>>>>>>>> In parse_misc_string
> >>>>>>>>>>> [Switching to Thread 182897610272 (LWP 11317)]
> >>>>>>>>>>>
> >>>>>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
> >>>>>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> >>>>>>>>>>> 1121 printf("In header_post_config\n");
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>> 1120 {
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>> 1121 printf("In header_post_config\n");
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>> In header_post_config
> >>>>>>>>>>> 1122 header_ssl_lookup =
> >>>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>> 1124 }
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
> >>>>>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> >>>>>>>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>>
> >>>>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
> >>>>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> So, it's actually blowing up in "BN_num_bits()" in
> >>>>>>>>>>> /lib64/libcrypto.so.4?
> >>>>>>>>>>>
> >>>>>>>>>>> Jim
> >>>>>>>>>> When you see the :
> >>>>>>>>>>
> >>>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
> >>>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> in gdb, type "bt" and hit enter. It should show the back trace
> >>>>>>>>>> and how
> >>>>>>>>>> you got to the BN_num_bits() function.
> >>>>>>>>>>
> >>>>>>>>>> Joe
> >>>>>>>>>> --
> >>>>>>>>>> http://www.silverhawk.net/
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> Here's the bt full:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> In header_inout_cmd
> >>>>>>>>> In parse_format_tag
> >>>>>>>>> In parse_misc_string
> >>>>>>>>> [Switching to Thread 182897610272 (LWP 6676)]
> >>>>>>>>>
> >>>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
> >>>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> >>>>>>>>> 1121 printf("In header_post_config\n");
> >>>>>>>>> (gdb) n
> >>>>>>>>> 1120 {
> >>>>>>>>> (gdb) n
> >>>>>>>>> 1121 printf("In header_post_config\n");
> >>>>>>>>> (gdb) n
> >>>>>>>>> In header_post_config
> >>>>>>>>> 1122 header_ssl_lookup =
> >>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>>> (gdb) n
> >>>>>>>>> 1124 }
> >>>>>>>>> (gdb) n
> >>>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
> >>>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> >>>>>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >>>>>>>>> (gdb) n
> >>>>>>>>>
> >>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
> >>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>>>>>> (gdb) bt full
> >>>>>>>>> #0 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #1 0x0000003518da8f4e in X509_ATTRIBUTE_create () from
> >>>>>>>>> /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #2 0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #3 0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #4 0x0000003518dae0e1 in ASN1_item_ex_i2d () from
> >>>>>>>>> /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #5 0x0000003518dae5f2 in ASN1_template_i2d () from
> >>>>>>>>> /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #6 0x0000003518dae28e in ASN1_item_ex_i2d () from
> >>>>>>>>> /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #7 0x0000003518dae3c6 in ASN1_item_i2d () from /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #8 0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8,
> >>>>>>>>> p=0x5a72d8) at ssl_engine_pphrase.c:505
> >>>>>>>>> mc = (SSLModConfigRec *) 0x571738
> >>>>>>>>> sc = (SSLSrvConfigRec *) 0x668c38
> >>>>>>>>> pServ = (server_rec *) 0x65fa48
> >>>>>>>>> cpVHostID = 0x60add0 "www.example.com:443"
> >>>>>>>>> szPath =
> >>>>>>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
> >>>>>>>>> '\0'<rep eats 15
> >>>>>>>>> times>,
> >>>>>>>>> "\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
> >>>>>>>>> '\0'<repeats 35 times>, "à\224k",
> >>>>>>>>> ' ---Type<return>
> >>>>>>>>> to continue, or q<return> to quit---
> >>>>>>>>> \0'<repeats 13 times>,
> >>>>>>>>> "øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
> >>>>>>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
> >>>>>>>>> '\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
> >>>>>>>>> '\0'<repeats 35 times>...
> >>>>>>>>> pPrivateKey = (EVP_PKEY *) 0x6ba670
> >>>>>>>>> asn1 = Variable "asn1" is not available.
> >>>>>>>>> (gdb)
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Not really sure what to make of that though :(...
> >>>>>>>>>
> >>>>>>>>> Jim
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> Ok, something interesting. I noticed that the servername in the
> >>>>>>>> gdb output was example.whatever.com, so I changed ServerName in
> >>>>>>>> extras/httpd-ssl.conf to "apache1.whatever.com". Apache still
> >>>>>>>> segfaulted after that.
> >>>>>>>>
> >>>>>>>> I noticed that igdb was showing my router IP address for
> >>>>>>>> "mod_unique_id" (whatever that is), so I thought it was because I
> >>>>>>>> didn't have the hostname in /etc/hosts. So, I added
> >>>>>>>> "apache1.whatever.com" to /etc/hosts, and also turned Apache
> >>>>>>>> LogLevel to debug, and now I get a different error:
> >>>>>>>>
> >>>>>>>> [root@apachemodule dev]# /apps/httpd2222/bin/apachectl -k start -X
> >>>>>>>> mod_headers-jl V0.09 - start calling OAM API
> >>>>>>>> In register_hooks
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In header_cmd
> >>>>>>>> In header_inout_cmd
> >>>>>>>> In parse_format_tag
> >>>>>>>> In parse_misc_string
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In header_cmd
> >>>>>>>> In header_inout_cmd
> >>>>>>>> In parse_format_tag
> >>>>>>>> In parse_misc_string
> >>>>>>>> In header_post_config
> >>>>>>>> *** glibc detected *** corrupted double-linked list:
> >>>>>>>> 0x00000000006b9710 ***
> >>>>>>>> /apps/httpd2222/bin/apachectl: line 78: 7599
> >>>>>>>> Aborted $HTTPD $ARGV
> >>>>>>>> [root@apachemodule dev]#
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Here's the Apache error_log:
> >>>>>>>>
> >>>>>>>> [Thu Jun 21 23:24:11 2012] [info] mod_unique_id: using ip addr
> >>>>>>>> 72.215.225.9
> >>>>>>>> [Thu Jun 21 23:24:12 2012] [info] Init: Seeding PRNG with 144 bytes
> >>>>>>>> of entropy
> >>>>>>>> [Thu Jun 21 23:24:12 2012] [info] Loading certificate& private
> >>>>>>>> key of SSL-aware server
> >>>>>>>> [Thu Jun 21 23:29:51 2012] [info] mod_unique_id: using ip addr
> >>>>>>>> 72.215.225.9
> >>>>>>>> [Thu Jun 21 23:29:52 2012] [info] Init: Seeding PRNG with 144 bytes
> >>>>>>>> of entropy
> >>>>>>>> [Thu Jun 21 23:29:52 2012] [info] Loading certificate& private
> >>>>>>>> key of SSL-aware server
> >>>>>>>> [Thu Jun 21 23:29:56 2012] [info] mod_unique_id: using ip addr
> >>>>>>>> 72.215.225.9
> >>>>>>>> [Thu Jun 21 23:29:57 2012] [info] Init: Seeding PRNG with 144 bytes
> >>>>>>>> of entropy
> >>>>>>>> [Thu Jun 21 23:29:57 2012] [info] Loading certificate& private
> >>>>>>>> key of SSL-aware server
> >>>>>>>> [root@apachemodule dev]#
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> and here's the gdb backtrace:
> >>>>>>>>
> >>>>>>>> (gdb) b header_post_config
> >>>>>>>> Function "header_post_config" not defined.
> >>>>>>>> Make breakpoint pending on future shared library load? (y or [n]) y
> >>>>>>>> Breakpoint 1 (header_post_config) pending.
> >>>>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> >>>>>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f
> >>>>>>>> /apps/httpd2222/conf/httpd.conf
> >>>>>>>> [Thread debugging using libthread_db enabled]
> >>>>>>>> [New Thread 182897610272 (LWP 7644)]
> >>>>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> >>>>>>>> Pending breakpoint "header_post_config" resolved
> >>>>>>>> mod_headers-jl V0.09 - start calling OAM API
> >>>>>>>> In register_hooks
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In header_cmd
> >>>>>>>> In header_inout_cmd
> >>>>>>>> In parse_format_tag
> >>>>>>>> In parse_misc_string
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In header_cmd
> >>>>>>>> In header_inout_cmd
> >>>>>>>> In parse_format_tag
> >>>>>>>> In parse_misc_string
> >>>>>>>> [Switching to Thread 182897610272 (LWP 7644)]
> >>>>>>>>
> >>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
> >>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> >>>>>>>> 1121 printf("In header_post_config\n");
> >>>>>>>> (gdb) n
> >>>>>>>> 1120 {
> >>>>>>>> (gdb) n
> >>>>>>>> 1121 printf("In header_post_config\n");
> >>>>>>>> (gdb) n
> >>>>>>>> In header_post_config
> >>>>>>>> 1122 header_ssl_lookup =
> >>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>> (gdb) n
> >>>>>>>> 1124 }
> >>>>>>>> (gdb) n
> >>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
> >>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> >>>>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >>>>>>>> (gdb) n
> >>>>>>>> *** glibc detected *** corrupted double-linked list:
> >>>>>>>> 0x00000000006b9710 ***
> >>>>>>>>
> >>>>>>>> Program received signal SIGABRT, Aborted.
> >>>>>>>> 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> >>>>>>>> (gdb) bt full
> >>>>>>>> #0 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> >>>>>>>> No symbol table info available.
> >>>>>>>> #1 0x000000351432fa6e in abort () from /lib64/tls/libc.so.6
> >>>>>>>> No symbol table info available.
> >>>>>>>> #2 0x0000003514363641 in __libc_message () from /lib64/tls/libc.so.6
> >>>>>>>> No symbol table info available.
> >>>>>>>> #3 0x0000003514369512 in _int_free () from /lib64/tls/libc.so.6
> >>>>>>>> No symbol table info available.
> >>>>>>>> #4 0x0000003514369846 in free () from /lib64/tls/libc.so.6
> >>>>>>>> No symbol table info available.
> >>>>>>>> #5 0x0000002a9790b6ba in R_free () from
> >>>>>>>> /apps/netpoint/lib64/libobaccess.so
> >>>>>>>> No symbol table info available.
> >>>>>>>> #6 0x0000002a9792bc41 in X509_CINF_free () from
> >>>>>>>> /apps/netpoint/lib64/libobaccess.so
> >>>>>>>> No symbol table info available.
> >>>>>>>> #7 0x0000002a9790ed98 in X509_free () from
> >>>>>>>> /apps/netpoint/lib64/libobaccess.so
> >>>>>>>> No symbol table info available.
> >>>>>>>> #8 0x0000002a987d97b6 in ssl_pphrase_Handle (s=0x59d3a8,
> >>>>>>>> p=0x5a72d8) at ssl_engine_pphrase.c:243
> >>>>>>>> mc = (SSLModConfigRec *) 0x571738
> >>>>>>>> sc = (SSLSrvConfigRec *) 0x668c38
> >>>>>>>> pServ = (server_rec *) 0x65fa48
> >>>>>>>> cpVHostID = 0x60add0 "apache1.whatever.com:443"
> >>>>>>>> szPath =
> >>>>>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.crt\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
> >>>>>>>> '\0'<repeats 15 times>,
> >>>>>>>> "\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
> >>>>>>>> '\0'<repeats 35 times>, "à\224k", '\0'<repeats 13 times>,
> >>>>>>>> "øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
> >>>>>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
> >>>>>>>> '\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
> >>>>>>>> '\0'<repeats 35 times>...
> >>>>>>>> pPrivateKey = Variable "pPrivateKey" is not available.
> >>>>>>>> (gdb)
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Jim
> >>>>>>>>
> >>>>>>> I guess the thing that I'm puzzled about is why it goes from
> >>>>>>> ssl_pphrase_Handle (which is part of Apache code?) to X509_free ()
> >>>>>>> in the libobaccess.so? Apache shouldn't even be aware of the
> >>>>>>> libobaccess stuff yet (since I think that Apache is still trying to
> >>>>>>> initialize SSL processing)?
> >>>>>>>
> >>>>>>> Jim
> >>>>>> It shouldn't unless you've declared a shutdown hook or a clean up hook
> >>>>>> of sorts. The looks of that stack trace look strikingly like an
> >>>>>> exception handler firing off. What are all of the hooks you have
> >>>>>> running?
> >>>>>>
> >>>>>> Joe
> >>>>>> --
> >>>>>> http://www.silverhawk.net/
> >>>>>>
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> I'm basically using the original mod_headers.c as a starter code, and
> >>>>> stuck some stuff in there. Haven't done anything with the hooks etc.
> >>>>> yet, so whatever was there is there:
> >>>>>
> >>>>> static void register_hooks(apr_pool_t *p)
> >>>>> {
> >>>>> printf("mod_headers-jl V0.09 - start calling OAM API\n");
> >>>>> printf("In register_hooks\n");
> >>>>> ap_register_output_filter("FIXUP_HEADERS_OUT",
> >>>>> ap_headers_output_filter,
> >>>>> NULL, AP_FTYPE_CONTENT_SET);
> >>>>> ap_register_output_filter("FIXUP_HEADERS_ERR",
> >>>>> ap_headers_error_filter,
> >>>>> NULL, AP_FTYPE_CONTENT_SET);
> >>>>> ap_hook_pre_config(header_pre_config,NULL,NULL,APR_HOOK_MIDDLE);
> >>>>> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
> >>>>> ap_hook_insert_filter(ap_headers_insert_output_filter, NULL,
> >>>>> NULL, APR_HOOK_LAST);
> >>>>> ap_hook_insert_error_filter(ap_headers_insert_error_filter,
> >>>>> NULL, NULL, APR_HOOK_LAST);
> >>>>> ap_hook_fixups(ap_headers_fixup, NULL, NULL, APR_HOOK_LAST);
> >>>>> ap_hook_post_read_request(ap_headers_early, NULL, NULL,
> >>>>> APR_HOOK_FIRST);
> >>>>> }
> >>>>>
> >>>>> module AP_MODULE_DECLARE_DATA headers_module =
> >>>>> {
> >>>>> STANDARD20_MODULE_STUFF,
> >>>>> create_headers_dir_config, /* dir config creater */
> >>>>> merge_headers_config, /* dir merger --- default is to
> >>>>> override */
> >>>>> NULL, /* server config */
> >>>>> NULL, /* merge server configs */
> >>>>> headers_cmds, /* command apr_table_t */
> >>>>> register_hooks /* register hooks */
> >>>>> };
> >>>>>
> >>>>> I guess that there is a:
> >>>>>
> >>>>> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
> >>>>>
> >>>>> in that register_hooks(), but it was there from the original code.
> >>>>>
> >>>>> BTW, I was checking, and it looks like there's an X509_free() function
> >>>>> in both openssl and in libobaccess.so (used 'nm --dynamic<file>), so
> >>>>> I'm wondering if things are confused, and maybe Apache code expects to
> >>>>> call the openssl X509_free, but somehow ends up calling the
> >>>>> X509_free() in libobaccess.so?
> >>>>>
> >>>>> What would cause something like that? If two .so files have the same
> >>>>> function, how to avoid conflicts?
> >>>>>
> >>>>> Jim
> >>>>
> >>>> Jim, you may have hit that on the head. I'm not sure of the load order
> >>>> and looking up symbols, but it could very well change things. Try
> >>>> re-ordering or adding a LoadFile before the SSL module and see if that
> >>>> changes things.
> >>>
> >>> Look up order usually is load order. So if mod_ssl loads first and loads
> >>> its dependency libcrypto, the symbol will always be found there even if
> >>> the implementation in libobaccess.so is needed - and vice versa.
> >>>
> >>> On Solaris there is a -Bdirect linker flag which changes runtime search
> >>> order so that each module would find a needed symbol first inits direct
> >>> dependencies. Unfortunately AFAIK notihing similar exists for Linux.
> >>>
> >>> You would need to find ssl libs and libobaccess with compatible symbols.
> >>> Best would be a libobaccess which is dynamically linked against a
> >>> compatible version of libcrypto instead of - as it seems - being
> >>> statically linked against it.
> >>>
> >>> Regards,
> >>>
> >>> Rainer
> >>>
> >>
> >>
> >> Hi Rainer,
> >>
> >> Here's exactly what I found:
> >>
> >> [root@apachemodule ~]# ldd /apps/netpoint/lib64/libobaccess.so
> >> libnsl.so.1 => /lib64/libnsl.so.1 (0x0000002a95ab8000)
> >> libdl.so.2 => /lib64/libdl.so.2 (0x0000002a95bd0000)
> >> libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x0000002a95cd3000)
> >> libstdc++.so.6 => /lib64/libstdc++.so.6 (0x0000002a95de9000)
> >> libm.so.6 => /lib64/tls/libm.so.6 (0x0000002a95fd9000)
> >> libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000002a9615f000)
> >> libc.so.6 => /lib64/tls/libc.so.6 (0x0000002a9626d000)
> >> /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
> >> [root@apachemodule ~]#
> >> [root@apachemodule ~]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "X509_free"
> >> 00000000002d8d60 T X509_free
> >>
> >>
> >> [root@apachemodule ~]# ldd /lib64/libssl.so.0.9.7a
> >> libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003518700000)
> >> libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003519000000)
> >> libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003518500000)
> >> libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003518900000)
> >> libcrypto.so.4 => /lib64/libcrypto.so.4 (0x0000003518d00000)
> >> libdl.so.2 => /lib64/libdl.so.2 (0x0000003514800000)
> >> libz.so.1 => /usr/lib64/libz.so.1 (0x0000003514c00000)
> >> libc.so.6 => /lib64/tls/libc.so.6 (0x0000003514300000)
> >> libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003518300000)
> >> /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
> >> [root@apachemodule ~]#
> >> [root@apachemodule ~]# nm --dynamic /lib64/libssl.so.0.9.7a | grep "X509_free"
> >> U X509_free
> >>
> >>
> >> [root@apachemodule ~]# ldd /lib64/libcrypto.so.0.9.7a
> >> libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003518700000)
> >> libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003519000000)
> >> libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003518500000)
> >> libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003518900000)
> >> libdl.so.2 => /lib64/libdl.so.2 (0x0000003514800000)
> >> libz.so.1 => /usr/lib64/libz.so.1 (0x0000003514c00000)
> >> libc.so.6 => /lib64/tls/libc.so.6 (0x0000003514300000)
> >> libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003518300000)
> >> /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
> >> [root@apachemodule ~]#
> >> [root@apachemodule ~]# nm --dynamic /lib64/libcrypto.so.0.9.7a | grep "X509_free"
> >> 0000003518da9860 T X509_free
> >>
> >>
> >> So:
> >>
> >> - liboaccess.so doesn't REFERENCE libcrypto, but it HAS an X509_free() function in it.
> >> - libssl.so.REFERENCES libcrypto, AND it HAS an X509_free function in it.
> >> - libcrypto also has an X509_free function in it.
> >>
> >> From your msg above, are you saying that this situation can't be resolved by trying to change load order?
> >>
> >> And the only way is what you said in your last paragraph (find another libobccess.so)?
>
> I don't know a way how to fix this apart from using mod_ssl and
> libobccess which have both bin build against the same major version of
> OpenSSL. In your case mod_ssl seems to use the platform OpenSSL, so
> 0.9.7(a) (which is pretty old), and we don't know what OpenSSL version
> was linked in statically into your libobccess.so.
>
> Load order most likely doesn't help. Either mod_header/liboccess.so will
> habe a problem or mod_ssl.
>
> > Also, BTW, the call to X509_free is apparently because in ssl_engine_pphrase.c, which it seems is part of mod_ssl, it has:
> >
> > /*
> > * Free the X509 structure
> > */
> > X509_free(pX509Cert);
> >
> > but that's calling the X509_free() in libobaccess.so, rather than the X509_free() in libcrypt.so?
>
> It's just calling X509_free(). The runtime linker will search through
> the httpd binary, all loaded modules and libraries in load order and
> call the symbol wherever it found it first. So you can change load order
> and load mod_ssl and OpenSSL libs before libobccess.so. But then when
> libobccess runs it will itself call symbols from within OpenSSL, e.g.
> X509_free() and again the runtime linker will search for the symbol
> through the loaded files *and not dircetly use the one inside libobccess
> - even if it is caled form inside this file*.
>
> Regards,
>
> Rainer
Rainer and Joe,
SUCCESS (though I'm not calling it that yet because of earlier false positive :))!
Moving the LoadFile obaccess.so after didn't work because then got undefined symbol.
So, I moved the LoadModule mod_ssl *in front of* the LoadModule libobaccess.so (so I guess that makes the libobaccess.so "first"???).
Anyway, after that, Apache starts without aborting.
I've done some quick testing and client-authenticated SSL still seems to be functioning (that was what I was afraid I'd break by moving the LoadModule for mod_ssl).
Here's part of my httpd.conf, for the record:
LoadModule log_forensic_module modules/mod_log_forensic.so
LoadModule logio_module modules/mod_logio.so
LoadModule env_module modules/mod_env.so
LoadModule mime_magic_module modules/mod_mime_magic.so
LoadModule cern_meta_module modules/mod_cern_meta.so
LoadModule expires_module modules/mod_expires.so
#LoadFile /lib64/libcrypto.so.4
LoadModule ssl_module modules/mod_ssl.so
#LoadFile /lib64/libcrypto.so.4
#LoadFile /apps/netpoint/AccessServerSDK/oblix/lib/libxmlengine.so
LoadFile /apps/netpoint/lib64/libobaccess.so
LoadModule headers_module modules/mod_headers.so
#LoadFile /apps/netpoint/lib64/libobaccess.so
LoadModule ident_module modules/mod_ident.so
LoadModule usertrack_module modules/mod_usertrack.so
LoadModule unique_id_module modules/mod_unique_id.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule version_module modules/mod_version.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_scgi_module modules/mod_proxy_scgi.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
# ORIGINAL LOCATION OF LOADMODULE mod_ssl.so
#LoadFile /lib64/libcrypto.so.4
#LoadModule ssl_module modules/mod_ssl.so
#LoadFile /lib64/libcrypto.so.4
LoadModule mime_module modules/mod_mime.so
LoadModule dav_module modules/mod_dav.so
LoadModule status_module modules/mod_status.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule asis_module modules/mod_asis.so
LoadModule info_module modules/mod_info.so
LoadModule cgi_module modules/mod_cgi.so
LoadModule dav_fs_module modules/mod_dav_fs.so
LoadModule vhost_alias_module modules/mod_vhost_alias.so
LoadModule negotiation_module modules/mod_negotiation.so
LoadModule dir_module modules/mod_dir.so
LoadModule imagemap_module modules/mod_imagemap.so
LoadModule actions_module modules/mod_actions.so
LoadModule speling_module modules/mod_speling.so
LoadModule userdir_module modules/mod_userdir.so
So:
- LoadModule for mod_ssl in front of LoadModule for libobaccess.so, then
- LoadFile libaccess.so, then
- LoadModule for mod_headers.so
(A preliminary) THANKS!!
Now I gotta debug my calls...
Jim
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use Apache
module that uses shared library?
Posted by Rainer Jung <ra...@kippdata.de>.
On 22.06.2012 14:16, ohaya@cox.net wrote:
>
> ---- ohaya@cox.net wrote:
>>
>> ---- Rainer Jung <ra...@kippdata.de> wrote:
>>> On 22.06.2012 06:10, Joe Lewis wrote:
>>>> On 6/21/12 10:02 PM, ohaya@cox.net wrote:
>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>>>> On 6/21/12 9:39 PM, ohaya@cox.net wrote:
>>>>>>> ---- ohaya@cox.net wrote:
>>>>>>>> ---- ohaya@cox.net wrote:
>>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>>>>>>>> On 6/21/12 7:32 PM, ohaya@cox.net wrote:
>>>>>>>>>>> ---- ohaya@cox.net wrote:
>>>>>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>>>>>>>>>>> On 6/21/12 6:46 PM, ohaya@cox.net wrote:
>>>>>>>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>>>>>>>>>>>>> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
>>>>>>>>>>>>>>>> ---- ohaya@cox.net wrote:
>>>>>>>>>>>>>>>>> ---- Sorin Manolache<so...@gmail.com> wrote:
>>>>>>>>>>>>>>>>>> And I forgot to say: run gdb in some sort of environment
>>>>>>>>>>>>>>>>>> where you see
>>>>>>>>>>>>>>>>>> your current source code line and a couple of surrounding
>>>>>>>>>>>>>>>>>> lines. You
>>>>>>>>>>>>>>>>>> could achieve this with the "list" command, but I prefer
>>>>>>>>>>>>>>>>>> running gdb in
>>>>>>>>>>>>>>>>>> emacs and let emacs do the nice listing of source code in
>>>>>>>>>>>>>>>>>> a different panel.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> S
>>>>>>>>>>>>>>>>> Here's the function from my source. It's the original
>>>>>>>>>>>>>>>>> from mod_headers.c, plus my printf:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> static int header_post_config(apr_pool_t *pconf,
>>>>>>>>>>>>>>>>> apr_pool_t *plog,
>>>>>>>>>>>>>>>>> apr_pool_t *ptemp,
>>>>>>>>>>>>>>>>> server_rec *s)
>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>> printf("In header_post_config\n");
>>>>>>>>>>>>>>>>> header_ssl_lookup =
>>>>>>>>>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>>>>>>>>>>> return OK;
>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Jim
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I was able to get the segfault to go away. Here's what I
>>>>>>>>>>>>>>>> had to do:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> - Created /etc/ld.so.conf.d/my.conf, and added the
>>>>>>>>>>>>>>>> directory where my libobaccess.so was
>>>>>>>>>>>>>>>> - Run 'ldconfig' to activate.
>>>>>>>>>>>>>>>> - In the apxs command, DON'T include the -L and -l arguments
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> After that, Apache appears to start ok, without segfault :)!!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks for all of the great help, esp. the suggestion about
>>>>>>>>>>>>>>>> checking "ldconfig -p". I still don't understand why, but
>>>>>>>>>>>>>>>> I'm just glad that I can get past this piece so now I can
>>>>>>>>>>>>>>>> debug my module :)...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Later,
>>>>>>>>>>>>>>>> Jim
>>>>>>>>>>>>>>> I'm just glad this list is as good as it is!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> FYI, the ldconfig is the dynamic linker control, and those
>>>>>>>>>>>>>>> /etc/ld.so.conf.d files provide additional search
>>>>>>>>>>>>>>> directories for the
>>>>>>>>>>>>>>> linker to check in when loading a library.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> http://www.silverhawk.net/
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sorry to report, but my earlier report was a "false positive"
>>>>>>>>>>>>>> :)...
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I forgot that the mod_headers.c that I was doing the earlier
>>>>>>>>>>>>>> testing with had all references to the libobaccess.so removed
>>>>>>>>>>>>>> :(!!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So, I'm still stuck with basically the same problem, now,
>>>>>>>>>>>>>> working with my "full" code, with the calls in it:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - If I compile with -L and -l, Apache segfaults when it starts
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - If I compile without -L and -l, then I get "undefined
>>>>>>>>>>>>>> symbol" errors when I try to start Apache, e.g.:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [root@apachemodule build-mod_headers]#
>>>>>>>>>>>>>> /apps/httpd2222/bin/apachectl -k start -X
>>>>>>>>>>>>>> httpd: Syntax error on line 84 of
>>>>>>>>>>>>>> /apps/httpd2222/conf/httpd.conf: Cannot load
>>>>>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so into server:
>>>>>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so: undefined symbol:
>>>>>>>>>>>>>> ObResource_isProtected
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> That "ObResource_isProtected" should be a symbol in
>>>>>>>>>>>>>> libobaccess.so, and in fact, if I do "nm --dynamic", I get:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [root@apachemodule build-mod_headers]# nm --dynamic
>>>>>>>>>>>>>> /apps/netpoint/lib64/libobaccess.so | grep
>>>>>>>>>>>>>> "ObResource_isProtected"
>>>>>>>>>>>>>> 00000000000a6d80 T ObResource_isProtected
>>>>>>>>>>>>>> [root@apachemodule build-mod_headers]#
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm *assuming* that the reason for the "undefined symbol"
>>>>>>>>>>>>>> error is that libobaccess.so is actually not being loaded,
>>>>>>>>>>>>>> but then when I try to load libobaccess.so, either via -L and
>>>>>>>>>>>>>> -l in the apxs, or using LoadFile in httpd.conf, I get the
>>>>>>>>>>>>>> segfault (same gdb info, BTW).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Catch-22?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sorry for the false alarm :(!!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Jim
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>> Not a catch-22. The -L and -l specify linker options when
>>>>>>>>>>>>> assembling
>>>>>>>>>>>>> the code. The ldconfig is a run-time thing. If you are
>>>>>>>>>>>>> getting the
>>>>>>>>>>>>> stderr messages, you are making it all the way into your
>>>>>>>>>>>>> library. I'd
>>>>>>>>>>>>> suggest commenting out the following line and see if you get
>>>>>>>>>>>>> farther :
>>>>>>>>>>>>>
>>>>>>>>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>>>>>>>
>>>>>>>>>>>>> That should tell you if the problem is the ssl_var_lookup.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Joe
>>>>>>>>>>>>> --
>>>>>>>>>>>>> http://www.silverhawk.net/
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for the suggestion. I just tried what you suggested,
>>>>>>>>>>>> and got a segfault when I started Apache with the modified module.
>>>>>>>>>>>>
>>>>>>>>>>>> Jim
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> As a reminder, here's the gdb with the library loaded:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> (gdb) b header_post_config
>>>>>>>>>>> Function "header_post_config" not defined.
>>>>>>>>>>> Make breakpoint pending on future shared library load? (y or [n]) y
>>>>>>>>>>> Breakpoint 1 (header_post_config) pending.
>>>>>>>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
>>>>>>>>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/
>>>>>>>>>>> -f /apps/httpd2222/conf/httpd.conf
>>>>>>>>>>> [Thread debugging using libthread_db enabled]
>>>>>>>>>>> [New Thread 182897610272 (LWP 11317)]
>>>>>>>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
>>>>>>>>>>> Pending breakpoint "header_post_config" resolved
>>>>>>>>>>> mod_headers-jl V0.09 - start calling OAM API
>>>>>>>>>>> In register_hooks
>>>>>>>>>>> In create_headers_dir_config
>>>>>>>>>>> In create_headers_dir_config
>>>>>>>>>>> In header_cmd
>>>>>>>>>>> In header_inout_cmd
>>>>>>>>>>> In parse_format_tag
>>>>>>>>>>> In parse_misc_string
>>>>>>>>>>> In create_headers_dir_config
>>>>>>>>>>> In header_cmd
>>>>>>>>>>> In header_inout_cmd
>>>>>>>>>>> In parse_format_tag
>>>>>>>>>>> In parse_misc_string
>>>>>>>>>>> [Switching to Thread 182897610272 (LWP 11317)]
>>>>>>>>>>>
>>>>>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
>>>>>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>>>>>>>>>>> 1121 printf("In header_post_config\n");
>>>>>>>>>>> (gdb) n
>>>>>>>>>>> 1120 {
>>>>>>>>>>> (gdb) n
>>>>>>>>>>> 1121 printf("In header_post_config\n");
>>>>>>>>>>> (gdb) n
>>>>>>>>>>> In header_post_config
>>>>>>>>>>> 1122 header_ssl_lookup =
>>>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>>>>> (gdb) n
>>>>>>>>>>> 1124 }
>>>>>>>>>>> (gdb) n
>>>>>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
>>>>>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
>>>>>>>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>>>>>>>>>>> (gdb) n
>>>>>>>>>>>
>>>>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> So, it's actually blowing up in "BN_num_bits()" in
>>>>>>>>>>> /lib64/libcrypto.so.4?
>>>>>>>>>>>
>>>>>>>>>>> Jim
>>>>>>>>>> When you see the :
>>>>>>>>>>
>>>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> in gdb, type "bt" and hit enter. It should show the back trace
>>>>>>>>>> and how
>>>>>>>>>> you got to the BN_num_bits() function.
>>>>>>>>>>
>>>>>>>>>> Joe
>>>>>>>>>> --
>>>>>>>>>> http://www.silverhawk.net/
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Here's the bt full:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> In header_inout_cmd
>>>>>>>>> In parse_format_tag
>>>>>>>>> In parse_misc_string
>>>>>>>>> [Switching to Thread 182897610272 (LWP 6676)]
>>>>>>>>>
>>>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
>>>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>>>>>>>>> 1121 printf("In header_post_config\n");
>>>>>>>>> (gdb) n
>>>>>>>>> 1120 {
>>>>>>>>> (gdb) n
>>>>>>>>> 1121 printf("In header_post_config\n");
>>>>>>>>> (gdb) n
>>>>>>>>> In header_post_config
>>>>>>>>> 1122 header_ssl_lookup =
>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>>> (gdb) n
>>>>>>>>> 1124 }
>>>>>>>>> (gdb) n
>>>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
>>>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
>>>>>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>>>>>>>>> (gdb) n
>>>>>>>>>
>>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>>>>> (gdb) bt full
>>>>>>>>> #0 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #1 0x0000003518da8f4e in X509_ATTRIBUTE_create () from
>>>>>>>>> /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #2 0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #3 0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #4 0x0000003518dae0e1 in ASN1_item_ex_i2d () from
>>>>>>>>> /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #5 0x0000003518dae5f2 in ASN1_template_i2d () from
>>>>>>>>> /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #6 0x0000003518dae28e in ASN1_item_ex_i2d () from
>>>>>>>>> /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #7 0x0000003518dae3c6 in ASN1_item_i2d () from /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #8 0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8,
>>>>>>>>> p=0x5a72d8) at ssl_engine_pphrase.c:505
>>>>>>>>> mc = (SSLModConfigRec *) 0x571738
>>>>>>>>> sc = (SSLSrvConfigRec *) 0x668c38
>>>>>>>>> pServ = (server_rec *) 0x65fa48
>>>>>>>>> cpVHostID = 0x60add0 "www.example.com:443"
>>>>>>>>> szPath =
>>>>>>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
>>>>>>>>> '\0'<rep eats 15
>>>>>>>>> times>,
>>>>>>>>> "\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
>>>>>>>>> '\0'<repeats 35 times>, "à\224k",
>>>>>>>>> ' ---Type<return>
>>>>>>>>> to continue, or q<return> to quit---
>>>>>>>>> \0'<repeats 13 times>,
>>>>>>>>> "øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
>>>>>>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
>>>>>>>>> '\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
>>>>>>>>> '\0'<repeats 35 times>...
>>>>>>>>> pPrivateKey = (EVP_PKEY *) 0x6ba670
>>>>>>>>> asn1 = Variable "asn1" is not available.
>>>>>>>>> (gdb)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Not really sure what to make of that though :(...
>>>>>>>>>
>>>>>>>>> Jim
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Ok, something interesting. I noticed that the servername in the
>>>>>>>> gdb output was example.whatever.com, so I changed ServerName in
>>>>>>>> extras/httpd-ssl.conf to "apache1.whatever.com". Apache still
>>>>>>>> segfaulted after that.
>>>>>>>>
>>>>>>>> I noticed that igdb was showing my router IP address for
>>>>>>>> "mod_unique_id" (whatever that is), so I thought it was because I
>>>>>>>> didn't have the hostname in /etc/hosts. So, I added
>>>>>>>> "apache1.whatever.com" to /etc/hosts, and also turned Apache
>>>>>>>> LogLevel to debug, and now I get a different error:
>>>>>>>>
>>>>>>>> [root@apachemodule dev]# /apps/httpd2222/bin/apachectl -k start -X
>>>>>>>> mod_headers-jl V0.09 - start calling OAM API
>>>>>>>> In register_hooks
>>>>>>>> In create_headers_dir_config
>>>>>>>> In create_headers_dir_config
>>>>>>>> In header_cmd
>>>>>>>> In header_inout_cmd
>>>>>>>> In parse_format_tag
>>>>>>>> In parse_misc_string
>>>>>>>> In create_headers_dir_config
>>>>>>>> In header_cmd
>>>>>>>> In header_inout_cmd
>>>>>>>> In parse_format_tag
>>>>>>>> In parse_misc_string
>>>>>>>> In header_post_config
>>>>>>>> *** glibc detected *** corrupted double-linked list:
>>>>>>>> 0x00000000006b9710 ***
>>>>>>>> /apps/httpd2222/bin/apachectl: line 78: 7599
>>>>>>>> Aborted $HTTPD $ARGV
>>>>>>>> [root@apachemodule dev]#
>>>>>>>>
>>>>>>>>
>>>>>>>> Here's the Apache error_log:
>>>>>>>>
>>>>>>>> [Thu Jun 21 23:24:11 2012] [info] mod_unique_id: using ip addr
>>>>>>>> 72.215.225.9
>>>>>>>> [Thu Jun 21 23:24:12 2012] [info] Init: Seeding PRNG with 144 bytes
>>>>>>>> of entropy
>>>>>>>> [Thu Jun 21 23:24:12 2012] [info] Loading certificate& private
>>>>>>>> key of SSL-aware server
>>>>>>>> [Thu Jun 21 23:29:51 2012] [info] mod_unique_id: using ip addr
>>>>>>>> 72.215.225.9
>>>>>>>> [Thu Jun 21 23:29:52 2012] [info] Init: Seeding PRNG with 144 bytes
>>>>>>>> of entropy
>>>>>>>> [Thu Jun 21 23:29:52 2012] [info] Loading certificate& private
>>>>>>>> key of SSL-aware server
>>>>>>>> [Thu Jun 21 23:29:56 2012] [info] mod_unique_id: using ip addr
>>>>>>>> 72.215.225.9
>>>>>>>> [Thu Jun 21 23:29:57 2012] [info] Init: Seeding PRNG with 144 bytes
>>>>>>>> of entropy
>>>>>>>> [Thu Jun 21 23:29:57 2012] [info] Loading certificate& private
>>>>>>>> key of SSL-aware server
>>>>>>>> [root@apachemodule dev]#
>>>>>>>>
>>>>>>>>
>>>>>>>> and here's the gdb backtrace:
>>>>>>>>
>>>>>>>> (gdb) b header_post_config
>>>>>>>> Function "header_post_config" not defined.
>>>>>>>> Make breakpoint pending on future shared library load? (y or [n]) y
>>>>>>>> Breakpoint 1 (header_post_config) pending.
>>>>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
>>>>>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f
>>>>>>>> /apps/httpd2222/conf/httpd.conf
>>>>>>>> [Thread debugging using libthread_db enabled]
>>>>>>>> [New Thread 182897610272 (LWP 7644)]
>>>>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
>>>>>>>> Pending breakpoint "header_post_config" resolved
>>>>>>>> mod_headers-jl V0.09 - start calling OAM API
>>>>>>>> In register_hooks
>>>>>>>> In create_headers_dir_config
>>>>>>>> In create_headers_dir_config
>>>>>>>> In header_cmd
>>>>>>>> In header_inout_cmd
>>>>>>>> In parse_format_tag
>>>>>>>> In parse_misc_string
>>>>>>>> In create_headers_dir_config
>>>>>>>> In header_cmd
>>>>>>>> In header_inout_cmd
>>>>>>>> In parse_format_tag
>>>>>>>> In parse_misc_string
>>>>>>>> [Switching to Thread 182897610272 (LWP 7644)]
>>>>>>>>
>>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
>>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>>>>>>>> 1121 printf("In header_post_config\n");
>>>>>>>> (gdb) n
>>>>>>>> 1120 {
>>>>>>>> (gdb) n
>>>>>>>> 1121 printf("In header_post_config\n");
>>>>>>>> (gdb) n
>>>>>>>> In header_post_config
>>>>>>>> 1122 header_ssl_lookup =
>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>> (gdb) n
>>>>>>>> 1124 }
>>>>>>>> (gdb) n
>>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
>>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
>>>>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>>>>>>>> (gdb) n
>>>>>>>> *** glibc detected *** corrupted double-linked list:
>>>>>>>> 0x00000000006b9710 ***
>>>>>>>>
>>>>>>>> Program received signal SIGABRT, Aborted.
>>>>>>>> 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
>>>>>>>> (gdb) bt full
>>>>>>>> #0 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
>>>>>>>> No symbol table info available.
>>>>>>>> #1 0x000000351432fa6e in abort () from /lib64/tls/libc.so.6
>>>>>>>> No symbol table info available.
>>>>>>>> #2 0x0000003514363641 in __libc_message () from /lib64/tls/libc.so.6
>>>>>>>> No symbol table info available.
>>>>>>>> #3 0x0000003514369512 in _int_free () from /lib64/tls/libc.so.6
>>>>>>>> No symbol table info available.
>>>>>>>> #4 0x0000003514369846 in free () from /lib64/tls/libc.so.6
>>>>>>>> No symbol table info available.
>>>>>>>> #5 0x0000002a9790b6ba in R_free () from
>>>>>>>> /apps/netpoint/lib64/libobaccess.so
>>>>>>>> No symbol table info available.
>>>>>>>> #6 0x0000002a9792bc41 in X509_CINF_free () from
>>>>>>>> /apps/netpoint/lib64/libobaccess.so
>>>>>>>> No symbol table info available.
>>>>>>>> #7 0x0000002a9790ed98 in X509_free () from
>>>>>>>> /apps/netpoint/lib64/libobaccess.so
>>>>>>>> No symbol table info available.
>>>>>>>> #8 0x0000002a987d97b6 in ssl_pphrase_Handle (s=0x59d3a8,
>>>>>>>> p=0x5a72d8) at ssl_engine_pphrase.c:243
>>>>>>>> mc = (SSLModConfigRec *) 0x571738
>>>>>>>> sc = (SSLSrvConfigRec *) 0x668c38
>>>>>>>> pServ = (server_rec *) 0x65fa48
>>>>>>>> cpVHostID = 0x60add0 "apache1.whatever.com:443"
>>>>>>>> szPath =
>>>>>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.crt\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
>>>>>>>> '\0'<repeats 15 times>,
>>>>>>>> "\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
>>>>>>>> '\0'<repeats 35 times>, "à\224k", '\0'<repeats 13 times>,
>>>>>>>> "øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
>>>>>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
>>>>>>>> '\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
>>>>>>>> '\0'<repeats 35 times>...
>>>>>>>> pPrivateKey = Variable "pPrivateKey" is not available.
>>>>>>>> (gdb)
>>>>>>>>
>>>>>>>>
>>>>>>>> Jim
>>>>>>>>
>>>>>>> I guess the thing that I'm puzzled about is why it goes from
>>>>>>> ssl_pphrase_Handle (which is part of Apache code?) to X509_free ()
>>>>>>> in the libobaccess.so? Apache shouldn't even be aware of the
>>>>>>> libobaccess stuff yet (since I think that Apache is still trying to
>>>>>>> initialize SSL processing)?
>>>>>>>
>>>>>>> Jim
>>>>>> It shouldn't unless you've declared a shutdown hook or a clean up hook
>>>>>> of sorts. The looks of that stack trace look strikingly like an
>>>>>> exception handler firing off. What are all of the hooks you have
>>>>>> running?
>>>>>>
>>>>>> Joe
>>>>>> --
>>>>>> http://www.silverhawk.net/
>>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm basically using the original mod_headers.c as a starter code, and
>>>>> stuck some stuff in there. Haven't done anything with the hooks etc.
>>>>> yet, so whatever was there is there:
>>>>>
>>>>> static void register_hooks(apr_pool_t *p)
>>>>> {
>>>>> printf("mod_headers-jl V0.09 - start calling OAM API\n");
>>>>> printf("In register_hooks\n");
>>>>> ap_register_output_filter("FIXUP_HEADERS_OUT",
>>>>> ap_headers_output_filter,
>>>>> NULL, AP_FTYPE_CONTENT_SET);
>>>>> ap_register_output_filter("FIXUP_HEADERS_ERR",
>>>>> ap_headers_error_filter,
>>>>> NULL, AP_FTYPE_CONTENT_SET);
>>>>> ap_hook_pre_config(header_pre_config,NULL,NULL,APR_HOOK_MIDDLE);
>>>>> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
>>>>> ap_hook_insert_filter(ap_headers_insert_output_filter, NULL,
>>>>> NULL, APR_HOOK_LAST);
>>>>> ap_hook_insert_error_filter(ap_headers_insert_error_filter,
>>>>> NULL, NULL, APR_HOOK_LAST);
>>>>> ap_hook_fixups(ap_headers_fixup, NULL, NULL, APR_HOOK_LAST);
>>>>> ap_hook_post_read_request(ap_headers_early, NULL, NULL,
>>>>> APR_HOOK_FIRST);
>>>>> }
>>>>>
>>>>> module AP_MODULE_DECLARE_DATA headers_module =
>>>>> {
>>>>> STANDARD20_MODULE_STUFF,
>>>>> create_headers_dir_config, /* dir config creater */
>>>>> merge_headers_config, /* dir merger --- default is to
>>>>> override */
>>>>> NULL, /* server config */
>>>>> NULL, /* merge server configs */
>>>>> headers_cmds, /* command apr_table_t */
>>>>> register_hooks /* register hooks */
>>>>> };
>>>>>
>>>>> I guess that there is a:
>>>>>
>>>>> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
>>>>>
>>>>> in that register_hooks(), but it was there from the original code.
>>>>>
>>>>> BTW, I was checking, and it looks like there's an X509_free() function
>>>>> in both openssl and in libobaccess.so (used 'nm --dynamic<file>), so
>>>>> I'm wondering if things are confused, and maybe Apache code expects to
>>>>> call the openssl X509_free, but somehow ends up calling the
>>>>> X509_free() in libobaccess.so?
>>>>>
>>>>> What would cause something like that? If two .so files have the same
>>>>> function, how to avoid conflicts?
>>>>>
>>>>> Jim
>>>>
>>>> Jim, you may have hit that on the head. I'm not sure of the load order
>>>> and looking up symbols, but it could very well change things. Try
>>>> re-ordering or adding a LoadFile before the SSL module and see if that
>>>> changes things.
>>>
>>> Look up order usually is load order. So if mod_ssl loads first and loads
>>> its dependency libcrypto, the symbol will always be found there even if
>>> the implementation in libobaccess.so is needed - and vice versa.
>>>
>>> On Solaris there is a -Bdirect linker flag which changes runtime search
>>> order so that each module would find a needed symbol first inits direct
>>> dependencies. Unfortunately AFAIK notihing similar exists for Linux.
>>>
>>> You would need to find ssl libs and libobaccess with compatible symbols.
>>> Best would be a libobaccess which is dynamically linked against a
>>> compatible version of libcrypto instead of - as it seems - being
>>> statically linked against it.
>>>
>>> Regards,
>>>
>>> Rainer
>>>
>>
>>
>> Hi Rainer,
>>
>> Here's exactly what I found:
>>
>> [root@apachemodule ~]# ldd /apps/netpoint/lib64/libobaccess.so
>> libnsl.so.1 => /lib64/libnsl.so.1 (0x0000002a95ab8000)
>> libdl.so.2 => /lib64/libdl.so.2 (0x0000002a95bd0000)
>> libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x0000002a95cd3000)
>> libstdc++.so.6 => /lib64/libstdc++.so.6 (0x0000002a95de9000)
>> libm.so.6 => /lib64/tls/libm.so.6 (0x0000002a95fd9000)
>> libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000002a9615f000)
>> libc.so.6 => /lib64/tls/libc.so.6 (0x0000002a9626d000)
>> /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
>> [root@apachemodule ~]#
>> [root@apachemodule ~]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "X509_free"
>> 00000000002d8d60 T X509_free
>>
>>
>> [root@apachemodule ~]# ldd /lib64/libssl.so.0.9.7a
>> libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003518700000)
>> libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003519000000)
>> libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003518500000)
>> libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003518900000)
>> libcrypto.so.4 => /lib64/libcrypto.so.4 (0x0000003518d00000)
>> libdl.so.2 => /lib64/libdl.so.2 (0x0000003514800000)
>> libz.so.1 => /usr/lib64/libz.so.1 (0x0000003514c00000)
>> libc.so.6 => /lib64/tls/libc.so.6 (0x0000003514300000)
>> libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003518300000)
>> /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
>> [root@apachemodule ~]#
>> [root@apachemodule ~]# nm --dynamic /lib64/libssl.so.0.9.7a | grep "X509_free"
>> U X509_free
>>
>>
>> [root@apachemodule ~]# ldd /lib64/libcrypto.so.0.9.7a
>> libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003518700000)
>> libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003519000000)
>> libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003518500000)
>> libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003518900000)
>> libdl.so.2 => /lib64/libdl.so.2 (0x0000003514800000)
>> libz.so.1 => /usr/lib64/libz.so.1 (0x0000003514c00000)
>> libc.so.6 => /lib64/tls/libc.so.6 (0x0000003514300000)
>> libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003518300000)
>> /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
>> [root@apachemodule ~]#
>> [root@apachemodule ~]# nm --dynamic /lib64/libcrypto.so.0.9.7a | grep "X509_free"
>> 0000003518da9860 T X509_free
>>
>>
>> So:
>>
>> - liboaccess.so doesn't REFERENCE libcrypto, but it HAS an X509_free() function in it.
>> - libssl.so.REFERENCES libcrypto, AND it HAS an X509_free function in it.
>> - libcrypto also has an X509_free function in it.
>>
>> From your msg above, are you saying that this situation can't be resolved by trying to change load order?
>>
>> And the only way is what you said in your last paragraph (find another libobccess.so)?
I don't know a way how to fix this apart from using mod_ssl and
libobccess which have both bin build against the same major version of
OpenSSL. In your case mod_ssl seems to use the platform OpenSSL, so
0.9.7(a) (which is pretty old), and we don't know what OpenSSL version
was linked in statically into your libobccess.so.
Load order most likely doesn't help. Either mod_header/liboccess.so will
habe a problem or mod_ssl.
> Also, BTW, the call to X509_free is apparently because in ssl_engine_pphrase.c, which it seems is part of mod_ssl, it has:
>
> /*
> * Free the X509 structure
> */
> X509_free(pX509Cert);
>
> but that's calling the X509_free() in libobaccess.so, rather than the X509_free() in libcrypt.so?
It's just calling X509_free(). The runtime linker will search through
the httpd binary, all loaded modules and libraries in load order and
call the symbol wherever it found it first. So you can change load order
and load mod_ssl and OpenSSL libs before libobccess.so. But then when
libobccess runs it will itself call symbols from within OpenSSL, e.g.
X509_free() and again the runtime linker will search for the symbol
through the loaded files *and not dircetly use the one inside libobccess
- even if it is caled form inside this file*.
Regards,
Rainer
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use
Apache module that uses shared library?
Posted by oh...@cox.net.
---- Joe Lewis <jo...@joe-lewis.com> wrote:
> On 6/22/12 6:16 AM, ohaya@cox.net wrote:
> > ---- ohaya@cox.net wrote:
> >> ---- Rainer Jung<ra...@kippdata.de> wrote:
> >>> On 22.06.2012 06:10, Joe Lewis wrote:
> >>>> On 6/21/12 10:02 PM, ohaya@cox.net wrote:
> >>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >>>>>> On 6/21/12 9:39 PM, ohaya@cox.net wrote:
> >>>>>>> ---- ohaya@cox.net wrote:
> >>>>>>>> ---- ohaya@cox.net wrote:
> >>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >>>>>>>>>> On 6/21/12 7:32 PM, ohaya@cox.net wrote:
> >>>>>>>>>>> ---- ohaya@cox.net wrote:
> >>>>>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >>>>>>>>>>>>> On 6/21/12 6:46 PM, ohaya@cox.net wrote:
> >>>>>>>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >>>>>>>>>>>>>>> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
> >>>>>>>>>>>>>>>> ---- ohaya@cox.net wrote:
> >>>>>>>>>>>>>>>>> ---- Sorin Manolache<so...@gmail.com> wrote:
> >>>>>>>>>>>>>>>>>> And I forgot to say: run gdb in some sort of environment
> >>>>>>>>>>>>>>>>>> where you see
> >>>>>>>>>>>>>>>>>> your current source code line and a couple of surrounding
> >>>>>>>>>>>>>>>>>> lines. You
> >>>>>>>>>>>>>>>>>> could achieve this with the "list" command, but I prefer
> >>>>>>>>>>>>>>>>>> running gdb in
> >>>>>>>>>>>>>>>>>> emacs and let emacs do the nice listing of source code in
> >>>>>>>>>>>>>>>>>> a different panel.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> S
> >>>>>>>>>>>>>>>>> Here's the function from my source. It's the original
> >>>>>>>>>>>>>>>>> from mod_headers.c, plus my printf:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> static int header_post_config(apr_pool_t *pconf,
> >>>>>>>>>>>>>>>>> apr_pool_t *plog,
> >>>>>>>>>>>>>>>>> apr_pool_t *ptemp,
> >>>>>>>>>>>>>>>>> server_rec *s)
> >>>>>>>>>>>>>>>>> {
> >>>>>>>>>>>>>>>>> printf("In header_post_config\n");
> >>>>>>>>>>>>>>>>> header_ssl_lookup =
> >>>>>>>>>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>>>>>>>>>>> return OK;
> >>>>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Jim
> >>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I was able to get the segfault to go away. Here's what I
> >>>>>>>>>>>>>>>> had to do:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> - Created /etc/ld.so.conf.d/my.conf, and added the
> >>>>>>>>>>>>>>>> directory where my libobaccess.so was
> >>>>>>>>>>>>>>>> - Run 'ldconfig' to activate.
> >>>>>>>>>>>>>>>> - In the apxs command, DON'T include the -L and -l arguments
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> After that, Apache appears to start ok, without segfault :)!!
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks for all of the great help, esp. the suggestion about
> >>>>>>>>>>>>>>>> checking "ldconfig -p". I still don't understand why, but
> >>>>>>>>>>>>>>>> I'm just glad that I can get past this piece so now I can
> >>>>>>>>>>>>>>>> debug my module :)...
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Later,
> >>>>>>>>>>>>>>>> Jim
> >>>>>>>>>>>>>>> I'm just glad this list is as good as it is!
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> FYI, the ldconfig is the dynamic linker control, and those
> >>>>>>>>>>>>>>> /etc/ld.so.conf.d files provide additional search
> >>>>>>>>>>>>>>> directories for the
> >>>>>>>>>>>>>>> linker to check in when loading a library.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Joe
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> http://www.silverhawk.net/
> >>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Sorry to report, but my earlier report was a "false positive"
> >>>>>>>>>>>>>> :)...
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I forgot that the mod_headers.c that I was doing the earlier
> >>>>>>>>>>>>>> testing with had all references to the libobaccess.so removed
> >>>>>>>>>>>>>> :(!!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> So, I'm still stuck with basically the same problem, now,
> >>>>>>>>>>>>>> working with my "full" code, with the calls in it:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> - If I compile with -L and -l, Apache segfaults when it starts
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> - If I compile without -L and -l, then I get "undefined
> >>>>>>>>>>>>>> symbol" errors when I try to start Apache, e.g.:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> [root@apachemodule build-mod_headers]#
> >>>>>>>>>>>>>> /apps/httpd2222/bin/apachectl -k start -X
> >>>>>>>>>>>>>> httpd: Syntax error on line 84 of
> >>>>>>>>>>>>>> /apps/httpd2222/conf/httpd.conf: Cannot load
> >>>>>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so into server:
> >>>>>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so: undefined symbol:
> >>>>>>>>>>>>>> ObResource_isProtected
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> That "ObResource_isProtected" should be a symbol in
> >>>>>>>>>>>>>> libobaccess.so, and in fact, if I do "nm --dynamic", I get:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> [root@apachemodule build-mod_headers]# nm --dynamic
> >>>>>>>>>>>>>> /apps/netpoint/lib64/libobaccess.so | grep
> >>>>>>>>>>>>>> "ObResource_isProtected"
> >>>>>>>>>>>>>> 00000000000a6d80 T ObResource_isProtected
> >>>>>>>>>>>>>> [root@apachemodule build-mod_headers]#
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I'm *assuming* that the reason for the "undefined symbol"
> >>>>>>>>>>>>>> error is that libobaccess.so is actually not being loaded,
> >>>>>>>>>>>>>> but then when I try to load libobaccess.so, either via -L and
> >>>>>>>>>>>>>> -l in the apxs, or using LoadFile in httpd.conf, I get the
> >>>>>>>>>>>>>> segfault (same gdb info, BTW).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Catch-22?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Sorry for the false alarm :(!!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Jim
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> Not a catch-22. The -L and -l specify linker options when
> >>>>>>>>>>>>> assembling
> >>>>>>>>>>>>> the code. The ldconfig is a run-time thing. If you are
> >>>>>>>>>>>>> getting the
> >>>>>>>>>>>>> stderr messages, you are making it all the way into your
> >>>>>>>>>>>>> library. I'd
> >>>>>>>>>>>>> suggest commenting out the following line and see if you get
> >>>>>>>>>>>>> farther :
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> That should tell you if the problem is the ssl_var_lookup.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Joe
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> http://www.silverhawk.net/
> >>>>>>>>>>>> Hi,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks for the suggestion. I just tried what you suggested,
> >>>>>>>>>>>> and got a segfault when I started Apache with the modified module.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Jim
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>> Hi,
> >>>>>>>>>>>
> >>>>>>>>>>> As a reminder, here's the gdb with the library loaded:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> (gdb) b header_post_config
> >>>>>>>>>>> Function "header_post_config" not defined.
> >>>>>>>>>>> Make breakpoint pending on future shared library load? (y or [n]) y
> >>>>>>>>>>> Breakpoint 1 (header_post_config) pending.
> >>>>>>>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> >>>>>>>>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/
> >>>>>>>>>>> -f /apps/httpd2222/conf/httpd.conf
> >>>>>>>>>>> [Thread debugging using libthread_db enabled]
> >>>>>>>>>>> [New Thread 182897610272 (LWP 11317)]
> >>>>>>>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> >>>>>>>>>>> Pending breakpoint "header_post_config" resolved
> >>>>>>>>>>> mod_headers-jl V0.09 - start calling OAM API
> >>>>>>>>>>> In register_hooks
> >>>>>>>>>>> In create_headers_dir_config
> >>>>>>>>>>> In create_headers_dir_config
> >>>>>>>>>>> In header_cmd
> >>>>>>>>>>> In header_inout_cmd
> >>>>>>>>>>> In parse_format_tag
> >>>>>>>>>>> In parse_misc_string
> >>>>>>>>>>> In create_headers_dir_config
> >>>>>>>>>>> In header_cmd
> >>>>>>>>>>> In header_inout_cmd
> >>>>>>>>>>> In parse_format_tag
> >>>>>>>>>>> In parse_misc_string
> >>>>>>>>>>> [Switching to Thread 182897610272 (LWP 11317)]
> >>>>>>>>>>>
> >>>>>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
> >>>>>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> >>>>>>>>>>> 1121 printf("In header_post_config\n");
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>> 1120 {
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>> 1121 printf("In header_post_config\n");
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>> In header_post_config
> >>>>>>>>>>> 1122 header_ssl_lookup =
> >>>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>> 1124 }
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
> >>>>>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> >>>>>>>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >>>>>>>>>>> (gdb) n
> >>>>>>>>>>>
> >>>>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
> >>>>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> So, it's actually blowing up in "BN_num_bits()" in
> >>>>>>>>>>> /lib64/libcrypto.so.4?
> >>>>>>>>>>>
> >>>>>>>>>>> Jim
> >>>>>>>>>> When you see the :
> >>>>>>>>>>
> >>>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
> >>>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> in gdb, type "bt" and hit enter. It should show the back trace
> >>>>>>>>>> and how
> >>>>>>>>>> you got to the BN_num_bits() function.
> >>>>>>>>>>
> >>>>>>>>>> Joe
> >>>>>>>>>> --
> >>>>>>>>>> http://www.silverhawk.net/
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> Here's the bt full:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> In header_inout_cmd
> >>>>>>>>> In parse_format_tag
> >>>>>>>>> In parse_misc_string
> >>>>>>>>> [Switching to Thread 182897610272 (LWP 6676)]
> >>>>>>>>>
> >>>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
> >>>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> >>>>>>>>> 1121 printf("In header_post_config\n");
> >>>>>>>>> (gdb) n
> >>>>>>>>> 1120 {
> >>>>>>>>> (gdb) n
> >>>>>>>>> 1121 printf("In header_post_config\n");
> >>>>>>>>> (gdb) n
> >>>>>>>>> In header_post_config
> >>>>>>>>> 1122 header_ssl_lookup =
> >>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>>> (gdb) n
> >>>>>>>>> 1124 }
> >>>>>>>>> (gdb) n
> >>>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
> >>>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> >>>>>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >>>>>>>>> (gdb) n
> >>>>>>>>>
> >>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
> >>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>>>>>> (gdb) bt full
> >>>>>>>>> #0 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #1 0x0000003518da8f4e in X509_ATTRIBUTE_create () from
> >>>>>>>>> /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #2 0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #3 0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #4 0x0000003518dae0e1 in ASN1_item_ex_i2d () from
> >>>>>>>>> /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #5 0x0000003518dae5f2 in ASN1_template_i2d () from
> >>>>>>>>> /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #6 0x0000003518dae28e in ASN1_item_ex_i2d () from
> >>>>>>>>> /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #7 0x0000003518dae3c6 in ASN1_item_i2d () from /lib64/libcrypto.so.4
> >>>>>>>>> No symbol table info available.
> >>>>>>>>> #8 0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8,
> >>>>>>>>> p=0x5a72d8) at ssl_engine_pphrase.c:505
> >>>>>>>>> mc = (SSLModConfigRec *) 0x571738
> >>>>>>>>> sc = (SSLSrvConfigRec *) 0x668c38
> >>>>>>>>> pServ = (server_rec *) 0x65fa48
> >>>>>>>>> cpVHostID = 0x60add0 "www.example.com:443"
> >>>>>>>>> szPath =
> >>>>>>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
> >>>>>>>>> '\0'<rep eats 15
> >>>>>>>>> times>,
> >>>>>>>>> "\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
> >>>>>>>>> '\0'<repeats 35 times>, "à\224k",
> >>>>>>>>> ' ---Type<return>
> >>>>>>>>> to continue, or q<return> to quit---
> >>>>>>>>> \0'<repeats 13 times>,
> >>>>>>>>> "øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
> >>>>>>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
> >>>>>>>>> '\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
> >>>>>>>>> '\0'<repeats 35 times>...
> >>>>>>>>> pPrivateKey = (EVP_PKEY *) 0x6ba670
> >>>>>>>>> asn1 = Variable "asn1" is not available.
> >>>>>>>>> (gdb)
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Not really sure what to make of that though :(...
> >>>>>>>>>
> >>>>>>>>> Jim
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> Ok, something interesting. I noticed that the servername in the
> >>>>>>>> gdb output was example.whatever.com, so I changed ServerName in
> >>>>>>>> extras/httpd-ssl.conf to "apache1.whatever.com". Apache still
> >>>>>>>> segfaulted after that.
> >>>>>>>>
> >>>>>>>> I noticed that igdb was showing my router IP address for
> >>>>>>>> "mod_unique_id" (whatever that is), so I thought it was because I
> >>>>>>>> didn't have the hostname in /etc/hosts. So, I added
> >>>>>>>> "apache1.whatever.com" to /etc/hosts, and also turned Apache
> >>>>>>>> LogLevel to debug, and now I get a different error:
> >>>>>>>>
> >>>>>>>> [root@apachemodule dev]# /apps/httpd2222/bin/apachectl -k start -X
> >>>>>>>> mod_headers-jl V0.09 - start calling OAM API
> >>>>>>>> In register_hooks
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In header_cmd
> >>>>>>>> In header_inout_cmd
> >>>>>>>> In parse_format_tag
> >>>>>>>> In parse_misc_string
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In header_cmd
> >>>>>>>> In header_inout_cmd
> >>>>>>>> In parse_format_tag
> >>>>>>>> In parse_misc_string
> >>>>>>>> In header_post_config
> >>>>>>>> *** glibc detected *** corrupted double-linked list:
> >>>>>>>> 0x00000000006b9710 ***
> >>>>>>>> /apps/httpd2222/bin/apachectl: line 78: 7599
> >>>>>>>> Aborted $HTTPD $ARGV
> >>>>>>>> [root@apachemodule dev]#
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Here's the Apache error_log:
> >>>>>>>>
> >>>>>>>> [Thu Jun 21 23:24:11 2012] [info] mod_unique_id: using ip addr
> >>>>>>>> 72.215.225.9
> >>>>>>>> [Thu Jun 21 23:24:12 2012] [info] Init: Seeding PRNG with 144 bytes
> >>>>>>>> of entropy
> >>>>>>>> [Thu Jun 21 23:24:12 2012] [info] Loading certificate& private
> >>>>>>>> key of SSL-aware server
> >>>>>>>> [Thu Jun 21 23:29:51 2012] [info] mod_unique_id: using ip addr
> >>>>>>>> 72.215.225.9
> >>>>>>>> [Thu Jun 21 23:29:52 2012] [info] Init: Seeding PRNG with 144 bytes
> >>>>>>>> of entropy
> >>>>>>>> [Thu Jun 21 23:29:52 2012] [info] Loading certificate& private
> >>>>>>>> key of SSL-aware server
> >>>>>>>> [Thu Jun 21 23:29:56 2012] [info] mod_unique_id: using ip addr
> >>>>>>>> 72.215.225.9
> >>>>>>>> [Thu Jun 21 23:29:57 2012] [info] Init: Seeding PRNG with 144 bytes
> >>>>>>>> of entropy
> >>>>>>>> [Thu Jun 21 23:29:57 2012] [info] Loading certificate& private
> >>>>>>>> key of SSL-aware server
> >>>>>>>> [root@apachemodule dev]#
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> and here's the gdb backtrace:
> >>>>>>>>
> >>>>>>>> (gdb) b header_post_config
> >>>>>>>> Function "header_post_config" not defined.
> >>>>>>>> Make breakpoint pending on future shared library load? (y or [n]) y
> >>>>>>>> Breakpoint 1 (header_post_config) pending.
> >>>>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> >>>>>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f
> >>>>>>>> /apps/httpd2222/conf/httpd.conf
> >>>>>>>> [Thread debugging using libthread_db enabled]
> >>>>>>>> [New Thread 182897610272 (LWP 7644)]
> >>>>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> >>>>>>>> Pending breakpoint "header_post_config" resolved
> >>>>>>>> mod_headers-jl V0.09 - start calling OAM API
> >>>>>>>> In register_hooks
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In header_cmd
> >>>>>>>> In header_inout_cmd
> >>>>>>>> In parse_format_tag
> >>>>>>>> In parse_misc_string
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In header_cmd
> >>>>>>>> In header_inout_cmd
> >>>>>>>> In parse_format_tag
> >>>>>>>> In parse_misc_string
> >>>>>>>> [Switching to Thread 182897610272 (LWP 7644)]
> >>>>>>>>
> >>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
> >>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> >>>>>>>> 1121 printf("In header_post_config\n");
> >>>>>>>> (gdb) n
> >>>>>>>> 1120 {
> >>>>>>>> (gdb) n
> >>>>>>>> 1121 printf("In header_post_config\n");
> >>>>>>>> (gdb) n
> >>>>>>>> In header_post_config
> >>>>>>>> 1122 header_ssl_lookup =
> >>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>> (gdb) n
> >>>>>>>> 1124 }
> >>>>>>>> (gdb) n
> >>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
> >>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> >>>>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >>>>>>>> (gdb) n
> >>>>>>>> *** glibc detected *** corrupted double-linked list:
> >>>>>>>> 0x00000000006b9710 ***
> >>>>>>>>
> >>>>>>>> Program received signal SIGABRT, Aborted.
> >>>>>>>> 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> >>>>>>>> (gdb) bt full
> >>>>>>>> #0 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> >>>>>>>> No symbol table info available.
> >>>>>>>> #1 0x000000351432fa6e in abort () from /lib64/tls/libc.so.6
> >>>>>>>> No symbol table info available.
> >>>>>>>> #2 0x0000003514363641 in __libc_message () from /lib64/tls/libc.so.6
> >>>>>>>> No symbol table info available.
> >>>>>>>> #3 0x0000003514369512 in _int_free () from /lib64/tls/libc.so.6
> >>>>>>>> No symbol table info available.
> >>>>>>>> #4 0x0000003514369846 in free () from /lib64/tls/libc.so.6
> >>>>>>>> No symbol table info available.
> >>>>>>>> #5 0x0000002a9790b6ba in R_free () from
> >>>>>>>> /apps/netpoint/lib64/libobaccess.so
> >>>>>>>> No symbol table info available.
> >>>>>>>> #6 0x0000002a9792bc41 in X509_CINF_free () from
> >>>>>>>> /apps/netpoint/lib64/libobaccess.so
> >>>>>>>> No symbol table info available.
> >>>>>>>> #7 0x0000002a9790ed98 in X509_free () from
> >>>>>>>> /apps/netpoint/lib64/libobaccess.so
> >>>>>>>> No symbol table info available.
> >>>>>>>> #8 0x0000002a987d97b6 in ssl_pphrase_Handle (s=0x59d3a8,
> >>>>>>>> p=0x5a72d8) at ssl_engine_pphrase.c:243
> >>>>>>>> mc = (SSLModConfigRec *) 0x571738
> >>>>>>>> sc = (SSLSrvConfigRec *) 0x668c38
> >>>>>>>> pServ = (server_rec *) 0x65fa48
> >>>>>>>> cpVHostID = 0x60add0 "apache1.whatever.com:443"
> >>>>>>>> szPath =
> >>>>>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.crt\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
> >>>>>>>> '\0'<repeats 15 times>,
> >>>>>>>> "\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
> >>>>>>>> '\0'<repeats 35 times>, "à\224k", '\0'<repeats 13 times>,
> >>>>>>>> "øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
> >>>>>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
> >>>>>>>> '\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
> >>>>>>>> '\0'<repeats 35 times>...
> >>>>>>>> pPrivateKey = Variable "pPrivateKey" is not available.
> >>>>>>>> (gdb)
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Jim
> >>>>>>>>
> >>>>>>> I guess the thing that I'm puzzled about is why it goes from
> >>>>>>> ssl_pphrase_Handle (which is part of Apache code?) to X509_free ()
> >>>>>>> in the libobaccess.so? Apache shouldn't even be aware of the
> >>>>>>> libobaccess stuff yet (since I think that Apache is still trying to
> >>>>>>> initialize SSL processing)?
> >>>>>>>
> >>>>>>> Jim
> >>>>>> It shouldn't unless you've declared a shutdown hook or a clean up hook
> >>>>>> of sorts. The looks of that stack trace look strikingly like an
> >>>>>> exception handler firing off. What are all of the hooks you have
> >>>>>> running?
> >>>>>>
> >>>>>> Joe
> >>>>>> --
> >>>>>> http://www.silverhawk.net/
> >>>>>>
> >>>>> Hi,
> >>>>>
> >>>>> I'm basically using the original mod_headers.c as a starter code, and
> >>>>> stuck some stuff in there. Haven't done anything with the hooks etc.
> >>>>> yet, so whatever was there is there:
> >>>>>
> >>>>> static void register_hooks(apr_pool_t *p)
> >>>>> {
> >>>>> printf("mod_headers-jl V0.09 - start calling OAM API\n");
> >>>>> printf("In register_hooks\n");
> >>>>> ap_register_output_filter("FIXUP_HEADERS_OUT",
> >>>>> ap_headers_output_filter,
> >>>>> NULL, AP_FTYPE_CONTENT_SET);
> >>>>> ap_register_output_filter("FIXUP_HEADERS_ERR",
> >>>>> ap_headers_error_filter,
> >>>>> NULL, AP_FTYPE_CONTENT_SET);
> >>>>> ap_hook_pre_config(header_pre_config,NULL,NULL,APR_HOOK_MIDDLE);
> >>>>> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
> >>>>> ap_hook_insert_filter(ap_headers_insert_output_filter, NULL,
> >>>>> NULL, APR_HOOK_LAST);
> >>>>> ap_hook_insert_error_filter(ap_headers_insert_error_filter,
> >>>>> NULL, NULL, APR_HOOK_LAST);
> >>>>> ap_hook_fixups(ap_headers_fixup, NULL, NULL, APR_HOOK_LAST);
> >>>>> ap_hook_post_read_request(ap_headers_early, NULL, NULL,
> >>>>> APR_HOOK_FIRST);
> >>>>> }
> >>>>>
> >>>>> module AP_MODULE_DECLARE_DATA headers_module =
> >>>>> {
> >>>>> STANDARD20_MODULE_STUFF,
> >>>>> create_headers_dir_config, /* dir config creater */
> >>>>> merge_headers_config, /* dir merger --- default is to
> >>>>> override */
> >>>>> NULL, /* server config */
> >>>>> NULL, /* merge server configs */
> >>>>> headers_cmds, /* command apr_table_t */
> >>>>> register_hooks /* register hooks */
> >>>>> };
> >>>>>
> >>>>> I guess that there is a:
> >>>>>
> >>>>> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
> >>>>>
> >>>>> in that register_hooks(), but it was there from the original code.
> >>>>>
> >>>>> BTW, I was checking, and it looks like there's an X509_free() function
> >>>>> in both openssl and in libobaccess.so (used 'nm --dynamic<file>), so
> >>>>> I'm wondering if things are confused, and maybe Apache code expects to
> >>>>> call the openssl X509_free, but somehow ends up calling the
> >>>>> X509_free() in libobaccess.so?
> >>>>>
> >>>>> What would cause something like that? If two .so files have the same
> >>>>> function, how to avoid conflicts?
> >>>>>
> >>>>> Jim
> >>>> Jim, you may have hit that on the head. I'm not sure of the load order
> >>>> and looking up symbols, but it could very well change things. Try
> >>>> re-ordering or adding a LoadFile before the SSL module and see if that
> >>>> changes things.
> >>> Look up order usually is load order. So if mod_ssl loads first and loads
> >>> its dependency libcrypto, the symbol will always be found there even if
> >>> the implementation in libobaccess.so is needed - and vice versa.
> >>>
> >>> On Solaris there is a -Bdirect linker flag which changes runtime search
> >>> order so that each module would find a needed symbol first inits direct
> >>> dependencies. Unfortunately AFAIK notihing similar exists for Linux.
> >>>
> >>> You would need to find ssl libs and libobaccess with compatible symbols.
> >>> Best would be a libobaccess which is dynamically linked against a
> >>> compatible version of libcrypto instead of - as it seems - being
> >>> statically linked against it.
> >>>
> >>> Regards,
> >>>
> >>> Rainer
> >>>
> >>
> >> Hi Rainer,
> >>
> >> Here's exactly what I found:
> >>
> >> [root@apachemodule ~]# ldd /apps/netpoint/lib64/libobaccess.so
> >> libnsl.so.1 => /lib64/libnsl.so.1 (0x0000002a95ab8000)
> >> libdl.so.2 => /lib64/libdl.so.2 (0x0000002a95bd0000)
> >> libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x0000002a95cd3000)
> >> libstdc++.so.6 => /lib64/libstdc++.so.6 (0x0000002a95de9000)
> >> libm.so.6 => /lib64/tls/libm.so.6 (0x0000002a95fd9000)
> >> libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000002a9615f000)
> >> libc.so.6 => /lib64/tls/libc.so.6 (0x0000002a9626d000)
> >> /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
> >> [root@apachemodule ~]#
> >> [root@apachemodule ~]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "X509_free"
> >> 00000000002d8d60 T X509_free
> >>
> >>
> >> [root@apachemodule ~]# ldd /lib64/libssl.so.0.9.7a
> >> libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003518700000)
> >> libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003519000000)
> >> libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003518500000)
> >> libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003518900000)
> >> libcrypto.so.4 => /lib64/libcrypto.so.4 (0x0000003518d00000)
> >> libdl.so.2 => /lib64/libdl.so.2 (0x0000003514800000)
> >> libz.so.1 => /usr/lib64/libz.so.1 (0x0000003514c00000)
> >> libc.so.6 => /lib64/tls/libc.so.6 (0x0000003514300000)
> >> libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003518300000)
> >> /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
> >> [root@apachemodule ~]#
> >> [root@apachemodule ~]# nm --dynamic /lib64/libssl.so.0.9.7a | grep "X509_free"
> >> U X509_free
> >>
> >>
> >> [root@apachemodule ~]# ldd /lib64/libcrypto.so.0.9.7a
> >> libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003518700000)
> >> libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003519000000)
> >> libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003518500000)
> >> libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003518900000)
> >> libdl.so.2 => /lib64/libdl.so.2 (0x0000003514800000)
> >> libz.so.1 => /usr/lib64/libz.so.1 (0x0000003514c00000)
> >> libc.so.6 => /lib64/tls/libc.so.6 (0x0000003514300000)
> >> libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003518300000)
> >> /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
> >> [root@apachemodule ~]#
> >> [root@apachemodule ~]# nm --dynamic /lib64/libcrypto.so.0.9.7a | grep "X509_free"
> >> 0000003518da9860 T X509_free
> >>
> >>
> >> So:
> >>
> >> - liboaccess.so doesn't REFERENCE libcrypto, but it HAS an X509_free() function in it.
> >> - libssl.so.REFERENCES libcrypto, AND it HAS an X509_free function in it.
> >> - libcrypto also has an X509_free function in it.
> >>
> >> From your msg above, are you saying that this situation can't be resolved by trying to change load order?
> >>
> >> And the only way is what you said in your last paragraph (find another libobccess.so)?
> >>
> >> Jim
> >>
> >>
> > Hi,
> >
> > Also, BTW, the call to X509_free is apparently because in ssl_engine_pphrase.c, which it seems is part of mod_ssl, it has:
> >
> > /*
> > * Free the X509 structure
> > */
> > X509_free(pX509Cert);
> >
> > but that's calling the X509_free() in libobaccess.so, rather than the X509_free() in libcrypt.so?
> >
> > Jim
>
> The way I read it, the load order can help. Did you try changing
> locations of the LoadFile for libobaccess.so to before or after the
> LoadModule for the SSL engine?
>
> Joe
> --
> http://www.silverhawk.net/
Hi,
No, not yet. I was waiting for more feedback from here :). But I will do it now and post back. Right now the LoadFile for libobaccess.so is in front of (earlier in httpd.conf) the Loadmodule.
Jim
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use Apache
module that uses shared library?
Posted by Joe Lewis <jo...@joe-lewis.com>.
On 6/22/12 6:16 AM, ohaya@cox.net wrote:
> ---- ohaya@cox.net wrote:
>> ---- Rainer Jung<ra...@kippdata.de> wrote:
>>> On 22.06.2012 06:10, Joe Lewis wrote:
>>>> On 6/21/12 10:02 PM, ohaya@cox.net wrote:
>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>>>> On 6/21/12 9:39 PM, ohaya@cox.net wrote:
>>>>>>> ---- ohaya@cox.net wrote:
>>>>>>>> ---- ohaya@cox.net wrote:
>>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>>>>>>>> On 6/21/12 7:32 PM, ohaya@cox.net wrote:
>>>>>>>>>>> ---- ohaya@cox.net wrote:
>>>>>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>>>>>>>>>>> On 6/21/12 6:46 PM, ohaya@cox.net wrote:
>>>>>>>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>>>>>>>>>>>>> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
>>>>>>>>>>>>>>>> ---- ohaya@cox.net wrote:
>>>>>>>>>>>>>>>>> ---- Sorin Manolache<so...@gmail.com> wrote:
>>>>>>>>>>>>>>>>>> And I forgot to say: run gdb in some sort of environment
>>>>>>>>>>>>>>>>>> where you see
>>>>>>>>>>>>>>>>>> your current source code line and a couple of surrounding
>>>>>>>>>>>>>>>>>> lines. You
>>>>>>>>>>>>>>>>>> could achieve this with the "list" command, but I prefer
>>>>>>>>>>>>>>>>>> running gdb in
>>>>>>>>>>>>>>>>>> emacs and let emacs do the nice listing of source code in
>>>>>>>>>>>>>>>>>> a different panel.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> S
>>>>>>>>>>>>>>>>> Here's the function from my source. It's the original
>>>>>>>>>>>>>>>>> from mod_headers.c, plus my printf:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> static int header_post_config(apr_pool_t *pconf,
>>>>>>>>>>>>>>>>> apr_pool_t *plog,
>>>>>>>>>>>>>>>>> apr_pool_t *ptemp,
>>>>>>>>>>>>>>>>> server_rec *s)
>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>> printf("In header_post_config\n");
>>>>>>>>>>>>>>>>> header_ssl_lookup =
>>>>>>>>>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>>>>>>>>>>> return OK;
>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Jim
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I was able to get the segfault to go away. Here's what I
>>>>>>>>>>>>>>>> had to do:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> - Created /etc/ld.so.conf.d/my.conf, and added the
>>>>>>>>>>>>>>>> directory where my libobaccess.so was
>>>>>>>>>>>>>>>> - Run 'ldconfig' to activate.
>>>>>>>>>>>>>>>> - In the apxs command, DON'T include the -L and -l arguments
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> After that, Apache appears to start ok, without segfault :)!!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks for all of the great help, esp. the suggestion about
>>>>>>>>>>>>>>>> checking "ldconfig -p". I still don't understand why, but
>>>>>>>>>>>>>>>> I'm just glad that I can get past this piece so now I can
>>>>>>>>>>>>>>>> debug my module :)...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Later,
>>>>>>>>>>>>>>>> Jim
>>>>>>>>>>>>>>> I'm just glad this list is as good as it is!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> FYI, the ldconfig is the dynamic linker control, and those
>>>>>>>>>>>>>>> /etc/ld.so.conf.d files provide additional search
>>>>>>>>>>>>>>> directories for the
>>>>>>>>>>>>>>> linker to check in when loading a library.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> http://www.silverhawk.net/
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sorry to report, but my earlier report was a "false positive"
>>>>>>>>>>>>>> :)...
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I forgot that the mod_headers.c that I was doing the earlier
>>>>>>>>>>>>>> testing with had all references to the libobaccess.so removed
>>>>>>>>>>>>>> :(!!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So, I'm still stuck with basically the same problem, now,
>>>>>>>>>>>>>> working with my "full" code, with the calls in it:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - If I compile with -L and -l, Apache segfaults when it starts
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - If I compile without -L and -l, then I get "undefined
>>>>>>>>>>>>>> symbol" errors when I try to start Apache, e.g.:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [root@apachemodule build-mod_headers]#
>>>>>>>>>>>>>> /apps/httpd2222/bin/apachectl -k start -X
>>>>>>>>>>>>>> httpd: Syntax error on line 84 of
>>>>>>>>>>>>>> /apps/httpd2222/conf/httpd.conf: Cannot load
>>>>>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so into server:
>>>>>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so: undefined symbol:
>>>>>>>>>>>>>> ObResource_isProtected
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> That "ObResource_isProtected" should be a symbol in
>>>>>>>>>>>>>> libobaccess.so, and in fact, if I do "nm --dynamic", I get:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [root@apachemodule build-mod_headers]# nm --dynamic
>>>>>>>>>>>>>> /apps/netpoint/lib64/libobaccess.so | grep
>>>>>>>>>>>>>> "ObResource_isProtected"
>>>>>>>>>>>>>> 00000000000a6d80 T ObResource_isProtected
>>>>>>>>>>>>>> [root@apachemodule build-mod_headers]#
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm *assuming* that the reason for the "undefined symbol"
>>>>>>>>>>>>>> error is that libobaccess.so is actually not being loaded,
>>>>>>>>>>>>>> but then when I try to load libobaccess.so, either via -L and
>>>>>>>>>>>>>> -l in the apxs, or using LoadFile in httpd.conf, I get the
>>>>>>>>>>>>>> segfault (same gdb info, BTW).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Catch-22?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sorry for the false alarm :(!!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Jim
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>> Not a catch-22. The -L and -l specify linker options when
>>>>>>>>>>>>> assembling
>>>>>>>>>>>>> the code. The ldconfig is a run-time thing. If you are
>>>>>>>>>>>>> getting the
>>>>>>>>>>>>> stderr messages, you are making it all the way into your
>>>>>>>>>>>>> library. I'd
>>>>>>>>>>>>> suggest commenting out the following line and see if you get
>>>>>>>>>>>>> farther :
>>>>>>>>>>>>>
>>>>>>>>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>>>>>>>
>>>>>>>>>>>>> That should tell you if the problem is the ssl_var_lookup.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Joe
>>>>>>>>>>>>> --
>>>>>>>>>>>>> http://www.silverhawk.net/
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for the suggestion. I just tried what you suggested,
>>>>>>>>>>>> and got a segfault when I started Apache with the modified module.
>>>>>>>>>>>>
>>>>>>>>>>>> Jim
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> As a reminder, here's the gdb with the library loaded:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> (gdb) b header_post_config
>>>>>>>>>>> Function "header_post_config" not defined.
>>>>>>>>>>> Make breakpoint pending on future shared library load? (y or [n]) y
>>>>>>>>>>> Breakpoint 1 (header_post_config) pending.
>>>>>>>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
>>>>>>>>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/
>>>>>>>>>>> -f /apps/httpd2222/conf/httpd.conf
>>>>>>>>>>> [Thread debugging using libthread_db enabled]
>>>>>>>>>>> [New Thread 182897610272 (LWP 11317)]
>>>>>>>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
>>>>>>>>>>> Pending breakpoint "header_post_config" resolved
>>>>>>>>>>> mod_headers-jl V0.09 - start calling OAM API
>>>>>>>>>>> In register_hooks
>>>>>>>>>>> In create_headers_dir_config
>>>>>>>>>>> In create_headers_dir_config
>>>>>>>>>>> In header_cmd
>>>>>>>>>>> In header_inout_cmd
>>>>>>>>>>> In parse_format_tag
>>>>>>>>>>> In parse_misc_string
>>>>>>>>>>> In create_headers_dir_config
>>>>>>>>>>> In header_cmd
>>>>>>>>>>> In header_inout_cmd
>>>>>>>>>>> In parse_format_tag
>>>>>>>>>>> In parse_misc_string
>>>>>>>>>>> [Switching to Thread 182897610272 (LWP 11317)]
>>>>>>>>>>>
>>>>>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
>>>>>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>>>>>>>>>>> 1121 printf("In header_post_config\n");
>>>>>>>>>>> (gdb) n
>>>>>>>>>>> 1120 {
>>>>>>>>>>> (gdb) n
>>>>>>>>>>> 1121 printf("In header_post_config\n");
>>>>>>>>>>> (gdb) n
>>>>>>>>>>> In header_post_config
>>>>>>>>>>> 1122 header_ssl_lookup =
>>>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>>>>> (gdb) n
>>>>>>>>>>> 1124 }
>>>>>>>>>>> (gdb) n
>>>>>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
>>>>>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
>>>>>>>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>>>>>>>>>>> (gdb) n
>>>>>>>>>>>
>>>>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> So, it's actually blowing up in "BN_num_bits()" in
>>>>>>>>>>> /lib64/libcrypto.so.4?
>>>>>>>>>>>
>>>>>>>>>>> Jim
>>>>>>>>>> When you see the :
>>>>>>>>>>
>>>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> in gdb, type "bt" and hit enter. It should show the back trace
>>>>>>>>>> and how
>>>>>>>>>> you got to the BN_num_bits() function.
>>>>>>>>>>
>>>>>>>>>> Joe
>>>>>>>>>> --
>>>>>>>>>> http://www.silverhawk.net/
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Here's the bt full:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> In header_inout_cmd
>>>>>>>>> In parse_format_tag
>>>>>>>>> In parse_misc_string
>>>>>>>>> [Switching to Thread 182897610272 (LWP 6676)]
>>>>>>>>>
>>>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
>>>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>>>>>>>>> 1121 printf("In header_post_config\n");
>>>>>>>>> (gdb) n
>>>>>>>>> 1120 {
>>>>>>>>> (gdb) n
>>>>>>>>> 1121 printf("In header_post_config\n");
>>>>>>>>> (gdb) n
>>>>>>>>> In header_post_config
>>>>>>>>> 1122 header_ssl_lookup =
>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>>> (gdb) n
>>>>>>>>> 1124 }
>>>>>>>>> (gdb) n
>>>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
>>>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
>>>>>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>>>>>>>>> (gdb) n
>>>>>>>>>
>>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>>>>> (gdb) bt full
>>>>>>>>> #0 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #1 0x0000003518da8f4e in X509_ATTRIBUTE_create () from
>>>>>>>>> /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #2 0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #3 0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #4 0x0000003518dae0e1 in ASN1_item_ex_i2d () from
>>>>>>>>> /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #5 0x0000003518dae5f2 in ASN1_template_i2d () from
>>>>>>>>> /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #6 0x0000003518dae28e in ASN1_item_ex_i2d () from
>>>>>>>>> /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #7 0x0000003518dae3c6 in ASN1_item_i2d () from /lib64/libcrypto.so.4
>>>>>>>>> No symbol table info available.
>>>>>>>>> #8 0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8,
>>>>>>>>> p=0x5a72d8) at ssl_engine_pphrase.c:505
>>>>>>>>> mc = (SSLModConfigRec *) 0x571738
>>>>>>>>> sc = (SSLSrvConfigRec *) 0x668c38
>>>>>>>>> pServ = (server_rec *) 0x65fa48
>>>>>>>>> cpVHostID = 0x60add0 "www.example.com:443"
>>>>>>>>> szPath =
>>>>>>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
>>>>>>>>> '\0'<rep eats 15
>>>>>>>>> times>,
>>>>>>>>> "\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
>>>>>>>>> '\0'<repeats 35 times>, "à\224k",
>>>>>>>>> ' ---Type<return>
>>>>>>>>> to continue, or q<return> to quit---
>>>>>>>>> \0'<repeats 13 times>,
>>>>>>>>> "øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
>>>>>>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
>>>>>>>>> '\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
>>>>>>>>> '\0'<repeats 35 times>...
>>>>>>>>> pPrivateKey = (EVP_PKEY *) 0x6ba670
>>>>>>>>> asn1 = Variable "asn1" is not available.
>>>>>>>>> (gdb)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Not really sure what to make of that though :(...
>>>>>>>>>
>>>>>>>>> Jim
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Ok, something interesting. I noticed that the servername in the
>>>>>>>> gdb output was example.whatever.com, so I changed ServerName in
>>>>>>>> extras/httpd-ssl.conf to "apache1.whatever.com". Apache still
>>>>>>>> segfaulted after that.
>>>>>>>>
>>>>>>>> I noticed that igdb was showing my router IP address for
>>>>>>>> "mod_unique_id" (whatever that is), so I thought it was because I
>>>>>>>> didn't have the hostname in /etc/hosts. So, I added
>>>>>>>> "apache1.whatever.com" to /etc/hosts, and also turned Apache
>>>>>>>> LogLevel to debug, and now I get a different error:
>>>>>>>>
>>>>>>>> [root@apachemodule dev]# /apps/httpd2222/bin/apachectl -k start -X
>>>>>>>> mod_headers-jl V0.09 - start calling OAM API
>>>>>>>> In register_hooks
>>>>>>>> In create_headers_dir_config
>>>>>>>> In create_headers_dir_config
>>>>>>>> In header_cmd
>>>>>>>> In header_inout_cmd
>>>>>>>> In parse_format_tag
>>>>>>>> In parse_misc_string
>>>>>>>> In create_headers_dir_config
>>>>>>>> In header_cmd
>>>>>>>> In header_inout_cmd
>>>>>>>> In parse_format_tag
>>>>>>>> In parse_misc_string
>>>>>>>> In header_post_config
>>>>>>>> *** glibc detected *** corrupted double-linked list:
>>>>>>>> 0x00000000006b9710 ***
>>>>>>>> /apps/httpd2222/bin/apachectl: line 78: 7599
>>>>>>>> Aborted $HTTPD $ARGV
>>>>>>>> [root@apachemodule dev]#
>>>>>>>>
>>>>>>>>
>>>>>>>> Here's the Apache error_log:
>>>>>>>>
>>>>>>>> [Thu Jun 21 23:24:11 2012] [info] mod_unique_id: using ip addr
>>>>>>>> 72.215.225.9
>>>>>>>> [Thu Jun 21 23:24:12 2012] [info] Init: Seeding PRNG with 144 bytes
>>>>>>>> of entropy
>>>>>>>> [Thu Jun 21 23:24:12 2012] [info] Loading certificate& private
>>>>>>>> key of SSL-aware server
>>>>>>>> [Thu Jun 21 23:29:51 2012] [info] mod_unique_id: using ip addr
>>>>>>>> 72.215.225.9
>>>>>>>> [Thu Jun 21 23:29:52 2012] [info] Init: Seeding PRNG with 144 bytes
>>>>>>>> of entropy
>>>>>>>> [Thu Jun 21 23:29:52 2012] [info] Loading certificate& private
>>>>>>>> key of SSL-aware server
>>>>>>>> [Thu Jun 21 23:29:56 2012] [info] mod_unique_id: using ip addr
>>>>>>>> 72.215.225.9
>>>>>>>> [Thu Jun 21 23:29:57 2012] [info] Init: Seeding PRNG with 144 bytes
>>>>>>>> of entropy
>>>>>>>> [Thu Jun 21 23:29:57 2012] [info] Loading certificate& private
>>>>>>>> key of SSL-aware server
>>>>>>>> [root@apachemodule dev]#
>>>>>>>>
>>>>>>>>
>>>>>>>> and here's the gdb backtrace:
>>>>>>>>
>>>>>>>> (gdb) b header_post_config
>>>>>>>> Function "header_post_config" not defined.
>>>>>>>> Make breakpoint pending on future shared library load? (y or [n]) y
>>>>>>>> Breakpoint 1 (header_post_config) pending.
>>>>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
>>>>>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f
>>>>>>>> /apps/httpd2222/conf/httpd.conf
>>>>>>>> [Thread debugging using libthread_db enabled]
>>>>>>>> [New Thread 182897610272 (LWP 7644)]
>>>>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
>>>>>>>> Pending breakpoint "header_post_config" resolved
>>>>>>>> mod_headers-jl V0.09 - start calling OAM API
>>>>>>>> In register_hooks
>>>>>>>> In create_headers_dir_config
>>>>>>>> In create_headers_dir_config
>>>>>>>> In header_cmd
>>>>>>>> In header_inout_cmd
>>>>>>>> In parse_format_tag
>>>>>>>> In parse_misc_string
>>>>>>>> In create_headers_dir_config
>>>>>>>> In header_cmd
>>>>>>>> In header_inout_cmd
>>>>>>>> In parse_format_tag
>>>>>>>> In parse_misc_string
>>>>>>>> [Switching to Thread 182897610272 (LWP 7644)]
>>>>>>>>
>>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
>>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>>>>>>>> 1121 printf("In header_post_config\n");
>>>>>>>> (gdb) n
>>>>>>>> 1120 {
>>>>>>>> (gdb) n
>>>>>>>> 1121 printf("In header_post_config\n");
>>>>>>>> (gdb) n
>>>>>>>> In header_post_config
>>>>>>>> 1122 header_ssl_lookup =
>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>> (gdb) n
>>>>>>>> 1124 }
>>>>>>>> (gdb) n
>>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
>>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
>>>>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>>>>>>>> (gdb) n
>>>>>>>> *** glibc detected *** corrupted double-linked list:
>>>>>>>> 0x00000000006b9710 ***
>>>>>>>>
>>>>>>>> Program received signal SIGABRT, Aborted.
>>>>>>>> 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
>>>>>>>> (gdb) bt full
>>>>>>>> #0 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
>>>>>>>> No symbol table info available.
>>>>>>>> #1 0x000000351432fa6e in abort () from /lib64/tls/libc.so.6
>>>>>>>> No symbol table info available.
>>>>>>>> #2 0x0000003514363641 in __libc_message () from /lib64/tls/libc.so.6
>>>>>>>> No symbol table info available.
>>>>>>>> #3 0x0000003514369512 in _int_free () from /lib64/tls/libc.so.6
>>>>>>>> No symbol table info available.
>>>>>>>> #4 0x0000003514369846 in free () from /lib64/tls/libc.so.6
>>>>>>>> No symbol table info available.
>>>>>>>> #5 0x0000002a9790b6ba in R_free () from
>>>>>>>> /apps/netpoint/lib64/libobaccess.so
>>>>>>>> No symbol table info available.
>>>>>>>> #6 0x0000002a9792bc41 in X509_CINF_free () from
>>>>>>>> /apps/netpoint/lib64/libobaccess.so
>>>>>>>> No symbol table info available.
>>>>>>>> #7 0x0000002a9790ed98 in X509_free () from
>>>>>>>> /apps/netpoint/lib64/libobaccess.so
>>>>>>>> No symbol table info available.
>>>>>>>> #8 0x0000002a987d97b6 in ssl_pphrase_Handle (s=0x59d3a8,
>>>>>>>> p=0x5a72d8) at ssl_engine_pphrase.c:243
>>>>>>>> mc = (SSLModConfigRec *) 0x571738
>>>>>>>> sc = (SSLSrvConfigRec *) 0x668c38
>>>>>>>> pServ = (server_rec *) 0x65fa48
>>>>>>>> cpVHostID = 0x60add0 "apache1.whatever.com:443"
>>>>>>>> szPath =
>>>>>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.crt\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
>>>>>>>> '\0'<repeats 15 times>,
>>>>>>>> "\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
>>>>>>>> '\0'<repeats 35 times>, "à\224k", '\0'<repeats 13 times>,
>>>>>>>> "øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
>>>>>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
>>>>>>>> '\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
>>>>>>>> '\0'<repeats 35 times>...
>>>>>>>> pPrivateKey = Variable "pPrivateKey" is not available.
>>>>>>>> (gdb)
>>>>>>>>
>>>>>>>>
>>>>>>>> Jim
>>>>>>>>
>>>>>>> I guess the thing that I'm puzzled about is why it goes from
>>>>>>> ssl_pphrase_Handle (which is part of Apache code?) to X509_free ()
>>>>>>> in the libobaccess.so? Apache shouldn't even be aware of the
>>>>>>> libobaccess stuff yet (since I think that Apache is still trying to
>>>>>>> initialize SSL processing)?
>>>>>>>
>>>>>>> Jim
>>>>>> It shouldn't unless you've declared a shutdown hook or a clean up hook
>>>>>> of sorts. The looks of that stack trace look strikingly like an
>>>>>> exception handler firing off. What are all of the hooks you have
>>>>>> running?
>>>>>>
>>>>>> Joe
>>>>>> --
>>>>>> http://www.silverhawk.net/
>>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm basically using the original mod_headers.c as a starter code, and
>>>>> stuck some stuff in there. Haven't done anything with the hooks etc.
>>>>> yet, so whatever was there is there:
>>>>>
>>>>> static void register_hooks(apr_pool_t *p)
>>>>> {
>>>>> printf("mod_headers-jl V0.09 - start calling OAM API\n");
>>>>> printf("In register_hooks\n");
>>>>> ap_register_output_filter("FIXUP_HEADERS_OUT",
>>>>> ap_headers_output_filter,
>>>>> NULL, AP_FTYPE_CONTENT_SET);
>>>>> ap_register_output_filter("FIXUP_HEADERS_ERR",
>>>>> ap_headers_error_filter,
>>>>> NULL, AP_FTYPE_CONTENT_SET);
>>>>> ap_hook_pre_config(header_pre_config,NULL,NULL,APR_HOOK_MIDDLE);
>>>>> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
>>>>> ap_hook_insert_filter(ap_headers_insert_output_filter, NULL,
>>>>> NULL, APR_HOOK_LAST);
>>>>> ap_hook_insert_error_filter(ap_headers_insert_error_filter,
>>>>> NULL, NULL, APR_HOOK_LAST);
>>>>> ap_hook_fixups(ap_headers_fixup, NULL, NULL, APR_HOOK_LAST);
>>>>> ap_hook_post_read_request(ap_headers_early, NULL, NULL,
>>>>> APR_HOOK_FIRST);
>>>>> }
>>>>>
>>>>> module AP_MODULE_DECLARE_DATA headers_module =
>>>>> {
>>>>> STANDARD20_MODULE_STUFF,
>>>>> create_headers_dir_config, /* dir config creater */
>>>>> merge_headers_config, /* dir merger --- default is to
>>>>> override */
>>>>> NULL, /* server config */
>>>>> NULL, /* merge server configs */
>>>>> headers_cmds, /* command apr_table_t */
>>>>> register_hooks /* register hooks */
>>>>> };
>>>>>
>>>>> I guess that there is a:
>>>>>
>>>>> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
>>>>>
>>>>> in that register_hooks(), but it was there from the original code.
>>>>>
>>>>> BTW, I was checking, and it looks like there's an X509_free() function
>>>>> in both openssl and in libobaccess.so (used 'nm --dynamic<file>), so
>>>>> I'm wondering if things are confused, and maybe Apache code expects to
>>>>> call the openssl X509_free, but somehow ends up calling the
>>>>> X509_free() in libobaccess.so?
>>>>>
>>>>> What would cause something like that? If two .so files have the same
>>>>> function, how to avoid conflicts?
>>>>>
>>>>> Jim
>>>> Jim, you may have hit that on the head. I'm not sure of the load order
>>>> and looking up symbols, but it could very well change things. Try
>>>> re-ordering or adding a LoadFile before the SSL module and see if that
>>>> changes things.
>>> Look up order usually is load order. So if mod_ssl loads first and loads
>>> its dependency libcrypto, the symbol will always be found there even if
>>> the implementation in libobaccess.so is needed - and vice versa.
>>>
>>> On Solaris there is a -Bdirect linker flag which changes runtime search
>>> order so that each module would find a needed symbol first inits direct
>>> dependencies. Unfortunately AFAIK notihing similar exists for Linux.
>>>
>>> You would need to find ssl libs and libobaccess with compatible symbols.
>>> Best would be a libobaccess which is dynamically linked against a
>>> compatible version of libcrypto instead of - as it seems - being
>>> statically linked against it.
>>>
>>> Regards,
>>>
>>> Rainer
>>>
>>
>> Hi Rainer,
>>
>> Here's exactly what I found:
>>
>> [root@apachemodule ~]# ldd /apps/netpoint/lib64/libobaccess.so
>> libnsl.so.1 => /lib64/libnsl.so.1 (0x0000002a95ab8000)
>> libdl.so.2 => /lib64/libdl.so.2 (0x0000002a95bd0000)
>> libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x0000002a95cd3000)
>> libstdc++.so.6 => /lib64/libstdc++.so.6 (0x0000002a95de9000)
>> libm.so.6 => /lib64/tls/libm.so.6 (0x0000002a95fd9000)
>> libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000002a9615f000)
>> libc.so.6 => /lib64/tls/libc.so.6 (0x0000002a9626d000)
>> /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
>> [root@apachemodule ~]#
>> [root@apachemodule ~]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "X509_free"
>> 00000000002d8d60 T X509_free
>>
>>
>> [root@apachemodule ~]# ldd /lib64/libssl.so.0.9.7a
>> libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003518700000)
>> libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003519000000)
>> libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003518500000)
>> libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003518900000)
>> libcrypto.so.4 => /lib64/libcrypto.so.4 (0x0000003518d00000)
>> libdl.so.2 => /lib64/libdl.so.2 (0x0000003514800000)
>> libz.so.1 => /usr/lib64/libz.so.1 (0x0000003514c00000)
>> libc.so.6 => /lib64/tls/libc.so.6 (0x0000003514300000)
>> libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003518300000)
>> /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
>> [root@apachemodule ~]#
>> [root@apachemodule ~]# nm --dynamic /lib64/libssl.so.0.9.7a | grep "X509_free"
>> U X509_free
>>
>>
>> [root@apachemodule ~]# ldd /lib64/libcrypto.so.0.9.7a
>> libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003518700000)
>> libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003519000000)
>> libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003518500000)
>> libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003518900000)
>> libdl.so.2 => /lib64/libdl.so.2 (0x0000003514800000)
>> libz.so.1 => /usr/lib64/libz.so.1 (0x0000003514c00000)
>> libc.so.6 => /lib64/tls/libc.so.6 (0x0000003514300000)
>> libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003518300000)
>> /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
>> [root@apachemodule ~]#
>> [root@apachemodule ~]# nm --dynamic /lib64/libcrypto.so.0.9.7a | grep "X509_free"
>> 0000003518da9860 T X509_free
>>
>>
>> So:
>>
>> - liboaccess.so doesn't REFERENCE libcrypto, but it HAS an X509_free() function in it.
>> - libssl.so.REFERENCES libcrypto, AND it HAS an X509_free function in it.
>> - libcrypto also has an X509_free function in it.
>>
>> From your msg above, are you saying that this situation can't be resolved by trying to change load order?
>>
>> And the only way is what you said in your last paragraph (find another libobccess.so)?
>>
>> Jim
>>
>>
> Hi,
>
> Also, BTW, the call to X509_free is apparently because in ssl_engine_pphrase.c, which it seems is part of mod_ssl, it has:
>
> /*
> * Free the X509 structure
> */
> X509_free(pX509Cert);
>
> but that's calling the X509_free() in libobaccess.so, rather than the X509_free() in libcrypt.so?
>
> Jim
The way I read it, the load order can help. Did you try changing
locations of the LoadFile for libobaccess.so to before or after the
LoadModule for the SSL engine?
Joe
--
http://www.silverhawk.net/
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use
Apache module that uses shared library?
Posted by oh...@cox.net.
---- ohaya@cox.net wrote:
>
> ---- Rainer Jung <ra...@kippdata.de> wrote:
> > On 22.06.2012 06:10, Joe Lewis wrote:
> > > On 6/21/12 10:02 PM, ohaya@cox.net wrote:
> > >> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> > >>> On 6/21/12 9:39 PM, ohaya@cox.net wrote:
> > >>>> ---- ohaya@cox.net wrote:
> > >>>>> ---- ohaya@cox.net wrote:
> > >>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> > >>>>>>> On 6/21/12 7:32 PM, ohaya@cox.net wrote:
> > >>>>>>>> ---- ohaya@cox.net wrote:
> > >>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> > >>>>>>>>>> On 6/21/12 6:46 PM, ohaya@cox.net wrote:
> > >>>>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> > >>>>>>>>>>>> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
> > >>>>>>>>>>>>> ---- ohaya@cox.net wrote:
> > >>>>>>>>>>>>>> ---- Sorin Manolache<so...@gmail.com> wrote:
> > >>>>>>>>>>>>>>> And I forgot to say: run gdb in some sort of environment
> > >>>>>>>>>>>>>>> where you see
> > >>>>>>>>>>>>>>> your current source code line and a couple of surrounding
> > >>>>>>>>>>>>>>> lines. You
> > >>>>>>>>>>>>>>> could achieve this with the "list" command, but I prefer
> > >>>>>>>>>>>>>>> running gdb in
> > >>>>>>>>>>>>>>> emacs and let emacs do the nice listing of source code in
> > >>>>>>>>>>>>>>> a different panel.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> S
> > >>>>>>>>>>>>>> Here's the function from my source. It's the original
> > >>>>>>>>>>>>>> from mod_headers.c, plus my printf:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> static int header_post_config(apr_pool_t *pconf,
> > >>>>>>>>>>>>>> apr_pool_t *plog,
> > >>>>>>>>>>>>>> apr_pool_t *ptemp,
> > >>>>>>>>>>>>>> server_rec *s)
> > >>>>>>>>>>>>>> {
> > >>>>>>>>>>>>>> printf("In header_post_config\n");
> > >>>>>>>>>>>>>> header_ssl_lookup =
> > >>>>>>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > >>>>>>>>>>>>>> return OK;
> > >>>>>>>>>>>>>> }
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Jim
> > >>>>>>>>>>>>> Hi,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I was able to get the segfault to go away. Here's what I
> > >>>>>>>>>>>>> had to do:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> - Created /etc/ld.so.conf.d/my.conf, and added the
> > >>>>>>>>>>>>> directory where my libobaccess.so was
> > >>>>>>>>>>>>> - Run 'ldconfig' to activate.
> > >>>>>>>>>>>>> - In the apxs command, DON'T include the -L and -l arguments
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> After that, Apache appears to start ok, without segfault :)!!
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Thanks for all of the great help, esp. the suggestion about
> > >>>>>>>>>>>>> checking "ldconfig -p". I still don't understand why, but
> > >>>>>>>>>>>>> I'm just glad that I can get past this piece so now I can
> > >>>>>>>>>>>>> debug my module :)...
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Later,
> > >>>>>>>>>>>>> Jim
> > >>>>>>>>>>>> I'm just glad this list is as good as it is!
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> FYI, the ldconfig is the dynamic linker control, and those
> > >>>>>>>>>>>> /etc/ld.so.conf.d files provide additional search
> > >>>>>>>>>>>> directories for the
> > >>>>>>>>>>>> linker to check in when loading a library.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Joe
> > >>>>>>>>>>>> --
> > >>>>>>>>>>>> http://www.silverhawk.net/
> > >>>>>>>>>>> Hi,
> > >>>>>>>>>>>
> > >>>>>>>>>>> Sorry to report, but my earlier report was a "false positive"
> > >>>>>>>>>>> :)...
> > >>>>>>>>>>>
> > >>>>>>>>>>> I forgot that the mod_headers.c that I was doing the earlier
> > >>>>>>>>>>> testing with had all references to the libobaccess.so removed
> > >>>>>>>>>>> :(!!
> > >>>>>>>>>>>
> > >>>>>>>>>>> So, I'm still stuck with basically the same problem, now,
> > >>>>>>>>>>> working with my "full" code, with the calls in it:
> > >>>>>>>>>>>
> > >>>>>>>>>>> - If I compile with -L and -l, Apache segfaults when it starts
> > >>>>>>>>>>>
> > >>>>>>>>>>> - If I compile without -L and -l, then I get "undefined
> > >>>>>>>>>>> symbol" errors when I try to start Apache, e.g.:
> > >>>>>>>>>>>
> > >>>>>>>>>>> [root@apachemodule build-mod_headers]#
> > >>>>>>>>>>> /apps/httpd2222/bin/apachectl -k start -X
> > >>>>>>>>>>> httpd: Syntax error on line 84 of
> > >>>>>>>>>>> /apps/httpd2222/conf/httpd.conf: Cannot load
> > >>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so into server:
> > >>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so: undefined symbol:
> > >>>>>>>>>>> ObResource_isProtected
> > >>>>>>>>>>>
> > >>>>>>>>>>> That "ObResource_isProtected" should be a symbol in
> > >>>>>>>>>>> libobaccess.so, and in fact, if I do "nm --dynamic", I get:
> > >>>>>>>>>>>
> > >>>>>>>>>>> [root@apachemodule build-mod_headers]# nm --dynamic
> > >>>>>>>>>>> /apps/netpoint/lib64/libobaccess.so | grep
> > >>>>>>>>>>> "ObResource_isProtected"
> > >>>>>>>>>>> 00000000000a6d80 T ObResource_isProtected
> > >>>>>>>>>>> [root@apachemodule build-mod_headers]#
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> I'm *assuming* that the reason for the "undefined symbol"
> > >>>>>>>>>>> error is that libobaccess.so is actually not being loaded,
> > >>>>>>>>>>> but then when I try to load libobaccess.so, either via -L and
> > >>>>>>>>>>> -l in the apxs, or using LoadFile in httpd.conf, I get the
> > >>>>>>>>>>> segfault (same gdb info, BTW).
> > >>>>>>>>>>>
> > >>>>>>>>>>> Catch-22?
> > >>>>>>>>>>>
> > >>>>>>>>>>> Sorry for the false alarm :(!!
> > >>>>>>>>>>>
> > >>>>>>>>>>> Jim
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>> Not a catch-22. The -L and -l specify linker options when
> > >>>>>>>>>> assembling
> > >>>>>>>>>> the code. The ldconfig is a run-time thing. If you are
> > >>>>>>>>>> getting the
> > >>>>>>>>>> stderr messages, you are making it all the way into your
> > >>>>>>>>>> library. I'd
> > >>>>>>>>>> suggest commenting out the following line and see if you get
> > >>>>>>>>>> farther :
> > >>>>>>>>>>
> > >>>>>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > >>>>>>>>>>
> > >>>>>>>>>> That should tell you if the problem is the ssl_var_lookup.
> > >>>>>>>>>>
> > >>>>>>>>>> Joe
> > >>>>>>>>>> --
> > >>>>>>>>>> http://www.silverhawk.net/
> > >>>>>>>>> Hi,
> > >>>>>>>>>
> > >>>>>>>>> Thanks for the suggestion. I just tried what you suggested,
> > >>>>>>>>> and got a segfault when I started Apache with the modified module.
> > >>>>>>>>>
> > >>>>>>>>> Jim
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>> Hi,
> > >>>>>>>>
> > >>>>>>>> As a reminder, here's the gdb with the library loaded:
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> (gdb) b header_post_config
> > >>>>>>>> Function "header_post_config" not defined.
> > >>>>>>>> Make breakpoint pending on future shared library load? (y or [n]) y
> > >>>>>>>> Breakpoint 1 (header_post_config) pending.
> > >>>>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> > >>>>>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/
> > >>>>>>>> -f /apps/httpd2222/conf/httpd.conf
> > >>>>>>>> [Thread debugging using libthread_db enabled]
> > >>>>>>>> [New Thread 182897610272 (LWP 11317)]
> > >>>>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> > >>>>>>>> Pending breakpoint "header_post_config" resolved
> > >>>>>>>> mod_headers-jl V0.09 - start calling OAM API
> > >>>>>>>> In register_hooks
> > >>>>>>>> In create_headers_dir_config
> > >>>>>>>> In create_headers_dir_config
> > >>>>>>>> In header_cmd
> > >>>>>>>> In header_inout_cmd
> > >>>>>>>> In parse_format_tag
> > >>>>>>>> In parse_misc_string
> > >>>>>>>> In create_headers_dir_config
> > >>>>>>>> In header_cmd
> > >>>>>>>> In header_inout_cmd
> > >>>>>>>> In parse_format_tag
> > >>>>>>>> In parse_misc_string
> > >>>>>>>> [Switching to Thread 182897610272 (LWP 11317)]
> > >>>>>>>>
> > >>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
> > >>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> > >>>>>>>> 1121 printf("In header_post_config\n");
> > >>>>>>>> (gdb) n
> > >>>>>>>> 1120 {
> > >>>>>>>> (gdb) n
> > >>>>>>>> 1121 printf("In header_post_config\n");
> > >>>>>>>> (gdb) n
> > >>>>>>>> In header_post_config
> > >>>>>>>> 1122 header_ssl_lookup =
> > >>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > >>>>>>>> (gdb) n
> > >>>>>>>> 1124 }
> > >>>>>>>> (gdb) n
> > >>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
> > >>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> > >>>>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> > >>>>>>>> (gdb) n
> > >>>>>>>>
> > >>>>>>>> Program received signal SIGSEGV, Segmentation fault.
> > >>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> So, it's actually blowing up in "BN_num_bits()" in
> > >>>>>>>> /lib64/libcrypto.so.4?
> > >>>>>>>>
> > >>>>>>>> Jim
> > >>>>>>> When you see the :
> > >>>>>>>
> > >>>>>>> Program received signal SIGSEGV, Segmentation fault.
> > >>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> in gdb, type "bt" and hit enter. It should show the back trace
> > >>>>>>> and how
> > >>>>>>> you got to the BN_num_bits() function.
> > >>>>>>>
> > >>>>>>> Joe
> > >>>>>>> --
> > >>>>>>> http://www.silverhawk.net/
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>> Here's the bt full:
> > >>>>>>
> > >>>>>>
> > >>>>>> In header_inout_cmd
> > >>>>>> In parse_format_tag
> > >>>>>> In parse_misc_string
> > >>>>>> [Switching to Thread 182897610272 (LWP 6676)]
> > >>>>>>
> > >>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
> > >>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> > >>>>>> 1121 printf("In header_post_config\n");
> > >>>>>> (gdb) n
> > >>>>>> 1120 {
> > >>>>>> (gdb) n
> > >>>>>> 1121 printf("In header_post_config\n");
> > >>>>>> (gdb) n
> > >>>>>> In header_post_config
> > >>>>>> 1122 header_ssl_lookup =
> > >>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > >>>>>> (gdb) n
> > >>>>>> 1124 }
> > >>>>>> (gdb) n
> > >>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
> > >>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> > >>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> > >>>>>> (gdb) n
> > >>>>>>
> > >>>>>> Program received signal SIGSEGV, Segmentation fault.
> > >>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> > >>>>>> (gdb) bt full
> > >>>>>> #0 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> > >>>>>> No symbol table info available.
> > >>>>>> #1 0x0000003518da8f4e in X509_ATTRIBUTE_create () from
> > >>>>>> /lib64/libcrypto.so.4
> > >>>>>> No symbol table info available.
> > >>>>>> #2 0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> > >>>>>> No symbol table info available.
> > >>>>>> #3 0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> > >>>>>> No symbol table info available.
> > >>>>>> #4 0x0000003518dae0e1 in ASN1_item_ex_i2d () from
> > >>>>>> /lib64/libcrypto.so.4
> > >>>>>> No symbol table info available.
> > >>>>>> #5 0x0000003518dae5f2 in ASN1_template_i2d () from
> > >>>>>> /lib64/libcrypto.so.4
> > >>>>>> No symbol table info available.
> > >>>>>> #6 0x0000003518dae28e in ASN1_item_ex_i2d () from
> > >>>>>> /lib64/libcrypto.so.4
> > >>>>>> No symbol table info available.
> > >>>>>> #7 0x0000003518dae3c6 in ASN1_item_i2d () from /lib64/libcrypto.so.4
> > >>>>>> No symbol table info available.
> > >>>>>> #8 0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8,
> > >>>>>> p=0x5a72d8) at ssl_engine_pphrase.c:505
> > >>>>>> mc = (SSLModConfigRec *) 0x571738
> > >>>>>> sc = (SSLSrvConfigRec *) 0x668c38
> > >>>>>> pServ = (server_rec *) 0x65fa48
> > >>>>>> cpVHostID = 0x60add0 "www.example.com:443"
> > >>>>>> szPath =
> > >>>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
> > >>>>>> '\0'<rep eats 15
> > >>>>>> times>,
> > >>>>>> "\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
> > >>>>>> '\0'<repeats 35 times>, "à\224k",
> > >>>>>> ' ---Type<return>
> > >>>>>> to continue, or q<return> to quit---
> > >>>>>> \0'<repeats 13 times>,
> > >>>>>> "øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
> > >>>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
> > >>>>>> '\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
> > >>>>>> '\0'<repeats 35 times>...
> > >>>>>> pPrivateKey = (EVP_PKEY *) 0x6ba670
> > >>>>>> asn1 = Variable "asn1" is not available.
> > >>>>>> (gdb)
> > >>>>>>
> > >>>>>>
> > >>>>>> Not really sure what to make of that though :(...
> > >>>>>>
> > >>>>>> Jim
> > >>>>> Hi,
> > >>>>>
> > >>>>> Ok, something interesting. I noticed that the servername in the
> > >>>>> gdb output was example.whatever.com, so I changed ServerName in
> > >>>>> extras/httpd-ssl.conf to "apache1.whatever.com". Apache still
> > >>>>> segfaulted after that.
> > >>>>>
> > >>>>> I noticed that igdb was showing my router IP address for
> > >>>>> "mod_unique_id" (whatever that is), so I thought it was because I
> > >>>>> didn't have the hostname in /etc/hosts. So, I added
> > >>>>> "apache1.whatever.com" to /etc/hosts, and also turned Apache
> > >>>>> LogLevel to debug, and now I get a different error:
> > >>>>>
> > >>>>> [root@apachemodule dev]# /apps/httpd2222/bin/apachectl -k start -X
> > >>>>> mod_headers-jl V0.09 - start calling OAM API
> > >>>>> In register_hooks
> > >>>>> In create_headers_dir_config
> > >>>>> In create_headers_dir_config
> > >>>>> In header_cmd
> > >>>>> In header_inout_cmd
> > >>>>> In parse_format_tag
> > >>>>> In parse_misc_string
> > >>>>> In create_headers_dir_config
> > >>>>> In header_cmd
> > >>>>> In header_inout_cmd
> > >>>>> In parse_format_tag
> > >>>>> In parse_misc_string
> > >>>>> In header_post_config
> > >>>>> *** glibc detected *** corrupted double-linked list:
> > >>>>> 0x00000000006b9710 ***
> > >>>>> /apps/httpd2222/bin/apachectl: line 78: 7599
> > >>>>> Aborted $HTTPD $ARGV
> > >>>>> [root@apachemodule dev]#
> > >>>>>
> > >>>>>
> > >>>>> Here's the Apache error_log:
> > >>>>>
> > >>>>> [Thu Jun 21 23:24:11 2012] [info] mod_unique_id: using ip addr
> > >>>>> 72.215.225.9
> > >>>>> [Thu Jun 21 23:24:12 2012] [info] Init: Seeding PRNG with 144 bytes
> > >>>>> of entropy
> > >>>>> [Thu Jun 21 23:24:12 2012] [info] Loading certificate& private
> > >>>>> key of SSL-aware server
> > >>>>> [Thu Jun 21 23:29:51 2012] [info] mod_unique_id: using ip addr
> > >>>>> 72.215.225.9
> > >>>>> [Thu Jun 21 23:29:52 2012] [info] Init: Seeding PRNG with 144 bytes
> > >>>>> of entropy
> > >>>>> [Thu Jun 21 23:29:52 2012] [info] Loading certificate& private
> > >>>>> key of SSL-aware server
> > >>>>> [Thu Jun 21 23:29:56 2012] [info] mod_unique_id: using ip addr
> > >>>>> 72.215.225.9
> > >>>>> [Thu Jun 21 23:29:57 2012] [info] Init: Seeding PRNG with 144 bytes
> > >>>>> of entropy
> > >>>>> [Thu Jun 21 23:29:57 2012] [info] Loading certificate& private
> > >>>>> key of SSL-aware server
> > >>>>> [root@apachemodule dev]#
> > >>>>>
> > >>>>>
> > >>>>> and here's the gdb backtrace:
> > >>>>>
> > >>>>> (gdb) b header_post_config
> > >>>>> Function "header_post_config" not defined.
> > >>>>> Make breakpoint pending on future shared library load? (y or [n]) y
> > >>>>> Breakpoint 1 (header_post_config) pending.
> > >>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> > >>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f
> > >>>>> /apps/httpd2222/conf/httpd.conf
> > >>>>> [Thread debugging using libthread_db enabled]
> > >>>>> [New Thread 182897610272 (LWP 7644)]
> > >>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> > >>>>> Pending breakpoint "header_post_config" resolved
> > >>>>> mod_headers-jl V0.09 - start calling OAM API
> > >>>>> In register_hooks
> > >>>>> In create_headers_dir_config
> > >>>>> In create_headers_dir_config
> > >>>>> In header_cmd
> > >>>>> In header_inout_cmd
> > >>>>> In parse_format_tag
> > >>>>> In parse_misc_string
> > >>>>> In create_headers_dir_config
> > >>>>> In header_cmd
> > >>>>> In header_inout_cmd
> > >>>>> In parse_format_tag
> > >>>>> In parse_misc_string
> > >>>>> [Switching to Thread 182897610272 (LWP 7644)]
> > >>>>>
> > >>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
> > >>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> > >>>>> 1121 printf("In header_post_config\n");
> > >>>>> (gdb) n
> > >>>>> 1120 {
> > >>>>> (gdb) n
> > >>>>> 1121 printf("In header_post_config\n");
> > >>>>> (gdb) n
> > >>>>> In header_post_config
> > >>>>> 1122 header_ssl_lookup =
> > >>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > >>>>> (gdb) n
> > >>>>> 1124 }
> > >>>>> (gdb) n
> > >>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
> > >>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> > >>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> > >>>>> (gdb) n
> > >>>>> *** glibc detected *** corrupted double-linked list:
> > >>>>> 0x00000000006b9710 ***
> > >>>>>
> > >>>>> Program received signal SIGABRT, Aborted.
> > >>>>> 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> > >>>>> (gdb) bt full
> > >>>>> #0 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> > >>>>> No symbol table info available.
> > >>>>> #1 0x000000351432fa6e in abort () from /lib64/tls/libc.so.6
> > >>>>> No symbol table info available.
> > >>>>> #2 0x0000003514363641 in __libc_message () from /lib64/tls/libc.so.6
> > >>>>> No symbol table info available.
> > >>>>> #3 0x0000003514369512 in _int_free () from /lib64/tls/libc.so.6
> > >>>>> No symbol table info available.
> > >>>>> #4 0x0000003514369846 in free () from /lib64/tls/libc.so.6
> > >>>>> No symbol table info available.
> > >>>>> #5 0x0000002a9790b6ba in R_free () from
> > >>>>> /apps/netpoint/lib64/libobaccess.so
> > >>>>> No symbol table info available.
> > >>>>> #6 0x0000002a9792bc41 in X509_CINF_free () from
> > >>>>> /apps/netpoint/lib64/libobaccess.so
> > >>>>> No symbol table info available.
> > >>>>> #7 0x0000002a9790ed98 in X509_free () from
> > >>>>> /apps/netpoint/lib64/libobaccess.so
> > >>>>> No symbol table info available.
> > >>>>> #8 0x0000002a987d97b6 in ssl_pphrase_Handle (s=0x59d3a8,
> > >>>>> p=0x5a72d8) at ssl_engine_pphrase.c:243
> > >>>>> mc = (SSLModConfigRec *) 0x571738
> > >>>>> sc = (SSLSrvConfigRec *) 0x668c38
> > >>>>> pServ = (server_rec *) 0x65fa48
> > >>>>> cpVHostID = 0x60add0 "apache1.whatever.com:443"
> > >>>>> szPath =
> > >>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.crt\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
> > >>>>> '\0'<repeats 15 times>,
> > >>>>> "\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
> > >>>>> '\0'<repeats 35 times>, "à\224k", '\0'<repeats 13 times>,
> > >>>>> "øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
> > >>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
> > >>>>> '\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
> > >>>>> '\0'<repeats 35 times>...
> > >>>>> pPrivateKey = Variable "pPrivateKey" is not available.
> > >>>>> (gdb)
> > >>>>>
> > >>>>>
> > >>>>> Jim
> > >>>>>
> > >>>> I guess the thing that I'm puzzled about is why it goes from
> > >>>> ssl_pphrase_Handle (which is part of Apache code?) to X509_free ()
> > >>>> in the libobaccess.so? Apache shouldn't even be aware of the
> > >>>> libobaccess stuff yet (since I think that Apache is still trying to
> > >>>> initialize SSL processing)?
> > >>>>
> > >>>> Jim
> > >>> It shouldn't unless you've declared a shutdown hook or a clean up hook
> > >>> of sorts. The looks of that stack trace look strikingly like an
> > >>> exception handler firing off. What are all of the hooks you have
> > >>> running?
> > >>>
> > >>> Joe
> > >>> --
> > >>> http://www.silverhawk.net/
> > >>>
> > >>
> > >> Hi,
> > >>
> > >> I'm basically using the original mod_headers.c as a starter code, and
> > >> stuck some stuff in there. Haven't done anything with the hooks etc.
> > >> yet, so whatever was there is there:
> > >>
> > >> static void register_hooks(apr_pool_t *p)
> > >> {
> > >> printf("mod_headers-jl V0.09 - start calling OAM API\n");
> > >> printf("In register_hooks\n");
> > >> ap_register_output_filter("FIXUP_HEADERS_OUT",
> > >> ap_headers_output_filter,
> > >> NULL, AP_FTYPE_CONTENT_SET);
> > >> ap_register_output_filter("FIXUP_HEADERS_ERR",
> > >> ap_headers_error_filter,
> > >> NULL, AP_FTYPE_CONTENT_SET);
> > >> ap_hook_pre_config(header_pre_config,NULL,NULL,APR_HOOK_MIDDLE);
> > >> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
> > >> ap_hook_insert_filter(ap_headers_insert_output_filter, NULL,
> > >> NULL, APR_HOOK_LAST);
> > >> ap_hook_insert_error_filter(ap_headers_insert_error_filter,
> > >> NULL, NULL, APR_HOOK_LAST);
> > >> ap_hook_fixups(ap_headers_fixup, NULL, NULL, APR_HOOK_LAST);
> > >> ap_hook_post_read_request(ap_headers_early, NULL, NULL,
> > >> APR_HOOK_FIRST);
> > >> }
> > >>
> > >> module AP_MODULE_DECLARE_DATA headers_module =
> > >> {
> > >> STANDARD20_MODULE_STUFF,
> > >> create_headers_dir_config, /* dir config creater */
> > >> merge_headers_config, /* dir merger --- default is to
> > >> override */
> > >> NULL, /* server config */
> > >> NULL, /* merge server configs */
> > >> headers_cmds, /* command apr_table_t */
> > >> register_hooks /* register hooks */
> > >> };
> > >>
> > >> I guess that there is a:
> > >>
> > >> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
> > >>
> > >> in that register_hooks(), but it was there from the original code.
> > >>
> > >> BTW, I was checking, and it looks like there's an X509_free() function
> > >> in both openssl and in libobaccess.so (used 'nm --dynamic<file>), so
> > >> I'm wondering if things are confused, and maybe Apache code expects to
> > >> call the openssl X509_free, but somehow ends up calling the
> > >> X509_free() in libobaccess.so?
> > >>
> > >> What would cause something like that? If two .so files have the same
> > >> function, how to avoid conflicts?
> > >>
> > >> Jim
> > >
> > > Jim, you may have hit that on the head. I'm not sure of the load order
> > > and looking up symbols, but it could very well change things. Try
> > > re-ordering or adding a LoadFile before the SSL module and see if that
> > > changes things.
> >
> > Look up order usually is load order. So if mod_ssl loads first and loads
> > its dependency libcrypto, the symbol will always be found there even if
> > the implementation in libobaccess.so is needed - and vice versa.
> >
> > On Solaris there is a -Bdirect linker flag which changes runtime search
> > order so that each module would find a needed symbol first inits direct
> > dependencies. Unfortunately AFAIK notihing similar exists for Linux.
> >
> > You would need to find ssl libs and libobaccess with compatible symbols.
> > Best would be a libobaccess which is dynamically linked against a
> > compatible version of libcrypto instead of - as it seems - being
> > statically linked against it.
> >
> > Regards,
> >
> > Rainer
> >
>
>
> Hi Rainer,
>
> Here's exactly what I found:
>
> [root@apachemodule ~]# ldd /apps/netpoint/lib64/libobaccess.so
> libnsl.so.1 => /lib64/libnsl.so.1 (0x0000002a95ab8000)
> libdl.so.2 => /lib64/libdl.so.2 (0x0000002a95bd0000)
> libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x0000002a95cd3000)
> libstdc++.so.6 => /lib64/libstdc++.so.6 (0x0000002a95de9000)
> libm.so.6 => /lib64/tls/libm.so.6 (0x0000002a95fd9000)
> libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000002a9615f000)
> libc.so.6 => /lib64/tls/libc.so.6 (0x0000002a9626d000)
> /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
> [root@apachemodule ~]#
> [root@apachemodule ~]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "X509_free"
> 00000000002d8d60 T X509_free
>
>
> [root@apachemodule ~]# ldd /lib64/libssl.so.0.9.7a
> libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003518700000)
> libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003519000000)
> libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003518500000)
> libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003518900000)
> libcrypto.so.4 => /lib64/libcrypto.so.4 (0x0000003518d00000)
> libdl.so.2 => /lib64/libdl.so.2 (0x0000003514800000)
> libz.so.1 => /usr/lib64/libz.so.1 (0x0000003514c00000)
> libc.so.6 => /lib64/tls/libc.so.6 (0x0000003514300000)
> libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003518300000)
> /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
> [root@apachemodule ~]#
> [root@apachemodule ~]# nm --dynamic /lib64/libssl.so.0.9.7a | grep "X509_free"
> U X509_free
>
>
> [root@apachemodule ~]# ldd /lib64/libcrypto.so.0.9.7a
> libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003518700000)
> libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003519000000)
> libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003518500000)
> libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003518900000)
> libdl.so.2 => /lib64/libdl.so.2 (0x0000003514800000)
> libz.so.1 => /usr/lib64/libz.so.1 (0x0000003514c00000)
> libc.so.6 => /lib64/tls/libc.so.6 (0x0000003514300000)
> libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003518300000)
> /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
> [root@apachemodule ~]#
> [root@apachemodule ~]# nm --dynamic /lib64/libcrypto.so.0.9.7a | grep "X509_free"
> 0000003518da9860 T X509_free
>
>
> So:
>
> - liboaccess.so doesn't REFERENCE libcrypto, but it HAS an X509_free() function in it.
> - libssl.so.REFERENCES libcrypto, AND it HAS an X509_free function in it.
> - libcrypto also has an X509_free function in it.
>
> From your msg above, are you saying that this situation can't be resolved by trying to change load order?
>
> And the only way is what you said in your last paragraph (find another libobccess.so)?
>
> Jim
>
>
Hi,
Also, BTW, the call to X509_free is apparently because in ssl_engine_pphrase.c, which it seems is part of mod_ssl, it has:
/*
* Free the X509 structure
*/
X509_free(pX509Cert);
but that's calling the X509_free() in libobaccess.so, rather than the X509_free() in libcrypt.so?
Jim
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use
Apache module that uses shared library?
Posted by oh...@cox.net.
---- Rainer Jung <ra...@kippdata.de> wrote:
> On 22.06.2012 06:10, Joe Lewis wrote:
> > On 6/21/12 10:02 PM, ohaya@cox.net wrote:
> >> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >>> On 6/21/12 9:39 PM, ohaya@cox.net wrote:
> >>>> ---- ohaya@cox.net wrote:
> >>>>> ---- ohaya@cox.net wrote:
> >>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >>>>>>> On 6/21/12 7:32 PM, ohaya@cox.net wrote:
> >>>>>>>> ---- ohaya@cox.net wrote:
> >>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >>>>>>>>>> On 6/21/12 6:46 PM, ohaya@cox.net wrote:
> >>>>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >>>>>>>>>>>> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
> >>>>>>>>>>>>> ---- ohaya@cox.net wrote:
> >>>>>>>>>>>>>> ---- Sorin Manolache<so...@gmail.com> wrote:
> >>>>>>>>>>>>>>> And I forgot to say: run gdb in some sort of environment
> >>>>>>>>>>>>>>> where you see
> >>>>>>>>>>>>>>> your current source code line and a couple of surrounding
> >>>>>>>>>>>>>>> lines. You
> >>>>>>>>>>>>>>> could achieve this with the "list" command, but I prefer
> >>>>>>>>>>>>>>> running gdb in
> >>>>>>>>>>>>>>> emacs and let emacs do the nice listing of source code in
> >>>>>>>>>>>>>>> a different panel.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> S
> >>>>>>>>>>>>>> Here's the function from my source. It's the original
> >>>>>>>>>>>>>> from mod_headers.c, plus my printf:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> static int header_post_config(apr_pool_t *pconf,
> >>>>>>>>>>>>>> apr_pool_t *plog,
> >>>>>>>>>>>>>> apr_pool_t *ptemp,
> >>>>>>>>>>>>>> server_rec *s)
> >>>>>>>>>>>>>> {
> >>>>>>>>>>>>>> printf("In header_post_config\n");
> >>>>>>>>>>>>>> header_ssl_lookup =
> >>>>>>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>>>>>>>> return OK;
> >>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Jim
> >>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I was able to get the segfault to go away. Here's what I
> >>>>>>>>>>>>> had to do:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> - Created /etc/ld.so.conf.d/my.conf, and added the
> >>>>>>>>>>>>> directory where my libobaccess.so was
> >>>>>>>>>>>>> - Run 'ldconfig' to activate.
> >>>>>>>>>>>>> - In the apxs command, DON'T include the -L and -l arguments
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> After that, Apache appears to start ok, without segfault :)!!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks for all of the great help, esp. the suggestion about
> >>>>>>>>>>>>> checking "ldconfig -p". I still don't understand why, but
> >>>>>>>>>>>>> I'm just glad that I can get past this piece so now I can
> >>>>>>>>>>>>> debug my module :)...
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Later,
> >>>>>>>>>>>>> Jim
> >>>>>>>>>>>> I'm just glad this list is as good as it is!
> >>>>>>>>>>>>
> >>>>>>>>>>>> FYI, the ldconfig is the dynamic linker control, and those
> >>>>>>>>>>>> /etc/ld.so.conf.d files provide additional search
> >>>>>>>>>>>> directories for the
> >>>>>>>>>>>> linker to check in when loading a library.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Joe
> >>>>>>>>>>>> --
> >>>>>>>>>>>> http://www.silverhawk.net/
> >>>>>>>>>>> Hi,
> >>>>>>>>>>>
> >>>>>>>>>>> Sorry to report, but my earlier report was a "false positive"
> >>>>>>>>>>> :)...
> >>>>>>>>>>>
> >>>>>>>>>>> I forgot that the mod_headers.c that I was doing the earlier
> >>>>>>>>>>> testing with had all references to the libobaccess.so removed
> >>>>>>>>>>> :(!!
> >>>>>>>>>>>
> >>>>>>>>>>> So, I'm still stuck with basically the same problem, now,
> >>>>>>>>>>> working with my "full" code, with the calls in it:
> >>>>>>>>>>>
> >>>>>>>>>>> - If I compile with -L and -l, Apache segfaults when it starts
> >>>>>>>>>>>
> >>>>>>>>>>> - If I compile without -L and -l, then I get "undefined
> >>>>>>>>>>> symbol" errors when I try to start Apache, e.g.:
> >>>>>>>>>>>
> >>>>>>>>>>> [root@apachemodule build-mod_headers]#
> >>>>>>>>>>> /apps/httpd2222/bin/apachectl -k start -X
> >>>>>>>>>>> httpd: Syntax error on line 84 of
> >>>>>>>>>>> /apps/httpd2222/conf/httpd.conf: Cannot load
> >>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so into server:
> >>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so: undefined symbol:
> >>>>>>>>>>> ObResource_isProtected
> >>>>>>>>>>>
> >>>>>>>>>>> That "ObResource_isProtected" should be a symbol in
> >>>>>>>>>>> libobaccess.so, and in fact, if I do "nm --dynamic", I get:
> >>>>>>>>>>>
> >>>>>>>>>>> [root@apachemodule build-mod_headers]# nm --dynamic
> >>>>>>>>>>> /apps/netpoint/lib64/libobaccess.so | grep
> >>>>>>>>>>> "ObResource_isProtected"
> >>>>>>>>>>> 00000000000a6d80 T ObResource_isProtected
> >>>>>>>>>>> [root@apachemodule build-mod_headers]#
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> I'm *assuming* that the reason for the "undefined symbol"
> >>>>>>>>>>> error is that libobaccess.so is actually not being loaded,
> >>>>>>>>>>> but then when I try to load libobaccess.so, either via -L and
> >>>>>>>>>>> -l in the apxs, or using LoadFile in httpd.conf, I get the
> >>>>>>>>>>> segfault (same gdb info, BTW).
> >>>>>>>>>>>
> >>>>>>>>>>> Catch-22?
> >>>>>>>>>>>
> >>>>>>>>>>> Sorry for the false alarm :(!!
> >>>>>>>>>>>
> >>>>>>>>>>> Jim
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> Not a catch-22. The -L and -l specify linker options when
> >>>>>>>>>> assembling
> >>>>>>>>>> the code. The ldconfig is a run-time thing. If you are
> >>>>>>>>>> getting the
> >>>>>>>>>> stderr messages, you are making it all the way into your
> >>>>>>>>>> library. I'd
> >>>>>>>>>> suggest commenting out the following line and see if you get
> >>>>>>>>>> farther :
> >>>>>>>>>>
> >>>>>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>>>>
> >>>>>>>>>> That should tell you if the problem is the ssl_var_lookup.
> >>>>>>>>>>
> >>>>>>>>>> Joe
> >>>>>>>>>> --
> >>>>>>>>>> http://www.silverhawk.net/
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> Thanks for the suggestion. I just tried what you suggested,
> >>>>>>>>> and got a segfault when I started Apache with the modified module.
> >>>>>>>>>
> >>>>>>>>> Jim
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> As a reminder, here's the gdb with the library loaded:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> (gdb) b header_post_config
> >>>>>>>> Function "header_post_config" not defined.
> >>>>>>>> Make breakpoint pending on future shared library load? (y or [n]) y
> >>>>>>>> Breakpoint 1 (header_post_config) pending.
> >>>>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> >>>>>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/
> >>>>>>>> -f /apps/httpd2222/conf/httpd.conf
> >>>>>>>> [Thread debugging using libthread_db enabled]
> >>>>>>>> [New Thread 182897610272 (LWP 11317)]
> >>>>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> >>>>>>>> Pending breakpoint "header_post_config" resolved
> >>>>>>>> mod_headers-jl V0.09 - start calling OAM API
> >>>>>>>> In register_hooks
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In header_cmd
> >>>>>>>> In header_inout_cmd
> >>>>>>>> In parse_format_tag
> >>>>>>>> In parse_misc_string
> >>>>>>>> In create_headers_dir_config
> >>>>>>>> In header_cmd
> >>>>>>>> In header_inout_cmd
> >>>>>>>> In parse_format_tag
> >>>>>>>> In parse_misc_string
> >>>>>>>> [Switching to Thread 182897610272 (LWP 11317)]
> >>>>>>>>
> >>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
> >>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> >>>>>>>> 1121 printf("In header_post_config\n");
> >>>>>>>> (gdb) n
> >>>>>>>> 1120 {
> >>>>>>>> (gdb) n
> >>>>>>>> 1121 printf("In header_post_config\n");
> >>>>>>>> (gdb) n
> >>>>>>>> In header_post_config
> >>>>>>>> 1122 header_ssl_lookup =
> >>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>> (gdb) n
> >>>>>>>> 1124 }
> >>>>>>>> (gdb) n
> >>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
> >>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> >>>>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >>>>>>>> (gdb) n
> >>>>>>>>
> >>>>>>>> Program received signal SIGSEGV, Segmentation fault.
> >>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> So, it's actually blowing up in "BN_num_bits()" in
> >>>>>>>> /lib64/libcrypto.so.4?
> >>>>>>>>
> >>>>>>>> Jim
> >>>>>>> When you see the :
> >>>>>>>
> >>>>>>> Program received signal SIGSEGV, Segmentation fault.
> >>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>>>>
> >>>>>>>
> >>>>>>> in gdb, type "bt" and hit enter. It should show the back trace
> >>>>>>> and how
> >>>>>>> you got to the BN_num_bits() function.
> >>>>>>>
> >>>>>>> Joe
> >>>>>>> --
> >>>>>>> http://www.silverhawk.net/
> >>>>>> Hi,
> >>>>>>
> >>>>>> Here's the bt full:
> >>>>>>
> >>>>>>
> >>>>>> In header_inout_cmd
> >>>>>> In parse_format_tag
> >>>>>> In parse_misc_string
> >>>>>> [Switching to Thread 182897610272 (LWP 6676)]
> >>>>>>
> >>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
> >>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> >>>>>> 1121 printf("In header_post_config\n");
> >>>>>> (gdb) n
> >>>>>> 1120 {
> >>>>>> (gdb) n
> >>>>>> 1121 printf("In header_post_config\n");
> >>>>>> (gdb) n
> >>>>>> In header_post_config
> >>>>>> 1122 header_ssl_lookup =
> >>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>> (gdb) n
> >>>>>> 1124 }
> >>>>>> (gdb) n
> >>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
> >>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> >>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >>>>>> (gdb) n
> >>>>>>
> >>>>>> Program received signal SIGSEGV, Segmentation fault.
> >>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>>> (gdb) bt full
> >>>>>> #0 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>>> No symbol table info available.
> >>>>>> #1 0x0000003518da8f4e in X509_ATTRIBUTE_create () from
> >>>>>> /lib64/libcrypto.so.4
> >>>>>> No symbol table info available.
> >>>>>> #2 0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> >>>>>> No symbol table info available.
> >>>>>> #3 0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> >>>>>> No symbol table info available.
> >>>>>> #4 0x0000003518dae0e1 in ASN1_item_ex_i2d () from
> >>>>>> /lib64/libcrypto.so.4
> >>>>>> No symbol table info available.
> >>>>>> #5 0x0000003518dae5f2 in ASN1_template_i2d () from
> >>>>>> /lib64/libcrypto.so.4
> >>>>>> No symbol table info available.
> >>>>>> #6 0x0000003518dae28e in ASN1_item_ex_i2d () from
> >>>>>> /lib64/libcrypto.so.4
> >>>>>> No symbol table info available.
> >>>>>> #7 0x0000003518dae3c6 in ASN1_item_i2d () from /lib64/libcrypto.so.4
> >>>>>> No symbol table info available.
> >>>>>> #8 0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8,
> >>>>>> p=0x5a72d8) at ssl_engine_pphrase.c:505
> >>>>>> mc = (SSLModConfigRec *) 0x571738
> >>>>>> sc = (SSLSrvConfigRec *) 0x668c38
> >>>>>> pServ = (server_rec *) 0x65fa48
> >>>>>> cpVHostID = 0x60add0 "www.example.com:443"
> >>>>>> szPath =
> >>>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
> >>>>>> '\0'<rep eats 15
> >>>>>> times>,
> >>>>>> "\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
> >>>>>> '\0'<repeats 35 times>, "à\224k",
> >>>>>> ' ---Type<return>
> >>>>>> to continue, or q<return> to quit---
> >>>>>> \0'<repeats 13 times>,
> >>>>>> "øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
> >>>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
> >>>>>> '\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
> >>>>>> '\0'<repeats 35 times>...
> >>>>>> pPrivateKey = (EVP_PKEY *) 0x6ba670
> >>>>>> asn1 = Variable "asn1" is not available.
> >>>>>> (gdb)
> >>>>>>
> >>>>>>
> >>>>>> Not really sure what to make of that though :(...
> >>>>>>
> >>>>>> Jim
> >>>>> Hi,
> >>>>>
> >>>>> Ok, something interesting. I noticed that the servername in the
> >>>>> gdb output was example.whatever.com, so I changed ServerName in
> >>>>> extras/httpd-ssl.conf to "apache1.whatever.com". Apache still
> >>>>> segfaulted after that.
> >>>>>
> >>>>> I noticed that igdb was showing my router IP address for
> >>>>> "mod_unique_id" (whatever that is), so I thought it was because I
> >>>>> didn't have the hostname in /etc/hosts. So, I added
> >>>>> "apache1.whatever.com" to /etc/hosts, and also turned Apache
> >>>>> LogLevel to debug, and now I get a different error:
> >>>>>
> >>>>> [root@apachemodule dev]# /apps/httpd2222/bin/apachectl -k start -X
> >>>>> mod_headers-jl V0.09 - start calling OAM API
> >>>>> In register_hooks
> >>>>> In create_headers_dir_config
> >>>>> In create_headers_dir_config
> >>>>> In header_cmd
> >>>>> In header_inout_cmd
> >>>>> In parse_format_tag
> >>>>> In parse_misc_string
> >>>>> In create_headers_dir_config
> >>>>> In header_cmd
> >>>>> In header_inout_cmd
> >>>>> In parse_format_tag
> >>>>> In parse_misc_string
> >>>>> In header_post_config
> >>>>> *** glibc detected *** corrupted double-linked list:
> >>>>> 0x00000000006b9710 ***
> >>>>> /apps/httpd2222/bin/apachectl: line 78: 7599
> >>>>> Aborted $HTTPD $ARGV
> >>>>> [root@apachemodule dev]#
> >>>>>
> >>>>>
> >>>>> Here's the Apache error_log:
> >>>>>
> >>>>> [Thu Jun 21 23:24:11 2012] [info] mod_unique_id: using ip addr
> >>>>> 72.215.225.9
> >>>>> [Thu Jun 21 23:24:12 2012] [info] Init: Seeding PRNG with 144 bytes
> >>>>> of entropy
> >>>>> [Thu Jun 21 23:24:12 2012] [info] Loading certificate& private
> >>>>> key of SSL-aware server
> >>>>> [Thu Jun 21 23:29:51 2012] [info] mod_unique_id: using ip addr
> >>>>> 72.215.225.9
> >>>>> [Thu Jun 21 23:29:52 2012] [info] Init: Seeding PRNG with 144 bytes
> >>>>> of entropy
> >>>>> [Thu Jun 21 23:29:52 2012] [info] Loading certificate& private
> >>>>> key of SSL-aware server
> >>>>> [Thu Jun 21 23:29:56 2012] [info] mod_unique_id: using ip addr
> >>>>> 72.215.225.9
> >>>>> [Thu Jun 21 23:29:57 2012] [info] Init: Seeding PRNG with 144 bytes
> >>>>> of entropy
> >>>>> [Thu Jun 21 23:29:57 2012] [info] Loading certificate& private
> >>>>> key of SSL-aware server
> >>>>> [root@apachemodule dev]#
> >>>>>
> >>>>>
> >>>>> and here's the gdb backtrace:
> >>>>>
> >>>>> (gdb) b header_post_config
> >>>>> Function "header_post_config" not defined.
> >>>>> Make breakpoint pending on future shared library load? (y or [n]) y
> >>>>> Breakpoint 1 (header_post_config) pending.
> >>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> >>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f
> >>>>> /apps/httpd2222/conf/httpd.conf
> >>>>> [Thread debugging using libthread_db enabled]
> >>>>> [New Thread 182897610272 (LWP 7644)]
> >>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> >>>>> Pending breakpoint "header_post_config" resolved
> >>>>> mod_headers-jl V0.09 - start calling OAM API
> >>>>> In register_hooks
> >>>>> In create_headers_dir_config
> >>>>> In create_headers_dir_config
> >>>>> In header_cmd
> >>>>> In header_inout_cmd
> >>>>> In parse_format_tag
> >>>>> In parse_misc_string
> >>>>> In create_headers_dir_config
> >>>>> In header_cmd
> >>>>> In header_inout_cmd
> >>>>> In parse_format_tag
> >>>>> In parse_misc_string
> >>>>> [Switching to Thread 182897610272 (LWP 7644)]
> >>>>>
> >>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
> >>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> >>>>> 1121 printf("In header_post_config\n");
> >>>>> (gdb) n
> >>>>> 1120 {
> >>>>> (gdb) n
> >>>>> 1121 printf("In header_post_config\n");
> >>>>> (gdb) n
> >>>>> In header_post_config
> >>>>> 1122 header_ssl_lookup =
> >>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>> (gdb) n
> >>>>> 1124 }
> >>>>> (gdb) n
> >>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
> >>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> >>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >>>>> (gdb) n
> >>>>> *** glibc detected *** corrupted double-linked list:
> >>>>> 0x00000000006b9710 ***
> >>>>>
> >>>>> Program received signal SIGABRT, Aborted.
> >>>>> 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> >>>>> (gdb) bt full
> >>>>> #0 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> >>>>> No symbol table info available.
> >>>>> #1 0x000000351432fa6e in abort () from /lib64/tls/libc.so.6
> >>>>> No symbol table info available.
> >>>>> #2 0x0000003514363641 in __libc_message () from /lib64/tls/libc.so.6
> >>>>> No symbol table info available.
> >>>>> #3 0x0000003514369512 in _int_free () from /lib64/tls/libc.so.6
> >>>>> No symbol table info available.
> >>>>> #4 0x0000003514369846 in free () from /lib64/tls/libc.so.6
> >>>>> No symbol table info available.
> >>>>> #5 0x0000002a9790b6ba in R_free () from
> >>>>> /apps/netpoint/lib64/libobaccess.so
> >>>>> No symbol table info available.
> >>>>> #6 0x0000002a9792bc41 in X509_CINF_free () from
> >>>>> /apps/netpoint/lib64/libobaccess.so
> >>>>> No symbol table info available.
> >>>>> #7 0x0000002a9790ed98 in X509_free () from
> >>>>> /apps/netpoint/lib64/libobaccess.so
> >>>>> No symbol table info available.
> >>>>> #8 0x0000002a987d97b6 in ssl_pphrase_Handle (s=0x59d3a8,
> >>>>> p=0x5a72d8) at ssl_engine_pphrase.c:243
> >>>>> mc = (SSLModConfigRec *) 0x571738
> >>>>> sc = (SSLSrvConfigRec *) 0x668c38
> >>>>> pServ = (server_rec *) 0x65fa48
> >>>>> cpVHostID = 0x60add0 "apache1.whatever.com:443"
> >>>>> szPath =
> >>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.crt\000\177\000\000\000Ü׿\177\000\000\000à׿\177",
> >>>>> '\0'<repeats 15 times>,
> >>>>> "\001\000\000\000°å¿\177\000\000\000;\000\000\000+\000\000\000\020Û¿\177",
> >>>>> '\0'<repeats 35 times>, "à\224k", '\0'<repeats 13 times>,
> >>>>> "øé¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
> >>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°å¿\177",
> >>>>> '\0'<repeats 11 times>, "\020Û¿\177\000\000\000+\000\000\000;",
> >>>>> '\0'<repeats 35 times>...
> >>>>> pPrivateKey = Variable "pPrivateKey" is not available.
> >>>>> (gdb)
> >>>>>
> >>>>>
> >>>>> Jim
> >>>>>
> >>>> I guess the thing that I'm puzzled about is why it goes from
> >>>> ssl_pphrase_Handle (which is part of Apache code?) to X509_free ()
> >>>> in the libobaccess.so? Apache shouldn't even be aware of the
> >>>> libobaccess stuff yet (since I think that Apache is still trying to
> >>>> initialize SSL processing)?
> >>>>
> >>>> Jim
> >>> It shouldn't unless you've declared a shutdown hook or a clean up hook
> >>> of sorts. The looks of that stack trace look strikingly like an
> >>> exception handler firing off. What are all of the hooks you have
> >>> running?
> >>>
> >>> Joe
> >>> --
> >>> http://www.silverhawk.net/
> >>>
> >>
> >> Hi,
> >>
> >> I'm basically using the original mod_headers.c as a starter code, and
> >> stuck some stuff in there. Haven't done anything with the hooks etc.
> >> yet, so whatever was there is there:
> >>
> >> static void register_hooks(apr_pool_t *p)
> >> {
> >> printf("mod_headers-jl V0.09 - start calling OAM API\n");
> >> printf("In register_hooks\n");
> >> ap_register_output_filter("FIXUP_HEADERS_OUT",
> >> ap_headers_output_filter,
> >> NULL, AP_FTYPE_CONTENT_SET);
> >> ap_register_output_filter("FIXUP_HEADERS_ERR",
> >> ap_headers_error_filter,
> >> NULL, AP_FTYPE_CONTENT_SET);
> >> ap_hook_pre_config(header_pre_config,NULL,NULL,APR_HOOK_MIDDLE);
> >> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
> >> ap_hook_insert_filter(ap_headers_insert_output_filter, NULL,
> >> NULL, APR_HOOK_LAST);
> >> ap_hook_insert_error_filter(ap_headers_insert_error_filter,
> >> NULL, NULL, APR_HOOK_LAST);
> >> ap_hook_fixups(ap_headers_fixup, NULL, NULL, APR_HOOK_LAST);
> >> ap_hook_post_read_request(ap_headers_early, NULL, NULL,
> >> APR_HOOK_FIRST);
> >> }
> >>
> >> module AP_MODULE_DECLARE_DATA headers_module =
> >> {
> >> STANDARD20_MODULE_STUFF,
> >> create_headers_dir_config, /* dir config creater */
> >> merge_headers_config, /* dir merger --- default is to
> >> override */
> >> NULL, /* server config */
> >> NULL, /* merge server configs */
> >> headers_cmds, /* command apr_table_t */
> >> register_hooks /* register hooks */
> >> };
> >>
> >> I guess that there is a:
> >>
> >> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
> >>
> >> in that register_hooks(), but it was there from the original code.
> >>
> >> BTW, I was checking, and it looks like there's an X509_free() function
> >> in both openssl and in libobaccess.so (used 'nm --dynamic<file>), so
> >> I'm wondering if things are confused, and maybe Apache code expects to
> >> call the openssl X509_free, but somehow ends up calling the
> >> X509_free() in libobaccess.so?
> >>
> >> What would cause something like that? If two .so files have the same
> >> function, how to avoid conflicts?
> >>
> >> Jim
> >
> > Jim, you may have hit that on the head. I'm not sure of the load order
> > and looking up symbols, but it could very well change things. Try
> > re-ordering or adding a LoadFile before the SSL module and see if that
> > changes things.
>
> Look up order usually is load order. So if mod_ssl loads first and loads
> its dependency libcrypto, the symbol will always be found there even if
> the implementation in libobaccess.so is needed - and vice versa.
>
> On Solaris there is a -Bdirect linker flag which changes runtime search
> order so that each module would find a needed symbol first inits direct
> dependencies. Unfortunately AFAIK notihing similar exists for Linux.
>
> You would need to find ssl libs and libobaccess with compatible symbols.
> Best would be a libobaccess which is dynamically linked against a
> compatible version of libcrypto instead of - as it seems - being
> statically linked against it.
>
> Regards,
>
> Rainer
>
Hi Rainer,
Here's exactly what I found:
[root@apachemodule ~]# ldd /apps/netpoint/lib64/libobaccess.so
libnsl.so.1 => /lib64/libnsl.so.1 (0x0000002a95ab8000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000002a95bd0000)
libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x0000002a95cd3000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x0000002a95de9000)
libm.so.6 => /lib64/tls/libm.so.6 (0x0000002a95fd9000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000002a9615f000)
libc.so.6 => /lib64/tls/libc.so.6 (0x0000002a9626d000)
/lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
[root@apachemodule ~]#
[root@apachemodule ~]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "X509_free"
00000000002d8d60 T X509_free
[root@apachemodule ~]# ldd /lib64/libssl.so.0.9.7a
libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003518700000)
libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003519000000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003518500000)
libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003518900000)
libcrypto.so.4 => /lib64/libcrypto.so.4 (0x0000003518d00000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003514800000)
libz.so.1 => /usr/lib64/libz.so.1 (0x0000003514c00000)
libc.so.6 => /lib64/tls/libc.so.6 (0x0000003514300000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003518300000)
/lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
[root@apachemodule ~]#
[root@apachemodule ~]# nm --dynamic /lib64/libssl.so.0.9.7a | grep "X509_free"
U X509_free
[root@apachemodule ~]# ldd /lib64/libcrypto.so.0.9.7a
libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x0000003518700000)
libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x0000003519000000)
libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000003518500000)
libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x0000003518900000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003514800000)
libz.so.1 => /usr/lib64/libz.so.1 (0x0000003514c00000)
libc.so.6 => /lib64/tls/libc.so.6 (0x0000003514300000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x0000003518300000)
/lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)
[root@apachemodule ~]#
[root@apachemodule ~]# nm --dynamic /lib64/libcrypto.so.0.9.7a | grep "X509_free"
0000003518da9860 T X509_free
So:
- liboaccess.so doesn't REFERENCE libcrypto, but it HAS an X509_free() function in it.
- libssl.so.REFERENCES libcrypto, AND it HAS an X509_free function in it.
- libcrypto also has an X509_free function in it.
>From your msg above, are you saying that this situation can't be resolved by trying to change load order?
And the only way is what you said in your last paragraph (find another libobccess.so)?
Jim
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use Apache
module that uses shared library?
Posted by Rainer Jung <ra...@kippdata.de>.
On 22.06.2012 06:10, Joe Lewis wrote:
> On 6/21/12 10:02 PM, ohaya@cox.net wrote:
>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>> On 6/21/12 9:39 PM, ohaya@cox.net wrote:
>>>> ---- ohaya@cox.net wrote:
>>>>> ---- ohaya@cox.net wrote:
>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>>>>> On 6/21/12 7:32 PM, ohaya@cox.net wrote:
>>>>>>>> ---- ohaya@cox.net wrote:
>>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>>>>>>>> On 6/21/12 6:46 PM, ohaya@cox.net wrote:
>>>>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>>>>>>>>>> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
>>>>>>>>>>>>> ---- ohaya@cox.net wrote:
>>>>>>>>>>>>>> ---- Sorin Manolache<so...@gmail.com> wrote:
>>>>>>>>>>>>>>> And I forgot to say: run gdb in some sort of environment
>>>>>>>>>>>>>>> where you see
>>>>>>>>>>>>>>> your current source code line and a couple of surrounding
>>>>>>>>>>>>>>> lines. You
>>>>>>>>>>>>>>> could achieve this with the "list" command, but I prefer
>>>>>>>>>>>>>>> running gdb in
>>>>>>>>>>>>>>> emacs and let emacs do the nice listing of source code in
>>>>>>>>>>>>>>> a different panel.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> S
>>>>>>>>>>>>>> Here's the function from my source. It's the original
>>>>>>>>>>>>>> from mod_headers.c, plus my printf:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> static int header_post_config(apr_pool_t *pconf,
>>>>>>>>>>>>>> apr_pool_t *plog,
>>>>>>>>>>>>>> apr_pool_t *ptemp,
>>>>>>>>>>>>>> server_rec *s)
>>>>>>>>>>>>>> {
>>>>>>>>>>>>>> printf("In header_post_config\n");
>>>>>>>>>>>>>> header_ssl_lookup =
>>>>>>>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>>>>>>>> return OK;
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Jim
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I was able to get the segfault to go away. Here's what I
>>>>>>>>>>>>> had to do:
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Created /etc/ld.so.conf.d/my.conf, and added the
>>>>>>>>>>>>> directory where my libobaccess.so was
>>>>>>>>>>>>> - Run 'ldconfig' to activate.
>>>>>>>>>>>>> - In the apxs command, DON'T include the -L and -l arguments
>>>>>>>>>>>>>
>>>>>>>>>>>>> After that, Apache appears to start ok, without segfault :)!!
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for all of the great help, esp. the suggestion about
>>>>>>>>>>>>> checking "ldconfig -p". I still don't understand why, but
>>>>>>>>>>>>> I'm just glad that I can get past this piece so now I can
>>>>>>>>>>>>> debug my module :)...
>>>>>>>>>>>>>
>>>>>>>>>>>>> Later,
>>>>>>>>>>>>> Jim
>>>>>>>>>>>> I'm just glad this list is as good as it is!
>>>>>>>>>>>>
>>>>>>>>>>>> FYI, the ldconfig is the dynamic linker control, and those
>>>>>>>>>>>> /etc/ld.so.conf.d files provide additional search
>>>>>>>>>>>> directories for the
>>>>>>>>>>>> linker to check in when loading a library.
>>>>>>>>>>>>
>>>>>>>>>>>> Joe
>>>>>>>>>>>> --
>>>>>>>>>>>> http://www.silverhawk.net/
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Sorry to report, but my earlier report was a "false positive"
>>>>>>>>>>> :)...
>>>>>>>>>>>
>>>>>>>>>>> I forgot that the mod_headers.c that I was doing the earlier
>>>>>>>>>>> testing with had all references to the libobaccess.so removed
>>>>>>>>>>> :(!!
>>>>>>>>>>>
>>>>>>>>>>> So, I'm still stuck with basically the same problem, now,
>>>>>>>>>>> working with my "full" code, with the calls in it:
>>>>>>>>>>>
>>>>>>>>>>> - If I compile with -L and -l, Apache segfaults when it starts
>>>>>>>>>>>
>>>>>>>>>>> - If I compile without -L and -l, then I get "undefined
>>>>>>>>>>> symbol" errors when I try to start Apache, e.g.:
>>>>>>>>>>>
>>>>>>>>>>> [root@apachemodule build-mod_headers]#
>>>>>>>>>>> /apps/httpd2222/bin/apachectl -k start -X
>>>>>>>>>>> httpd: Syntax error on line 84 of
>>>>>>>>>>> /apps/httpd2222/conf/httpd.conf: Cannot load
>>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so into server:
>>>>>>>>>>> /apps/httpd2222/modules/mod_headers.so: undefined symbol:
>>>>>>>>>>> ObResource_isProtected
>>>>>>>>>>>
>>>>>>>>>>> That "ObResource_isProtected" should be a symbol in
>>>>>>>>>>> libobaccess.so, and in fact, if I do "nm --dynamic", I get:
>>>>>>>>>>>
>>>>>>>>>>> [root@apachemodule build-mod_headers]# nm --dynamic
>>>>>>>>>>> /apps/netpoint/lib64/libobaccess.so | grep
>>>>>>>>>>> "ObResource_isProtected"
>>>>>>>>>>> 00000000000a6d80 T ObResource_isProtected
>>>>>>>>>>> [root@apachemodule build-mod_headers]#
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I'm *assuming* that the reason for the "undefined symbol"
>>>>>>>>>>> error is that libobaccess.so is actually not being loaded,
>>>>>>>>>>> but then when I try to load libobaccess.so, either via -L and
>>>>>>>>>>> -l in the apxs, or using LoadFile in httpd.conf, I get the
>>>>>>>>>>> segfault (same gdb info, BTW).
>>>>>>>>>>>
>>>>>>>>>>> Catch-22?
>>>>>>>>>>>
>>>>>>>>>>> Sorry for the false alarm :(!!
>>>>>>>>>>>
>>>>>>>>>>> Jim
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> Not a catch-22. The -L and -l specify linker options when
>>>>>>>>>> assembling
>>>>>>>>>> the code. The ldconfig is a run-time thing. If you are
>>>>>>>>>> getting the
>>>>>>>>>> stderr messages, you are making it all the way into your
>>>>>>>>>> library. I'd
>>>>>>>>>> suggest commenting out the following line and see if you get
>>>>>>>>>> farther :
>>>>>>>>>>
>>>>>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>>>>
>>>>>>>>>> That should tell you if the problem is the ssl_var_lookup.
>>>>>>>>>>
>>>>>>>>>> Joe
>>>>>>>>>> --
>>>>>>>>>> http://www.silverhawk.net/
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Thanks for the suggestion. I just tried what you suggested,
>>>>>>>>> and got a segfault when I started Apache with the modified module.
>>>>>>>>>
>>>>>>>>> Jim
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> As a reminder, here's the gdb with the library loaded:
>>>>>>>>
>>>>>>>>
>>>>>>>> (gdb) b header_post_config
>>>>>>>> Function "header_post_config" not defined.
>>>>>>>> Make breakpoint pending on future shared library load? (y or [n]) y
>>>>>>>> Breakpoint 1 (header_post_config) pending.
>>>>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
>>>>>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/
>>>>>>>> -f /apps/httpd2222/conf/httpd.conf
>>>>>>>> [Thread debugging using libthread_db enabled]
>>>>>>>> [New Thread 182897610272 (LWP 11317)]
>>>>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
>>>>>>>> Pending breakpoint "header_post_config" resolved
>>>>>>>> mod_headers-jl V0.09 - start calling OAM API
>>>>>>>> In register_hooks
>>>>>>>> In create_headers_dir_config
>>>>>>>> In create_headers_dir_config
>>>>>>>> In header_cmd
>>>>>>>> In header_inout_cmd
>>>>>>>> In parse_format_tag
>>>>>>>> In parse_misc_string
>>>>>>>> In create_headers_dir_config
>>>>>>>> In header_cmd
>>>>>>>> In header_inout_cmd
>>>>>>>> In parse_format_tag
>>>>>>>> In parse_misc_string
>>>>>>>> [Switching to Thread 182897610272 (LWP 11317)]
>>>>>>>>
>>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
>>>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>>>>>>>> 1121 printf("In header_post_config\n");
>>>>>>>> (gdb) n
>>>>>>>> 1120 {
>>>>>>>> (gdb) n
>>>>>>>> 1121 printf("In header_post_config\n");
>>>>>>>> (gdb) n
>>>>>>>> In header_post_config
>>>>>>>> 1122 header_ssl_lookup =
>>>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>> (gdb) n
>>>>>>>> 1124 }
>>>>>>>> (gdb) n
>>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
>>>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
>>>>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>>>>>>>> (gdb) n
>>>>>>>>
>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>>>>
>>>>>>>>
>>>>>>>> So, it's actually blowing up in "BN_num_bits()" in
>>>>>>>> /lib64/libcrypto.so.4?
>>>>>>>>
>>>>>>>> Jim
>>>>>>> When you see the :
>>>>>>>
>>>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>>>
>>>>>>>
>>>>>>> in gdb, type "bt" and hit enter. It should show the back trace
>>>>>>> and how
>>>>>>> you got to the BN_num_bits() function.
>>>>>>>
>>>>>>> Joe
>>>>>>> --
>>>>>>> http://www.silverhawk.net/
>>>>>> Hi,
>>>>>>
>>>>>> Here's the bt full:
>>>>>>
>>>>>>
>>>>>> In header_inout_cmd
>>>>>> In parse_format_tag
>>>>>> In parse_misc_string
>>>>>> [Switching to Thread 182897610272 (LWP 6676)]
>>>>>>
>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
>>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>>>>>> 1121 printf("In header_post_config\n");
>>>>>> (gdb) n
>>>>>> 1120 {
>>>>>> (gdb) n
>>>>>> 1121 printf("In header_post_config\n");
>>>>>> (gdb) n
>>>>>> In header_post_config
>>>>>> 1122 header_ssl_lookup =
>>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>> (gdb) n
>>>>>> 1124 }
>>>>>> (gdb) n
>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
>>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
>>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>>>>>> (gdb) n
>>>>>>
>>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>> (gdb) bt full
>>>>>> #0 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>> No symbol table info available.
>>>>>> #1 0x0000003518da8f4e in X509_ATTRIBUTE_create () from
>>>>>> /lib64/libcrypto.so.4
>>>>>> No symbol table info available.
>>>>>> #2 0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
>>>>>> No symbol table info available.
>>>>>> #3 0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
>>>>>> No symbol table info available.
>>>>>> #4 0x0000003518dae0e1 in ASN1_item_ex_i2d () from
>>>>>> /lib64/libcrypto.so.4
>>>>>> No symbol table info available.
>>>>>> #5 0x0000003518dae5f2 in ASN1_template_i2d () from
>>>>>> /lib64/libcrypto.so.4
>>>>>> No symbol table info available.
>>>>>> #6 0x0000003518dae28e in ASN1_item_ex_i2d () from
>>>>>> /lib64/libcrypto.so.4
>>>>>> No symbol table info available.
>>>>>> #7 0x0000003518dae3c6 in ASN1_item_i2d () from /lib64/libcrypto.so.4
>>>>>> No symbol table info available.
>>>>>> #8 0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8,
>>>>>> p=0x5a72d8) at ssl_engine_pphrase.c:505
>>>>>> mc = (SSLModConfigRec *) 0x571738
>>>>>> sc = (SSLSrvConfigRec *) 0x668c38
>>>>>> pServ = (server_rec *) 0x65fa48
>>>>>> cpVHostID = 0x60add0 "www.example.com:443"
>>>>>> szPath =
>>>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü×ÿ¿\177\000\000\000à×ÿ¿\177",
>>>>>> '\0'<rep eats 15
>>>>>> times>,
>>>>>> "\001\000\000\000°åÿ¿\177\000\000\000;\000\000\000+\000\000\000\020Ûÿ¿\177",
>>>>>> '\0'<repeats 35 times>, "à\224k",
>>>>>> ' ---Type<return>
>>>>>> to continue, or q<return> to quit---
>>>>>> \0'<repeats 13 times>,
>>>>>> "øéÿ¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
>>>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°åÿ¿\177",
>>>>>> '\0'<repeats 11 times>, "\020Ûÿ¿\177\000\000\000+\000\000\000;",
>>>>>> '\0'<repeats 35 times>...
>>>>>> pPrivateKey = (EVP_PKEY *) 0x6ba670
>>>>>> asn1 = Variable "asn1" is not available.
>>>>>> (gdb)
>>>>>>
>>>>>>
>>>>>> Not really sure what to make of that though :(...
>>>>>>
>>>>>> Jim
>>>>> Hi,
>>>>>
>>>>> Ok, something interesting. I noticed that the servername in the
>>>>> gdb output was example.whatever.com, so I changed ServerName in
>>>>> extras/httpd-ssl.conf to "apache1.whatever.com". Apache still
>>>>> segfaulted after that.
>>>>>
>>>>> I noticed that igdb was showing my router IP address for
>>>>> "mod_unique_id" (whatever that is), so I thought it was because I
>>>>> didn't have the hostname in /etc/hosts. So, I added
>>>>> "apache1.whatever.com" to /etc/hosts, and also turned Apache
>>>>> LogLevel to debug, and now I get a different error:
>>>>>
>>>>> [root@apachemodule dev]# /apps/httpd2222/bin/apachectl -k start -X
>>>>> mod_headers-jl V0.09 - start calling OAM API
>>>>> In register_hooks
>>>>> In create_headers_dir_config
>>>>> In create_headers_dir_config
>>>>> In header_cmd
>>>>> In header_inout_cmd
>>>>> In parse_format_tag
>>>>> In parse_misc_string
>>>>> In create_headers_dir_config
>>>>> In header_cmd
>>>>> In header_inout_cmd
>>>>> In parse_format_tag
>>>>> In parse_misc_string
>>>>> In header_post_config
>>>>> *** glibc detected *** corrupted double-linked list:
>>>>> 0x00000000006b9710 ***
>>>>> /apps/httpd2222/bin/apachectl: line 78: 7599
>>>>> Aborted $HTTPD $ARGV
>>>>> [root@apachemodule dev]#
>>>>>
>>>>>
>>>>> Here's the Apache error_log:
>>>>>
>>>>> [Thu Jun 21 23:24:11 2012] [info] mod_unique_id: using ip addr
>>>>> 72.215.225.9
>>>>> [Thu Jun 21 23:24:12 2012] [info] Init: Seeding PRNG with 144 bytes
>>>>> of entropy
>>>>> [Thu Jun 21 23:24:12 2012] [info] Loading certificate& private
>>>>> key of SSL-aware server
>>>>> [Thu Jun 21 23:29:51 2012] [info] mod_unique_id: using ip addr
>>>>> 72.215.225.9
>>>>> [Thu Jun 21 23:29:52 2012] [info] Init: Seeding PRNG with 144 bytes
>>>>> of entropy
>>>>> [Thu Jun 21 23:29:52 2012] [info] Loading certificate& private
>>>>> key of SSL-aware server
>>>>> [Thu Jun 21 23:29:56 2012] [info] mod_unique_id: using ip addr
>>>>> 72.215.225.9
>>>>> [Thu Jun 21 23:29:57 2012] [info] Init: Seeding PRNG with 144 bytes
>>>>> of entropy
>>>>> [Thu Jun 21 23:29:57 2012] [info] Loading certificate& private
>>>>> key of SSL-aware server
>>>>> [root@apachemodule dev]#
>>>>>
>>>>>
>>>>> and here's the gdb backtrace:
>>>>>
>>>>> (gdb) b header_post_config
>>>>> Function "header_post_config" not defined.
>>>>> Make breakpoint pending on future shared library load? (y or [n]) y
>>>>> Breakpoint 1 (header_post_config) pending.
>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
>>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f
>>>>> /apps/httpd2222/conf/httpd.conf
>>>>> [Thread debugging using libthread_db enabled]
>>>>> [New Thread 182897610272 (LWP 7644)]
>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
>>>>> Pending breakpoint "header_post_config" resolved
>>>>> mod_headers-jl V0.09 - start calling OAM API
>>>>> In register_hooks
>>>>> In create_headers_dir_config
>>>>> In create_headers_dir_config
>>>>> In header_cmd
>>>>> In header_inout_cmd
>>>>> In parse_format_tag
>>>>> In parse_misc_string
>>>>> In create_headers_dir_config
>>>>> In header_cmd
>>>>> In header_inout_cmd
>>>>> In parse_format_tag
>>>>> In parse_misc_string
>>>>> [Switching to Thread 182897610272 (LWP 7644)]
>>>>>
>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8,
>>>>> ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>>>>> 1121 printf("In header_post_config\n");
>>>>> (gdb) n
>>>>> 1120 {
>>>>> (gdb) n
>>>>> 1121 printf("In header_post_config\n");
>>>>> (gdb) n
>>>>> In header_post_config
>>>>> 1122 header_ssl_lookup =
>>>>> APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>> (gdb) n
>>>>> 1124 }
>>>>> (gdb) n
>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138,
>>>>> plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>>>>> (gdb) n
>>>>> *** glibc detected *** corrupted double-linked list:
>>>>> 0x00000000006b9710 ***
>>>>>
>>>>> Program received signal SIGABRT, Aborted.
>>>>> 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
>>>>> (gdb) bt full
>>>>> #0 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
>>>>> No symbol table info available.
>>>>> #1 0x000000351432fa6e in abort () from /lib64/tls/libc.so.6
>>>>> No symbol table info available.
>>>>> #2 0x0000003514363641 in __libc_message () from /lib64/tls/libc.so.6
>>>>> No symbol table info available.
>>>>> #3 0x0000003514369512 in _int_free () from /lib64/tls/libc.so.6
>>>>> No symbol table info available.
>>>>> #4 0x0000003514369846 in free () from /lib64/tls/libc.so.6
>>>>> No symbol table info available.
>>>>> #5 0x0000002a9790b6ba in R_free () from
>>>>> /apps/netpoint/lib64/libobaccess.so
>>>>> No symbol table info available.
>>>>> #6 0x0000002a9792bc41 in X509_CINF_free () from
>>>>> /apps/netpoint/lib64/libobaccess.so
>>>>> No symbol table info available.
>>>>> #7 0x0000002a9790ed98 in X509_free () from
>>>>> /apps/netpoint/lib64/libobaccess.so
>>>>> No symbol table info available.
>>>>> #8 0x0000002a987d97b6 in ssl_pphrase_Handle (s=0x59d3a8,
>>>>> p=0x5a72d8) at ssl_engine_pphrase.c:243
>>>>> mc = (SSLModConfigRec *) 0x571738
>>>>> sc = (SSLSrvConfigRec *) 0x668c38
>>>>> pServ = (server_rec *) 0x65fa48
>>>>> cpVHostID = 0x60add0 "apache1.whatever.com:443"
>>>>> szPath =
>>>>> "/apps/httpd2222/conf/certs/apache1.whatever.com.crt\000\177\000\000\000Ü×ÿ¿\177\000\000\000à×ÿ¿\177",
>>>>> '\0'<repeats 15 times>,
>>>>> "\001\000\000\000°åÿ¿\177\000\000\000;\000\000\000+\000\000\000\020Ûÿ¿\177",
>>>>> '\0'<repeats 35 times>, "à\224k", '\0'<repeats 13 times>,
>>>>> "øéÿ¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001",
>>>>> '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°åÿ¿\177",
>>>>> '\0'<repeats 11 times>, "\020Ûÿ¿\177\000\000\000+\000\000\000;",
>>>>> '\0'<repeats 35 times>...
>>>>> pPrivateKey = Variable "pPrivateKey" is not available.
>>>>> (gdb)
>>>>>
>>>>>
>>>>> Jim
>>>>>
>>>> I guess the thing that I'm puzzled about is why it goes from
>>>> ssl_pphrase_Handle (which is part of Apache code?) to X509_free ()
>>>> in the libobaccess.so? Apache shouldn't even be aware of the
>>>> libobaccess stuff yet (since I think that Apache is still trying to
>>>> initialize SSL processing)?
>>>>
>>>> Jim
>>> It shouldn't unless you've declared a shutdown hook or a clean up hook
>>> of sorts. The looks of that stack trace look strikingly like an
>>> exception handler firing off. What are all of the hooks you have
>>> running?
>>>
>>> Joe
>>> --
>>> http://www.silverhawk.net/
>>>
>>
>> Hi,
>>
>> I'm basically using the original mod_headers.c as a starter code, and
>> stuck some stuff in there. Haven't done anything with the hooks etc.
>> yet, so whatever was there is there:
>>
>> static void register_hooks(apr_pool_t *p)
>> {
>> printf("mod_headers-jl V0.09 - start calling OAM API\n");
>> printf("In register_hooks\n");
>> ap_register_output_filter("FIXUP_HEADERS_OUT",
>> ap_headers_output_filter,
>> NULL, AP_FTYPE_CONTENT_SET);
>> ap_register_output_filter("FIXUP_HEADERS_ERR",
>> ap_headers_error_filter,
>> NULL, AP_FTYPE_CONTENT_SET);
>> ap_hook_pre_config(header_pre_config,NULL,NULL,APR_HOOK_MIDDLE);
>> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
>> ap_hook_insert_filter(ap_headers_insert_output_filter, NULL,
>> NULL, APR_HOOK_LAST);
>> ap_hook_insert_error_filter(ap_headers_insert_error_filter,
>> NULL, NULL, APR_HOOK_LAST);
>> ap_hook_fixups(ap_headers_fixup, NULL, NULL, APR_HOOK_LAST);
>> ap_hook_post_read_request(ap_headers_early, NULL, NULL,
>> APR_HOOK_FIRST);
>> }
>>
>> module AP_MODULE_DECLARE_DATA headers_module =
>> {
>> STANDARD20_MODULE_STUFF,
>> create_headers_dir_config, /* dir config creater */
>> merge_headers_config, /* dir merger --- default is to
>> override */
>> NULL, /* server config */
>> NULL, /* merge server configs */
>> headers_cmds, /* command apr_table_t */
>> register_hooks /* register hooks */
>> };
>>
>> I guess that there is a:
>>
>> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
>>
>> in that register_hooks(), but it was there from the original code.
>>
>> BTW, I was checking, and it looks like there's an X509_free() function
>> in both openssl and in libobaccess.so (used 'nm --dynamic<file>), so
>> I'm wondering if things are confused, and maybe Apache code expects to
>> call the openssl X509_free, but somehow ends up calling the
>> X509_free() in libobaccess.so?
>>
>> What would cause something like that? If two .so files have the same
>> function, how to avoid conflicts?
>>
>> Jim
>
> Jim, you may have hit that on the head. I'm not sure of the load order
> and looking up symbols, but it could very well change things. Try
> re-ordering or adding a LoadFile before the SSL module and see if that
> changes things.
Look up order usually is load order. So if mod_ssl loads first and loads
its dependency libcrypto, the symbol will always be found there even if
the implementation in libobaccess.so is needed - and vice versa.
On Solaris there is a -Bdirect linker flag which changes runtime search
order so that each module would find a needed symbol first inits direct
dependencies. Unfortunately AFAIK notihing similar exists for Linux.
You would need to find ssl libs and libobaccess with compatible symbols.
Best would be a libobaccess which is dynamically linked against a
compatible version of libcrypto instead of - as it seems - being
statically linked against it.
Regards,
Rainer
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use Apache
module that uses shared library?
Posted by Joe Lewis <jo...@joe-lewis.com>.
On 6/21/12 10:02 PM, ohaya@cox.net wrote:
> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>> On 6/21/12 9:39 PM, ohaya@cox.net wrote:
>>> ---- ohaya@cox.net wrote:
>>>> ---- ohaya@cox.net wrote:
>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>>>> On 6/21/12 7:32 PM, ohaya@cox.net wrote:
>>>>>>> ---- ohaya@cox.net wrote:
>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>>>>>>> On 6/21/12 6:46 PM, ohaya@cox.net wrote:
>>>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>>>>>>>>> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
>>>>>>>>>>>> ---- ohaya@cox.net wrote:
>>>>>>>>>>>>> ---- Sorin Manolache<so...@gmail.com> wrote:
>>>>>>>>>>>>>> And I forgot to say: run gdb in some sort of environment where you see
>>>>>>>>>>>>>> your current source code line and a couple of surrounding lines. You
>>>>>>>>>>>>>> could achieve this with the "list" command, but I prefer running gdb in
>>>>>>>>>>>>>> emacs and let emacs do the nice listing of source code in a different panel.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> S
>>>>>>>>>>>>> Here's the function from my source. It's the original from mod_headers.c, plus my printf:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> static int header_post_config(apr_pool_t *pconf, apr_pool_t *plog,
>>>>>>>>>>>>> apr_pool_t *ptemp, server_rec *s)
>>>>>>>>>>>>> {
>>>>>>>>>>>>> printf("In header_post_config\n");
>>>>>>>>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>>>>>>> return OK;
>>>>>>>>>>>>> }
>>>>>>>>>>>>>
>>>>>>>>>>>>> Jim
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> I was able to get the segfault to go away. Here's what I had to do:
>>>>>>>>>>>>
>>>>>>>>>>>> - Created /etc/ld.so.conf.d/my.conf, and added the directory where my libobaccess.so was
>>>>>>>>>>>> - Run 'ldconfig' to activate.
>>>>>>>>>>>> - In the apxs command, DON'T include the -L and -l arguments
>>>>>>>>>>>>
>>>>>>>>>>>> After that, Apache appears to start ok, without segfault :)!!
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for all of the great help, esp. the suggestion about checking "ldconfig -p". I still don't understand why, but I'm just glad that I can get past this piece so now I can debug my module :)...
>>>>>>>>>>>>
>>>>>>>>>>>> Later,
>>>>>>>>>>>> Jim
>>>>>>>>>>> I'm just glad this list is as good as it is!
>>>>>>>>>>>
>>>>>>>>>>> FYI, the ldconfig is the dynamic linker control, and those
>>>>>>>>>>> /etc/ld.so.conf.d files provide additional search directories for the
>>>>>>>>>>> linker to check in when loading a library.
>>>>>>>>>>>
>>>>>>>>>>> Joe
>>>>>>>>>>> --
>>>>>>>>>>> http://www.silverhawk.net/
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> Sorry to report, but my earlier report was a "false positive" :)...
>>>>>>>>>>
>>>>>>>>>> I forgot that the mod_headers.c that I was doing the earlier testing with had all references to the libobaccess.so removed :(!!
>>>>>>>>>>
>>>>>>>>>> So, I'm still stuck with basically the same problem, now, working with my "full" code, with the calls in it:
>>>>>>>>>>
>>>>>>>>>> - If I compile with -L and -l, Apache segfaults when it starts
>>>>>>>>>>
>>>>>>>>>> - If I compile without -L and -l, then I get "undefined symbol" errors when I try to start Apache, e.g.:
>>>>>>>>>>
>>>>>>>>>> [root@apachemodule build-mod_headers]# /apps/httpd2222/bin/apachectl -k start -X
>>>>>>>>>> httpd: Syntax error on line 84 of /apps/httpd2222/conf/httpd.conf: Cannot load /apps/httpd2222/modules/mod_headers.so into server: /apps/httpd2222/modules/mod_headers.so: undefined symbol: ObResource_isProtected
>>>>>>>>>>
>>>>>>>>>> That "ObResource_isProtected" should be a symbol in libobaccess.so, and in fact, if I do "nm --dynamic", I get:
>>>>>>>>>>
>>>>>>>>>> [root@apachemodule build-mod_headers]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "ObResource_isProtected"
>>>>>>>>>> 00000000000a6d80 T ObResource_isProtected
>>>>>>>>>> [root@apachemodule build-mod_headers]#
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I'm *assuming* that the reason for the "undefined symbol" error is that libobaccess.so is actually not being loaded, but then when I try to load libobaccess.so, either via -L and -l in the apxs, or using LoadFile in httpd.conf, I get the segfault (same gdb info, BTW).
>>>>>>>>>>
>>>>>>>>>> Catch-22?
>>>>>>>>>>
>>>>>>>>>> Sorry for the false alarm :(!!
>>>>>>>>>>
>>>>>>>>>> Jim
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> Not a catch-22. The -L and -l specify linker options when assembling
>>>>>>>>> the code. The ldconfig is a run-time thing. If you are getting the
>>>>>>>>> stderr messages, you are making it all the way into your library. I'd
>>>>>>>>> suggest commenting out the following line and see if you get farther :
>>>>>>>>>
>>>>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>>>
>>>>>>>>> That should tell you if the problem is the ssl_var_lookup.
>>>>>>>>>
>>>>>>>>> Joe
>>>>>>>>> --
>>>>>>>>> http://www.silverhawk.net/
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Thanks for the suggestion. I just tried what you suggested, and got a segfault when I started Apache with the modified module.
>>>>>>>>
>>>>>>>> Jim
>>>>>>>>
>>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> As a reminder, here's the gdb with the library loaded:
>>>>>>>
>>>>>>>
>>>>>>> (gdb) b header_post_config
>>>>>>> Function "header_post_config" not defined.
>>>>>>> Make breakpoint pending on future shared library load? (y or [n]) y
>>>>>>> Breakpoint 1 (header_post_config) pending.
>>>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
>>>>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
>>>>>>> [Thread debugging using libthread_db enabled]
>>>>>>> [New Thread 182897610272 (LWP 11317)]
>>>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
>>>>>>> Pending breakpoint "header_post_config" resolved
>>>>>>> mod_headers-jl V0.09 - start calling OAM API
>>>>>>> In register_hooks
>>>>>>> In create_headers_dir_config
>>>>>>> In create_headers_dir_config
>>>>>>> In header_cmd
>>>>>>> In header_inout_cmd
>>>>>>> In parse_format_tag
>>>>>>> In parse_misc_string
>>>>>>> In create_headers_dir_config
>>>>>>> In header_cmd
>>>>>>> In header_inout_cmd
>>>>>>> In parse_format_tag
>>>>>>> In parse_misc_string
>>>>>>> [Switching to Thread 182897610272 (LWP 11317)]
>>>>>>>
>>>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>>>>>>> 1121 printf("In header_post_config\n");
>>>>>>> (gdb) n
>>>>>>> 1120 {
>>>>>>> (gdb) n
>>>>>>> 1121 printf("In header_post_config\n");
>>>>>>> (gdb) n
>>>>>>> In header_post_config
>>>>>>> 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>> (gdb) n
>>>>>>> 1124 }
>>>>>>> (gdb) n
>>>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
>>>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>>>>>>> (gdb) n
>>>>>>>
>>>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>>>
>>>>>>>
>>>>>>> So, it's actually blowing up in "BN_num_bits()" in /lib64/libcrypto.so.4?
>>>>>>>
>>>>>>> Jim
>>>>>> When you see the :
>>>>>>
>>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>>
>>>>>>
>>>>>> in gdb, type "bt" and hit enter. It should show the back trace and how
>>>>>> you got to the BN_num_bits() function.
>>>>>>
>>>>>> Joe
>>>>>> --
>>>>>> http://www.silverhawk.net/
>>>>> Hi,
>>>>>
>>>>> Here's the bt full:
>>>>>
>>>>>
>>>>> In header_inout_cmd
>>>>> In parse_format_tag
>>>>> In parse_misc_string
>>>>> [Switching to Thread 182897610272 (LWP 6676)]
>>>>>
>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>>>>> 1121 printf("In header_post_config\n");
>>>>> (gdb) n
>>>>> 1120 {
>>>>> (gdb) n
>>>>> 1121 printf("In header_post_config\n");
>>>>> (gdb) n
>>>>> In header_post_config
>>>>> 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>> (gdb) n
>>>>> 1124 }
>>>>> (gdb) n
>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>>>>> (gdb) n
>>>>>
>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>> (gdb) bt full
>>>>> #0 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>> No symbol table info available.
>>>>> #1 0x0000003518da8f4e in X509_ATTRIBUTE_create () from /lib64/libcrypto.so.4
>>>>> No symbol table info available.
>>>>> #2 0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
>>>>> No symbol table info available.
>>>>> #3 0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
>>>>> No symbol table info available.
>>>>> #4 0x0000003518dae0e1 in ASN1_item_ex_i2d () from /lib64/libcrypto.so.4
>>>>> No symbol table info available.
>>>>> #5 0x0000003518dae5f2 in ASN1_template_i2d () from /lib64/libcrypto.so.4
>>>>> No symbol table info available.
>>>>> #6 0x0000003518dae28e in ASN1_item_ex_i2d () from /lib64/libcrypto.so.4
>>>>> No symbol table info available.
>>>>> #7 0x0000003518dae3c6 in ASN1_item_i2d () from /lib64/libcrypto.so.4
>>>>> No symbol table info available.
>>>>> #8 0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8, p=0x5a72d8) at ssl_engine_pphrase.c:505
>>>>> mc = (SSLModConfigRec *) 0x571738
>>>>> sc = (SSLSrvConfigRec *) 0x668c38
>>>>> pServ = (server_rec *) 0x65fa48
>>>>> cpVHostID = 0x60add0 "www.example.com:443"
>>>>> szPath = "/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü×ÿ¿\177\000\000\000à×ÿ¿\177", '\0'<rep eats 15 times>, "\001\000\000\000°åÿ¿\177\000\000\000;\000\000\000+\000\000\000\020Ûÿ¿\177", '\0'<repeats 35 times>, "à\224k", ' ---Type<return> to continue, or q<return> to quit---
>>>>> \0'<repeats 13 times>, "øéÿ¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001", '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°åÿ¿\177", '\0'<repeats 11 times>, "\020Ûÿ¿\177\000\000\000+\000\000\000;", '\0'<repeats 35 times>...
>>>>> pPrivateKey = (EVP_PKEY *) 0x6ba670
>>>>> asn1 = Variable "asn1" is not available.
>>>>> (gdb)
>>>>>
>>>>>
>>>>> Not really sure what to make of that though :(...
>>>>>
>>>>> Jim
>>>> Hi,
>>>>
>>>> Ok, something interesting. I noticed that the servername in the gdb output was example.whatever.com, so I changed ServerName in extras/httpd-ssl.conf to "apache1.whatever.com". Apache still segfaulted after that.
>>>>
>>>> I noticed that igdb was showing my router IP address for "mod_unique_id" (whatever that is), so I thought it was because I didn't have the hostname in /etc/hosts. So, I added "apache1.whatever.com" to /etc/hosts, and also turned Apache LogLevel to debug, and now I get a different error:
>>>>
>>>> [root@apachemodule dev]# /apps/httpd2222/bin/apachectl -k start -X
>>>> mod_headers-jl V0.09 - start calling OAM API
>>>> In register_hooks
>>>> In create_headers_dir_config
>>>> In create_headers_dir_config
>>>> In header_cmd
>>>> In header_inout_cmd
>>>> In parse_format_tag
>>>> In parse_misc_string
>>>> In create_headers_dir_config
>>>> In header_cmd
>>>> In header_inout_cmd
>>>> In parse_format_tag
>>>> In parse_misc_string
>>>> In header_post_config
>>>> *** glibc detected *** corrupted double-linked list: 0x00000000006b9710 ***
>>>> /apps/httpd2222/bin/apachectl: line 78: 7599 Aborted $HTTPD $ARGV
>>>> [root@apachemodule dev]#
>>>>
>>>>
>>>> Here's the Apache error_log:
>>>>
>>>> [Thu Jun 21 23:24:11 2012] [info] mod_unique_id: using ip addr 72.215.225.9
>>>> [Thu Jun 21 23:24:12 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
>>>> [Thu Jun 21 23:24:12 2012] [info] Loading certificate& private key of SSL-aware server
>>>> [Thu Jun 21 23:29:51 2012] [info] mod_unique_id: using ip addr 72.215.225.9
>>>> [Thu Jun 21 23:29:52 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
>>>> [Thu Jun 21 23:29:52 2012] [info] Loading certificate& private key of SSL-aware server
>>>> [Thu Jun 21 23:29:56 2012] [info] mod_unique_id: using ip addr 72.215.225.9
>>>> [Thu Jun 21 23:29:57 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
>>>> [Thu Jun 21 23:29:57 2012] [info] Loading certificate& private key of SSL-aware server
>>>> [root@apachemodule dev]#
>>>>
>>>>
>>>> and here's the gdb backtrace:
>>>>
>>>> (gdb) b header_post_config
>>>> Function "header_post_config" not defined.
>>>> Make breakpoint pending on future shared library load? (y or [n]) y
>>>> Breakpoint 1 (header_post_config) pending.
>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
>>>> [Thread debugging using libthread_db enabled]
>>>> [New Thread 182897610272 (LWP 7644)]
>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
>>>> Pending breakpoint "header_post_config" resolved
>>>> mod_headers-jl V0.09 - start calling OAM API
>>>> In register_hooks
>>>> In create_headers_dir_config
>>>> In create_headers_dir_config
>>>> In header_cmd
>>>> In header_inout_cmd
>>>> In parse_format_tag
>>>> In parse_misc_string
>>>> In create_headers_dir_config
>>>> In header_cmd
>>>> In header_inout_cmd
>>>> In parse_format_tag
>>>> In parse_misc_string
>>>> [Switching to Thread 182897610272 (LWP 7644)]
>>>>
>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>>>> 1121 printf("In header_post_config\n");
>>>> (gdb) n
>>>> 1120 {
>>>> (gdb) n
>>>> 1121 printf("In header_post_config\n");
>>>> (gdb) n
>>>> In header_post_config
>>>> 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>> (gdb) n
>>>> 1124 }
>>>> (gdb) n
>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>>>> (gdb) n
>>>> *** glibc detected *** corrupted double-linked list: 0x00000000006b9710 ***
>>>>
>>>> Program received signal SIGABRT, Aborted.
>>>> 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
>>>> (gdb) bt full
>>>> #0 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
>>>> No symbol table info available.
>>>> #1 0x000000351432fa6e in abort () from /lib64/tls/libc.so.6
>>>> No symbol table info available.
>>>> #2 0x0000003514363641 in __libc_message () from /lib64/tls/libc.so.6
>>>> No symbol table info available.
>>>> #3 0x0000003514369512 in _int_free () from /lib64/tls/libc.so.6
>>>> No symbol table info available.
>>>> #4 0x0000003514369846 in free () from /lib64/tls/libc.so.6
>>>> No symbol table info available.
>>>> #5 0x0000002a9790b6ba in R_free () from /apps/netpoint/lib64/libobaccess.so
>>>> No symbol table info available.
>>>> #6 0x0000002a9792bc41 in X509_CINF_free () from /apps/netpoint/lib64/libobaccess.so
>>>> No symbol table info available.
>>>> #7 0x0000002a9790ed98 in X509_free () from /apps/netpoint/lib64/libobaccess.so
>>>> No symbol table info available.
>>>> #8 0x0000002a987d97b6 in ssl_pphrase_Handle (s=0x59d3a8, p=0x5a72d8) at ssl_engine_pphrase.c:243
>>>> mc = (SSLModConfigRec *) 0x571738
>>>> sc = (SSLSrvConfigRec *) 0x668c38
>>>> pServ = (server_rec *) 0x65fa48
>>>> cpVHostID = 0x60add0 "apache1.whatever.com:443"
>>>> szPath = "/apps/httpd2222/conf/certs/apache1.whatever.com.crt\000\177\000\000\000Ü×ÿ¿\177\000\000\000à×ÿ¿\177", '\0'<repeats 15 times>, "\001\000\000\000°åÿ¿\177\000\000\000;\000\000\000+\000\000\000\020Ûÿ¿\177", '\0'<repeats 35 times>, "à\224k", '\0'<repeats 13 times>, "øéÿ¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001", '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°åÿ¿\177", '\0'<repeats 11 times>, "\020Ûÿ¿\177\000\000\000+\000\000\000;", '\0'<repeats 35 times>...
>>>> pPrivateKey = Variable "pPrivateKey" is not available.
>>>> (gdb)
>>>>
>>>>
>>>> Jim
>>>>
>>> I guess the thing that I'm puzzled about is why it goes from ssl_pphrase_Handle (which is part of Apache code?) to X509_free () in the libobaccess.so? Apache shouldn't even be aware of the libobaccess stuff yet (since I think that Apache is still trying to initialize SSL processing)?
>>>
>>> Jim
>> It shouldn't unless you've declared a shutdown hook or a clean up hook
>> of sorts. The looks of that stack trace look strikingly like an
>> exception handler firing off. What are all of the hooks you have running?
>>
>> Joe
>> --
>> http://www.silverhawk.net/
>>
>
> Hi,
>
> I'm basically using the original mod_headers.c as a starter code, and stuck some stuff in there. Haven't done anything with the hooks etc. yet, so whatever was there is there:
>
> static void register_hooks(apr_pool_t *p)
> {
> printf("mod_headers-jl V0.09 - start calling OAM API\n");
> printf("In register_hooks\n");
> ap_register_output_filter("FIXUP_HEADERS_OUT", ap_headers_output_filter,
> NULL, AP_FTYPE_CONTENT_SET);
> ap_register_output_filter("FIXUP_HEADERS_ERR", ap_headers_error_filter,
> NULL, AP_FTYPE_CONTENT_SET);
> ap_hook_pre_config(header_pre_config,NULL,NULL,APR_HOOK_MIDDLE);
> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
> ap_hook_insert_filter(ap_headers_insert_output_filter, NULL, NULL, APR_HOOK_LAST);
> ap_hook_insert_error_filter(ap_headers_insert_error_filter,
> NULL, NULL, APR_HOOK_LAST);
> ap_hook_fixups(ap_headers_fixup, NULL, NULL, APR_HOOK_LAST);
> ap_hook_post_read_request(ap_headers_early, NULL, NULL, APR_HOOK_FIRST);
> }
>
> module AP_MODULE_DECLARE_DATA headers_module =
> {
> STANDARD20_MODULE_STUFF,
> create_headers_dir_config, /* dir config creater */
> merge_headers_config, /* dir merger --- default is to override */
> NULL, /* server config */
> NULL, /* merge server configs */
> headers_cmds, /* command apr_table_t */
> register_hooks /* register hooks */
> };
>
> I guess that there is a:
>
> ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
>
> in that register_hooks(), but it was there from the original code.
>
> BTW, I was checking, and it looks like there's an X509_free() function in both openssl and in libobaccess.so (used 'nm --dynamic<file>), so I'm wondering if things are confused, and maybe Apache code expects to call the openssl X509_free, but somehow ends up calling the X509_free() in libobaccess.so?
>
> What would cause something like that? If two .so files have the same function, how to avoid conflicts?
>
> Jim
Jim, you may have hit that on the head. I'm not sure of the load order
and looking up symbols, but it could very well change things. Try
re-ordering or adding a LoadFile before the SSL module and see if that
changes things.
Joe
--
http://www.silverhawk.net/
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use
Apache module that uses shared library?
Posted by oh...@cox.net.
---- Joe Lewis <jo...@joe-lewis.com> wrote:
> On 6/21/12 9:39 PM, ohaya@cox.net wrote:
> > ---- ohaya@cox.net wrote:
> >> ---- ohaya@cox.net wrote:
> >>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >>>> On 6/21/12 7:32 PM, ohaya@cox.net wrote:
> >>>>> ---- ohaya@cox.net wrote:
> >>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >>>>>>> On 6/21/12 6:46 PM, ohaya@cox.net wrote:
> >>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >>>>>>>>> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
> >>>>>>>>>> ---- ohaya@cox.net wrote:
> >>>>>>>>>>> ---- Sorin Manolache<so...@gmail.com> wrote:
> >>>>>>>>>>>> And I forgot to say: run gdb in some sort of environment where you see
> >>>>>>>>>>>> your current source code line and a couple of surrounding lines. You
> >>>>>>>>>>>> could achieve this with the "list" command, but I prefer running gdb in
> >>>>>>>>>>>> emacs and let emacs do the nice listing of source code in a different panel.
> >>>>>>>>>>>>
> >>>>>>>>>>>> S
> >>>>>>>>>>> Here's the function from my source. It's the original from mod_headers.c, plus my printf:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> static int header_post_config(apr_pool_t *pconf, apr_pool_t *plog,
> >>>>>>>>>>> apr_pool_t *ptemp, server_rec *s)
> >>>>>>>>>>> {
> >>>>>>>>>>> printf("In header_post_config\n");
> >>>>>>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>>>>> return OK;
> >>>>>>>>>>> }
> >>>>>>>>>>>
> >>>>>>>>>>> Jim
> >>>>>>>>>> Hi,
> >>>>>>>>>>
> >>>>>>>>>> I was able to get the segfault to go away. Here's what I had to do:
> >>>>>>>>>>
> >>>>>>>>>> - Created /etc/ld.so.conf.d/my.conf, and added the directory where my libobaccess.so was
> >>>>>>>>>> - Run 'ldconfig' to activate.
> >>>>>>>>>> - In the apxs command, DON'T include the -L and -l arguments
> >>>>>>>>>>
> >>>>>>>>>> After that, Apache appears to start ok, without segfault :)!!
> >>>>>>>>>>
> >>>>>>>>>> Thanks for all of the great help, esp. the suggestion about checking "ldconfig -p". I still don't understand why, but I'm just glad that I can get past this piece so now I can debug my module :)...
> >>>>>>>>>>
> >>>>>>>>>> Later,
> >>>>>>>>>> Jim
> >>>>>>>>> I'm just glad this list is as good as it is!
> >>>>>>>>>
> >>>>>>>>> FYI, the ldconfig is the dynamic linker control, and those
> >>>>>>>>> /etc/ld.so.conf.d files provide additional search directories for the
> >>>>>>>>> linker to check in when loading a library.
> >>>>>>>>>
> >>>>>>>>> Joe
> >>>>>>>>> --
> >>>>>>>>> http://www.silverhawk.net/
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> Sorry to report, but my earlier report was a "false positive" :)...
> >>>>>>>>
> >>>>>>>> I forgot that the mod_headers.c that I was doing the earlier testing with had all references to the libobaccess.so removed :(!!
> >>>>>>>>
> >>>>>>>> So, I'm still stuck with basically the same problem, now, working with my "full" code, with the calls in it:
> >>>>>>>>
> >>>>>>>> - If I compile with -L and -l, Apache segfaults when it starts
> >>>>>>>>
> >>>>>>>> - If I compile without -L and -l, then I get "undefined symbol" errors when I try to start Apache, e.g.:
> >>>>>>>>
> >>>>>>>> [root@apachemodule build-mod_headers]# /apps/httpd2222/bin/apachectl -k start -X
> >>>>>>>> httpd: Syntax error on line 84 of /apps/httpd2222/conf/httpd.conf: Cannot load /apps/httpd2222/modules/mod_headers.so into server: /apps/httpd2222/modules/mod_headers.so: undefined symbol: ObResource_isProtected
> >>>>>>>>
> >>>>>>>> That "ObResource_isProtected" should be a symbol in libobaccess.so, and in fact, if I do "nm --dynamic", I get:
> >>>>>>>>
> >>>>>>>> [root@apachemodule build-mod_headers]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "ObResource_isProtected"
> >>>>>>>> 00000000000a6d80 T ObResource_isProtected
> >>>>>>>> [root@apachemodule build-mod_headers]#
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> I'm *assuming* that the reason for the "undefined symbol" error is that libobaccess.so is actually not being loaded, but then when I try to load libobaccess.so, either via -L and -l in the apxs, or using LoadFile in httpd.conf, I get the segfault (same gdb info, BTW).
> >>>>>>>>
> >>>>>>>> Catch-22?
> >>>>>>>>
> >>>>>>>> Sorry for the false alarm :(!!
> >>>>>>>>
> >>>>>>>> Jim
> >>>>>>>>
> >>>>>>>>
> >>>>>>> Not a catch-22. The -L and -l specify linker options when assembling
> >>>>>>> the code. The ldconfig is a run-time thing. If you are getting the
> >>>>>>> stderr messages, you are making it all the way into your library. I'd
> >>>>>>> suggest commenting out the following line and see if you get farther :
> >>>>>>>
> >>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>>
> >>>>>>> That should tell you if the problem is the ssl_var_lookup.
> >>>>>>>
> >>>>>>> Joe
> >>>>>>> --
> >>>>>>> http://www.silverhawk.net/
> >>>>>> Hi,
> >>>>>>
> >>>>>> Thanks for the suggestion. I just tried what you suggested, and got a segfault when I started Apache with the modified module.
> >>>>>>
> >>>>>> Jim
> >>>>>>
> >>>>>>
> >>>>> Hi,
> >>>>>
> >>>>> As a reminder, here's the gdb with the library loaded:
> >>>>>
> >>>>>
> >>>>> (gdb) b header_post_config
> >>>>> Function "header_post_config" not defined.
> >>>>> Make breakpoint pending on future shared library load? (y or [n]) y
> >>>>> Breakpoint 1 (header_post_config) pending.
> >>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> >>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> >>>>> [Thread debugging using libthread_db enabled]
> >>>>> [New Thread 182897610272 (LWP 11317)]
> >>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> >>>>> Pending breakpoint "header_post_config" resolved
> >>>>> mod_headers-jl V0.09 - start calling OAM API
> >>>>> In register_hooks
> >>>>> In create_headers_dir_config
> >>>>> In create_headers_dir_config
> >>>>> In header_cmd
> >>>>> In header_inout_cmd
> >>>>> In parse_format_tag
> >>>>> In parse_misc_string
> >>>>> In create_headers_dir_config
> >>>>> In header_cmd
> >>>>> In header_inout_cmd
> >>>>> In parse_format_tag
> >>>>> In parse_misc_string
> >>>>> [Switching to Thread 182897610272 (LWP 11317)]
> >>>>>
> >>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> >>>>> 1121 printf("In header_post_config\n");
> >>>>> (gdb) n
> >>>>> 1120 {
> >>>>> (gdb) n
> >>>>> 1121 printf("In header_post_config\n");
> >>>>> (gdb) n
> >>>>> In header_post_config
> >>>>> 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>> (gdb) n
> >>>>> 1124 }
> >>>>> (gdb) n
> >>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> >>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >>>>> (gdb) n
> >>>>>
> >>>>> Program received signal SIGSEGV, Segmentation fault.
> >>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>>
> >>>>>
> >>>>> So, it's actually blowing up in "BN_num_bits()" in /lib64/libcrypto.so.4?
> >>>>>
> >>>>> Jim
> >>>> When you see the :
> >>>>
> >>>> Program received signal SIGSEGV, Segmentation fault.
> >>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>>>
> >>>>
> >>>> in gdb, type "bt" and hit enter. It should show the back trace and how
> >>>> you got to the BN_num_bits() function.
> >>>>
> >>>> Joe
> >>>> --
> >>>> http://www.silverhawk.net/
> >>>
> >>> Hi,
> >>>
> >>> Here's the bt full:
> >>>
> >>>
> >>> In header_inout_cmd
> >>> In parse_format_tag
> >>> In parse_misc_string
> >>> [Switching to Thread 182897610272 (LWP 6676)]
> >>>
> >>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> >>> 1121 printf("In header_post_config\n");
> >>> (gdb) n
> >>> 1120 {
> >>> (gdb) n
> >>> 1121 printf("In header_post_config\n");
> >>> (gdb) n
> >>> In header_post_config
> >>> 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>> (gdb) n
> >>> 1124 }
> >>> (gdb) n
> >>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> >>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >>> (gdb) n
> >>>
> >>> Program received signal SIGSEGV, Segmentation fault.
> >>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>> (gdb) bt full
> >>> #0 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >>> No symbol table info available.
> >>> #1 0x0000003518da8f4e in X509_ATTRIBUTE_create () from /lib64/libcrypto.so.4
> >>> No symbol table info available.
> >>> #2 0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> >>> No symbol table info available.
> >>> #3 0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> >>> No symbol table info available.
> >>> #4 0x0000003518dae0e1 in ASN1_item_ex_i2d () from /lib64/libcrypto.so.4
> >>> No symbol table info available.
> >>> #5 0x0000003518dae5f2 in ASN1_template_i2d () from /lib64/libcrypto.so.4
> >>> No symbol table info available.
> >>> #6 0x0000003518dae28e in ASN1_item_ex_i2d () from /lib64/libcrypto.so.4
> >>> No symbol table info available.
> >>> #7 0x0000003518dae3c6 in ASN1_item_i2d () from /lib64/libcrypto.so.4
> >>> No symbol table info available.
> >>> #8 0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8, p=0x5a72d8) at ssl_engine_pphrase.c:505
> >>> mc = (SSLModConfigRec *) 0x571738
> >>> sc = (SSLSrvConfigRec *) 0x668c38
> >>> pServ = (server_rec *) 0x65fa48
> >>> cpVHostID = 0x60add0 "www.example.com:443"
> >>> szPath = "/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü×ÿ¿\177\000\000\000à×ÿ¿\177", '\0'<rep eats 15 times>, "\001\000\000\000°åÿ¿\177\000\000\000;\000\000\000+\000\000\000\020Ûÿ¿\177", '\0'<repeats 35 times>, "à\224k", ' ---Type<return> to continue, or q<return> to quit---
> >>> \0'<repeats 13 times>, "øéÿ¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001", '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°åÿ¿\177", '\0'<repeats 11 times>, "\020Ûÿ¿\177\000\000\000+\000\000\000;", '\0'<repeats 35 times>...
> >>> pPrivateKey = (EVP_PKEY *) 0x6ba670
> >>> asn1 = Variable "asn1" is not available.
> >>> (gdb)
> >>>
> >>>
> >>> Not really sure what to make of that though :(...
> >>>
> >>> Jim
> >>
> >> Hi,
> >>
> >> Ok, something interesting. I noticed that the servername in the gdb output was example.whatever.com, so I changed ServerName in extras/httpd-ssl.conf to "apache1.whatever.com". Apache still segfaulted after that.
> >>
> >> I noticed that igdb was showing my router IP address for "mod_unique_id" (whatever that is), so I thought it was because I didn't have the hostname in /etc/hosts. So, I added "apache1.whatever.com" to /etc/hosts, and also turned Apache LogLevel to debug, and now I get a different error:
> >>
> >> [root@apachemodule dev]# /apps/httpd2222/bin/apachectl -k start -X
> >> mod_headers-jl V0.09 - start calling OAM API
> >> In register_hooks
> >> In create_headers_dir_config
> >> In create_headers_dir_config
> >> In header_cmd
> >> In header_inout_cmd
> >> In parse_format_tag
> >> In parse_misc_string
> >> In create_headers_dir_config
> >> In header_cmd
> >> In header_inout_cmd
> >> In parse_format_tag
> >> In parse_misc_string
> >> In header_post_config
> >> *** glibc detected *** corrupted double-linked list: 0x00000000006b9710 ***
> >> /apps/httpd2222/bin/apachectl: line 78: 7599 Aborted $HTTPD $ARGV
> >> [root@apachemodule dev]#
> >>
> >>
> >> Here's the Apache error_log:
> >>
> >> [Thu Jun 21 23:24:11 2012] [info] mod_unique_id: using ip addr 72.215.225.9
> >> [Thu Jun 21 23:24:12 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
> >> [Thu Jun 21 23:24:12 2012] [info] Loading certificate& private key of SSL-aware server
> >> [Thu Jun 21 23:29:51 2012] [info] mod_unique_id: using ip addr 72.215.225.9
> >> [Thu Jun 21 23:29:52 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
> >> [Thu Jun 21 23:29:52 2012] [info] Loading certificate& private key of SSL-aware server
> >> [Thu Jun 21 23:29:56 2012] [info] mod_unique_id: using ip addr 72.215.225.9
> >> [Thu Jun 21 23:29:57 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
> >> [Thu Jun 21 23:29:57 2012] [info] Loading certificate& private key of SSL-aware server
> >> [root@apachemodule dev]#
> >>
> >>
> >> and here's the gdb backtrace:
> >>
> >> (gdb) b header_post_config
> >> Function "header_post_config" not defined.
> >> Make breakpoint pending on future shared library load? (y or [n]) y
> >> Breakpoint 1 (header_post_config) pending.
> >> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> >> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> >> [Thread debugging using libthread_db enabled]
> >> [New Thread 182897610272 (LWP 7644)]
> >> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> >> Pending breakpoint "header_post_config" resolved
> >> mod_headers-jl V0.09 - start calling OAM API
> >> In register_hooks
> >> In create_headers_dir_config
> >> In create_headers_dir_config
> >> In header_cmd
> >> In header_inout_cmd
> >> In parse_format_tag
> >> In parse_misc_string
> >> In create_headers_dir_config
> >> In header_cmd
> >> In header_inout_cmd
> >> In parse_format_tag
> >> In parse_misc_string
> >> [Switching to Thread 182897610272 (LWP 7644)]
> >>
> >> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> >> 1121 printf("In header_post_config\n");
> >> (gdb) n
> >> 1120 {
> >> (gdb) n
> >> 1121 printf("In header_post_config\n");
> >> (gdb) n
> >> In header_post_config
> >> 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >> (gdb) n
> >> 1124 }
> >> (gdb) n
> >> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> >> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> >> (gdb) n
> >> *** glibc detected *** corrupted double-linked list: 0x00000000006b9710 ***
> >>
> >> Program received signal SIGABRT, Aborted.
> >> 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> >> (gdb) bt full
> >> #0 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> >> No symbol table info available.
> >> #1 0x000000351432fa6e in abort () from /lib64/tls/libc.so.6
> >> No symbol table info available.
> >> #2 0x0000003514363641 in __libc_message () from /lib64/tls/libc.so.6
> >> No symbol table info available.
> >> #3 0x0000003514369512 in _int_free () from /lib64/tls/libc.so.6
> >> No symbol table info available.
> >> #4 0x0000003514369846 in free () from /lib64/tls/libc.so.6
> >> No symbol table info available.
> >> #5 0x0000002a9790b6ba in R_free () from /apps/netpoint/lib64/libobaccess.so
> >> No symbol table info available.
> >> #6 0x0000002a9792bc41 in X509_CINF_free () from /apps/netpoint/lib64/libobaccess.so
> >> No symbol table info available.
> >> #7 0x0000002a9790ed98 in X509_free () from /apps/netpoint/lib64/libobaccess.so
> >> No symbol table info available.
> >> #8 0x0000002a987d97b6 in ssl_pphrase_Handle (s=0x59d3a8, p=0x5a72d8) at ssl_engine_pphrase.c:243
> >> mc = (SSLModConfigRec *) 0x571738
> >> sc = (SSLSrvConfigRec *) 0x668c38
> >> pServ = (server_rec *) 0x65fa48
> >> cpVHostID = 0x60add0 "apache1.whatever.com:443"
> >> szPath = "/apps/httpd2222/conf/certs/apache1.whatever.com.crt\000\177\000\000\000Ü×ÿ¿\177\000\000\000à×ÿ¿\177", '\0'<repeats 15 times>, "\001\000\000\000°åÿ¿\177\000\000\000;\000\000\000+\000\000\000\020Ûÿ¿\177", '\0'<repeats 35 times>, "à\224k", '\0'<repeats 13 times>, "øéÿ¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001", '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°åÿ¿\177", '\0'<repeats 11 times>, "\020Ûÿ¿\177\000\000\000+\000\000\000;", '\0'<repeats 35 times>...
> >> pPrivateKey = Variable "pPrivateKey" is not available.
> >> (gdb)
> >>
> >>
> >> Jim
> >>
> >
> > I guess the thing that I'm puzzled about is why it goes from ssl_pphrase_Handle (which is part of Apache code?) to X509_free () in the libobaccess.so? Apache shouldn't even be aware of the libobaccess stuff yet (since I think that Apache is still trying to initialize SSL processing)?
> >
> > Jim
>
> It shouldn't unless you've declared a shutdown hook or a clean up hook
> of sorts. The looks of that stack trace look strikingly like an
> exception handler firing off. What are all of the hooks you have running?
>
> Joe
> --
> http://www.silverhawk.net/
>
Hi,
I'm basically using the original mod_headers.c as a starter code, and stuck some stuff in there. Haven't done anything with the hooks etc. yet, so whatever was there is there:
static void register_hooks(apr_pool_t *p)
{
printf("mod_headers-jl V0.09 - start calling OAM API\n");
printf("In register_hooks\n");
ap_register_output_filter("FIXUP_HEADERS_OUT", ap_headers_output_filter,
NULL, AP_FTYPE_CONTENT_SET);
ap_register_output_filter("FIXUP_HEADERS_ERR", ap_headers_error_filter,
NULL, AP_FTYPE_CONTENT_SET);
ap_hook_pre_config(header_pre_config,NULL,NULL,APR_HOOK_MIDDLE);
ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
ap_hook_insert_filter(ap_headers_insert_output_filter, NULL, NULL, APR_HOOK_LAST);
ap_hook_insert_error_filter(ap_headers_insert_error_filter,
NULL, NULL, APR_HOOK_LAST);
ap_hook_fixups(ap_headers_fixup, NULL, NULL, APR_HOOK_LAST);
ap_hook_post_read_request(ap_headers_early, NULL, NULL, APR_HOOK_FIRST);
}
module AP_MODULE_DECLARE_DATA headers_module =
{
STANDARD20_MODULE_STUFF,
create_headers_dir_config, /* dir config creater */
merge_headers_config, /* dir merger --- default is to override */
NULL, /* server config */
NULL, /* merge server configs */
headers_cmds, /* command apr_table_t */
register_hooks /* register hooks */
};
I guess that there is a:
ap_hook_post_config(header_post_config,NULL,NULL,APR_HOOK_MIDDLE);
in that register_hooks(), but it was there from the original code.
BTW, I was checking, and it looks like there's an X509_free() function in both openssl and in libobaccess.so (used 'nm --dynamic <file>), so I'm wondering if things are confused, and maybe Apache code expects to call the openssl X509_free, but somehow ends up calling the X509_free() in libobaccess.so?
What would cause something like that? If two .so files have the same function, how to avoid conflicts?
Jim
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use Apache
module that uses shared library?
Posted by Joe Lewis <jo...@joe-lewis.com>.
On 6/21/12 9:39 PM, ohaya@cox.net wrote:
> ---- ohaya@cox.net wrote:
>> ---- ohaya@cox.net wrote:
>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>> On 6/21/12 7:32 PM, ohaya@cox.net wrote:
>>>>> ---- ohaya@cox.net wrote:
>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>>>>> On 6/21/12 6:46 PM, ohaya@cox.net wrote:
>>>>>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>>>>>>> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
>>>>>>>>>> ---- ohaya@cox.net wrote:
>>>>>>>>>>> ---- Sorin Manolache<so...@gmail.com> wrote:
>>>>>>>>>>>> And I forgot to say: run gdb in some sort of environment where you see
>>>>>>>>>>>> your current source code line and a couple of surrounding lines. You
>>>>>>>>>>>> could achieve this with the "list" command, but I prefer running gdb in
>>>>>>>>>>>> emacs and let emacs do the nice listing of source code in a different panel.
>>>>>>>>>>>>
>>>>>>>>>>>> S
>>>>>>>>>>> Here's the function from my source. It's the original from mod_headers.c, plus my printf:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> static int header_post_config(apr_pool_t *pconf, apr_pool_t *plog,
>>>>>>>>>>> apr_pool_t *ptemp, server_rec *s)
>>>>>>>>>>> {
>>>>>>>>>>> printf("In header_post_config\n");
>>>>>>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>>>>> return OK;
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> Jim
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I was able to get the segfault to go away. Here's what I had to do:
>>>>>>>>>>
>>>>>>>>>> - Created /etc/ld.so.conf.d/my.conf, and added the directory where my libobaccess.so was
>>>>>>>>>> - Run 'ldconfig' to activate.
>>>>>>>>>> - In the apxs command, DON'T include the -L and -l arguments
>>>>>>>>>>
>>>>>>>>>> After that, Apache appears to start ok, without segfault :)!!
>>>>>>>>>>
>>>>>>>>>> Thanks for all of the great help, esp. the suggestion about checking "ldconfig -p". I still don't understand why, but I'm just glad that I can get past this piece so now I can debug my module :)...
>>>>>>>>>>
>>>>>>>>>> Later,
>>>>>>>>>> Jim
>>>>>>>>> I'm just glad this list is as good as it is!
>>>>>>>>>
>>>>>>>>> FYI, the ldconfig is the dynamic linker control, and those
>>>>>>>>> /etc/ld.so.conf.d files provide additional search directories for the
>>>>>>>>> linker to check in when loading a library.
>>>>>>>>>
>>>>>>>>> Joe
>>>>>>>>> --
>>>>>>>>> http://www.silverhawk.net/
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Sorry to report, but my earlier report was a "false positive" :)...
>>>>>>>>
>>>>>>>> I forgot that the mod_headers.c that I was doing the earlier testing with had all references to the libobaccess.so removed :(!!
>>>>>>>>
>>>>>>>> So, I'm still stuck with basically the same problem, now, working with my "full" code, with the calls in it:
>>>>>>>>
>>>>>>>> - If I compile with -L and -l, Apache segfaults when it starts
>>>>>>>>
>>>>>>>> - If I compile without -L and -l, then I get "undefined symbol" errors when I try to start Apache, e.g.:
>>>>>>>>
>>>>>>>> [root@apachemodule build-mod_headers]# /apps/httpd2222/bin/apachectl -k start -X
>>>>>>>> httpd: Syntax error on line 84 of /apps/httpd2222/conf/httpd.conf: Cannot load /apps/httpd2222/modules/mod_headers.so into server: /apps/httpd2222/modules/mod_headers.so: undefined symbol: ObResource_isProtected
>>>>>>>>
>>>>>>>> That "ObResource_isProtected" should be a symbol in libobaccess.so, and in fact, if I do "nm --dynamic", I get:
>>>>>>>>
>>>>>>>> [root@apachemodule build-mod_headers]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "ObResource_isProtected"
>>>>>>>> 00000000000a6d80 T ObResource_isProtected
>>>>>>>> [root@apachemodule build-mod_headers]#
>>>>>>>>
>>>>>>>>
>>>>>>>> I'm *assuming* that the reason for the "undefined symbol" error is that libobaccess.so is actually not being loaded, but then when I try to load libobaccess.so, either via -L and -l in the apxs, or using LoadFile in httpd.conf, I get the segfault (same gdb info, BTW).
>>>>>>>>
>>>>>>>> Catch-22?
>>>>>>>>
>>>>>>>> Sorry for the false alarm :(!!
>>>>>>>>
>>>>>>>> Jim
>>>>>>>>
>>>>>>>>
>>>>>>> Not a catch-22. The -L and -l specify linker options when assembling
>>>>>>> the code. The ldconfig is a run-time thing. If you are getting the
>>>>>>> stderr messages, you are making it all the way into your library. I'd
>>>>>>> suggest commenting out the following line and see if you get farther :
>>>>>>>
>>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>>
>>>>>>> That should tell you if the problem is the ssl_var_lookup.
>>>>>>>
>>>>>>> Joe
>>>>>>> --
>>>>>>> http://www.silverhawk.net/
>>>>>> Hi,
>>>>>>
>>>>>> Thanks for the suggestion. I just tried what you suggested, and got a segfault when I started Apache with the modified module.
>>>>>>
>>>>>> Jim
>>>>>>
>>>>>>
>>>>> Hi,
>>>>>
>>>>> As a reminder, here's the gdb with the library loaded:
>>>>>
>>>>>
>>>>> (gdb) b header_post_config
>>>>> Function "header_post_config" not defined.
>>>>> Make breakpoint pending on future shared library load? (y or [n]) y
>>>>> Breakpoint 1 (header_post_config) pending.
>>>>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
>>>>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
>>>>> [Thread debugging using libthread_db enabled]
>>>>> [New Thread 182897610272 (LWP 11317)]
>>>>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
>>>>> Pending breakpoint "header_post_config" resolved
>>>>> mod_headers-jl V0.09 - start calling OAM API
>>>>> In register_hooks
>>>>> In create_headers_dir_config
>>>>> In create_headers_dir_config
>>>>> In header_cmd
>>>>> In header_inout_cmd
>>>>> In parse_format_tag
>>>>> In parse_misc_string
>>>>> In create_headers_dir_config
>>>>> In header_cmd
>>>>> In header_inout_cmd
>>>>> In parse_format_tag
>>>>> In parse_misc_string
>>>>> [Switching to Thread 182897610272 (LWP 11317)]
>>>>>
>>>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>>>>> 1121 printf("In header_post_config\n");
>>>>> (gdb) n
>>>>> 1120 {
>>>>> (gdb) n
>>>>> 1121 printf("In header_post_config\n");
>>>>> (gdb) n
>>>>> In header_post_config
>>>>> 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>> (gdb) n
>>>>> 1124 }
>>>>> (gdb) n
>>>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
>>>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>>>>> (gdb) n
>>>>>
>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>>
>>>>>
>>>>> So, it's actually blowing up in "BN_num_bits()" in /lib64/libcrypto.so.4?
>>>>>
>>>>> Jim
>>>> When you see the :
>>>>
>>>> Program received signal SIGSEGV, Segmentation fault.
>>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>>>
>>>>
>>>> in gdb, type "bt" and hit enter. It should show the back trace and how
>>>> you got to the BN_num_bits() function.
>>>>
>>>> Joe
>>>> --
>>>> http://www.silverhawk.net/
>>>
>>> Hi,
>>>
>>> Here's the bt full:
>>>
>>>
>>> In header_inout_cmd
>>> In parse_format_tag
>>> In parse_misc_string
>>> [Switching to Thread 182897610272 (LWP 6676)]
>>>
>>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>>> 1121 printf("In header_post_config\n");
>>> (gdb) n
>>> 1120 {
>>> (gdb) n
>>> 1121 printf("In header_post_config\n");
>>> (gdb) n
>>> In header_post_config
>>> 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>> (gdb) n
>>> 1124 }
>>> (gdb) n
>>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
>>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>>> (gdb) n
>>>
>>> Program received signal SIGSEGV, Segmentation fault.
>>> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>> (gdb) bt full
>>> #0 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>>> No symbol table info available.
>>> #1 0x0000003518da8f4e in X509_ATTRIBUTE_create () from /lib64/libcrypto.so.4
>>> No symbol table info available.
>>> #2 0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
>>> No symbol table info available.
>>> #3 0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
>>> No symbol table info available.
>>> #4 0x0000003518dae0e1 in ASN1_item_ex_i2d () from /lib64/libcrypto.so.4
>>> No symbol table info available.
>>> #5 0x0000003518dae5f2 in ASN1_template_i2d () from /lib64/libcrypto.so.4
>>> No symbol table info available.
>>> #6 0x0000003518dae28e in ASN1_item_ex_i2d () from /lib64/libcrypto.so.4
>>> No symbol table info available.
>>> #7 0x0000003518dae3c6 in ASN1_item_i2d () from /lib64/libcrypto.so.4
>>> No symbol table info available.
>>> #8 0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8, p=0x5a72d8) at ssl_engine_pphrase.c:505
>>> mc = (SSLModConfigRec *) 0x571738
>>> sc = (SSLSrvConfigRec *) 0x668c38
>>> pServ = (server_rec *) 0x65fa48
>>> cpVHostID = 0x60add0 "www.example.com:443"
>>> szPath = "/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü×ÿ¿\177\000\000\000à×ÿ¿\177", '\0'<rep eats 15 times>, "\001\000\000\000°åÿ¿\177\000\000\000;\000\000\000+\000\000\000\020Ûÿ¿\177", '\0'<repeats 35 times>, "à\224k", ' ---Type<return> to continue, or q<return> to quit---
>>> \0'<repeats 13 times>, "øéÿ¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001", '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°åÿ¿\177", '\0'<repeats 11 times>, "\020Ûÿ¿\177\000\000\000+\000\000\000;", '\0'<repeats 35 times>...
>>> pPrivateKey = (EVP_PKEY *) 0x6ba670
>>> asn1 = Variable "asn1" is not available.
>>> (gdb)
>>>
>>>
>>> Not really sure what to make of that though :(...
>>>
>>> Jim
>>
>> Hi,
>>
>> Ok, something interesting. I noticed that the servername in the gdb output was example.whatever.com, so I changed ServerName in extras/httpd-ssl.conf to "apache1.whatever.com". Apache still segfaulted after that.
>>
>> I noticed that igdb was showing my router IP address for "mod_unique_id" (whatever that is), so I thought it was because I didn't have the hostname in /etc/hosts. So, I added "apache1.whatever.com" to /etc/hosts, and also turned Apache LogLevel to debug, and now I get a different error:
>>
>> [root@apachemodule dev]# /apps/httpd2222/bin/apachectl -k start -X
>> mod_headers-jl V0.09 - start calling OAM API
>> In register_hooks
>> In create_headers_dir_config
>> In create_headers_dir_config
>> In header_cmd
>> In header_inout_cmd
>> In parse_format_tag
>> In parse_misc_string
>> In create_headers_dir_config
>> In header_cmd
>> In header_inout_cmd
>> In parse_format_tag
>> In parse_misc_string
>> In header_post_config
>> *** glibc detected *** corrupted double-linked list: 0x00000000006b9710 ***
>> /apps/httpd2222/bin/apachectl: line 78: 7599 Aborted $HTTPD $ARGV
>> [root@apachemodule dev]#
>>
>>
>> Here's the Apache error_log:
>>
>> [Thu Jun 21 23:24:11 2012] [info] mod_unique_id: using ip addr 72.215.225.9
>> [Thu Jun 21 23:24:12 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
>> [Thu Jun 21 23:24:12 2012] [info] Loading certificate& private key of SSL-aware server
>> [Thu Jun 21 23:29:51 2012] [info] mod_unique_id: using ip addr 72.215.225.9
>> [Thu Jun 21 23:29:52 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
>> [Thu Jun 21 23:29:52 2012] [info] Loading certificate& private key of SSL-aware server
>> [Thu Jun 21 23:29:56 2012] [info] mod_unique_id: using ip addr 72.215.225.9
>> [Thu Jun 21 23:29:57 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
>> [Thu Jun 21 23:29:57 2012] [info] Loading certificate& private key of SSL-aware server
>> [root@apachemodule dev]#
>>
>>
>> and here's the gdb backtrace:
>>
>> (gdb) b header_post_config
>> Function "header_post_config" not defined.
>> Make breakpoint pending on future shared library load? (y or [n]) y
>> Breakpoint 1 (header_post_config) pending.
>> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
>> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
>> [Thread debugging using libthread_db enabled]
>> [New Thread 182897610272 (LWP 7644)]
>> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
>> Pending breakpoint "header_post_config" resolved
>> mod_headers-jl V0.09 - start calling OAM API
>> In register_hooks
>> In create_headers_dir_config
>> In create_headers_dir_config
>> In header_cmd
>> In header_inout_cmd
>> In parse_format_tag
>> In parse_misc_string
>> In create_headers_dir_config
>> In header_cmd
>> In header_inout_cmd
>> In parse_format_tag
>> In parse_misc_string
>> [Switching to Thread 182897610272 (LWP 7644)]
>>
>> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
>> 1121 printf("In header_post_config\n");
>> (gdb) n
>> 1120 {
>> (gdb) n
>> 1121 printf("In header_post_config\n");
>> (gdb) n
>> In header_post_config
>> 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>> (gdb) n
>> 1124 }
>> (gdb) n
>> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
>> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
>> (gdb) n
>> *** glibc detected *** corrupted double-linked list: 0x00000000006b9710 ***
>>
>> Program received signal SIGABRT, Aborted.
>> 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
>> (gdb) bt full
>> #0 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
>> No symbol table info available.
>> #1 0x000000351432fa6e in abort () from /lib64/tls/libc.so.6
>> No symbol table info available.
>> #2 0x0000003514363641 in __libc_message () from /lib64/tls/libc.so.6
>> No symbol table info available.
>> #3 0x0000003514369512 in _int_free () from /lib64/tls/libc.so.6
>> No symbol table info available.
>> #4 0x0000003514369846 in free () from /lib64/tls/libc.so.6
>> No symbol table info available.
>> #5 0x0000002a9790b6ba in R_free () from /apps/netpoint/lib64/libobaccess.so
>> No symbol table info available.
>> #6 0x0000002a9792bc41 in X509_CINF_free () from /apps/netpoint/lib64/libobaccess.so
>> No symbol table info available.
>> #7 0x0000002a9790ed98 in X509_free () from /apps/netpoint/lib64/libobaccess.so
>> No symbol table info available.
>> #8 0x0000002a987d97b6 in ssl_pphrase_Handle (s=0x59d3a8, p=0x5a72d8) at ssl_engine_pphrase.c:243
>> mc = (SSLModConfigRec *) 0x571738
>> sc = (SSLSrvConfigRec *) 0x668c38
>> pServ = (server_rec *) 0x65fa48
>> cpVHostID = 0x60add0 "apache1.whatever.com:443"
>> szPath = "/apps/httpd2222/conf/certs/apache1.whatever.com.crt\000\177\000\000\000Ü×ÿ¿\177\000\000\000à×ÿ¿\177", '\0'<repeats 15 times>, "\001\000\000\000°åÿ¿\177\000\000\000;\000\000\000+\000\000\000\020Ûÿ¿\177", '\0'<repeats 35 times>, "à\224k", '\0'<repeats 13 times>, "øéÿ¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001", '\0'<repeats 15 times>, "n\000\000\000\000\004\000\000°åÿ¿\177", '\0'<repeats 11 times>, "\020Ûÿ¿\177\000\000\000+\000\000\000;", '\0'<repeats 35 times>...
>> pPrivateKey = Variable "pPrivateKey" is not available.
>> (gdb)
>>
>>
>> Jim
>>
>
> I guess the thing that I'm puzzled about is why it goes from ssl_pphrase_Handle (which is part of Apache code?) to X509_free () in the libobaccess.so? Apache shouldn't even be aware of the libobaccess stuff yet (since I think that Apache is still trying to initialize SSL processing)?
>
> Jim
It shouldn't unless you've declared a shutdown hook or a clean up hook
of sorts. The looks of that stack trace look strikingly like an
exception handler firing off. What are all of the hooks you have running?
Joe
--
http://www.silverhawk.net/
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use
Apache module that uses shared library?
Posted by oh...@cox.net.
---- ohaya@cox.net wrote:
>
> ---- ohaya@cox.net wrote:
> >
> > ---- Joe Lewis <jo...@joe-lewis.com> wrote:
> > > On 6/21/12 7:32 PM, ohaya@cox.net wrote:
> > > > ---- ohaya@cox.net wrote:
> > > >> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> > > >>> On 6/21/12 6:46 PM, ohaya@cox.net wrote:
> > > >>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> > > >>>>> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
> > > >>>>>> ---- ohaya@cox.net wrote:
> > > >>>>>>> ---- Sorin Manolache<so...@gmail.com> wrote:
> > > >>>>>>>> And I forgot to say: run gdb in some sort of environment where you see
> > > >>>>>>>> your current source code line and a couple of surrounding lines. You
> > > >>>>>>>> could achieve this with the "list" command, but I prefer running gdb in
> > > >>>>>>>> emacs and let emacs do the nice listing of source code in a different panel.
> > > >>>>>>>>
> > > >>>>>>>> S
> > > >>>>>>> Here's the function from my source. It's the original from mod_headers.c, plus my printf:
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>> static int header_post_config(apr_pool_t *pconf, apr_pool_t *plog,
> > > >>>>>>> apr_pool_t *ptemp, server_rec *s)
> > > >>>>>>> {
> > > >>>>>>> printf("In header_post_config\n");
> > > >>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > > >>>>>>> return OK;
> > > >>>>>>> }
> > > >>>>>>>
> > > >>>>>>> Jim
> > > >>>>>> Hi,
> > > >>>>>>
> > > >>>>>> I was able to get the segfault to go away. Here's what I had to do:
> > > >>>>>>
> > > >>>>>> - Created /etc/ld.so.conf.d/my.conf, and added the directory where my libobaccess.so was
> > > >>>>>> - Run 'ldconfig' to activate.
> > > >>>>>> - In the apxs command, DON'T include the -L and -l arguments
> > > >>>>>>
> > > >>>>>> After that, Apache appears to start ok, without segfault :)!!
> > > >>>>>>
> > > >>>>>> Thanks for all of the great help, esp. the suggestion about checking "ldconfig -p". I still don't understand why, but I'm just glad that I can get past this piece so now I can debug my module :)...
> > > >>>>>>
> > > >>>>>> Later,
> > > >>>>>> Jim
> > > >>>>> I'm just glad this list is as good as it is!
> > > >>>>>
> > > >>>>> FYI, the ldconfig is the dynamic linker control, and those
> > > >>>>> /etc/ld.so.conf.d files provide additional search directories for the
> > > >>>>> linker to check in when loading a library.
> > > >>>>>
> > > >>>>> Joe
> > > >>>>> --
> > > >>>>> http://www.silverhawk.net/
> > > >>>> Hi,
> > > >>>>
> > > >>>> Sorry to report, but my earlier report was a "false positive" :)...
> > > >>>>
> > > >>>> I forgot that the mod_headers.c that I was doing the earlier testing with had all references to the libobaccess.so removed :(!!
> > > >>>>
> > > >>>> So, I'm still stuck with basically the same problem, now, working with my "full" code, with the calls in it:
> > > >>>>
> > > >>>> - If I compile with -L and -l, Apache segfaults when it starts
> > > >>>>
> > > >>>> - If I compile without -L and -l, then I get "undefined symbol" errors when I try to start Apache, e.g.:
> > > >>>>
> > > >>>> [root@apachemodule build-mod_headers]# /apps/httpd2222/bin/apachectl -k start -X
> > > >>>> httpd: Syntax error on line 84 of /apps/httpd2222/conf/httpd.conf: Cannot load /apps/httpd2222/modules/mod_headers.so into server: /apps/httpd2222/modules/mod_headers.so: undefined symbol: ObResource_isProtected
> > > >>>>
> > > >>>> That "ObResource_isProtected" should be a symbol in libobaccess.so, and in fact, if I do "nm --dynamic", I get:
> > > >>>>
> > > >>>> [root@apachemodule build-mod_headers]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "ObResource_isProtected"
> > > >>>> 00000000000a6d80 T ObResource_isProtected
> > > >>>> [root@apachemodule build-mod_headers]#
> > > >>>>
> > > >>>>
> > > >>>> I'm *assuming* that the reason for the "undefined symbol" error is that libobaccess.so is actually not being loaded, but then when I try to load libobaccess.so, either via -L and -l in the apxs, or using LoadFile in httpd.conf, I get the segfault (same gdb info, BTW).
> > > >>>>
> > > >>>> Catch-22?
> > > >>>>
> > > >>>> Sorry for the false alarm :(!!
> > > >>>>
> > > >>>> Jim
> > > >>>>
> > > >>>>
> > > >>> Not a catch-22. The -L and -l specify linker options when assembling
> > > >>> the code. The ldconfig is a run-time thing. If you are getting the
> > > >>> stderr messages, you are making it all the way into your library. I'd
> > > >>> suggest commenting out the following line and see if you get farther :
> > > >>>
> > > >>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > > >>>
> > > >>> That should tell you if the problem is the ssl_var_lookup.
> > > >>>
> > > >>> Joe
> > > >>> --
> > > >>> http://www.silverhawk.net/
> > > >>
> > > >> Hi,
> > > >>
> > > >> Thanks for the suggestion. I just tried what you suggested, and got a segfault when I started Apache with the modified module.
> > > >>
> > > >> Jim
> > > >>
> > > >>
> > > > Hi,
> > > >
> > > > As a reminder, here's the gdb with the library loaded:
> > > >
> > > >
> > > > (gdb) b header_post_config
> > > > Function "header_post_config" not defined.
> > > > Make breakpoint pending on future shared library load? (y or [n]) y
> > > > Breakpoint 1 (header_post_config) pending.
> > > > (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> > > > Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> > > > [Thread debugging using libthread_db enabled]
> > > > [New Thread 182897610272 (LWP 11317)]
> > > > Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> > > > Pending breakpoint "header_post_config" resolved
> > > > mod_headers-jl V0.09 - start calling OAM API
> > > > In register_hooks
> > > > In create_headers_dir_config
> > > > In create_headers_dir_config
> > > > In header_cmd
> > > > In header_inout_cmd
> > > > In parse_format_tag
> > > > In parse_misc_string
> > > > In create_headers_dir_config
> > > > In header_cmd
> > > > In header_inout_cmd
> > > > In parse_format_tag
> > > > In parse_misc_string
> > > > [Switching to Thread 182897610272 (LWP 11317)]
> > > >
> > > > Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> > > > 1121 printf("In header_post_config\n");
> > > > (gdb) n
> > > > 1120 {
> > > > (gdb) n
> > > > 1121 printf("In header_post_config\n");
> > > > (gdb) n
> > > > In header_post_config
> > > > 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > > > (gdb) n
> > > > 1124 }
> > > > (gdb) n
> > > > 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> > > > 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> > > > (gdb) n
> > > >
> > > > Program received signal SIGSEGV, Segmentation fault.
> > > > 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> > > >
> > > >
> > > > So, it's actually blowing up in "BN_num_bits()" in /lib64/libcrypto.so.4?
> > > >
> > > > Jim
> > > When you see the :
> > >
> > > Program received signal SIGSEGV, Segmentation fault.
> > > 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> > >
> > >
> > > in gdb, type "bt" and hit enter. It should show the back trace and how
> > > you got to the BN_num_bits() function.
> > >
> > > Joe
> > > --
> > > http://www.silverhawk.net/
> >
> >
> > Hi,
> >
> > Here's the bt full:
> >
> >
> > In header_inout_cmd
> > In parse_format_tag
> > In parse_misc_string
> > [Switching to Thread 182897610272 (LWP 6676)]
> >
> > Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> > 1121 printf("In header_post_config\n");
> > (gdb) n
> > 1120 {
> > (gdb) n
> > 1121 printf("In header_post_config\n");
> > (gdb) n
> > In header_post_config
> > 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > (gdb) n
> > 1124 }
> > (gdb) n
> > 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> > 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> > (gdb) n
> >
> > Program received signal SIGSEGV, Segmentation fault.
> > 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> > (gdb) bt full
> > #0 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> > No symbol table info available.
> > #1 0x0000003518da8f4e in X509_ATTRIBUTE_create () from /lib64/libcrypto.so.4
> > No symbol table info available.
> > #2 0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> > No symbol table info available.
> > #3 0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> > No symbol table info available.
> > #4 0x0000003518dae0e1 in ASN1_item_ex_i2d () from /lib64/libcrypto.so.4
> > No symbol table info available.
> > #5 0x0000003518dae5f2 in ASN1_template_i2d () from /lib64/libcrypto.so.4
> > No symbol table info available.
> > #6 0x0000003518dae28e in ASN1_item_ex_i2d () from /lib64/libcrypto.so.4
> > No symbol table info available.
> > #7 0x0000003518dae3c6 in ASN1_item_i2d () from /lib64/libcrypto.so.4
> > No symbol table info available.
> > #8 0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8, p=0x5a72d8) at ssl_engine_pphrase.c:505
> > mc = (SSLModConfigRec *) 0x571738
> > sc = (SSLSrvConfigRec *) 0x668c38
> > pServ = (server_rec *) 0x65fa48
> > cpVHostID = 0x60add0 "www.example.com:443"
> > szPath = "/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü×ÿ¿\177\000\000\000à×ÿ¿\177", '\0' <rep eats 15 times>, "\001\000\000\000°åÿ¿\177\000\000\000;\000\000\000+\000\000\000\020Ûÿ¿\177", '\0' <repeats 35 times>, "à\224k", ' ---Type <return> to continue, or q <return> to quit---
> > \0' <repeats 13 times>, "øéÿ¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001", '\0' <repeats 15 times>, "n\000\000\000\000\004\000\000°åÿ¿\177", '\0' <repeats 11 times>, "\020Ûÿ¿\177\000\000\000+\000\000\000;", '\0' <repeats 35 times>...
> > pPrivateKey = (EVP_PKEY *) 0x6ba670
> > asn1 = Variable "asn1" is not available.
> > (gdb)
> >
> >
> > Not really sure what to make of that though :(...
> >
> > Jim
>
>
> Hi,
>
> Ok, something interesting. I noticed that the servername in the gdb output was example.whatever.com, so I changed ServerName in extras/httpd-ssl.conf to "apache1.whatever.com". Apache still segfaulted after that.
>
> I noticed that igdb was showing my router IP address for "mod_unique_id" (whatever that is), so I thought it was because I didn't have the hostname in /etc/hosts. So, I added "apache1.whatever.com" to /etc/hosts, and also turned Apache LogLevel to debug, and now I get a different error:
>
> [root@apachemodule dev]# /apps/httpd2222/bin/apachectl -k start -X
> mod_headers-jl V0.09 - start calling OAM API
> In register_hooks
> In create_headers_dir_config
> In create_headers_dir_config
> In header_cmd
> In header_inout_cmd
> In parse_format_tag
> In parse_misc_string
> In create_headers_dir_config
> In header_cmd
> In header_inout_cmd
> In parse_format_tag
> In parse_misc_string
> In header_post_config
> *** glibc detected *** corrupted double-linked list: 0x00000000006b9710 ***
> /apps/httpd2222/bin/apachectl: line 78: 7599 Aborted $HTTPD $ARGV
> [root@apachemodule dev]#
>
>
> Here's the Apache error_log:
>
> [Thu Jun 21 23:24:11 2012] [info] mod_unique_id: using ip addr 72.215.225.9
> [Thu Jun 21 23:24:12 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
> [Thu Jun 21 23:24:12 2012] [info] Loading certificate & private key of SSL-aware server
> [Thu Jun 21 23:29:51 2012] [info] mod_unique_id: using ip addr 72.215.225.9
> [Thu Jun 21 23:29:52 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
> [Thu Jun 21 23:29:52 2012] [info] Loading certificate & private key of SSL-aware server
> [Thu Jun 21 23:29:56 2012] [info] mod_unique_id: using ip addr 72.215.225.9
> [Thu Jun 21 23:29:57 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
> [Thu Jun 21 23:29:57 2012] [info] Loading certificate & private key of SSL-aware server
> [root@apachemodule dev]#
>
>
> and here's the gdb backtrace:
>
> (gdb) b header_post_config
> Function "header_post_config" not defined.
> Make breakpoint pending on future shared library load? (y or [n]) y
> Breakpoint 1 (header_post_config) pending.
> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> [Thread debugging using libthread_db enabled]
> [New Thread 182897610272 (LWP 7644)]
> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> Pending breakpoint "header_post_config" resolved
> mod_headers-jl V0.09 - start calling OAM API
> In register_hooks
> In create_headers_dir_config
> In create_headers_dir_config
> In header_cmd
> In header_inout_cmd
> In parse_format_tag
> In parse_misc_string
> In create_headers_dir_config
> In header_cmd
> In header_inout_cmd
> In parse_format_tag
> In parse_misc_string
> [Switching to Thread 182897610272 (LWP 7644)]
>
> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> 1121 printf("In header_post_config\n");
> (gdb) n
> 1120 {
> (gdb) n
> 1121 printf("In header_post_config\n");
> (gdb) n
> In header_post_config
> 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> (gdb) n
> 1124 }
> (gdb) n
> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> (gdb) n
> *** glibc detected *** corrupted double-linked list: 0x00000000006b9710 ***
>
> Program received signal SIGABRT, Aborted.
> 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> (gdb) bt full
> #0 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
> No symbol table info available.
> #1 0x000000351432fa6e in abort () from /lib64/tls/libc.so.6
> No symbol table info available.
> #2 0x0000003514363641 in __libc_message () from /lib64/tls/libc.so.6
> No symbol table info available.
> #3 0x0000003514369512 in _int_free () from /lib64/tls/libc.so.6
> No symbol table info available.
> #4 0x0000003514369846 in free () from /lib64/tls/libc.so.6
> No symbol table info available.
> #5 0x0000002a9790b6ba in R_free () from /apps/netpoint/lib64/libobaccess.so
> No symbol table info available.
> #6 0x0000002a9792bc41 in X509_CINF_free () from /apps/netpoint/lib64/libobaccess.so
> No symbol table info available.
> #7 0x0000002a9790ed98 in X509_free () from /apps/netpoint/lib64/libobaccess.so
> No symbol table info available.
> #8 0x0000002a987d97b6 in ssl_pphrase_Handle (s=0x59d3a8, p=0x5a72d8) at ssl_engine_pphrase.c:243
> mc = (SSLModConfigRec *) 0x571738
> sc = (SSLSrvConfigRec *) 0x668c38
> pServ = (server_rec *) 0x65fa48
> cpVHostID = 0x60add0 "apache1.whatever.com:443"
> szPath = "/apps/httpd2222/conf/certs/apache1.whatever.com.crt\000\177\000\000\000Ü×ÿ¿\177\000\000\000à×ÿ¿\177", '\0' <repeats 15 times>, "\001\000\000\000°åÿ¿\177\000\000\000;\000\000\000+\000\000\000\020Ûÿ¿\177", '\0' <repeats 35 times>, "à\224k", '\0' <repeats 13 times>, "øéÿ¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001", '\0' <repeats 15 times>, "n\000\000\000\000\004\000\000°åÿ¿\177", '\0' <repeats 11 times>, "\020Ûÿ¿\177\000\000\000+\000\000\000;", '\0' <repeats 35 times>...
> pPrivateKey = Variable "pPrivateKey" is not available.
> (gdb)
>
>
> Jim
>
I guess the thing that I'm puzzled about is why it goes from ssl_pphrase_Handle (which is part of Apache code?) to X509_free () in the libobaccess.so? Apache shouldn't even be aware of the libobaccess stuff yet (since I think that Apache is still trying to initialize SSL processing)?
Jim
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use
Apache module that uses shared library?
Posted by oh...@cox.net.
---- ohaya@cox.net wrote:
>
> ---- Joe Lewis <jo...@joe-lewis.com> wrote:
> > On 6/21/12 7:32 PM, ohaya@cox.net wrote:
> > > ---- ohaya@cox.net wrote:
> > >> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> > >>> On 6/21/12 6:46 PM, ohaya@cox.net wrote:
> > >>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> > >>>>> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
> > >>>>>> ---- ohaya@cox.net wrote:
> > >>>>>>> ---- Sorin Manolache<so...@gmail.com> wrote:
> > >>>>>>>> And I forgot to say: run gdb in some sort of environment where you see
> > >>>>>>>> your current source code line and a couple of surrounding lines. You
> > >>>>>>>> could achieve this with the "list" command, but I prefer running gdb in
> > >>>>>>>> emacs and let emacs do the nice listing of source code in a different panel.
> > >>>>>>>>
> > >>>>>>>> S
> > >>>>>>> Here's the function from my source. It's the original from mod_headers.c, plus my printf:
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> static int header_post_config(apr_pool_t *pconf, apr_pool_t *plog,
> > >>>>>>> apr_pool_t *ptemp, server_rec *s)
> > >>>>>>> {
> > >>>>>>> printf("In header_post_config\n");
> > >>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > >>>>>>> return OK;
> > >>>>>>> }
> > >>>>>>>
> > >>>>>>> Jim
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>> I was able to get the segfault to go away. Here's what I had to do:
> > >>>>>>
> > >>>>>> - Created /etc/ld.so.conf.d/my.conf, and added the directory where my libobaccess.so was
> > >>>>>> - Run 'ldconfig' to activate.
> > >>>>>> - In the apxs command, DON'T include the -L and -l arguments
> > >>>>>>
> > >>>>>> After that, Apache appears to start ok, without segfault :)!!
> > >>>>>>
> > >>>>>> Thanks for all of the great help, esp. the suggestion about checking "ldconfig -p". I still don't understand why, but I'm just glad that I can get past this piece so now I can debug my module :)...
> > >>>>>>
> > >>>>>> Later,
> > >>>>>> Jim
> > >>>>> I'm just glad this list is as good as it is!
> > >>>>>
> > >>>>> FYI, the ldconfig is the dynamic linker control, and those
> > >>>>> /etc/ld.so.conf.d files provide additional search directories for the
> > >>>>> linker to check in when loading a library.
> > >>>>>
> > >>>>> Joe
> > >>>>> --
> > >>>>> http://www.silverhawk.net/
> > >>>> Hi,
> > >>>>
> > >>>> Sorry to report, but my earlier report was a "false positive" :)...
> > >>>>
> > >>>> I forgot that the mod_headers.c that I was doing the earlier testing with had all references to the libobaccess.so removed :(!!
> > >>>>
> > >>>> So, I'm still stuck with basically the same problem, now, working with my "full" code, with the calls in it:
> > >>>>
> > >>>> - If I compile with -L and -l, Apache segfaults when it starts
> > >>>>
> > >>>> - If I compile without -L and -l, then I get "undefined symbol" errors when I try to start Apache, e.g.:
> > >>>>
> > >>>> [root@apachemodule build-mod_headers]# /apps/httpd2222/bin/apachectl -k start -X
> > >>>> httpd: Syntax error on line 84 of /apps/httpd2222/conf/httpd.conf: Cannot load /apps/httpd2222/modules/mod_headers.so into server: /apps/httpd2222/modules/mod_headers.so: undefined symbol: ObResource_isProtected
> > >>>>
> > >>>> That "ObResource_isProtected" should be a symbol in libobaccess.so, and in fact, if I do "nm --dynamic", I get:
> > >>>>
> > >>>> [root@apachemodule build-mod_headers]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "ObResource_isProtected"
> > >>>> 00000000000a6d80 T ObResource_isProtected
> > >>>> [root@apachemodule build-mod_headers]#
> > >>>>
> > >>>>
> > >>>> I'm *assuming* that the reason for the "undefined symbol" error is that libobaccess.so is actually not being loaded, but then when I try to load libobaccess.so, either via -L and -l in the apxs, or using LoadFile in httpd.conf, I get the segfault (same gdb info, BTW).
> > >>>>
> > >>>> Catch-22?
> > >>>>
> > >>>> Sorry for the false alarm :(!!
> > >>>>
> > >>>> Jim
> > >>>>
> > >>>>
> > >>> Not a catch-22. The -L and -l specify linker options when assembling
> > >>> the code. The ldconfig is a run-time thing. If you are getting the
> > >>> stderr messages, you are making it all the way into your library. I'd
> > >>> suggest commenting out the following line and see if you get farther :
> > >>>
> > >>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > >>>
> > >>> That should tell you if the problem is the ssl_var_lookup.
> > >>>
> > >>> Joe
> > >>> --
> > >>> http://www.silverhawk.net/
> > >>
> > >> Hi,
> > >>
> > >> Thanks for the suggestion. I just tried what you suggested, and got a segfault when I started Apache with the modified module.
> > >>
> > >> Jim
> > >>
> > >>
> > > Hi,
> > >
> > > As a reminder, here's the gdb with the library loaded:
> > >
> > >
> > > (gdb) b header_post_config
> > > Function "header_post_config" not defined.
> > > Make breakpoint pending on future shared library load? (y or [n]) y
> > > Breakpoint 1 (header_post_config) pending.
> > > (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> > > Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> > > [Thread debugging using libthread_db enabled]
> > > [New Thread 182897610272 (LWP 11317)]
> > > Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> > > Pending breakpoint "header_post_config" resolved
> > > mod_headers-jl V0.09 - start calling OAM API
> > > In register_hooks
> > > In create_headers_dir_config
> > > In create_headers_dir_config
> > > In header_cmd
> > > In header_inout_cmd
> > > In parse_format_tag
> > > In parse_misc_string
> > > In create_headers_dir_config
> > > In header_cmd
> > > In header_inout_cmd
> > > In parse_format_tag
> > > In parse_misc_string
> > > [Switching to Thread 182897610272 (LWP 11317)]
> > >
> > > Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> > > 1121 printf("In header_post_config\n");
> > > (gdb) n
> > > 1120 {
> > > (gdb) n
> > > 1121 printf("In header_post_config\n");
> > > (gdb) n
> > > In header_post_config
> > > 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > > (gdb) n
> > > 1124 }
> > > (gdb) n
> > > 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> > > 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> > > (gdb) n
> > >
> > > Program received signal SIGSEGV, Segmentation fault.
> > > 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> > >
> > >
> > > So, it's actually blowing up in "BN_num_bits()" in /lib64/libcrypto.so.4?
> > >
> > > Jim
> > When you see the :
> >
> > Program received signal SIGSEGV, Segmentation fault.
> > 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >
> >
> > in gdb, type "bt" and hit enter. It should show the back trace and how
> > you got to the BN_num_bits() function.
> >
> > Joe
> > --
> > http://www.silverhawk.net/
>
>
> Hi,
>
> Here's the bt full:
>
>
> In header_inout_cmd
> In parse_format_tag
> In parse_misc_string
> [Switching to Thread 182897610272 (LWP 6676)]
>
> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> 1121 printf("In header_post_config\n");
> (gdb) n
> 1120 {
> (gdb) n
> 1121 printf("In header_post_config\n");
> (gdb) n
> In header_post_config
> 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> (gdb) n
> 1124 }
> (gdb) n
> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> (gdb) n
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> (gdb) bt full
> #0 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> No symbol table info available.
> #1 0x0000003518da8f4e in X509_ATTRIBUTE_create () from /lib64/libcrypto.so.4
> No symbol table info available.
> #2 0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> No symbol table info available.
> #3 0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
> No symbol table info available.
> #4 0x0000003518dae0e1 in ASN1_item_ex_i2d () from /lib64/libcrypto.so.4
> No symbol table info available.
> #5 0x0000003518dae5f2 in ASN1_template_i2d () from /lib64/libcrypto.so.4
> No symbol table info available.
> #6 0x0000003518dae28e in ASN1_item_ex_i2d () from /lib64/libcrypto.so.4
> No symbol table info available.
> #7 0x0000003518dae3c6 in ASN1_item_i2d () from /lib64/libcrypto.so.4
> No symbol table info available.
> #8 0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8, p=0x5a72d8) at ssl_engine_pphrase.c:505
> mc = (SSLModConfigRec *) 0x571738
> sc = (SSLSrvConfigRec *) 0x668c38
> pServ = (server_rec *) 0x65fa48
> cpVHostID = 0x60add0 "www.example.com:443"
> szPath = "/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü×ÿ¿\177\000\000\000à×ÿ¿\177", '\0' <rep eats 15 times>, "\001\000\000\000°åÿ¿\177\000\000\000;\000\000\000+\000\000\000\020Ûÿ¿\177", '\0' <repeats 35 times>, "à\224k", ' ---Type <return> to continue, or q <return> to quit---
> \0' <repeats 13 times>, "øéÿ¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001", '\0' <repeats 15 times>, "n\000\000\000\000\004\000\000°åÿ¿\177", '\0' <repeats 11 times>, "\020Ûÿ¿\177\000\000\000+\000\000\000;", '\0' <repeats 35 times>...
> pPrivateKey = (EVP_PKEY *) 0x6ba670
> asn1 = Variable "asn1" is not available.
> (gdb)
>
>
> Not really sure what to make of that though :(...
>
> Jim
Hi,
Ok, something interesting. I noticed that the servername in the gdb output was example.whatever.com, so I changed ServerName in extras/httpd-ssl.conf to "apache1.whatever.com". Apache still segfaulted after that.
I noticed that igdb was showing my router IP address for "mod_unique_id" (whatever that is), so I thought it was because I didn't have the hostname in /etc/hosts. So, I added "apache1.whatever.com" to /etc/hosts, and also turned Apache LogLevel to debug, and now I get a different error:
[root@apachemodule dev]# /apps/httpd2222/bin/apachectl -k start -X
mod_headers-jl V0.09 - start calling OAM API
In register_hooks
In create_headers_dir_config
In create_headers_dir_config
In header_cmd
In header_inout_cmd
In parse_format_tag
In parse_misc_string
In create_headers_dir_config
In header_cmd
In header_inout_cmd
In parse_format_tag
In parse_misc_string
In header_post_config
*** glibc detected *** corrupted double-linked list: 0x00000000006b9710 ***
/apps/httpd2222/bin/apachectl: line 78: 7599 Aborted $HTTPD $ARGV
[root@apachemodule dev]#
Here's the Apache error_log:
[Thu Jun 21 23:24:11 2012] [info] mod_unique_id: using ip addr 72.215.225.9
[Thu Jun 21 23:24:12 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
[Thu Jun 21 23:24:12 2012] [info] Loading certificate & private key of SSL-aware server
[Thu Jun 21 23:29:51 2012] [info] mod_unique_id: using ip addr 72.215.225.9
[Thu Jun 21 23:29:52 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
[Thu Jun 21 23:29:52 2012] [info] Loading certificate & private key of SSL-aware server
[Thu Jun 21 23:29:56 2012] [info] mod_unique_id: using ip addr 72.215.225.9
[Thu Jun 21 23:29:57 2012] [info] Init: Seeding PRNG with 144 bytes of entropy
[Thu Jun 21 23:29:57 2012] [info] Loading certificate & private key of SSL-aware server
[root@apachemodule dev]#
and here's the gdb backtrace:
(gdb) b header_post_config
Function "header_post_config" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (header_post_config) pending.
(gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
[Thread debugging using libthread_db enabled]
[New Thread 182897610272 (LWP 7644)]
Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
Pending breakpoint "header_post_config" resolved
mod_headers-jl V0.09 - start calling OAM API
In register_hooks
In create_headers_dir_config
In create_headers_dir_config
In header_cmd
In header_inout_cmd
In parse_format_tag
In parse_misc_string
In create_headers_dir_config
In header_cmd
In header_inout_cmd
In parse_format_tag
In parse_misc_string
[Switching to Thread 182897610272 (LWP 7644)]
Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
1121 printf("In header_post_config\n");
(gdb) n
1120 {
(gdb) n
1121 printf("In header_post_config\n");
(gdb) n
In header_post_config
1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
(gdb) n
1124 }
(gdb) n
0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
(gdb) n
*** glibc detected *** corrupted double-linked list: 0x00000000006b9710 ***
Program received signal SIGABRT, Aborted.
0x000000351432e26d in raise () from /lib64/tls/libc.so.6
(gdb) bt full
#0 0x000000351432e26d in raise () from /lib64/tls/libc.so.6
No symbol table info available.
#1 0x000000351432fa6e in abort () from /lib64/tls/libc.so.6
No symbol table info available.
#2 0x0000003514363641 in __libc_message () from /lib64/tls/libc.so.6
No symbol table info available.
#3 0x0000003514369512 in _int_free () from /lib64/tls/libc.so.6
No symbol table info available.
#4 0x0000003514369846 in free () from /lib64/tls/libc.so.6
No symbol table info available.
#5 0x0000002a9790b6ba in R_free () from /apps/netpoint/lib64/libobaccess.so
No symbol table info available.
#6 0x0000002a9792bc41 in X509_CINF_free () from /apps/netpoint/lib64/libobaccess.so
No symbol table info available.
#7 0x0000002a9790ed98 in X509_free () from /apps/netpoint/lib64/libobaccess.so
No symbol table info available.
#8 0x0000002a987d97b6 in ssl_pphrase_Handle (s=0x59d3a8, p=0x5a72d8) at ssl_engine_pphrase.c:243
mc = (SSLModConfigRec *) 0x571738
sc = (SSLSrvConfigRec *) 0x668c38
pServ = (server_rec *) 0x65fa48
cpVHostID = 0x60add0 "apache1.whatever.com:443"
szPath = "/apps/httpd2222/conf/certs/apache1.whatever.com.crt\000\177\000\000\000Ü×ÿ¿\177\000\000\000à×ÿ¿\177", '\0' <repeats 15 times>, "\001\000\000\000°åÿ¿\177\000\000\000;\000\000\000+\000\000\000\020Ûÿ¿\177", '\0' <repeats 35 times>, "à\224k", '\0' <repeats 13 times>, "øéÿ¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001", '\0' <repeats 15 times>, "n\000\000\000\000\004\000\000°åÿ¿\177", '\0' <repeats 11 times>, "\020Ûÿ¿\177\000\000\000+\000\000\000;", '\0' <repeats 35 times>...
pPrivateKey = Variable "pPrivateKey" is not available.
(gdb)
Jim
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use
Apache module that uses shared library?
Posted by oh...@cox.net.
---- Joe Lewis <jo...@joe-lewis.com> wrote:
> On 6/21/12 7:32 PM, ohaya@cox.net wrote:
> > ---- ohaya@cox.net wrote:
> >> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >>> On 6/21/12 6:46 PM, ohaya@cox.net wrote:
> >>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >>>>> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
> >>>>>> ---- ohaya@cox.net wrote:
> >>>>>>> ---- Sorin Manolache<so...@gmail.com> wrote:
> >>>>>>>> And I forgot to say: run gdb in some sort of environment where you see
> >>>>>>>> your current source code line and a couple of surrounding lines. You
> >>>>>>>> could achieve this with the "list" command, but I prefer running gdb in
> >>>>>>>> emacs and let emacs do the nice listing of source code in a different panel.
> >>>>>>>>
> >>>>>>>> S
> >>>>>>> Here's the function from my source. It's the original from mod_headers.c, plus my printf:
> >>>>>>>
> >>>>>>>
> >>>>>>> static int header_post_config(apr_pool_t *pconf, apr_pool_t *plog,
> >>>>>>> apr_pool_t *ptemp, server_rec *s)
> >>>>>>> {
> >>>>>>> printf("In header_post_config\n");
> >>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>>>>> return OK;
> >>>>>>> }
> >>>>>>>
> >>>>>>> Jim
> >>>>>> Hi,
> >>>>>>
> >>>>>> I was able to get the segfault to go away. Here's what I had to do:
> >>>>>>
> >>>>>> - Created /etc/ld.so.conf.d/my.conf, and added the directory where my libobaccess.so was
> >>>>>> - Run 'ldconfig' to activate.
> >>>>>> - In the apxs command, DON'T include the -L and -l arguments
> >>>>>>
> >>>>>> After that, Apache appears to start ok, without segfault :)!!
> >>>>>>
> >>>>>> Thanks for all of the great help, esp. the suggestion about checking "ldconfig -p". I still don't understand why, but I'm just glad that I can get past this piece so now I can debug my module :)...
> >>>>>>
> >>>>>> Later,
> >>>>>> Jim
> >>>>> I'm just glad this list is as good as it is!
> >>>>>
> >>>>> FYI, the ldconfig is the dynamic linker control, and those
> >>>>> /etc/ld.so.conf.d files provide additional search directories for the
> >>>>> linker to check in when loading a library.
> >>>>>
> >>>>> Joe
> >>>>> --
> >>>>> http://www.silverhawk.net/
> >>>> Hi,
> >>>>
> >>>> Sorry to report, but my earlier report was a "false positive" :)...
> >>>>
> >>>> I forgot that the mod_headers.c that I was doing the earlier testing with had all references to the libobaccess.so removed :(!!
> >>>>
> >>>> So, I'm still stuck with basically the same problem, now, working with my "full" code, with the calls in it:
> >>>>
> >>>> - If I compile with -L and -l, Apache segfaults when it starts
> >>>>
> >>>> - If I compile without -L and -l, then I get "undefined symbol" errors when I try to start Apache, e.g.:
> >>>>
> >>>> [root@apachemodule build-mod_headers]# /apps/httpd2222/bin/apachectl -k start -X
> >>>> httpd: Syntax error on line 84 of /apps/httpd2222/conf/httpd.conf: Cannot load /apps/httpd2222/modules/mod_headers.so into server: /apps/httpd2222/modules/mod_headers.so: undefined symbol: ObResource_isProtected
> >>>>
> >>>> That "ObResource_isProtected" should be a symbol in libobaccess.so, and in fact, if I do "nm --dynamic", I get:
> >>>>
> >>>> [root@apachemodule build-mod_headers]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "ObResource_isProtected"
> >>>> 00000000000a6d80 T ObResource_isProtected
> >>>> [root@apachemodule build-mod_headers]#
> >>>>
> >>>>
> >>>> I'm *assuming* that the reason for the "undefined symbol" error is that libobaccess.so is actually not being loaded, but then when I try to load libobaccess.so, either via -L and -l in the apxs, or using LoadFile in httpd.conf, I get the segfault (same gdb info, BTW).
> >>>>
> >>>> Catch-22?
> >>>>
> >>>> Sorry for the false alarm :(!!
> >>>>
> >>>> Jim
> >>>>
> >>>>
> >>> Not a catch-22. The -L and -l specify linker options when assembling
> >>> the code. The ldconfig is a run-time thing. If you are getting the
> >>> stderr messages, you are making it all the way into your library. I'd
> >>> suggest commenting out the following line and see if you get farther :
> >>>
> >>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>
> >>> That should tell you if the problem is the ssl_var_lookup.
> >>>
> >>> Joe
> >>> --
> >>> http://www.silverhawk.net/
> >>
> >> Hi,
> >>
> >> Thanks for the suggestion. I just tried what you suggested, and got a segfault when I started Apache with the modified module.
> >>
> >> Jim
> >>
> >>
> > Hi,
> >
> > As a reminder, here's the gdb with the library loaded:
> >
> >
> > (gdb) b header_post_config
> > Function "header_post_config" not defined.
> > Make breakpoint pending on future shared library load? (y or [n]) y
> > Breakpoint 1 (header_post_config) pending.
> > (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> > Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> > [Thread debugging using libthread_db enabled]
> > [New Thread 182897610272 (LWP 11317)]
> > Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> > Pending breakpoint "header_post_config" resolved
> > mod_headers-jl V0.09 - start calling OAM API
> > In register_hooks
> > In create_headers_dir_config
> > In create_headers_dir_config
> > In header_cmd
> > In header_inout_cmd
> > In parse_format_tag
> > In parse_misc_string
> > In create_headers_dir_config
> > In header_cmd
> > In header_inout_cmd
> > In parse_format_tag
> > In parse_misc_string
> > [Switching to Thread 182897610272 (LWP 11317)]
> >
> > Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> > 1121 printf("In header_post_config\n");
> > (gdb) n
> > 1120 {
> > (gdb) n
> > 1121 printf("In header_post_config\n");
> > (gdb) n
> > In header_post_config
> > 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > (gdb) n
> > 1124 }
> > (gdb) n
> > 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> > 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> > (gdb) n
> >
> > Program received signal SIGSEGV, Segmentation fault.
> > 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
> >
> >
> > So, it's actually blowing up in "BN_num_bits()" in /lib64/libcrypto.so.4?
> >
> > Jim
> When you see the :
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>
>
> in gdb, type "bt" and hit enter. It should show the back trace and how
> you got to the BN_num_bits() function.
>
> Joe
> --
> http://www.silverhawk.net/
Hi,
Here's the bt full:
In header_inout_cmd
In parse_format_tag
In parse_misc_string
[Switching to Thread 182897610272 (LWP 6676)]
Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
1121 printf("In header_post_config\n");
(gdb) n
1120 {
(gdb) n
1121 printf("In header_post_config\n");
(gdb) n
In header_post_config
1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
(gdb) n
1124 }
(gdb) n
0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
(gdb) n
Program received signal SIGSEGV, Segmentation fault.
0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
(gdb) bt full
#0 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
No symbol table info available.
#1 0x0000003518da8f4e in X509_ATTRIBUTE_create () from /lib64/libcrypto.so.4
No symbol table info available.
#2 0x0000003518dadea2 in asn1_ex_i2c () from /lib64/libcrypto.so.4
No symbol table info available.
#3 0x0000003518dadf79 in asn1_ex_i2c () from /lib64/libcrypto.so.4
No symbol table info available.
#4 0x0000003518dae0e1 in ASN1_item_ex_i2d () from /lib64/libcrypto.so.4
No symbol table info available.
#5 0x0000003518dae5f2 in ASN1_template_i2d () from /lib64/libcrypto.so.4
No symbol table info available.
#6 0x0000003518dae28e in ASN1_item_ex_i2d () from /lib64/libcrypto.so.4
No symbol table info available.
#7 0x0000003518dae3c6 in ASN1_item_i2d () from /lib64/libcrypto.so.4
No symbol table info available.
#8 0x0000002a987d9d3a in ssl_pphrase_Handle (s=0x59d3a8, p=0x5a72d8) at ssl_engine_pphrase.c:505
mc = (SSLModConfigRec *) 0x571738
sc = (SSLSrvConfigRec *) 0x668c38
pServ = (server_rec *) 0x65fa48
cpVHostID = 0x60add0 "www.example.com:443"
szPath = "/apps/httpd2222/conf/certs/apache1.whatever.com.key\000\177\000\000\000Ü×ÿ¿\177\000\000\000à×ÿ¿\177", '\0' <rep eats 15 times>, "\001\000\000\000°åÿ¿\177\000\000\000;\000\000\000+\000\000\000\020Ûÿ¿\177", '\0' <repeats 35 times>, "à\224k", ' ---Type <return> to continue, or q <return> to quit---
\0' <repeats 13 times>, "øéÿ¿\177\000\000\000\020\000\000\000;\000\000\000\016\000\000\000\001\000\001\000\001", '\0' <repeats 15 times>, "n\000\000\000\000\004\000\000°åÿ¿\177", '\0' <repeats 11 times>, "\020Ûÿ¿\177\000\000\000+\000\000\000;", '\0' <repeats 35 times>...
pPrivateKey = (EVP_PKEY *) 0x6ba670
asn1 = Variable "asn1" is not available.
(gdb)
Not really sure what to make of that though :(...
Jim
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use Apache
module that uses shared library?
Posted by Joe Lewis <jo...@joe-lewis.com>.
On 6/21/12 7:32 PM, ohaya@cox.net wrote:
> ---- ohaya@cox.net wrote:
>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>> On 6/21/12 6:46 PM, ohaya@cox.net wrote:
>>>> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>>>>> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
>>>>>> ---- ohaya@cox.net wrote:
>>>>>>> ---- Sorin Manolache<so...@gmail.com> wrote:
>>>>>>>> And I forgot to say: run gdb in some sort of environment where you see
>>>>>>>> your current source code line and a couple of surrounding lines. You
>>>>>>>> could achieve this with the "list" command, but I prefer running gdb in
>>>>>>>> emacs and let emacs do the nice listing of source code in a different panel.
>>>>>>>>
>>>>>>>> S
>>>>>>> Here's the function from my source. It's the original from mod_headers.c, plus my printf:
>>>>>>>
>>>>>>>
>>>>>>> static int header_post_config(apr_pool_t *pconf, apr_pool_t *plog,
>>>>>>> apr_pool_t *ptemp, server_rec *s)
>>>>>>> {
>>>>>>> printf("In header_post_config\n");
>>>>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>>>>> return OK;
>>>>>>> }
>>>>>>>
>>>>>>> Jim
>>>>>> Hi,
>>>>>>
>>>>>> I was able to get the segfault to go away. Here's what I had to do:
>>>>>>
>>>>>> - Created /etc/ld.so.conf.d/my.conf, and added the directory where my libobaccess.so was
>>>>>> - Run 'ldconfig' to activate.
>>>>>> - In the apxs command, DON'T include the -L and -l arguments
>>>>>>
>>>>>> After that, Apache appears to start ok, without segfault :)!!
>>>>>>
>>>>>> Thanks for all of the great help, esp. the suggestion about checking "ldconfig -p". I still don't understand why, but I'm just glad that I can get past this piece so now I can debug my module :)...
>>>>>>
>>>>>> Later,
>>>>>> Jim
>>>>> I'm just glad this list is as good as it is!
>>>>>
>>>>> FYI, the ldconfig is the dynamic linker control, and those
>>>>> /etc/ld.so.conf.d files provide additional search directories for the
>>>>> linker to check in when loading a library.
>>>>>
>>>>> Joe
>>>>> --
>>>>> http://www.silverhawk.net/
>>>> Hi,
>>>>
>>>> Sorry to report, but my earlier report was a "false positive" :)...
>>>>
>>>> I forgot that the mod_headers.c that I was doing the earlier testing with had all references to the libobaccess.so removed :(!!
>>>>
>>>> So, I'm still stuck with basically the same problem, now, working with my "full" code, with the calls in it:
>>>>
>>>> - If I compile with -L and -l, Apache segfaults when it starts
>>>>
>>>> - If I compile without -L and -l, then I get "undefined symbol" errors when I try to start Apache, e.g.:
>>>>
>>>> [root@apachemodule build-mod_headers]# /apps/httpd2222/bin/apachectl -k start -X
>>>> httpd: Syntax error on line 84 of /apps/httpd2222/conf/httpd.conf: Cannot load /apps/httpd2222/modules/mod_headers.so into server: /apps/httpd2222/modules/mod_headers.so: undefined symbol: ObResource_isProtected
>>>>
>>>> That "ObResource_isProtected" should be a symbol in libobaccess.so, and in fact, if I do "nm --dynamic", I get:
>>>>
>>>> [root@apachemodule build-mod_headers]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "ObResource_isProtected"
>>>> 00000000000a6d80 T ObResource_isProtected
>>>> [root@apachemodule build-mod_headers]#
>>>>
>>>>
>>>> I'm *assuming* that the reason for the "undefined symbol" error is that libobaccess.so is actually not being loaded, but then when I try to load libobaccess.so, either via -L and -l in the apxs, or using LoadFile in httpd.conf, I get the segfault (same gdb info, BTW).
>>>>
>>>> Catch-22?
>>>>
>>>> Sorry for the false alarm :(!!
>>>>
>>>> Jim
>>>>
>>>>
>>> Not a catch-22. The -L and -l specify linker options when assembling
>>> the code. The ldconfig is a run-time thing. If you are getting the
>>> stderr messages, you are making it all the way into your library. I'd
>>> suggest commenting out the following line and see if you get farther :
>>>
>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>
>>> That should tell you if the problem is the ssl_var_lookup.
>>>
>>> Joe
>>> --
>>> http://www.silverhawk.net/
>>
>> Hi,
>>
>> Thanks for the suggestion. I just tried what you suggested, and got a segfault when I started Apache with the modified module.
>>
>> Jim
>>
>>
> Hi,
>
> As a reminder, here's the gdb with the library loaded:
>
>
> (gdb) b header_post_config
> Function "header_post_config" not defined.
> Make breakpoint pending on future shared library load? (y or [n]) y
> Breakpoint 1 (header_post_config) pending.
> (gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
> [Thread debugging using libthread_db enabled]
> [New Thread 182897610272 (LWP 11317)]
> Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
> Pending breakpoint "header_post_config" resolved
> mod_headers-jl V0.09 - start calling OAM API
> In register_hooks
> In create_headers_dir_config
> In create_headers_dir_config
> In header_cmd
> In header_inout_cmd
> In parse_format_tag
> In parse_misc_string
> In create_headers_dir_config
> In header_cmd
> In header_inout_cmd
> In parse_format_tag
> In parse_misc_string
> [Switching to Thread 182897610272 (LWP 11317)]
>
> Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
> 1121 printf("In header_post_config\n");
> (gdb) n
> 1120 {
> (gdb) n
> 1121 printf("In header_post_config\n");
> (gdb) n
> In header_post_config
> 1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> (gdb) n
> 1124 }
> (gdb) n
> 0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
> 91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
> (gdb) n
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
>
>
> So, it's actually blowing up in "BN_num_bits()" in /lib64/libcrypto.so.4?
>
> Jim
When you see the :
Program received signal SIGSEGV, Segmentation fault.
0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
in gdb, type "bt" and hit enter. It should show the back trace and how
you got to the BN_num_bits() function.
Joe
--
http://www.silverhawk.net/
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use
Apache module that uses shared library?
Posted by oh...@cox.net.
---- ohaya@cox.net wrote:
>
> ---- Joe Lewis <jo...@joe-lewis.com> wrote:
> > On 6/21/12 6:46 PM, ohaya@cox.net wrote:
> > > ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> > >> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
> > >>> ---- ohaya@cox.net wrote:
> > >>>> ---- Sorin Manolache<so...@gmail.com> wrote:
> > >>>>> And I forgot to say: run gdb in some sort of environment where you see
> > >>>>> your current source code line and a couple of surrounding lines. You
> > >>>>> could achieve this with the "list" command, but I prefer running gdb in
> > >>>>> emacs and let emacs do the nice listing of source code in a different panel.
> > >>>>>
> > >>>>> S
> > >>>> Here's the function from my source. It's the original from mod_headers.c, plus my printf:
> > >>>>
> > >>>>
> > >>>> static int header_post_config(apr_pool_t *pconf, apr_pool_t *plog,
> > >>>> apr_pool_t *ptemp, server_rec *s)
> > >>>> {
> > >>>> printf("In header_post_config\n");
> > >>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > >>>> return OK;
> > >>>> }
> > >>>>
> > >>>> Jim
> > >>> Hi,
> > >>>
> > >>> I was able to get the segfault to go away. Here's what I had to do:
> > >>>
> > >>> - Created /etc/ld.so.conf.d/my.conf, and added the directory where my libobaccess.so was
> > >>> - Run 'ldconfig' to activate.
> > >>> - In the apxs command, DON'T include the -L and -l arguments
> > >>>
> > >>> After that, Apache appears to start ok, without segfault :)!!
> > >>>
> > >>> Thanks for all of the great help, esp. the suggestion about checking "ldconfig -p". I still don't understand why, but I'm just glad that I can get past this piece so now I can debug my module :)...
> > >>>
> > >>> Later,
> > >>> Jim
> > >> I'm just glad this list is as good as it is!
> > >>
> > >> FYI, the ldconfig is the dynamic linker control, and those
> > >> /etc/ld.so.conf.d files provide additional search directories for the
> > >> linker to check in when loading a library.
> > >>
> > >> Joe
> > >> --
> > >> http://www.silverhawk.net/
> > >
> > > Hi,
> > >
> > > Sorry to report, but my earlier report was a "false positive" :)...
> > >
> > > I forgot that the mod_headers.c that I was doing the earlier testing with had all references to the libobaccess.so removed :(!!
> > >
> > > So, I'm still stuck with basically the same problem, now, working with my "full" code, with the calls in it:
> > >
> > > - If I compile with -L and -l, Apache segfaults when it starts
> > >
> > > - If I compile without -L and -l, then I get "undefined symbol" errors when I try to start Apache, e.g.:
> > >
> > > [root@apachemodule build-mod_headers]# /apps/httpd2222/bin/apachectl -k start -X
> > > httpd: Syntax error on line 84 of /apps/httpd2222/conf/httpd.conf: Cannot load /apps/httpd2222/modules/mod_headers.so into server: /apps/httpd2222/modules/mod_headers.so: undefined symbol: ObResource_isProtected
> > >
> > > That "ObResource_isProtected" should be a symbol in libobaccess.so, and in fact, if I do "nm --dynamic", I get:
> > >
> > > [root@apachemodule build-mod_headers]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "ObResource_isProtected"
> > > 00000000000a6d80 T ObResource_isProtected
> > > [root@apachemodule build-mod_headers]#
> > >
> > >
> > > I'm *assuming* that the reason for the "undefined symbol" error is that libobaccess.so is actually not being loaded, but then when I try to load libobaccess.so, either via -L and -l in the apxs, or using LoadFile in httpd.conf, I get the segfault (same gdb info, BTW).
> > >
> > > Catch-22?
> > >
> > > Sorry for the false alarm :(!!
> > >
> > > Jim
> > >
> > >
> > Not a catch-22. The -L and -l specify linker options when assembling
> > the code. The ldconfig is a run-time thing. If you are getting the
> > stderr messages, you are making it all the way into your library. I'd
> > suggest commenting out the following line and see if you get farther :
> >
> > header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >
> > That should tell you if the problem is the ssl_var_lookup.
> >
> > Joe
> > --
> > http://www.silverhawk.net/
>
>
> Hi,
>
> Thanks for the suggestion. I just tried what you suggested, and got a segfault when I started Apache with the modified module.
>
> Jim
>
>
Hi,
As a reminder, here's the gdb with the library loaded:
(gdb) b header_post_config
Function "header_post_config" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (header_post_config) pending.
(gdb) run -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
Starting program: /apps/httpd2222/bin/httpd -d /apps/httpd2222/ -f /apps/httpd2222/conf/httpd.conf
[Thread debugging using libthread_db enabled]
[New Thread 182897610272 (LWP 11317)]
Breakpoint 2 at 0x2a9751ea90: file mod_headers.c, line 1121.
Pending breakpoint "header_post_config" resolved
mod_headers-jl V0.09 - start calling OAM API
In register_hooks
In create_headers_dir_config
In create_headers_dir_config
In header_cmd
In header_inout_cmd
In parse_format_tag
In parse_misc_string
In create_headers_dir_config
In header_cmd
In header_inout_cmd
In parse_format_tag
In parse_misc_string
[Switching to Thread 182897610272 (LWP 11317)]
Breakpoint 2, header_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at mod_headers.c:1121
1121 printf("In header_post_config\n");
(gdb) n
1120 {
(gdb) n
1121 printf("In header_post_config\n");
(gdb) n
In header_post_config
1122 header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
(gdb) n
1124 }
(gdb) n
0x00000000004360c7 in ap_run_post_config (pconf=0x573138, plog=0x5a52c8, ptemp=0x5a72d8, s=0x59d3a8) at config.c:91
91 AP_IMPLEMENT_HOOK_RUN_ALL(int, post_config,
(gdb) n
Program received signal SIGSEGV, Segmentation fault.
0x0000003518d6c1e1 in BN_num_bits () from /lib64/libcrypto.so.4
So, it's actually blowing up in "BN_num_bits()" in /lib64/libcrypto.so.4?
Jim
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use
Apache module that uses shared library?
Posted by oh...@cox.net.
---- Joe Lewis <jo...@joe-lewis.com> wrote:
> On 6/21/12 6:46 PM, ohaya@cox.net wrote:
> > ---- Joe Lewis<jo...@joe-lewis.com> wrote:
> >> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
> >>> ---- ohaya@cox.net wrote:
> >>>> ---- Sorin Manolache<so...@gmail.com> wrote:
> >>>>> And I forgot to say: run gdb in some sort of environment where you see
> >>>>> your current source code line and a couple of surrounding lines. You
> >>>>> could achieve this with the "list" command, but I prefer running gdb in
> >>>>> emacs and let emacs do the nice listing of source code in a different panel.
> >>>>>
> >>>>> S
> >>>> Here's the function from my source. It's the original from mod_headers.c, plus my printf:
> >>>>
> >>>>
> >>>> static int header_post_config(apr_pool_t *pconf, apr_pool_t *plog,
> >>>> apr_pool_t *ptemp, server_rec *s)
> >>>> {
> >>>> printf("In header_post_config\n");
> >>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >>>> return OK;
> >>>> }
> >>>>
> >>>> Jim
> >>> Hi,
> >>>
> >>> I was able to get the segfault to go away. Here's what I had to do:
> >>>
> >>> - Created /etc/ld.so.conf.d/my.conf, and added the directory where my libobaccess.so was
> >>> - Run 'ldconfig' to activate.
> >>> - In the apxs command, DON'T include the -L and -l arguments
> >>>
> >>> After that, Apache appears to start ok, without segfault :)!!
> >>>
> >>> Thanks for all of the great help, esp. the suggestion about checking "ldconfig -p". I still don't understand why, but I'm just glad that I can get past this piece so now I can debug my module :)...
> >>>
> >>> Later,
> >>> Jim
> >> I'm just glad this list is as good as it is!
> >>
> >> FYI, the ldconfig is the dynamic linker control, and those
> >> /etc/ld.so.conf.d files provide additional search directories for the
> >> linker to check in when loading a library.
> >>
> >> Joe
> >> --
> >> http://www.silverhawk.net/
> >
> > Hi,
> >
> > Sorry to report, but my earlier report was a "false positive" :)...
> >
> > I forgot that the mod_headers.c that I was doing the earlier testing with had all references to the libobaccess.so removed :(!!
> >
> > So, I'm still stuck with basically the same problem, now, working with my "full" code, with the calls in it:
> >
> > - If I compile with -L and -l, Apache segfaults when it starts
> >
> > - If I compile without -L and -l, then I get "undefined symbol" errors when I try to start Apache, e.g.:
> >
> > [root@apachemodule build-mod_headers]# /apps/httpd2222/bin/apachectl -k start -X
> > httpd: Syntax error on line 84 of /apps/httpd2222/conf/httpd.conf: Cannot load /apps/httpd2222/modules/mod_headers.so into server: /apps/httpd2222/modules/mod_headers.so: undefined symbol: ObResource_isProtected
> >
> > That "ObResource_isProtected" should be a symbol in libobaccess.so, and in fact, if I do "nm --dynamic", I get:
> >
> > [root@apachemodule build-mod_headers]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "ObResource_isProtected"
> > 00000000000a6d80 T ObResource_isProtected
> > [root@apachemodule build-mod_headers]#
> >
> >
> > I'm *assuming* that the reason for the "undefined symbol" error is that libobaccess.so is actually not being loaded, but then when I try to load libobaccess.so, either via -L and -l in the apxs, or using LoadFile in httpd.conf, I get the segfault (same gdb info, BTW).
> >
> > Catch-22?
> >
> > Sorry for the false alarm :(!!
> >
> > Jim
> >
> >
> Not a catch-22. The -L and -l specify linker options when assembling
> the code. The ldconfig is a run-time thing. If you are getting the
> stderr messages, you are making it all the way into your library. I'd
> suggest commenting out the following line and see if you get farther :
>
> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>
> That should tell you if the problem is the ssl_var_lookup.
>
> Joe
> --
> http://www.silverhawk.net/
Hi,
Thanks for the suggestion. I just tried what you suggested, and got a segfault when I started Apache with the modified module.
Jim
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use Apache
module that uses shared library?
Posted by Joe Lewis <jo...@joe-lewis.com>.
On 6/21/12 6:46 PM, ohaya@cox.net wrote:
> ---- Joe Lewis<jo...@joe-lewis.com> wrote:
>> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
>>> ---- ohaya@cox.net wrote:
>>>> ---- Sorin Manolache<so...@gmail.com> wrote:
>>>>> And I forgot to say: run gdb in some sort of environment where you see
>>>>> your current source code line and a couple of surrounding lines. You
>>>>> could achieve this with the "list" command, but I prefer running gdb in
>>>>> emacs and let emacs do the nice listing of source code in a different panel.
>>>>>
>>>>> S
>>>> Here's the function from my source. It's the original from mod_headers.c, plus my printf:
>>>>
>>>>
>>>> static int header_post_config(apr_pool_t *pconf, apr_pool_t *plog,
>>>> apr_pool_t *ptemp, server_rec *s)
>>>> {
>>>> printf("In header_post_config\n");
>>>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>>>> return OK;
>>>> }
>>>>
>>>> Jim
>>> Hi,
>>>
>>> I was able to get the segfault to go away. Here's what I had to do:
>>>
>>> - Created /etc/ld.so.conf.d/my.conf, and added the directory where my libobaccess.so was
>>> - Run 'ldconfig' to activate.
>>> - In the apxs command, DON'T include the -L and -l arguments
>>>
>>> After that, Apache appears to start ok, without segfault :)!!
>>>
>>> Thanks for all of the great help, esp. the suggestion about checking "ldconfig -p". I still don't understand why, but I'm just glad that I can get past this piece so now I can debug my module :)...
>>>
>>> Later,
>>> Jim
>> I'm just glad this list is as good as it is!
>>
>> FYI, the ldconfig is the dynamic linker control, and those
>> /etc/ld.so.conf.d files provide additional search directories for the
>> linker to check in when loading a library.
>>
>> Joe
>> --
>> http://www.silverhawk.net/
>
> Hi,
>
> Sorry to report, but my earlier report was a "false positive" :)...
>
> I forgot that the mod_headers.c that I was doing the earlier testing with had all references to the libobaccess.so removed :(!!
>
> So, I'm still stuck with basically the same problem, now, working with my "full" code, with the calls in it:
>
> - If I compile with -L and -l, Apache segfaults when it starts
>
> - If I compile without -L and -l, then I get "undefined symbol" errors when I try to start Apache, e.g.:
>
> [root@apachemodule build-mod_headers]# /apps/httpd2222/bin/apachectl -k start -X
> httpd: Syntax error on line 84 of /apps/httpd2222/conf/httpd.conf: Cannot load /apps/httpd2222/modules/mod_headers.so into server: /apps/httpd2222/modules/mod_headers.so: undefined symbol: ObResource_isProtected
>
> That "ObResource_isProtected" should be a symbol in libobaccess.so, and in fact, if I do "nm --dynamic", I get:
>
> [root@apachemodule build-mod_headers]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "ObResource_isProtected"
> 00000000000a6d80 T ObResource_isProtected
> [root@apachemodule build-mod_headers]#
>
>
> I'm *assuming* that the reason for the "undefined symbol" error is that libobaccess.so is actually not being loaded, but then when I try to load libobaccess.so, either via -L and -l in the apxs, or using LoadFile in httpd.conf, I get the segfault (same gdb info, BTW).
>
> Catch-22?
>
> Sorry for the false alarm :(!!
>
> Jim
>
>
Not a catch-22. The -L and -l specify linker options when assembling
the code. The ldconfig is a run-time thing. If you are getting the
stderr messages, you are making it all the way into your library. I'd
suggest commenting out the following line and see if you get farther :
header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
That should tell you if the problem is the ssl_var_lookup.
Joe
--
http://www.silverhawk.net/
Re: UNSOLVED was Re: SOLVED was Re: How to compiling/link/use
Apache module that uses shared library?
Posted by oh...@cox.net.
---- ohaya@cox.net wrote:
>
> ---- Joe Lewis <jo...@joe-lewis.com> wrote:
> > On 6/21/12 5:49 PM, ohaya@cox.net wrote:
> > > ---- ohaya@cox.net wrote:
> > >> ---- Sorin Manolache<so...@gmail.com> wrote:
> > >>> And I forgot to say: run gdb in some sort of environment where you see
> > >>> your current source code line and a couple of surrounding lines. You
> > >>> could achieve this with the "list" command, but I prefer running gdb in
> > >>> emacs and let emacs do the nice listing of source code in a different panel.
> > >>>
> > >>> S
> > >>
> > >> Here's the function from my source. It's the original from mod_headers.c, plus my printf:
> > >>
> > >>
> > >> static int header_post_config(apr_pool_t *pconf, apr_pool_t *plog,
> > >> apr_pool_t *ptemp, server_rec *s)
> > >> {
> > >> printf("In header_post_config\n");
> > >> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> > >> return OK;
> > >> }
> > >>
> > >> Jim
> > >
> > > Hi,
> > >
> > > I was able to get the segfault to go away. Here's what I had to do:
> > >
> > > - Created /etc/ld.so.conf.d/my.conf, and added the directory where my libobaccess.so was
> > > - Run 'ldconfig' to activate.
> > > - In the apxs command, DON'T include the -L and -l arguments
> > >
> > > After that, Apache appears to start ok, without segfault :)!!
> > >
> > > Thanks for all of the great help, esp. the suggestion about checking "ldconfig -p". I still don't understand why, but I'm just glad that I can get past this piece so now I can debug my module :)...
> > >
> > > Later,
> > > Jim
> > I'm just glad this list is as good as it is!
> >
> > FYI, the ldconfig is the dynamic linker control, and those
> > /etc/ld.so.conf.d files provide additional search directories for the
> > linker to check in when loading a library.
> >
> > Joe
> > --
> > http://www.silverhawk.net/
>
>
> Hi,
>
> Sorry to report, but my earlier report was a "false positive" :)...
>
> I forgot that the mod_headers.c that I was doing the earlier testing with had all references to the libobaccess.so removed :(!!
>
> So, I'm still stuck with basically the same problem, now, working with my "full" code, with the calls in it:
>
> - If I compile with -L and -l, Apache segfaults when it starts
>
> - If I compile without -L and -l, then I get "undefined symbol" errors when I try to start Apache, e.g.:
>
> [root@apachemodule build-mod_headers]# /apps/httpd2222/bin/apachectl -k start -X
> httpd: Syntax error on line 84 of /apps/httpd2222/conf/httpd.conf: Cannot load /apps/httpd2222/modules/mod_headers.so into server: /apps/httpd2222/modules/mod_headers.so: undefined symbol: ObResource_isProtected
>
> That "ObResource_isProtected" should be a symbol in libobaccess.so, and in fact, if I do "nm --dynamic", I get:
>
> [root@apachemodule build-mod_headers]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "ObResource_isProtected"
> 00000000000a6d80 T ObResource_isProtected
> [root@apachemodule build-mod_headers]#
>
>
> I'm *assuming* that the reason for the "undefined symbol" error is that libobaccess.so is actually not being loaded, but then when I try to load libobaccess.so, either via -L and -l in the apxs, or using LoadFile in httpd.conf, I get the segfault (same gdb info, BTW).
>
> Catch-22?
>
> Sorry for the false alarm :(!!
>
> Jim
>
>
Hi,
Also, BTW, now that I have the ld.so.conf.d setup, I can do "ldconfig -p" and get:
[root@apachemodule build-mod_headers]# ldconfig -p | grep libob
libobaccess.so (libc6,x86-64) => /apps/netpoint/lib64/libobaccess.so
The library TYPE ("libc6") is the same with other .so files that are use, e.g., the libssl*.so, when I list them with ldconfig.
Sill don't understand why loading that libobaccess.so causes Apache to segfault :(...
Jim
UNSOLVED was Re: SOLVED was Re: How to compiling/link/use Apache
module that uses shared library?
Posted by oh...@cox.net.
---- Joe Lewis <jo...@joe-lewis.com> wrote:
> On 6/21/12 5:49 PM, ohaya@cox.net wrote:
> > ---- ohaya@cox.net wrote:
> >> ---- Sorin Manolache<so...@gmail.com> wrote:
> >>> And I forgot to say: run gdb in some sort of environment where you see
> >>> your current source code line and a couple of surrounding lines. You
> >>> could achieve this with the "list" command, but I prefer running gdb in
> >>> emacs and let emacs do the nice listing of source code in a different panel.
> >>>
> >>> S
> >>
> >> Here's the function from my source. It's the original from mod_headers.c, plus my printf:
> >>
> >>
> >> static int header_post_config(apr_pool_t *pconf, apr_pool_t *plog,
> >> apr_pool_t *ptemp, server_rec *s)
> >> {
> >> printf("In header_post_config\n");
> >> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
> >> return OK;
> >> }
> >>
> >> Jim
> >
> > Hi,
> >
> > I was able to get the segfault to go away. Here's what I had to do:
> >
> > - Created /etc/ld.so.conf.d/my.conf, and added the directory where my libobaccess.so was
> > - Run 'ldconfig' to activate.
> > - In the apxs command, DON'T include the -L and -l arguments
> >
> > After that, Apache appears to start ok, without segfault :)!!
> >
> > Thanks for all of the great help, esp. the suggestion about checking "ldconfig -p". I still don't understand why, but I'm just glad that I can get past this piece so now I can debug my module :)...
> >
> > Later,
> > Jim
> I'm just glad this list is as good as it is!
>
> FYI, the ldconfig is the dynamic linker control, and those
> /etc/ld.so.conf.d files provide additional search directories for the
> linker to check in when loading a library.
>
> Joe
> --
> http://www.silverhawk.net/
Hi,
Sorry to report, but my earlier report was a "false positive" :)...
I forgot that the mod_headers.c that I was doing the earlier testing with had all references to the libobaccess.so removed :(!!
So, I'm still stuck with basically the same problem, now, working with my "full" code, with the calls in it:
- If I compile with -L and -l, Apache segfaults when it starts
- If I compile without -L and -l, then I get "undefined symbol" errors when I try to start Apache, e.g.:
[root@apachemodule build-mod_headers]# /apps/httpd2222/bin/apachectl -k start -X
httpd: Syntax error on line 84 of /apps/httpd2222/conf/httpd.conf: Cannot load /apps/httpd2222/modules/mod_headers.so into server: /apps/httpd2222/modules/mod_headers.so: undefined symbol: ObResource_isProtected
That "ObResource_isProtected" should be a symbol in libobaccess.so, and in fact, if I do "nm --dynamic", I get:
[root@apachemodule build-mod_headers]# nm --dynamic /apps/netpoint/lib64/libobaccess.so | grep "ObResource_isProtected"
00000000000a6d80 T ObResource_isProtected
[root@apachemodule build-mod_headers]#
I'm *assuming* that the reason for the "undefined symbol" error is that libobaccess.so is actually not being loaded, but then when I try to load libobaccess.so, either via -L and -l in the apxs, or using LoadFile in httpd.conf, I get the segfault (same gdb info, BTW).
Catch-22?
Sorry for the false alarm :(!!
Jim
Re: SOLVED was Re: How to compiling/link/use Apache module that uses
shared library?
Posted by Joe Lewis <jo...@joe-lewis.com>.
On 6/21/12 5:49 PM, ohaya@cox.net wrote:
> ---- ohaya@cox.net wrote:
>> ---- Sorin Manolache<so...@gmail.com> wrote:
>>> And I forgot to say: run gdb in some sort of environment where you see
>>> your current source code line and a couple of surrounding lines. You
>>> could achieve this with the "list" command, but I prefer running gdb in
>>> emacs and let emacs do the nice listing of source code in a different panel.
>>>
>>> S
>>
>> Here's the function from my source. It's the original from mod_headers.c, plus my printf:
>>
>>
>> static int header_post_config(apr_pool_t *pconf, apr_pool_t *plog,
>> apr_pool_t *ptemp, server_rec *s)
>> {
>> printf("In header_post_config\n");
>> header_ssl_lookup = APR_RETRIEVE_OPTIONAL_FN(ssl_var_lookup);
>> return OK;
>> }
>>
>> Jim
>
> Hi,
>
> I was able to get the segfault to go away. Here's what I had to do:
>
> - Created /etc/ld.so.conf.d/my.conf, and added the directory where my libobaccess.so was
> - Run 'ldconfig' to activate.
> - In the apxs command, DON'T include the -L and -l arguments
>
> After that, Apache appears to start ok, without segfault :)!!
>
> Thanks for all of the great help, esp. the suggestion about checking "ldconfig -p". I still don't understand why, but I'm just glad that I can get past this piece so now I can debug my module :)...
>
> Later,
> Jim
I'm just glad this list is as good as it is!
FYI, the ldconfig is the dynamic linker control, and those
/etc/ld.so.conf.d files provide additional search directories for the
linker to check in when loading a library.
Joe
--
http://www.silverhawk.net/