You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Luis Neves <lu...@hotmail.com> on 2010/04/29 16:33:37 UTC
[users@httpd] Incorrect conversion of UTF-8 characters comming from X.509
certificates, please help
Hi there,
I am trying to match the values coming from apache/mod_ssl/mod_authz_ldap against some fields (subjectDN and issuerDN) in an Openldap directory
the problem is that Apache is receiving certificate data that contains UTF8 encoded chars
That chars are being incorrectly encoded with '\x' characters (deprecated source code? bug?) and this is making the effect of mod_authz_ldap failing the query with "bad search filter" error
Here some example data on the ssl_error.log
http://www.mail-archive.com/openssl-users@openssl.org/msg60934.html
I need help on solving this, Iam sucked and dont know what to do to put this thing working
Can someboby help me please?
PS: Im using Apache 2.2.3 on a Centos 5.4, against openldap
Luis
_________________________________________________________________
Hotmail: Trusted email with Microsoft’s powerful SPAM protection.
https://signup.live.com/signup.aspx?id=60969
RE: [users@httpd] UTF-8 strings through ap_log_cerror
Posted by Luis Neves <lu...@hotmail.com>.
Hi Eric
I am having this error on /var/log/http/ssl_error.log
[Fri Apr
16 12:30:40 2010] [error] [client 10.14.149.196] [16695] search from
'ou=AuthzLDAPCertmap,dc=cm-lisboa,dc=pt' for
'(&(issuerDN=/C=PT/O=Cart\\xC3\\xA3o de
Cidad\\xC3\\xA3o/OU=subECEstado/CN=EC de Autentica\\xC3\\xA7\\xC3\\xA3o
do Cart\\xC3\\xA3o de Cidad\\xC3\\xA3o
0003)(serialNumber=5FD933E0F2F95D0F)(objectClass=authzLDAPmap))'
returns -7 = 'Bad search filter'
This is mod_authz_ldap trying to query the directory and getting the 'Bad search filter' error
Luis
> Date: Fri, 30 Apr 2010 16:39:29 -0400
> From: covener@gmail.com
> To: users@httpd.apache.org
> Subject: Re: [users@httpd] UTF-8 strings through ap_log_cerror
>
> On Fri, Apr 30, 2010 at 4:28 PM, Luis Neves <lu...@hotmail.com> wrote:
>
> > I think this issue is not only related to the logs apache is creating "in
> > ssl_engine_kernel.c" but must be in another place as well, i say this
> > because the mod_authz_ldap is using the incorreclty values (with '\x') to
> > query the ldap directory.
>
> How do you know LDAP is using the wrong info? Where does the info come
> from that LDAP is putting on the wire?
> --
> Eric Covener
> covener@gmail.com
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
_________________________________________________________________
Your E-mail and More On-the-Go. Get Windows Live Hotmail Free.
https://signup.live.com/signup.aspx?id=60969
Re: [users@httpd] UTF-8 strings through ap_log_cerror
Posted by Eric Covener <co...@gmail.com>.
On Fri, Apr 30, 2010 at 4:28 PM, Luis Neves <lu...@hotmail.com> wrote:
> I think this issue is not only related to the logs apache is creating "in
> ssl_engine_kernel.c" but must be in another place as well, i say this
> because the mod_authz_ldap is using the incorreclty values (with '\x') to
> query the ldap directory.
How do you know LDAP is using the wrong info? Where does the info come
from that LDAP is putting on the wire?
--
Eric Covener
covener@gmail.com
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
[users@httpd] Re: {SPAM?} RE: [users@httpd] UTF-8 strings through ap_log_cerror
Posted by Sander Temme <sc...@apache.org>.
On May 1, 2010, at 10:32 AM, Luis Neves wrote:
> mod_authz_ldap has another way to validate a certificate. Instead of using the issuer and subject fields I will try to use all the client certificate as the validation field, this way I dont have to deal with utf8 problems, I hope...
That might work: the unparsed certificate would not have been mangled by httpd. If whichever program ends up parsing it does adhere to the RFC, you'd end up with the right data.
S.
--
Sander Temme
sctemme@apache.org
PGP FP: 51B4 8727 466A 0BC3 69F4 B7B8 B2BE BC40 1529 24AF
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: [users@httpd] UTF-8 strings through ap_log_cerror
Posted by Luis Neves <lu...@hotmail.com>.
Thanks for the most compreensive reply so far Sander.
I see, the problem is not easy at all, Now Im curious to know how other people dealed with this problem, if they discovered it at all. if I were not in a country with accented characters probably this would never be a problem to me.
I will try to follow the results of this patch you've showed me , hope it works ok, but meanwhile I wlil change strategy
(now that I know is not any missing configuration on my system)
mod_authz_ldap has another way to validate a certificate. Instead of using the issuer and subject fields I will try to use all the client certificate as the validation field, this way I dont have to deal with utf8 problems, I hope...
Luis
> From: sctemme@apache.org
> Date: Sat, 1 May 2010 09:29:21 -0500
> To: users@httpd.apache.org
> Subject: Re: [users@httpd] UTF-8 strings through ap_log_cerror
>
> Luis,
>
> On Apr 30, 2010, at 3:28 PM, Luis Neves wrote:
>
> > Hi list members,
> >
> > see here http://marc.info/?l=apache-httpd-dev&m=127242179232546&w=2
> >
> > I am the original poster of this issue, unfortunally so far I have no answers to my problem and maybe you can give me some clues
> >
> > I think this issue is not only related to the logs apache is creating "in ssl_engine_kernel.c" but must be in another place as well, i say this because the mod_authz_ldap is using the incorreclty values (with '\x') to query the ldap directory.
> > So IMHO it needs fixing somewere else too.....
>
> I am not sure, but I think what's going on is that modules/ssl/ssl_engine_vars.c calls X509_NAME_oneline() on line 382 and 388 (in trunk). This populates the "environment" variables that I assume you have configured to pass on to mod_authz_ldap. Per Stephen Henson on the openssl-users list, the right function to use is X509_NAME_print_ex() using (per Kaspar Brand on the dev@httpd list) a memory BIO and XN_FLAG_RFC2253 for formatting.
>
> Using X509_NAME_print_ex() in these spots would get the DN out of the certificate in a sensible format. The problem is, however, that the ssl_var_lookup_ssl_cert() function returns a char *, not a wchar_t *. So, even if we were to have get the certificate data in the right format, we couldn't pass it up the call stack without escaping it since ssl_var_lookup_ssl_cert(), ssl_var_lookup_ssl(), ssl_var_lookup() and all the functions that call it all expect a char *, and return a char *. This goes up all the way to the ssl_hook_Fixup() function registered as the ap_hook_fixups handler for mod_ssl: this is where the "environment" gets populated with stuff that mod_authz_ldap can use. To fix this issue would mean making this call stack Unicode clean all the way to the top. This of course may cause an avalanche of side effects throughout the code, so before you know it you're rewriting the entire web server.
>
> So regarding your last comment on the OpenSSL list, it's not that basic. mod_ssl has been part of Apache for ten years or more, and existed as a third party module before that. I would not be surprised if Unicode did not exist when this code was written, so the reason we use a legacy function there is that it is, really, legacy code.
>
> I doubt you're the first to run into this issue. However, apparently no one confronted with the problem of UTF-8 characters in a client certificate DN has had the time, acumen and energy to solve the problem.
>
> > I need somebody to confirm this because if it is the case I need to find other way to check the certificates
> > If not, so how do I am suposed to use the correct values on the other modules?
> > and lastly: should a BUG be filed for this?
>
> As it happens, there is a partial patch in bug 48780:
>
> https://issues.apache.org/bugzilla/show_bug.cgi?id=48780
>
> However, I don't know if you can stuff Unicode DNs into a char * like Peter is doing. And we'd have to do the same thing for the Issuer field. But if this works, maybe we should entertain it. Luis, would this solve your problem?
>
> S.
>
> > Thank you a lot
> > Luis
> >
> > just for context, heres my first post on this problem:
> >
> > I am trying to match the values coming from apache/mod_ssl/mod_authz_ldap against some fields (subjectDN and issuerDN) in an Openldap directory
> > the problem is that Apache is receiving certificate data that contains UTF8 encoded chars
> >
> > That chars are being incorrectly encoded with '\x' characters (deprecated source code? bug?) and this is making the effect of mod_authz_ldap failing the query with "bad search filter" error
> >
> > Here some example data on the ssl_error.log
> > http://www.mail-archive.com/openssl-users@openssl.org/msg60934.html
> >
> > I need help on solving this, Iam sucked and dont know what to do to put this thing working
> > Can someboby help me please?
> >
> > PS: Im using Apache 2.2.3 on a Centos 5.4, against openldap
> >
> > Hotmail: Trusted email with Microsoft’s powerful SPAM protection. Sign up now.
>
>
>
> --
> Sander Temme
> sctemme@apache.org
> PGP FP: 51B4 8727 466A 0BC3 69F4 B7B8 B2BE BC40 1529 24AF
>
>
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
_________________________________________________________________
Hotmail: Trusted email with powerful SPAM protection.
https://signup.live.com/signup.aspx?id=60969
Re: [users@httpd] UTF-8 strings through ap_log_cerror
Posted by Sander Temme <sc...@apache.org>.
Luis,
On Apr 30, 2010, at 3:28 PM, Luis Neves wrote:
> Hi list members,
>
> see here http://marc.info/?l=apache-httpd-dev&m=127242179232546&w=2
>
> I am the original poster of this issue, unfortunally so far I have no answers to my problem and maybe you can give me some clues
>
> I think this issue is not only related to the logs apache is creating "in ssl_engine_kernel.c" but must be in another place as well, i say this because the mod_authz_ldap is using the incorreclty values (with '\x') to query the ldap directory.
> So IMHO it needs fixing somewere else too.....
I am not sure, but I think what's going on is that modules/ssl/ssl_engine_vars.c calls X509_NAME_oneline() on line 382 and 388 (in trunk). This populates the "environment" variables that I assume you have configured to pass on to mod_authz_ldap. Per Stephen Henson on the openssl-users list, the right function to use is X509_NAME_print_ex() using (per Kaspar Brand on the dev@httpd list) a memory BIO and XN_FLAG_RFC2253 for formatting.
Using X509_NAME_print_ex() in these spots would get the DN out of the certificate in a sensible format. The problem is, however, that the ssl_var_lookup_ssl_cert() function returns a char *, not a wchar_t *. So, even if we were to have get the certificate data in the right format, we couldn't pass it up the call stack without escaping it since ssl_var_lookup_ssl_cert(), ssl_var_lookup_ssl(), ssl_var_lookup() and all the functions that call it all expect a char *, and return a char *. This goes up all the way to the ssl_hook_Fixup() function registered as the ap_hook_fixups handler for mod_ssl: this is where the "environment" gets populated with stuff that mod_authz_ldap can use. To fix this issue would mean making this call stack Unicode clean all the way to the top. This of course may cause an avalanche of side effects throughout the code, so before you know it you're rewriting the entire web server.
So regarding your last comment on the OpenSSL list, it's not that basic. mod_ssl has been part of Apache for ten years or more, and existed as a third party module before that. I would not be surprised if Unicode did not exist when this code was written, so the reason we use a legacy function there is that it is, really, legacy code.
I doubt you're the first to run into this issue. However, apparently no one confronted with the problem of UTF-8 characters in a client certificate DN has had the time, acumen and energy to solve the problem.
> I need somebody to confirm this because if it is the case I need to find other way to check the certificates
> If not, so how do I am suposed to use the correct values on the other modules?
> and lastly: should a BUG be filed for this?
As it happens, there is a partial patch in bug 48780:
https://issues.apache.org/bugzilla/show_bug.cgi?id=48780
However, I don't know if you can stuff Unicode DNs into a char * like Peter is doing. And we'd have to do the same thing for the Issuer field. But if this works, maybe we should entertain it. Luis, would this solve your problem?
S.
> Thank you a lot
> Luis
>
> just for context, heres my first post on this problem:
>
> I am trying to match the values coming from apache/mod_ssl/mod_authz_ldap against some fields (subjectDN and issuerDN) in an Openldap directory
> the problem is that Apache is receiving certificate data that contains UTF8 encoded chars
>
> That chars are being incorrectly encoded with '\x' characters (deprecated source code? bug?) and this is making the effect of mod_authz_ldap failing the query with "bad search filter" error
>
> Here some example data on the ssl_error.log
> http://www.mail-archive.com/openssl-users@openssl.org/msg60934.html
>
> I need help on solving this, Iam sucked and dont know what to do to put this thing working
> Can someboby help me please?
>
> PS: Im using Apache 2.2.3 on a Centos 5.4, against openldap
>
> Hotmail: Trusted email with Microsoft’s powerful SPAM protection. Sign up now.
--
Sander Temme
sctemme@apache.org
PGP FP: 51B4 8727 466A 0BC3 69F4 B7B8 B2BE BC40 1529 24AF
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
[users@httpd] UTF-8 strings through ap_log_cerror
Posted by Luis Neves <lu...@hotmail.com>.
Hi list members,
see here http://marc.info/?l=apache-httpd-dev&m=127242179232546&w=2
I am the original poster of this issue, unfortunally so far I have no answers to my problem and maybe you can give me some clues
I think this issue is not only related to the logs apache is creating "in ssl_engine_kernel.c" but must be in another place as well, i say this because the mod_authz_ldap is using the incorreclty values (with '\x') to query the ldap directory.
So IMHO it needs fixing somewere else too.....
I need somebody to confirm this because if it is the case I need to find other way to check the certificates
If not, so how do I am suposed to use the correct values on the other modules?
and lastly: should a BUG be filed for this?
Thank you a lot
Luis
just for context, heres my first post on this problem:
I am trying to match the values coming from apache/mod_ssl/mod_authz_ldap against some fields (subjectDN and issuerDN) in an Openldap directory
the problem is that Apache is receiving certificate data that contains UTF8 encoded chars
That chars are being incorrectly encoded with '\x' characters (deprecated source code? bug?) and this is making the effect of mod_authz_ldap failing the query with "bad search filter" error
Here some example data on the ssl_error.log
http://www.mail-archive.com/openssl-users@openssl.org/msg60934.html
I need help on solving this, Iam sucked and dont know what to do to put this thing working
Can someboby help me please?
PS: Im using Apache 2.2.3 on a Centos 5.4, against openldap
_________________________________________________________________
Hotmail: Trusted email with Microsoft’s powerful SPAM protection.
https://signup.live.com/signup.aspx?id=60969
RE: [users@httpd] Incorrect conversion of UTF-8 characters comming from X.509 certificates, please help
Posted by "Thomas, Peter" <pt...@HPTI.com>.
This may help you; I patched mod_ssl to retrieve the certificate DN in
RFC2253 [LDAP-compliant] format, instead of the deprecated method
currently used:
--- http-2.2.15-baseline/modules/ssl//ssl_engine_vars.c Sat Feb 27
16:00:58 2010
--- http-2.2.15/modules/ssl//ssl_engine_vars.c Tue Mar 23 14:22:53 2010
@@ -367,10 +367,20 @@
}
else if (strcEQ(var, "S_DN")) {
xsname = X509_get_subject_name(xs);
- cp = X509_NAME_oneline(xsname, NULL, 0);
- result = apr_pstrdup(p, cp);
- modssl_free(cp);
- resdup = FALSE;
+ BIO *bio;
+ int n;
+
+ if ((bio = BIO_new(BIO_s_mem())) == NULL) {
+ result = NULL;
+ } else {
+ X509_NAME_print_ex(bio, xsname, 0, XN_FLAG_RFC2253);
+ n = BIO_pending(bio);
+ result = apr_pcalloc(p, n+1);
+ n = BIO_read(bio, result, n);
+ result[n] = NUL;
+ BIO_free(bio);
+ resdup = FALSE;
+ }
}
else if (strlen(var) > 5 && strcEQn(var, "S_DN_", 5)) {
xsname = X509_get_subject_name(xs)
See also:
http://mail-archives.apache.org/mod_mbox/httpd-modules-dev/201003.mbox/%
3C5045A4D718CAB644BA24979206486B60067A4179@hptimail03.HPTI.COM%3E
________________________________
From: Luis Neves [mailto:luisneves@hotmail.com]
Sent: Thursday, April 29, 2010 10:34 AM
To: users@httpd.apache.org
Subject: [users@httpd] Incorrect conversion of UTF-8 characters
comming from X.509 certificates, please help
Hi there,
I am trying to match the values coming from
apache/mod_ssl/mod_authz_ldap against some fields (subjectDN and
issuerDN) in an Openldap directory
the problem is that Apache is receiving certificate data that
contains UTF8 encoded chars
That chars are being incorrectly encoded with '\x' characters
(deprecated source code? bug?) and this is making the effect of
mod_authz_ldap failing the query with "bad search filter" error
Here some example data on the ssl_error.log
http://www.mail-archive.com/openssl-users@openssl.org/msg60934.html
I need help on solving this, Iam sucked and dont know what to do
to put this thing working
Can someboby help me please?
PS: Im using Apache 2.2.3 on a Centos 5.4, against openldap
Luis
________________________________
Hotmail: Trusted email with Microsoft's powerful SPAM
protection. Sign up now. <https://signup.live.com/signup.aspx?id=60969>