You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficserver.apache.org by Brian Geffon <br...@apache.org> on 2012/03/26 08:06:58 UTC

Cleaning up ink_gethostbyname_r and ink_gethostbyaddr_r

Hi all,
In an effort to get ATS to build on OpenBSD, I noticed that we're using
gethostbyname_r and gethostbyaddr_r (see ink_inet.cc), according to the
manpage both of these APIs are obsolete and getaddrinfo should be used
(which is reentrant by default), I only noticed this because OpenBSD 5.0
doesn't support them. What does everyone think about doing the following:

As a first step I will create a new function called ink_getaddrinfo().
Next, I will rewrite ink_gethostbyname_r() and ink_gethostbyaddr_r() to
wrap ink_getaddrinfo(), this will be tricky because I will need to manually
create the hostent structs to not break existing code that uses
ink_gethostbyname_r and inkgethostbyaddr_r, does that seem hacky? The nice
thing about getaddrinfo is that it appears we will be able to get rid of
several defines such as RENTRENT_GETHOSTBYNAME and GETHOSTBYNAME_R_GLIBC2
since the API seems to be consistant across BSD, Linux, and Darwin, unless
there is something i'm missing? Lastly, I will try to switch all usages of
ink_gethostbyname/addr_r over to the new ink_getaddrinfo(), at which point
I will remove them entirely.

Thoughts?

Brian

Re: Cleaning up ink_gethostbyname_r and ink_gethostbyaddr_r

Posted by "Alan M. Carroll" <am...@network-geographics.com>.
Gah! The ats_ip_getbestaddrinfo bug is TS-1170. Not TS-1070.


Re: Cleaning up ink_gethostbyname_r and ink_gethostbyaddr_r

Posted by "Alan M. Carroll" <am...@network-geographics.com>.
ats_getbestaddrinfo is already implemented. It's in lib/ts/ink_inet.cc. It's a bit hard to use, I need to improve it :-) but you can see examples if you look at the commits for those issues. The issue is that you can get back both an IPv4 and an IPv6 address. I've filed TS-1070 to remind me to clean it up. If you have suggestions, please put them in comments on that issue.

For some of them I used ats_getbestaddrinfo, others I used getaddrinfo directly, and one I just removed the method because it wasn't used.

At this point I don't see much benefit to wrapping getaddrinfo.

Saturday, March 31, 2012, 8:56:40 PM, you wrote:

> Sure, on all of those tickets you just moved directly to getaddrinfo(), and
> I still need to write ats_getbestaddrinfo(), correct?

> Brian


Re: Cleaning up ink_gethostbyname_r and ink_gethostbyaddr_r

Posted by Brian Geffon <br...@apache.org>.
Sure, on all of those tickets you just moved directly to getaddrinfo(), and
I still need to write ats_getbestaddrinfo(), correct?

Brian

On Sat, Mar 31, 2012 at 6:49 PM, Alan M. Carroll <
amc@network-geographics.com> wrote:

> I have fixed TS-1166, TS-1167, TS-1168, TS-1017 all of which eliminate
> uses of gethostbyname. Looks to me like only clustering, web management,
> CoreAPIShared.cc in mgmt, and some test code are still problematic. Except
> for clustering those look like using ats_getbestaddrinfo() would work
> (since they just grab whatever's first in the gethostbyname return data).
> Brian, want to take a few of those?
>
>

Re: Cleaning up ink_gethostbyname_r and ink_gethostbyaddr_r

Posted by "Alan M. Carroll" <am...@network-geographics.com>.
I have fixed TS-1166, TS-1167, TS-1168, TS-1017 all of which eliminate uses of gethostbyname. Looks to me like only clustering, web management, CoreAPIShared.cc in mgmt, and some test code are still problematic. Except for clustering those look like using ats_getbestaddrinfo() would work (since they just grab whatever's first in the gethostbyname return data). Brian, want to take a few of those?


Re: Cleaning up ink_gethostbyname_r and ink_gethostbyaddr_r

Posted by John Plevyak <jp...@acm.org>.
Great.  Then it seems like it is time to make the move into the 21st
century.

john

On Tue, Mar 27, 2012 at 3:46 PM, Alan M. Carroll <
amc@network-geographics.com> wrote:

> I would presume that applies to the functions documented on the page, not
> getaddrinfo which is mentioned only in passing. This is reenforced by
> getaddrinfo having its own documentation page.
>
> Just for amusement value, here's a couple of bug reports of
> gethostbyaddr_r() crashing under load -
>
> Tuesday, March 27, 2012, 5:10:57 PM, you wrote:
>
> > Search for the line:
>
> > "The lookup functions above all have one in common: they are not
> reentrant
> > and therefore unusable in multi-threaded applications. Therefore provides
> > the GNU C library a new set of functions which can be used in this
> context."
>
> > Perhaps it operates differently if _POSIX_SOURCE is defined or this
> > documentation is old although the former doesn't seem to be the case.
>
>
>

Re: Cleaning up ink_gethostbyname_r and ink_gethostbyaddr_r

Posted by "Alan M. Carroll" <am...@network-geographics.com>.
I would presume that applies to the functions documented on the page, not getaddrinfo which is mentioned only in passing. This is reenforced by getaddrinfo having its own documentation page.

Just for amusement value, here's a couple of bug reports of gethostbyaddr_r() crashing under load -

Tuesday, March 27, 2012, 5:10:57 PM, you wrote:

> Search for the line:

> "The lookup functions above all have one in common: they are not reentrant
> and therefore unusable in multi-threaded applications. Therefore provides
> the GNU C library a new set of functions which can be used in this context."

> Perhaps it operates differently if _POSIX_SOURCE is defined or this
> documentation is old although the former doesn't seem to be the case.



Re: Cleaning up ink_gethostbyname_r and ink_gethostbyaddr_r

Posted by John Plevyak <jp...@acm.org>.
Search for the line:

"The lookup functions above all have one in common: they are not reentrant
and therefore unusable in multi-threaded applications. Therefore provides
the GNU C library a new set of functions which can be used in this context."

Perhaps it operates differently if _POSIX_SOURCE is defined or this
documentation is old although the former doesn't seem to be the case.


john


On Tue, Mar 27, 2012 at 10:52 AM, Alan M. Carroll <
amc@network-geographics.com> wrote:

> Where do you see that? My browser claims only one mention of getaddrinfo
> on that page
>
> "You can also use getaddrinfo and getnameinfo to obtain this information"
>
> which contains no mention of thread safety. The glibc documentation for
> getaddinfo
>
> http://www.gnu.org/software/gnulib/manual/html_node/getaddrinfo.html
>
> just refers to the POSIX specification
>
> http://pubs.opengroup.org/onlinepubs/9699919799/functions/getaddrinfo.html
>
> which states
>
> The freeaddrinfo() and getaddrinfo() functions shall be thread-safe.
>
>
> Tuesday, March 27, 2012, 11:27:34 AM, you wrote:
>
> > FYI: According to
>
> > http://www.gnu.org/software/libc/manual/html_node/Host-Names.html
>
> > The glibc versions of getaddrinfo are still not thread safe.
>
>

Re: Cleaning up ink_gethostbyname_r and ink_gethostbyaddr_r

Posted by "Alan M. Carroll" <am...@network-geographics.com>.
Where do you see that? My browser claims only one mention of getaddrinfo on that page

"You can also use getaddrinfo and getnameinfo to obtain this information"

which contains no mention of thread safety. The glibc documentation for getaddinfo

http://www.gnu.org/software/gnulib/manual/html_node/getaddrinfo.html

just refers to the POSIX specification

http://pubs.opengroup.org/onlinepubs/9699919799/functions/getaddrinfo.html

which states

The freeaddrinfo() and getaddrinfo() functions shall be thread-safe.


Tuesday, March 27, 2012, 11:27:34 AM, you wrote:

> FYI: According to

> http://www.gnu.org/software/libc/manual/html_node/Host-Names.html

> The glibc versions of getaddrinfo are still not thread safe.


Re: Cleaning up ink_gethostbyname_r and ink_gethostbyaddr_r

Posted by John Plevyak <jp...@acm.org>.
FYI: According to

http://www.gnu.org/software/libc/manual/html_node/Host-Names.html

The glibc versions of getaddrinfo are still not thread safe.

john

On Mon, Mar 26, 2012 at 7:58 PM, Alan M. Carroll <
amc@network-geographics.com> wrote:

> I filed TS-1166, TS-1167, TS-1168, to track some of these places. TS-990
> tracks the Cluster issue, and TS-1017 tracks it in logging.  I am working
> on the logging. I will try to get through these others although others are
> welcome to step up :-).
>
>

Re: Cleaning up ink_gethostbyname_r and ink_gethostbyaddr_r

Posted by "Alan M. Carroll" <am...@network-geographics.com>.
I filed TS-1166, TS-1167, TS-1168, to track some of these places. TS-990 tracks the Cluster issue, and TS-1017 tracks it in logging.  I am working on the logging. I will try to get through these others although others are welcome to step up :-).


Re: Cleaning up ink_gethostbyname_r and ink_gethostbyaddr_r

Posted by "Alan M. Carroll" <am...@network-geographics.com>.
Mostly reasonable, just a few things to do better:

1) Do not use the "ink_" prefix, we're moving to "ats_".

2) It's not clear to me we need to wrap getaddrinfo at all unless you can simplify the api (e.g., ats_getbestaddrinfo).

3) Looking at the call sites which still use ink_gethostbyname, I would recommend converting them directly, one by one, and not trying to simulate the hostent at all. In fact, it looks like some of them can just be replaced by ats_getbestaddrinfo since they just take whatever is in slot 0 of the hostent.

4) Frankly, it looks like most of these are places I failed to convert to IPv6. At least the UrlRewrite.cc one, and probably the two in proxy/logging. (TS-1017). And maybe clustering (TS-990). Let me see what I have done for those.

Monday, March 26, 2012, 1:06:58 AM, you wrote:

> As a first step I will create a new function called ink_getaddrinfo().
> Next, I will rewrite ink_gethostbyname_r() and ink_gethostbyaddr_r() to
> wrap ink_getaddrinfo(), this will be tricky because I will need to manually
> create the hostent structs to not break existing code that uses
> ink_gethostbyname_r and inkgethostbyaddr_r, does that seem hacky? The nice
> thing about getaddrinfo is that it appears we will be able to get rid of
> several defines such as RENTRENT_GETHOSTBYNAME and GETHOSTBYNAME_R_GLIBC2
> since the API seems to be consistant across BSD, Linux, and Darwin, unless
> there is something i'm missing? Lastly, I will try to switch all usages of
> ink_gethostbyname/addr_r over to the new ink_getaddrinfo(), at which point
> I will remove them entirely.

> Thoughts?

> Brian


Re: Cleaning up ink_gethostbyname_r and ink_gethostbyaddr_r

Posted by James Peach <ja...@me.com>.
On Mar 25, 2012, at 11:06 PM, Brian Geffon <br...@apache.org> wrote:

> Hi all,
> In an effort to get ATS to build on OpenBSD, I noticed that we're using
> gethostbyname_r and gethostbyaddr_r (see ink_inet.cc), according to the
> manpage both of these APIs are obsolete and getaddrinfo should be used
> (which is reentrant by default), I only noticed this because OpenBSD 5.0
> doesn't support them. What does everyone think about doing the following:
> 
> As a first step I will create a new function called ink_getaddrinfo().
> Next, I will rewrite ink_gethostbyname_r() and ink_gethostbyaddr_r() to
> wrap ink_getaddrinfo(), this will be tricky because I will need to manually
> create the hostent structs to not break existing code that uses
> ink_gethostbyname_r and inkgethostbyaddr_r, does that seem hacky? The nice
> thing about getaddrinfo is that it appears we will be able to get rid of
> several defines such as RENTRENT_GETHOSTBYNAME and GETHOSTBYNAME_R_GLIBC2
> since the API seems to be consistant across BSD, Linux, and Darwin, unless
> there is something i'm missing? Lastly, I will try to switch all usages of
> ink_gethostbyname/addr_r over to the new ink_getaddrinfo(), at which point
> I will remove them entirely.

+1 this sounds good

J

Re: Cleaning up ink_gethostbyname_r and ink_gethostbyaddr_r

Posted by Igor Galić <i....@brainsware.org>.

----- Original Message -----
> 
> 
> ----- Original Message -----
> > Hi all,
> > In an effort to get ATS to build on OpenBSD, I noticed that we're
> > using
> > gethostbyname_r and gethostbyaddr_r (see ink_inet.cc), according to
> > the
> > manpage both of these APIs are obsolete and getaddrinfo should be
> > used
> > (which is reentrant by default), I only noticed this because
> > OpenBSD
> > 5.0
> > doesn't support them. What does everyone think about doing the
> > following:

Wait.. why again are we supporting platforms that don't support
sensible threading?

[snip]

i

-- 
Igor Galić

Tel: +43 (0) 664 886 22 883
Mail: i.galic@brainsware.org
URL: http://brainsware.org/
GPG: 6880 4155 74BD FD7C B515  2EA5 4B1D 9E08 A097 C9AE


Re: Cleaning up ink_gethostbyname_r and ink_gethostbyaddr_r

Posted by Igor Galić <i....@brainsware.org>.

----- Original Message -----
> Hi all,
> In an effort to get ATS to build on OpenBSD, I noticed that we're
> using
> gethostbyname_r and gethostbyaddr_r (see ink_inet.cc), according to
> the
> manpage both of these APIs are obsolete and getaddrinfo should be
> used
> (which is reentrant by default), I only noticed this because OpenBSD
> 5.0
> doesn't support them. What does everyone think about doing the
> following:
> 
> As a first step I will create a new function called
> ink_getaddrinfo().
> Next, I will rewrite ink_gethostbyname_r() and ink_gethostbyaddr_r()
> to
> wrap ink_getaddrinfo(), this will be tricky because I will need to
> manually
> create the hostent structs to not break existing code that uses
> ink_gethostbyname_r and inkgethostbyaddr_r, does that seem hacky? The
> nice
> thing about getaddrinfo is that it appears we will be able to get rid
> of
> several defines such as RENTRENT_GETHOSTBYNAME and
> GETHOSTBYNAME_R_GLIBC2
> since the API seems to be consistant across BSD, Linux, and Darwin,
> unless
> there is something i'm missing? Lastly, I will try to switch all
> usages of
> ink_gethostbyname/addr_r over to the new ink_getaddrinfo(), at which
> point
> I will remove them entirely.
> 
> Thoughts?

The pattern is the same as for strlcpy() and other functions that
wrap because they don't exist on all platforms.

Geoff Greer's silver searcher uses some automagick to help this case
maybe we could adapt that too? (if we don't already do)

https://github.com/ggreer/the_silver_searcher/blob/master/src/util.h
https://github.com/ggreer/the_silver_searcher/blob/master/src/util.c
https://github.com/ggreer/the_silver_searcher/blob/master/configure.ac

That would skip at least one step in the process.

> Brian

So long,

i

-- 
Igor Galić

Tel: +43 (0) 664 886 22 883
Mail: i.galic@brainsware.org
URL: http://brainsware.org/
GPG: 6880 4155 74BD FD7C B515  2EA5 4B1D 9E08 A097 C9AE