You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by Vasile Alin <al...@gmail.com> on 2011/09/01 22:17:50 UTC
In-memory DNS resolver for HC
Hi,
I had some situations when working directly with http-client or JMeter in
restrictive environments, where I wasn't able to edit the local /etc/hosts
file for resolving some custom hostnames. Based on the HttpUnit solution
[1], I was thinking that would be great if httpclient can lookup a hostname
in a custom DNS Resolver, editable or programable in the Java environment,
and fall-back to the default system address resolver if this isn't found in
the first place. For example the above snippet will resolve the "
www.alin.com" hostname to the one of the Google IPs:
interface DnsResolver{
InetAddress[] resolve(String hostname);
}
class InMemoryDnsResolverImpl implements DnsResolver{
public InetAddress[] resolve(final String hostname){
try {
if ("www.alin.com".equals(hostname)) {
String ip = "74.125.39.105";
String[] ipParts = ip.split("\\.");
byte[] byteIpAddress = new byte[4];
for (int i = 0; i < 4; i++) {
byteIpAddress[i] = Byte.parseByte(ipParts[i]);
}
return new InetAddress[] { InetAddress
.getByAddress(byteIpAddress) };
} else {
return null;
}
} catch (UnknownHostException e) {
// log exception
return null;
}
}
}
public class DefaultClientConnectionOperator implements
ClientConnectionOperator {
....
private DnsResolver dnsResolver;
...
protected InetAddress[] resolveHostname(final String host) throws
UnknownHostException {
InetAddress[] resolvedAddress = null;
if(dnResolver != null){
resolvedAddress = dnResolver.resolve(host);
}
if(resolverAddress==null){
resolvedAddress = InetAddress.getAllByName(host);
}
return resolvedAddress;
}
}
This functionality can be extended to get the IP addresses from a
properties file through an PropertyFileDnsResolverImpl and so on. What do
you think?
Regards,
Alin
[1] http://httpunit.sourceforge.net/
Re: In-memory DNS resolver for HC
Posted by Vasile Alin <al...@gmail.com>.
Bill, Oleg, thanks for your feedback.
I've created the improvement issue 1123 against HttpClient (
https://issues.apache.org/jira/browse/HTTPCLIENT-1123) and I'll try to
provide a patch to prove the functionality I proposed.
Regards,
Alin
On 2 September 2011 18:52, Oleg Kalnichevski <ol...@apache.org> wrote:
> On Fri, 2011-09-02 at 17:58 +0300, Vasile Alin wrote:
> > On 2 September 2011 17:45, Bill Speirs <bi...@gmail.com> wrote:
> >
> > > That does make for a slightly more compelling argument, but I would
> > > say disable auto-redirects and handle them manually doing the DNS
> > > resolution?
> > >
> >
> > This is OK in most of the cases.I see that JMeter has the option to
> follow
> > redirects automatically (with the limitations defined in the RFC), maybe
> in
> > this case there is a nice feature to allow custom host entries in a
> section
> > in the Preferences section. The question is if this case is frequent and
> > worths the effort to change the httpclient library.
> >
> > Regarding the interface and implementation classes names, maybe
> DnsOverrider
> > and InMemoryDnsOverriderImpl are more appropriate and less missleading.
> >
> > Regards,
> > Alin
> >
>
> Alin
>
> This feature does sound quite useful. It would be a quite welcome
> contribution to the project.
>
> You know your way around HttpClient quite well, I think you should be
> perfectly capable of putting together a patch with all the necessary
> changes in it.
>
> Cheers
>
> Oleg
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
>
Re: In-memory DNS resolver for HC
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Fri, 2011-09-02 at 17:58 +0300, Vasile Alin wrote:
> On 2 September 2011 17:45, Bill Speirs <bi...@gmail.com> wrote:
>
> > That does make for a slightly more compelling argument, but I would
> > say disable auto-redirects and handle them manually doing the DNS
> > resolution?
> >
>
> This is OK in most of the cases.I see that JMeter has the option to follow
> redirects automatically (with the limitations defined in the RFC), maybe in
> this case there is a nice feature to allow custom host entries in a section
> in the Preferences section. The question is if this case is frequent and
> worths the effort to change the httpclient library.
>
> Regarding the interface and implementation classes names, maybe DnsOverrider
> and InMemoryDnsOverriderImpl are more appropriate and less missleading.
>
> Regards,
> Alin
>
Alin
This feature does sound quite useful. It would be a quite welcome
contribution to the project.
You know your way around HttpClient quite well, I think you should be
perfectly capable of putting together a patch with all the necessary
changes in it.
Cheers
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: In-memory DNS resolver for HC
Posted by Vasile Alin <al...@gmail.com>.
On 2 September 2011 17:45, Bill Speirs <bi...@gmail.com> wrote:
> That does make for a slightly more compelling argument, but I would
> say disable auto-redirects and handle them manually doing the DNS
> resolution?
>
This is OK in most of the cases.I see that JMeter has the option to follow
redirects automatically (with the limitations defined in the RFC), maybe in
this case there is a nice feature to allow custom host entries in a section
in the Preferences section. The question is if this case is frequent and
worths the effort to change the httpclient library.
Regarding the interface and implementation classes names, maybe DnsOverrider
and InMemoryDnsOverriderImpl are more appropriate and less missleading.
Regards,
Alin
>
> Bill-
>
> On Fri, Sep 2, 2011 at 10:32 AM, Vasile Alin <al...@gmail.com>
> wrote:
> > This is ok when the first request is made, but how about the situation
> when
> > HttpClient follows redirects automatically?
> >
> > On 2 September 2011 16:39, Bill Speirs <bi...@gmail.com> wrote:
> >
> >> Why wouldn't you just have the code around HttppClient do this
> translation
> >> for you? Then you setup your requests to IP addresses, set the Host
> header
> >> to whatever you want and go.
> >>
> >> Adding a DNS resolver seems like a big (and possibly confusing) thing to
> >> add
> >> to the library when you can code around it.
> >>
> >> Bill-
> >> On Sep 1, 2011 4:18 PM, "Vasile Alin" <al...@gmail.com> wrote:
> >> > Hi,
> >> >
> >> > I had some situations when working directly with http-client or JMeter
> in
> >> > restrictive environments, where I wasn't able to edit the local
> >> /etc/hosts
> >> > file for resolving some custom hostnames. Based on the HttpUnit
> solution
> >> > [1], I was thinking that would be great if httpclient can lookup a
> >> hostname
> >> > in a custom DNS Resolver, editable or programable in the Java
> >> environment,
> >> > and fall-back to the default system address resolver if this isn't
> found
> >> in
> >> > the first place. For example the above snippet will resolve the "
> >> > www.alin.com" hostname to the one of the Google IPs:
> >> >
> >> > interface DnsResolver{
> >> >
> >> > InetAddress[] resolve(String hostname);
> >> >
> >> > }
> >> >
> >> > class InMemoryDnsResolverImpl implements DnsResolver{
> >> >
> >> > public InetAddress[] resolve(final String hostname){
> >> > try {
> >> > if ("www.alin.com".equals(hostname)) {
> >> > String ip = "74.125.39.105";
> >> > String[] ipParts = ip.split("\\.");
> >> >
> >> > byte[] byteIpAddress = new byte[4];
> >> > for (int i = 0; i < 4; i++) {
> >> > byteIpAddress[i] = Byte.parseByte(ipParts[i]);
> >> > }
> >> >
> >> > return new InetAddress[] { InetAddress
> >> > .getByAddress(byteIpAddress) };
> >> > } else {
> >> > return null;
> >> > }
> >> > } catch (UnknownHostException e) {
> >> > // log exception
> >> > return null;
> >> > }
> >> > }
> >> >
> >> > }
> >> >
> >> > public class DefaultClientConnectionOperator implements
> >> > ClientConnectionOperator {
> >> >
> >> > ....
> >> >
> >> > private DnsResolver dnsResolver;
> >> >
> >> > ...
> >> >
> >> > protected InetAddress[] resolveHostname(final String host) throws
> >> > UnknownHostException {
> >> > InetAddress[] resolvedAddress = null;
> >> >
> >> > if(dnResolver != null){
> >> > resolvedAddress = dnResolver.resolve(host);
> >> > }
> >> >
> >> > if(resolverAddress==null){
> >> > resolvedAddress = InetAddress.getAllByName(host);
> >> > }
> >> >
> >> > return resolvedAddress;
> >> >
> >> > }
> >> >
> >> > }
> >> >
> >> >
> >> > This functionality can be extended to get the IP addresses from a
> >> > properties file through an PropertyFileDnsResolverImpl and so on. What
> do
> >> > you think?
> >> >
> >> >
> >> > Regards,
> >> > Alin
> >> >
> >> >
> >> > [1] http://httpunit.sourceforge.net/
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
>
Re: In-memory DNS resolver for HC
Posted by Bill Speirs <bi...@gmail.com>.
That does make for a slightly more compelling argument, but I would
say disable auto-redirects and handle them manually doing the DNS
resolution?
Bill-
On Fri, Sep 2, 2011 at 10:32 AM, Vasile Alin <al...@gmail.com> wrote:
> This is ok when the first request is made, but how about the situation when
> HttpClient follows redirects automatically?
>
> On 2 September 2011 16:39, Bill Speirs <bi...@gmail.com> wrote:
>
>> Why wouldn't you just have the code around HttppClient do this translation
>> for you? Then you setup your requests to IP addresses, set the Host header
>> to whatever you want and go.
>>
>> Adding a DNS resolver seems like a big (and possibly confusing) thing to
>> add
>> to the library when you can code around it.
>>
>> Bill-
>> On Sep 1, 2011 4:18 PM, "Vasile Alin" <al...@gmail.com> wrote:
>> > Hi,
>> >
>> > I had some situations when working directly with http-client or JMeter in
>> > restrictive environments, where I wasn't able to edit the local
>> /etc/hosts
>> > file for resolving some custom hostnames. Based on the HttpUnit solution
>> > [1], I was thinking that would be great if httpclient can lookup a
>> hostname
>> > in a custom DNS Resolver, editable or programable in the Java
>> environment,
>> > and fall-back to the default system address resolver if this isn't found
>> in
>> > the first place. For example the above snippet will resolve the "
>> > www.alin.com" hostname to the one of the Google IPs:
>> >
>> > interface DnsResolver{
>> >
>> > InetAddress[] resolve(String hostname);
>> >
>> > }
>> >
>> > class InMemoryDnsResolverImpl implements DnsResolver{
>> >
>> > public InetAddress[] resolve(final String hostname){
>> > try {
>> > if ("www.alin.com".equals(hostname)) {
>> > String ip = "74.125.39.105";
>> > String[] ipParts = ip.split("\\.");
>> >
>> > byte[] byteIpAddress = new byte[4];
>> > for (int i = 0; i < 4; i++) {
>> > byteIpAddress[i] = Byte.parseByte(ipParts[i]);
>> > }
>> >
>> > return new InetAddress[] { InetAddress
>> > .getByAddress(byteIpAddress) };
>> > } else {
>> > return null;
>> > }
>> > } catch (UnknownHostException e) {
>> > // log exception
>> > return null;
>> > }
>> > }
>> >
>> > }
>> >
>> > public class DefaultClientConnectionOperator implements
>> > ClientConnectionOperator {
>> >
>> > ....
>> >
>> > private DnsResolver dnsResolver;
>> >
>> > ...
>> >
>> > protected InetAddress[] resolveHostname(final String host) throws
>> > UnknownHostException {
>> > InetAddress[] resolvedAddress = null;
>> >
>> > if(dnResolver != null){
>> > resolvedAddress = dnResolver.resolve(host);
>> > }
>> >
>> > if(resolverAddress==null){
>> > resolvedAddress = InetAddress.getAllByName(host);
>> > }
>> >
>> > return resolvedAddress;
>> >
>> > }
>> >
>> > }
>> >
>> >
>> > This functionality can be extended to get the IP addresses from a
>> > properties file through an PropertyFileDnsResolverImpl and so on. What do
>> > you think?
>> >
>> >
>> > Regards,
>> > Alin
>> >
>> >
>> > [1] http://httpunit.sourceforge.net/
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: In-memory DNS resolver for HC
Posted by Vasile Alin <al...@gmail.com>.
This is ok when the first request is made, but how about the situation when
HttpClient follows redirects automatically?
On 2 September 2011 16:39, Bill Speirs <bi...@gmail.com> wrote:
> Why wouldn't you just have the code around HttppClient do this translation
> for you? Then you setup your requests to IP addresses, set the Host header
> to whatever you want and go.
>
> Adding a DNS resolver seems like a big (and possibly confusing) thing to
> add
> to the library when you can code around it.
>
> Bill-
> On Sep 1, 2011 4:18 PM, "Vasile Alin" <al...@gmail.com> wrote:
> > Hi,
> >
> > I had some situations when working directly with http-client or JMeter in
> > restrictive environments, where I wasn't able to edit the local
> /etc/hosts
> > file for resolving some custom hostnames. Based on the HttpUnit solution
> > [1], I was thinking that would be great if httpclient can lookup a
> hostname
> > in a custom DNS Resolver, editable or programable in the Java
> environment,
> > and fall-back to the default system address resolver if this isn't found
> in
> > the first place. For example the above snippet will resolve the "
> > www.alin.com" hostname to the one of the Google IPs:
> >
> > interface DnsResolver{
> >
> > InetAddress[] resolve(String hostname);
> >
> > }
> >
> > class InMemoryDnsResolverImpl implements DnsResolver{
> >
> > public InetAddress[] resolve(final String hostname){
> > try {
> > if ("www.alin.com".equals(hostname)) {
> > String ip = "74.125.39.105";
> > String[] ipParts = ip.split("\\.");
> >
> > byte[] byteIpAddress = new byte[4];
> > for (int i = 0; i < 4; i++) {
> > byteIpAddress[i] = Byte.parseByte(ipParts[i]);
> > }
> >
> > return new InetAddress[] { InetAddress
> > .getByAddress(byteIpAddress) };
> > } else {
> > return null;
> > }
> > } catch (UnknownHostException e) {
> > // log exception
> > return null;
> > }
> > }
> >
> > }
> >
> > public class DefaultClientConnectionOperator implements
> > ClientConnectionOperator {
> >
> > ....
> >
> > private DnsResolver dnsResolver;
> >
> > ...
> >
> > protected InetAddress[] resolveHostname(final String host) throws
> > UnknownHostException {
> > InetAddress[] resolvedAddress = null;
> >
> > if(dnResolver != null){
> > resolvedAddress = dnResolver.resolve(host);
> > }
> >
> > if(resolverAddress==null){
> > resolvedAddress = InetAddress.getAllByName(host);
> > }
> >
> > return resolvedAddress;
> >
> > }
> >
> > }
> >
> >
> > This functionality can be extended to get the IP addresses from a
> > properties file through an PropertyFileDnsResolverImpl and so on. What do
> > you think?
> >
> >
> > Regards,
> > Alin
> >
> >
> > [1] http://httpunit.sourceforge.net/
>
Re: In-memory DNS resolver for HC
Posted by Bill Speirs <bi...@gmail.com>.
Why wouldn't you just have the code around HttppClient do this translation
for you? Then you setup your requests to IP addresses, set the Host header
to whatever you want and go.
Adding a DNS resolver seems like a big (and possibly confusing) thing to add
to the library when you can code around it.
Bill-
On Sep 1, 2011 4:18 PM, "Vasile Alin" <al...@gmail.com> wrote:
> Hi,
>
> I had some situations when working directly with http-client or JMeter in
> restrictive environments, where I wasn't able to edit the local /etc/hosts
> file for resolving some custom hostnames. Based on the HttpUnit solution
> [1], I was thinking that would be great if httpclient can lookup a
hostname
> in a custom DNS Resolver, editable or programable in the Java environment,
> and fall-back to the default system address resolver if this isn't found
in
> the first place. For example the above snippet will resolve the "
> www.alin.com" hostname to the one of the Google IPs:
>
> interface DnsResolver{
>
> InetAddress[] resolve(String hostname);
>
> }
>
> class InMemoryDnsResolverImpl implements DnsResolver{
>
> public InetAddress[] resolve(final String hostname){
> try {
> if ("www.alin.com".equals(hostname)) {
> String ip = "74.125.39.105";
> String[] ipParts = ip.split("\\.");
>
> byte[] byteIpAddress = new byte[4];
> for (int i = 0; i < 4; i++) {
> byteIpAddress[i] = Byte.parseByte(ipParts[i]);
> }
>
> return new InetAddress[] { InetAddress
> .getByAddress(byteIpAddress) };
> } else {
> return null;
> }
> } catch (UnknownHostException e) {
> // log exception
> return null;
> }
> }
>
> }
>
> public class DefaultClientConnectionOperator implements
> ClientConnectionOperator {
>
> ....
>
> private DnsResolver dnsResolver;
>
> ...
>
> protected InetAddress[] resolveHostname(final String host) throws
> UnknownHostException {
> InetAddress[] resolvedAddress = null;
>
> if(dnResolver != null){
> resolvedAddress = dnResolver.resolve(host);
> }
>
> if(resolverAddress==null){
> resolvedAddress = InetAddress.getAllByName(host);
> }
>
> return resolvedAddress;
>
> }
>
> }
>
>
> This functionality can be extended to get the IP addresses from a
> properties file through an PropertyFileDnsResolverImpl and so on. What do
> you think?
>
>
> Regards,
> Alin
>
>
> [1] http://httpunit.sourceforge.net/