You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crunch.apache.org by Josh Wills <jw...@cloudera.com> on 2014/03/31 17:13:45 UTC
hbase error I don't understand
Hey devs,
I'm back from a long break and I'm getting an odd exception on my hbase
integration tests running on my mac--
INFO org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob
- java.lang.NumberFormatException: For input string: ":22e5:2aff:fe15:2686"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Integer.parseInt(Integer.java:449)
at java.lang.Integer.parseInt(Integer.java:499)
at com.sun.jndi.dns.DnsClient.<init>(DnsClient.java:103)
at com.sun.jndi.dns.Resolver.<init>(Resolver.java:44)
at com.sun.jndi.dns.DnsContext.getResolver(DnsContext.java:553)
at com.sun.jndi.dns.DnsContext.c_getAttributes(DnsContext.java:413)
at
com.sun.jndi.toolkit.ctx.ComponentDirContext.p_getAttributes(ComponentDirContext.java:213)
at
com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.getAttributes(PartialCompositeDirContext.java:121)
at
com.sun.jndi.toolkit.url.GenericURLDirContext.getAttributes(GenericURLDirContext.java:85)
at
javax.naming.directory.InitialDirContext.getAttributes(InitialDirContext.java:123)
at org.apache.hadoop.net.DNS.reverseDns(DNS.java:84)
at
org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.reverseDNS(TableInputFormatBase.java:223)
at
org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:189)
I get it w/both hadoop-1 and hadoop-2-- any clues as to what's going on
here? To be fair, I'm running this from the Cloudera UK office for the
first time evah, and I'm wondering if there's some thing I need to do
differently here.
Thanks,
J
--
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>
Re: hbase error I don't understand
Posted by Micah Whitacre <mk...@gmail.com>.
We've had issues with IPv6 (though I haven't had the problem with Crunch
specifically). When we last had the issue it was actually ZK that had
trouble connecting to IPv6.
There are two fixes we've found:
The first is to set surefire to use IPv4
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Xmx2048m -Djava.net.preferIPv4Stack=true</argLine>
</configuration>
</plugin>
The second is to change the devices /etc/hosts to be more explicit on
the IPv4 entries so it chose that over the the IPv6.
Obviously the first would probably be preferred for repeatable builds.
On Mon, Mar 31, 2014 at 10:24 AM, Josh Wills <jo...@gmail.com> wrote:
> Got it, thanks Gabriel. Will try it again when I'm @ my hotel.
>
> J
>
>
> On Mon, Mar 31, 2014 at 8:23 AM, Gabriel Reid <gabriel.reid@gmail.com
> >wrote:
>
> > Hi Josh,
> >
> > I've run into things like this before as well when running HBase
> > integration tests. As far as I can understand it, it has something to
> > do with IPv6 addresses being returned when doing a reverse DNS lookup.
> > I think that I've only gotten this intermittently when running things
> > from home, which has made me assume that my ISP was doing something
> > with their DNS setup at the time.
> >
> > Unfortunately, I don't have any suggestions as to a resolution. This
> > problem has always disappeared on its own before I've figured out how
> > to resolve it properly.
> >
> > - Gabriel
> >
> > On Mon, Mar 31, 2014 at 5:13 PM, Josh Wills <jw...@cloudera.com> wrote:
> > > Hey devs,
> > >
> > > I'm back from a long break and I'm getting an odd exception on my hbase
> > > integration tests running on my mac--
> > >
> > > INFO
> > org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob
> > > - java.lang.NumberFormatException: For input string:
> > ":22e5:2aff:fe15:2686"
> > > at
> > >
> >
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> > > at java.lang.Integer.parseInt(Integer.java:449)
> > > at java.lang.Integer.parseInt(Integer.java:499)
> > > at com.sun.jndi.dns.DnsClient.<init>(DnsClient.java:103)
> > > at com.sun.jndi.dns.Resolver.<init>(Resolver.java:44)
> > > at com.sun.jndi.dns.DnsContext.getResolver(DnsContext.java:553)
> > > at com.sun.jndi.dns.DnsContext.c_getAttributes(DnsContext.java:413)
> > > at
> > >
> >
> com.sun.jndi.toolkit.ctx.ComponentDirContext.p_getAttributes(ComponentDirContext.java:213)
> > > at
> > >
> >
> com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.getAttributes(PartialCompositeDirContext.java:121)
> > > at
> > >
> >
> com.sun.jndi.toolkit.url.GenericURLDirContext.getAttributes(GenericURLDirContext.java:85)
> > > at
> > >
> >
> javax.naming.directory.InitialDirContext.getAttributes(InitialDirContext.java:123)
> > > at org.apache.hadoop.net.DNS.reverseDns(DNS.java:84)
> > > at
> > >
> >
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.reverseDNS(TableInputFormatBase.java:223)
> > > at
> > >
> >
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:189)
> > >
> > > I get it w/both hadoop-1 and hadoop-2-- any clues as to what's going on
> > > here? To be fair, I'm running this from the Cloudera UK office for the
> > > first time evah, and I'm wondering if there's some thing I need to do
> > > differently here.
> > >
> > > Thanks,
> > > J
> > >
> > > --
> > > Director of Data Science
> > > Cloudera <http://www.cloudera.com>
> > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> >
>
Re: hbase error I don't understand
Posted by Josh Wills <jo...@gmail.com>.
Got it, thanks Gabriel. Will try it again when I'm @ my hotel.
J
On Mon, Mar 31, 2014 at 8:23 AM, Gabriel Reid <ga...@gmail.com>wrote:
> Hi Josh,
>
> I've run into things like this before as well when running HBase
> integration tests. As far as I can understand it, it has something to
> do with IPv6 addresses being returned when doing a reverse DNS lookup.
> I think that I've only gotten this intermittently when running things
> from home, which has made me assume that my ISP was doing something
> with their DNS setup at the time.
>
> Unfortunately, I don't have any suggestions as to a resolution. This
> problem has always disappeared on its own before I've figured out how
> to resolve it properly.
>
> - Gabriel
>
> On Mon, Mar 31, 2014 at 5:13 PM, Josh Wills <jw...@cloudera.com> wrote:
> > Hey devs,
> >
> > I'm back from a long break and I'm getting an odd exception on my hbase
> > integration tests running on my mac--
> >
> > INFO
> org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob
> > - java.lang.NumberFormatException: For input string:
> ":22e5:2aff:fe15:2686"
> > at
> >
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> > at java.lang.Integer.parseInt(Integer.java:449)
> > at java.lang.Integer.parseInt(Integer.java:499)
> > at com.sun.jndi.dns.DnsClient.<init>(DnsClient.java:103)
> > at com.sun.jndi.dns.Resolver.<init>(Resolver.java:44)
> > at com.sun.jndi.dns.DnsContext.getResolver(DnsContext.java:553)
> > at com.sun.jndi.dns.DnsContext.c_getAttributes(DnsContext.java:413)
> > at
> >
> com.sun.jndi.toolkit.ctx.ComponentDirContext.p_getAttributes(ComponentDirContext.java:213)
> > at
> >
> com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.getAttributes(PartialCompositeDirContext.java:121)
> > at
> >
> com.sun.jndi.toolkit.url.GenericURLDirContext.getAttributes(GenericURLDirContext.java:85)
> > at
> >
> javax.naming.directory.InitialDirContext.getAttributes(InitialDirContext.java:123)
> > at org.apache.hadoop.net.DNS.reverseDns(DNS.java:84)
> > at
> >
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.reverseDNS(TableInputFormatBase.java:223)
> > at
> >
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:189)
> >
> > I get it w/both hadoop-1 and hadoop-2-- any clues as to what's going on
> > here? To be fair, I'm running this from the Cloudera UK office for the
> > first time evah, and I'm wondering if there's some thing I need to do
> > differently here.
> >
> > Thanks,
> > J
> >
> > --
> > Director of Data Science
> > Cloudera <http://www.cloudera.com>
> > Twitter: @josh_wills <http://twitter.com/josh_wills>
>
Re: hbase error I don't understand
Posted by Gabriel Reid <ga...@gmail.com>.
Hi Josh,
I've run into things like this before as well when running HBase
integration tests. As far as I can understand it, it has something to
do with IPv6 addresses being returned when doing a reverse DNS lookup.
I think that I've only gotten this intermittently when running things
from home, which has made me assume that my ISP was doing something
with their DNS setup at the time.
Unfortunately, I don't have any suggestions as to a resolution. This
problem has always disappeared on its own before I've figured out how
to resolve it properly.
- Gabriel
On Mon, Mar 31, 2014 at 5:13 PM, Josh Wills <jw...@cloudera.com> wrote:
> Hey devs,
>
> I'm back from a long break and I'm getting an odd exception on my hbase
> integration tests running on my mac--
>
> INFO org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob
> - java.lang.NumberFormatException: For input string: ":22e5:2aff:fe15:2686"
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Integer.parseInt(Integer.java:449)
> at java.lang.Integer.parseInt(Integer.java:499)
> at com.sun.jndi.dns.DnsClient.<init>(DnsClient.java:103)
> at com.sun.jndi.dns.Resolver.<init>(Resolver.java:44)
> at com.sun.jndi.dns.DnsContext.getResolver(DnsContext.java:553)
> at com.sun.jndi.dns.DnsContext.c_getAttributes(DnsContext.java:413)
> at
> com.sun.jndi.toolkit.ctx.ComponentDirContext.p_getAttributes(ComponentDirContext.java:213)
> at
> com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.getAttributes(PartialCompositeDirContext.java:121)
> at
> com.sun.jndi.toolkit.url.GenericURLDirContext.getAttributes(GenericURLDirContext.java:85)
> at
> javax.naming.directory.InitialDirContext.getAttributes(InitialDirContext.java:123)
> at org.apache.hadoop.net.DNS.reverseDns(DNS.java:84)
> at
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.reverseDNS(TableInputFormatBase.java:223)
> at
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:189)
>
> I get it w/both hadoop-1 and hadoop-2-- any clues as to what's going on
> here? To be fair, I'm running this from the Cloudera UK office for the
> first time evah, and I'm wondering if there's some thing I need to do
> differently here.
>
> Thanks,
> J
>
> --
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>