You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crunch.apache.org by Danushka Menikkumbura <da...@gmail.com> on 2014/10/14 15:08:16 UTC
Test failure on the master branch
Hi,
I am getting the following test failure while building Crunch. Have you got
an idea as to what may be the issue here?.
34787 [Thread-2057] INFO
org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob -
Job status available at: http://localhost:8080/
44548 [M:0;danushka:38318.oldLogCleaner] ERROR
org.apache.hadoop.hbase.client.HConnectionManager - Connection not found
in the list, can't delete it (connection
key=HConnectionKey{properties={hbase.zookeeper.quorum=localhost,
hbase.rpc.timeout=60000, hbase.zookeeper.property.clientPort=57963,
zookeeper.znode.parent=/hbase, hbase.client.retries.number=350,
hbase.client.pause=100}, username='danushka'}). May be the key was modified?
java.lang.Exception
at
org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:493)
at
org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:429)
at
org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.stop(ReplicationLogCleaner.java:141)
at
org.apache.hadoop.hbase.master.cleaner.CleanerChore.cleanup(CleanerChore.java:276)
at org.apache.hadoop.hbase.Chore.run(Chore.java:94)
at java.lang.Thread.run(Thread.java:745)
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.625 sec
Thanks,
Danushka
Re: Test failure on the master branch
Posted by Josh Wills <jw...@cloudera.com>.
Yeah, I'm right: we needed the 5711 hack in there ~ a million years ago to
get the WordCountHBaseIT to work:
https://www.google.com/search?q=crunch+hbase-5711
I wonder why it's back now.
On Wed, Oct 15, 2014 at 5:33 AM, Josh Wills <jw...@cloudera.com> wrote:
> So applying the bit mask workaround from HBASE-5711 fixes the
> HFileTargetIT test, but I have this deja vu-like feeling that we ran into
> something like this before, but the Internet at this hotel isn't quite
> solid enough for me to figure out when it was. Will keep poking around.
>
> J
>
> On Tue, Oct 14, 2014 at 9:17 PM, Josh Wills <jw...@cloudera.com> wrote:
>
>> I can replicate the original error- looks like dfs.data.dir permissions
>> aren't set properly in test mode for some reason. Will investigate and
>> figure out a patch.
>>
>>
>> On Tuesday, October 14, 2014, Micah Whitacre <mk...@gmail.com> wrote:
>>
>>> One thing I noticed is that is when testing we don't force IPv4.
>>> Zookeeper
>>> does not handle IPv6 well and can actually significantly slow down tests
>>> trying to resolve the localhost. This doesn't affect everyone because
>>> your
>>> /etc/hosts could help it resolve faster but does make machine setup
>>> important. We could try adding an argline like this to surefire/failsafe
>>> to enforce IPv4.
>>>
>>> <argLine>-Xmx512m -XX:MaxPermSize=128m
>>> -Djava.net.preferIPv4Stack=true</argLine>
>>>
>>> On Tue, Oct 14, 2014 at 9:28 AM, Danushka Menikkumbura <
>>> danushka.menikkumbura@gmail.com> wrote:
>>>
>>> > Thanks J!
>>> >
>>> > On Tue, Oct 14, 2014 at 9:57 AM, Josh Wills <jw...@cloudera.com>
>>> wrote:
>>> >
>>> > > I'm still rocking 1.7. Will give 1.8 a whirl this evening.
>>> > >
>>> > > J
>>> > >
>>> > > On Tue, Oct 14, 2014 at 6:53 AM, Danushka Menikkumbura <
>>> > > danushka.menikkumbura@gmail.com> wrote:
>>> > >
>>> > > > I am on Ubuntu 14.04.1 LTS and Java 1.8.0_20.
>>> > > >
>>> > > > BTW maybe it is a result of the commit
>>> > > > 3f98411364cec32a0a8c6681dfaabd43caa4dd60?.
>>> > > >
>>> > > > Thanks,
>>> > > > Danushka
>>> > > >
>>> > > >
>>> > > >
>>> > > > On Tue, Oct 14, 2014 at 9:44 AM, Josh Wills <jw...@cloudera.com>
>>> > wrote:
>>> > > >
>>> > > > > No, master is compatible w/Hadoop 1; the error you're seeing is
>>> > caused
>>> > > by
>>> > > > > the HBase testing code being flaky. I've experienced the flaky
>>> > > > > HFileTargetIT test on my machine before, but never on a regular
>>> > basis.
>>> > > > Can
>>> > > > > you give me the basics of your setup-- OS, java version, etc.?
>>> > > > >
>>> > > > > On Tue, Oct 14, 2014 at 6:38 AM, Danushka Menikkumbura <
>>> > > > > danushka.menikkumbura@gmail.com> wrote:
>>> > > > >
>>> > > > > > No I don't.
>>> > > > > >
>>> > > > > > That means the master branch is not compatible with Hadoop 1?
>>> > > > > >
>>> > > > > > Thanks,
>>> > > > > > Danushka
>>> > > > > >
>>> > > > > > On Tue, Oct 14, 2014 at 9:26 AM, Josh Wills <
>>> jwills@cloudera.com>
>>> > > > wrote:
>>> > > > > >
>>> > > > > > > Hrm, okay. Do you get it if you use the -Dcrunch.platform=2
>>> > option?
>>> > > > > > >
>>> > > > > > > On Tue, Oct 14, 2014 at 6:23 AM, Danushka Menikkumbura <
>>> > > > > > > danushka.menikkumbura@gmail.com> wrote:
>>> > > > > > >
>>> > > > > > > > Yes. I get it every time I try to build HBase module.
>>> > > > > > > >
>>> > > > > > > > FYI : org.apache.crunch.io.hbase.HFileTargetIT is the test
>>> in
>>> > > > error.
>>> > > > > > > >
>>> > > > > > > > Thanks,
>>> > > > > > > > Danushka
>>> > > > > > > >
>>> > > > > > > > On Tue, Oct 14, 2014 at 9:15 AM, Josh Wills <
>>> > jwills@cloudera.com
>>> > > >
>>> > > > > > wrote:
>>> > > > > > > >
>>> > > > > > > > > That can happen intermittently if the local HBase cluster
>>> > gets
>>> > > > hung
>>> > > > > > > up--
>>> > > > > > > > do
>>> > > > > > > > > you get it regularly (i.e., every time you run?)
>>> > > > > > > > >
>>> > > > > > > > > J
>>> > > > > > > > >
>>> > > > > > > > > On Tue, Oct 14, 2014 at 6:08 AM, Danushka Menikkumbura <
>>> > > > > > > > > danushka.menikkumbura@gmail.com> wrote:
>>> > > > > > > > >
>>> > > > > > > > > > Hi,
>>> > > > > > > > > >
>>> > > > > > > > > > I am getting the following test failure while building
>>> > > Crunch.
>>> > > > > Have
>>> > > > > > > you
>>> > > > > > > > > got
>>> > > > > > > > > > an idea as to what may be the issue here?.
>>> > > > > > > > > >
>>> > > > > > > > > > 34787 [Thread-2057] INFO
>>> > > > > > > > > >
>>> > > > > > > >
>>> > > >
>>> org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob
>>> > > > > > -
>>> > > > > > > > > > Job status available at: http://localhost:8080/
>>> > > > > > > > > > 44548 [M:0;danushka:38318.oldLogCleaner] ERROR
>>> > > > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager -
>>> > > Connection
>>> > > > > not
>>> > > > > > > > found
>>> > > > > > > > > > in the list, can't delete it (connection
>>> > > > > > > > > >
>>> > > > key=HConnectionKey{properties={hbase.zookeeper.quorum=localhost,
>>> > > > > > > > > > hbase.rpc.timeout=60000,
>>> > > > > hbase.zookeeper.property.clientPort=57963,
>>> > > > > > > > > > zookeeper.znode.parent=/hbase,
>>> > > hbase.client.retries.number=350,
>>> > > > > > > > > > hbase.client.pause=100}, username='danushka'}). May be
>>> the
>>> > > key
>>> > > > > was
>>> > > > > > > > > > modified?
>>> > > > > > > > > > java.lang.Exception
>>> > > > > > > > > > at
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:493)
>>> > > > > > > > > > at
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:429)
>>> > > > > > > > > > at
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.stop(ReplicationLogCleaner.java:141)
>>> > > > > > > > > > at
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> org.apache.hadoop.hbase.master.cleaner.CleanerChore.cleanup(CleanerChore.java:276)
>>> > > > > > > > > > at org.apache.hadoop.hbase.Chore.run(Chore.java:94)
>>> > > > > > > > > > at java.lang.Thread.run(Thread.java:745)
>>> > > > > > > > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time
>>> > > elapsed:
>>> > > > > > > 21.625
>>> > > > > > > > > sec
>>> > > > > > > > > >
>>> > > > > > > > > > Thanks,
>>> > > > > > > > > > Danushka
>>> > > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > --
>>> > > > > > > > > Director of Data Science
>>> > > > > > > > > Cloudera <http://www.cloudera.com>
>>> > > > > > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
>>> > > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > --
>>> > > > > > > Director of Data Science
>>> > > > > > > Cloudera <http://www.cloudera.com>
>>> > > > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > --
>>> > > > > Director of Data Science
>>> > > > > Cloudera <http://www.cloudera.com>
>>> > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
>>> > > > >
>>> > > >
>>> > >
>>> > >
>>> > >
>>> > > --
>>> > > Director of Data Science
>>> > > Cloudera <http://www.cloudera.com>
>>> > > Twitter: @josh_wills <http://twitter.com/josh_wills>
>>> > >
>>> >
>>>
>>
>>
>> --
>> Director of Data Science
>> Cloudera <http://www.cloudera.com>
>> Twitter: @josh_wills <http://twitter.com/josh_wills>
>>
>>
>
>
> --
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>
>
--
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>
Re: Test failure on the master branch
Posted by Josh Wills <jw...@cloudera.com>.
So applying the bit mask workaround from HBASE-5711 fixes the HFileTargetIT
test, but I have this deja vu-like feeling that we ran into something like
this before, but the Internet at this hotel isn't quite solid enough for me
to figure out when it was. Will keep poking around.
J
On Tue, Oct 14, 2014 at 9:17 PM, Josh Wills <jw...@cloudera.com> wrote:
> I can replicate the original error- looks like dfs.data.dir permissions
> aren't set properly in test mode for some reason. Will investigate and
> figure out a patch.
>
>
> On Tuesday, October 14, 2014, Micah Whitacre <mk...@gmail.com> wrote:
>
>> One thing I noticed is that is when testing we don't force IPv4.
>> Zookeeper
>> does not handle IPv6 well and can actually significantly slow down tests
>> trying to resolve the localhost. This doesn't affect everyone because
>> your
>> /etc/hosts could help it resolve faster but does make machine setup
>> important. We could try adding an argline like this to surefire/failsafe
>> to enforce IPv4.
>>
>> <argLine>-Xmx512m -XX:MaxPermSize=128m
>> -Djava.net.preferIPv4Stack=true</argLine>
>>
>> On Tue, Oct 14, 2014 at 9:28 AM, Danushka Menikkumbura <
>> danushka.menikkumbura@gmail.com> wrote:
>>
>> > Thanks J!
>> >
>> > On Tue, Oct 14, 2014 at 9:57 AM, Josh Wills <jw...@cloudera.com>
>> wrote:
>> >
>> > > I'm still rocking 1.7. Will give 1.8 a whirl this evening.
>> > >
>> > > J
>> > >
>> > > On Tue, Oct 14, 2014 at 6:53 AM, Danushka Menikkumbura <
>> > > danushka.menikkumbura@gmail.com> wrote:
>> > >
>> > > > I am on Ubuntu 14.04.1 LTS and Java 1.8.0_20.
>> > > >
>> > > > BTW maybe it is a result of the commit
>> > > > 3f98411364cec32a0a8c6681dfaabd43caa4dd60?.
>> > > >
>> > > > Thanks,
>> > > > Danushka
>> > > >
>> > > >
>> > > >
>> > > > On Tue, Oct 14, 2014 at 9:44 AM, Josh Wills <jw...@cloudera.com>
>> > wrote:
>> > > >
>> > > > > No, master is compatible w/Hadoop 1; the error you're seeing is
>> > caused
>> > > by
>> > > > > the HBase testing code being flaky. I've experienced the flaky
>> > > > > HFileTargetIT test on my machine before, but never on a regular
>> > basis.
>> > > > Can
>> > > > > you give me the basics of your setup-- OS, java version, etc.?
>> > > > >
>> > > > > On Tue, Oct 14, 2014 at 6:38 AM, Danushka Menikkumbura <
>> > > > > danushka.menikkumbura@gmail.com> wrote:
>> > > > >
>> > > > > > No I don't.
>> > > > > >
>> > > > > > That means the master branch is not compatible with Hadoop 1?
>> > > > > >
>> > > > > > Thanks,
>> > > > > > Danushka
>> > > > > >
>> > > > > > On Tue, Oct 14, 2014 at 9:26 AM, Josh Wills <
>> jwills@cloudera.com>
>> > > > wrote:
>> > > > > >
>> > > > > > > Hrm, okay. Do you get it if you use the -Dcrunch.platform=2
>> > option?
>> > > > > > >
>> > > > > > > On Tue, Oct 14, 2014 at 6:23 AM, Danushka Menikkumbura <
>> > > > > > > danushka.menikkumbura@gmail.com> wrote:
>> > > > > > >
>> > > > > > > > Yes. I get it every time I try to build HBase module.
>> > > > > > > >
>> > > > > > > > FYI : org.apache.crunch.io.hbase.HFileTargetIT is the test
>> in
>> > > > error.
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > > Danushka
>> > > > > > > >
>> > > > > > > > On Tue, Oct 14, 2014 at 9:15 AM, Josh Wills <
>> > jwills@cloudera.com
>> > > >
>> > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > That can happen intermittently if the local HBase cluster
>> > gets
>> > > > hung
>> > > > > > > up--
>> > > > > > > > do
>> > > > > > > > > you get it regularly (i.e., every time you run?)
>> > > > > > > > >
>> > > > > > > > > J
>> > > > > > > > >
>> > > > > > > > > On Tue, Oct 14, 2014 at 6:08 AM, Danushka Menikkumbura <
>> > > > > > > > > danushka.menikkumbura@gmail.com> wrote:
>> > > > > > > > >
>> > > > > > > > > > Hi,
>> > > > > > > > > >
>> > > > > > > > > > I am getting the following test failure while building
>> > > Crunch.
>> > > > > Have
>> > > > > > > you
>> > > > > > > > > got
>> > > > > > > > > > an idea as to what may be the issue here?.
>> > > > > > > > > >
>> > > > > > > > > > 34787 [Thread-2057] INFO
>> > > > > > > > > >
>> > > > > > > >
>> > > >
>> org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob
>> > > > > > -
>> > > > > > > > > > Job status available at: http://localhost:8080/
>> > > > > > > > > > 44548 [M:0;danushka:38318.oldLogCleaner] ERROR
>> > > > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager -
>> > > Connection
>> > > > > not
>> > > > > > > > found
>> > > > > > > > > > in the list, can't delete it (connection
>> > > > > > > > > >
>> > > > key=HConnectionKey{properties={hbase.zookeeper.quorum=localhost,
>> > > > > > > > > > hbase.rpc.timeout=60000,
>> > > > > hbase.zookeeper.property.clientPort=57963,
>> > > > > > > > > > zookeeper.znode.parent=/hbase,
>> > > hbase.client.retries.number=350,
>> > > > > > > > > > hbase.client.pause=100}, username='danushka'}). May be
>> the
>> > > key
>> > > > > was
>> > > > > > > > > > modified?
>> > > > > > > > > > java.lang.Exception
>> > > > > > > > > > at
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:493)
>> > > > > > > > > > at
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:429)
>> > > > > > > > > > at
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.stop(ReplicationLogCleaner.java:141)
>> > > > > > > > > > at
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.master.cleaner.CleanerChore.cleanup(CleanerChore.java:276)
>> > > > > > > > > > at org.apache.hadoop.hbase.Chore.run(Chore.java:94)
>> > > > > > > > > > at java.lang.Thread.run(Thread.java:745)
>> > > > > > > > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time
>> > > elapsed:
>> > > > > > > 21.625
>> > > > > > > > > sec
>> > > > > > > > > >
>> > > > > > > > > > Thanks,
>> > > > > > > > > > Danushka
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > --
>> > > > > > > > > Director of Data Science
>> > > > > > > > > Cloudera <http://www.cloudera.com>
>> > > > > > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > --
>> > > > > > > Director of Data Science
>> > > > > > > Cloudera <http://www.cloudera.com>
>> > > > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
>> > > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Director of Data Science
>> > > > > Cloudera <http://www.cloudera.com>
>> > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
>> > > > >
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Director of Data Science
>> > > Cloudera <http://www.cloudera.com>
>> > > Twitter: @josh_wills <http://twitter.com/josh_wills>
>> > >
>> >
>>
>
>
> --
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>
>
>
--
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>
Re: Test failure on the master branch
Posted by Josh Wills <jw...@cloudera.com>.
I can replicate the original error- looks like dfs.data.dir permissions
aren't set properly in test mode for some reason. Will investigate and
figure out a patch.
On Tuesday, October 14, 2014, Micah Whitacre <mk...@gmail.com> wrote:
> One thing I noticed is that is when testing we don't force IPv4. Zookeeper
> does not handle IPv6 well and can actually significantly slow down tests
> trying to resolve the localhost. This doesn't affect everyone because your
> /etc/hosts could help it resolve faster but does make machine setup
> important. We could try adding an argline like this to surefire/failsafe
> to enforce IPv4.
>
> <argLine>-Xmx512m -XX:MaxPermSize=128m
> -Djava.net.preferIPv4Stack=true</argLine>
>
> On Tue, Oct 14, 2014 at 9:28 AM, Danushka Menikkumbura <
> danushka.menikkumbura@gmail.com <javascript:;>> wrote:
>
> > Thanks J!
> >
> > On Tue, Oct 14, 2014 at 9:57 AM, Josh Wills <jwills@cloudera.com
> <javascript:;>> wrote:
> >
> > > I'm still rocking 1.7. Will give 1.8 a whirl this evening.
> > >
> > > J
> > >
> > > On Tue, Oct 14, 2014 at 6:53 AM, Danushka Menikkumbura <
> > > danushka.menikkumbura@gmail.com <javascript:;>> wrote:
> > >
> > > > I am on Ubuntu 14.04.1 LTS and Java 1.8.0_20.
> > > >
> > > > BTW maybe it is a result of the commit
> > > > 3f98411364cec32a0a8c6681dfaabd43caa4dd60?.
> > > >
> > > > Thanks,
> > > > Danushka
> > > >
> > > >
> > > >
> > > > On Tue, Oct 14, 2014 at 9:44 AM, Josh Wills <jwills@cloudera.com
> <javascript:;>>
> > wrote:
> > > >
> > > > > No, master is compatible w/Hadoop 1; the error you're seeing is
> > caused
> > > by
> > > > > the HBase testing code being flaky. I've experienced the flaky
> > > > > HFileTargetIT test on my machine before, but never on a regular
> > basis.
> > > > Can
> > > > > you give me the basics of your setup-- OS, java version, etc.?
> > > > >
> > > > > On Tue, Oct 14, 2014 at 6:38 AM, Danushka Menikkumbura <
> > > > > danushka.menikkumbura@gmail.com <javascript:;>> wrote:
> > > > >
> > > > > > No I don't.
> > > > > >
> > > > > > That means the master branch is not compatible with Hadoop 1?
> > > > > >
> > > > > > Thanks,
> > > > > > Danushka
> > > > > >
> > > > > > On Tue, Oct 14, 2014 at 9:26 AM, Josh Wills <jwills@cloudera.com
> <javascript:;>>
> > > > wrote:
> > > > > >
> > > > > > > Hrm, okay. Do you get it if you use the -Dcrunch.platform=2
> > option?
> > > > > > >
> > > > > > > On Tue, Oct 14, 2014 at 6:23 AM, Danushka Menikkumbura <
> > > > > > > danushka.menikkumbura@gmail.com <javascript:;>> wrote:
> > > > > > >
> > > > > > > > Yes. I get it every time I try to build HBase module.
> > > > > > > >
> > > > > > > > FYI : org.apache.crunch.io.hbase.HFileTargetIT is the test in
> > > > error.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Danushka
> > > > > > > >
> > > > > > > > On Tue, Oct 14, 2014 at 9:15 AM, Josh Wills <
> > jwills@cloudera.com <javascript:;>
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > That can happen intermittently if the local HBase cluster
> > gets
> > > > hung
> > > > > > > up--
> > > > > > > > do
> > > > > > > > > you get it regularly (i.e., every time you run?)
> > > > > > > > >
> > > > > > > > > J
> > > > > > > > >
> > > > > > > > > On Tue, Oct 14, 2014 at 6:08 AM, Danushka Menikkumbura <
> > > > > > > > > danushka.menikkumbura@gmail.com <javascript:;>> wrote:
> > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > I am getting the following test failure while building
> > > Crunch.
> > > > > Have
> > > > > > > you
> > > > > > > > > got
> > > > > > > > > > an idea as to what may be the issue here?.
> > > > > > > > > >
> > > > > > > > > > 34787 [Thread-2057] INFO
> > > > > > > > > >
> > > > > > > >
> > > > org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob
> > > > > > -
> > > > > > > > > > Job status available at: http://localhost:8080/
> > > > > > > > > > 44548 [M:0;danushka:38318.oldLogCleaner] ERROR
> > > > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager -
> > > Connection
> > > > > not
> > > > > > > > found
> > > > > > > > > > in the list, can't delete it (connection
> > > > > > > > > >
> > > > key=HConnectionKey{properties={hbase.zookeeper.quorum=localhost,
> > > > > > > > > > hbase.rpc.timeout=60000,
> > > > > hbase.zookeeper.property.clientPort=57963,
> > > > > > > > > > zookeeper.znode.parent=/hbase,
> > > hbase.client.retries.number=350,
> > > > > > > > > > hbase.client.pause=100}, username='danushka'}). May be
> the
> > > key
> > > > > was
> > > > > > > > > > modified?
> > > > > > > > > > java.lang.Exception
> > > > > > > > > > at
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:493)
> > > > > > > > > > at
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:429)
> > > > > > > > > > at
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.stop(ReplicationLogCleaner.java:141)
> > > > > > > > > > at
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.cleanup(CleanerChore.java:276)
> > > > > > > > > > at org.apache.hadoop.hbase.Chore.run(Chore.java:94)
> > > > > > > > > > at java.lang.Thread.run(Thread.java:745)
> > > > > > > > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time
> > > elapsed:
> > > > > > > 21.625
> > > > > > > > > sec
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Danushka
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Director of Data Science
> > > > > > > > > Cloudera <http://www.cloudera.com>
> > > > > > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Director of Data Science
> > > > > > > Cloudera <http://www.cloudera.com>
> > > > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Director of Data Science
> > > > > Cloudera <http://www.cloudera.com>
> > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Director of Data Science
> > > Cloudera <http://www.cloudera.com>
> > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> > >
> >
>
--
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>
Re: Test failure on the master branch
Posted by Micah Whitacre <mk...@gmail.com>.
One thing I noticed is that is when testing we don't force IPv4. Zookeeper
does not handle IPv6 well and can actually significantly slow down tests
trying to resolve the localhost. This doesn't affect everyone because your
/etc/hosts could help it resolve faster but does make machine setup
important. We could try adding an argline like this to surefire/failsafe
to enforce IPv4.
<argLine>-Xmx512m -XX:MaxPermSize=128m
-Djava.net.preferIPv4Stack=true</argLine>
On Tue, Oct 14, 2014 at 9:28 AM, Danushka Menikkumbura <
danushka.menikkumbura@gmail.com> wrote:
> Thanks J!
>
> On Tue, Oct 14, 2014 at 9:57 AM, Josh Wills <jw...@cloudera.com> wrote:
>
> > I'm still rocking 1.7. Will give 1.8 a whirl this evening.
> >
> > J
> >
> > On Tue, Oct 14, 2014 at 6:53 AM, Danushka Menikkumbura <
> > danushka.menikkumbura@gmail.com> wrote:
> >
> > > I am on Ubuntu 14.04.1 LTS and Java 1.8.0_20.
> > >
> > > BTW maybe it is a result of the commit
> > > 3f98411364cec32a0a8c6681dfaabd43caa4dd60?.
> > >
> > > Thanks,
> > > Danushka
> > >
> > >
> > >
> > > On Tue, Oct 14, 2014 at 9:44 AM, Josh Wills <jw...@cloudera.com>
> wrote:
> > >
> > > > No, master is compatible w/Hadoop 1; the error you're seeing is
> caused
> > by
> > > > the HBase testing code being flaky. I've experienced the flaky
> > > > HFileTargetIT test on my machine before, but never on a regular
> basis.
> > > Can
> > > > you give me the basics of your setup-- OS, java version, etc.?
> > > >
> > > > On Tue, Oct 14, 2014 at 6:38 AM, Danushka Menikkumbura <
> > > > danushka.menikkumbura@gmail.com> wrote:
> > > >
> > > > > No I don't.
> > > > >
> > > > > That means the master branch is not compatible with Hadoop 1?
> > > > >
> > > > > Thanks,
> > > > > Danushka
> > > > >
> > > > > On Tue, Oct 14, 2014 at 9:26 AM, Josh Wills <jw...@cloudera.com>
> > > wrote:
> > > > >
> > > > > > Hrm, okay. Do you get it if you use the -Dcrunch.platform=2
> option?
> > > > > >
> > > > > > On Tue, Oct 14, 2014 at 6:23 AM, Danushka Menikkumbura <
> > > > > > danushka.menikkumbura@gmail.com> wrote:
> > > > > >
> > > > > > > Yes. I get it every time I try to build HBase module.
> > > > > > >
> > > > > > > FYI : org.apache.crunch.io.hbase.HFileTargetIT is the test in
> > > error.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Danushka
> > > > > > >
> > > > > > > On Tue, Oct 14, 2014 at 9:15 AM, Josh Wills <
> jwills@cloudera.com
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > That can happen intermittently if the local HBase cluster
> gets
> > > hung
> > > > > > up--
> > > > > > > do
> > > > > > > > you get it regularly (i.e., every time you run?)
> > > > > > > >
> > > > > > > > J
> > > > > > > >
> > > > > > > > On Tue, Oct 14, 2014 at 6:08 AM, Danushka Menikkumbura <
> > > > > > > > danushka.menikkumbura@gmail.com> wrote:
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > I am getting the following test failure while building
> > Crunch.
> > > > Have
> > > > > > you
> > > > > > > > got
> > > > > > > > > an idea as to what may be the issue here?.
> > > > > > > > >
> > > > > > > > > 34787 [Thread-2057] INFO
> > > > > > > > >
> > > > > > >
> > > org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob
> > > > > -
> > > > > > > > > Job status available at: http://localhost:8080/
> > > > > > > > > 44548 [M:0;danushka:38318.oldLogCleaner] ERROR
> > > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager -
> > Connection
> > > > not
> > > > > > > found
> > > > > > > > > in the list, can't delete it (connection
> > > > > > > > >
> > > key=HConnectionKey{properties={hbase.zookeeper.quorum=localhost,
> > > > > > > > > hbase.rpc.timeout=60000,
> > > > hbase.zookeeper.property.clientPort=57963,
> > > > > > > > > zookeeper.znode.parent=/hbase,
> > hbase.client.retries.number=350,
> > > > > > > > > hbase.client.pause=100}, username='danushka'}). May be the
> > key
> > > > was
> > > > > > > > > modified?
> > > > > > > > > java.lang.Exception
> > > > > > > > > at
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:493)
> > > > > > > > > at
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:429)
> > > > > > > > > at
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.stop(ReplicationLogCleaner.java:141)
> > > > > > > > > at
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.cleanup(CleanerChore.java:276)
> > > > > > > > > at org.apache.hadoop.hbase.Chore.run(Chore.java:94)
> > > > > > > > > at java.lang.Thread.run(Thread.java:745)
> > > > > > > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time
> > elapsed:
> > > > > > 21.625
> > > > > > > > sec
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Danushka
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Director of Data Science
> > > > > > > > Cloudera <http://www.cloudera.com>
> > > > > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Director of Data Science
> > > > > > Cloudera <http://www.cloudera.com>
> > > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Director of Data Science
> > > > Cloudera <http://www.cloudera.com>
> > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> > > >
> > >
> >
> >
> >
> > --
> > Director of Data Science
> > Cloudera <http://www.cloudera.com>
> > Twitter: @josh_wills <http://twitter.com/josh_wills>
> >
>
Re: Test failure on the master branch
Posted by Danushka Menikkumbura <da...@gmail.com>.
Thanks J!
On Tue, Oct 14, 2014 at 9:57 AM, Josh Wills <jw...@cloudera.com> wrote:
> I'm still rocking 1.7. Will give 1.8 a whirl this evening.
>
> J
>
> On Tue, Oct 14, 2014 at 6:53 AM, Danushka Menikkumbura <
> danushka.menikkumbura@gmail.com> wrote:
>
> > I am on Ubuntu 14.04.1 LTS and Java 1.8.0_20.
> >
> > BTW maybe it is a result of the commit
> > 3f98411364cec32a0a8c6681dfaabd43caa4dd60?.
> >
> > Thanks,
> > Danushka
> >
> >
> >
> > On Tue, Oct 14, 2014 at 9:44 AM, Josh Wills <jw...@cloudera.com> wrote:
> >
> > > No, master is compatible w/Hadoop 1; the error you're seeing is caused
> by
> > > the HBase testing code being flaky. I've experienced the flaky
> > > HFileTargetIT test on my machine before, but never on a regular basis.
> > Can
> > > you give me the basics of your setup-- OS, java version, etc.?
> > >
> > > On Tue, Oct 14, 2014 at 6:38 AM, Danushka Menikkumbura <
> > > danushka.menikkumbura@gmail.com> wrote:
> > >
> > > > No I don't.
> > > >
> > > > That means the master branch is not compatible with Hadoop 1?
> > > >
> > > > Thanks,
> > > > Danushka
> > > >
> > > > On Tue, Oct 14, 2014 at 9:26 AM, Josh Wills <jw...@cloudera.com>
> > wrote:
> > > >
> > > > > Hrm, okay. Do you get it if you use the -Dcrunch.platform=2 option?
> > > > >
> > > > > On Tue, Oct 14, 2014 at 6:23 AM, Danushka Menikkumbura <
> > > > > danushka.menikkumbura@gmail.com> wrote:
> > > > >
> > > > > > Yes. I get it every time I try to build HBase module.
> > > > > >
> > > > > > FYI : org.apache.crunch.io.hbase.HFileTargetIT is the test in
> > error.
> > > > > >
> > > > > > Thanks,
> > > > > > Danushka
> > > > > >
> > > > > > On Tue, Oct 14, 2014 at 9:15 AM, Josh Wills <jwills@cloudera.com
> >
> > > > wrote:
> > > > > >
> > > > > > > That can happen intermittently if the local HBase cluster gets
> > hung
> > > > > up--
> > > > > > do
> > > > > > > you get it regularly (i.e., every time you run?)
> > > > > > >
> > > > > > > J
> > > > > > >
> > > > > > > On Tue, Oct 14, 2014 at 6:08 AM, Danushka Menikkumbura <
> > > > > > > danushka.menikkumbura@gmail.com> wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > I am getting the following test failure while building
> Crunch.
> > > Have
> > > > > you
> > > > > > > got
> > > > > > > > an idea as to what may be the issue here?.
> > > > > > > >
> > > > > > > > 34787 [Thread-2057] INFO
> > > > > > > >
> > > > > >
> > org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob
> > > > -
> > > > > > > > Job status available at: http://localhost:8080/
> > > > > > > > 44548 [M:0;danushka:38318.oldLogCleaner] ERROR
> > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager -
> Connection
> > > not
> > > > > > found
> > > > > > > > in the list, can't delete it (connection
> > > > > > > >
> > key=HConnectionKey{properties={hbase.zookeeper.quorum=localhost,
> > > > > > > > hbase.rpc.timeout=60000,
> > > hbase.zookeeper.property.clientPort=57963,
> > > > > > > > zookeeper.znode.parent=/hbase,
> hbase.client.retries.number=350,
> > > > > > > > hbase.client.pause=100}, username='danushka'}). May be the
> key
> > > was
> > > > > > > > modified?
> > > > > > > > java.lang.Exception
> > > > > > > > at
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:493)
> > > > > > > > at
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:429)
> > > > > > > > at
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.stop(ReplicationLogCleaner.java:141)
> > > > > > > > at
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.cleanup(CleanerChore.java:276)
> > > > > > > > at org.apache.hadoop.hbase.Chore.run(Chore.java:94)
> > > > > > > > at java.lang.Thread.run(Thread.java:745)
> > > > > > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time
> elapsed:
> > > > > 21.625
> > > > > > > sec
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Danushka
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Director of Data Science
> > > > > > > Cloudera <http://www.cloudera.com>
> > > > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Director of Data Science
> > > > > Cloudera <http://www.cloudera.com>
> > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Director of Data Science
> > > Cloudera <http://www.cloudera.com>
> > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> > >
> >
>
>
>
> --
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>
>
Re: Test failure on the master branch
Posted by Josh Wills <jw...@cloudera.com>.
I'm still rocking 1.7. Will give 1.8 a whirl this evening.
J
On Tue, Oct 14, 2014 at 6:53 AM, Danushka Menikkumbura <
danushka.menikkumbura@gmail.com> wrote:
> I am on Ubuntu 14.04.1 LTS and Java 1.8.0_20.
>
> BTW maybe it is a result of the commit
> 3f98411364cec32a0a8c6681dfaabd43caa4dd60?.
>
> Thanks,
> Danushka
>
>
>
> On Tue, Oct 14, 2014 at 9:44 AM, Josh Wills <jw...@cloudera.com> wrote:
>
> > No, master is compatible w/Hadoop 1; the error you're seeing is caused by
> > the HBase testing code being flaky. I've experienced the flaky
> > HFileTargetIT test on my machine before, but never on a regular basis.
> Can
> > you give me the basics of your setup-- OS, java version, etc.?
> >
> > On Tue, Oct 14, 2014 at 6:38 AM, Danushka Menikkumbura <
> > danushka.menikkumbura@gmail.com> wrote:
> >
> > > No I don't.
> > >
> > > That means the master branch is not compatible with Hadoop 1?
> > >
> > > Thanks,
> > > Danushka
> > >
> > > On Tue, Oct 14, 2014 at 9:26 AM, Josh Wills <jw...@cloudera.com>
> wrote:
> > >
> > > > Hrm, okay. Do you get it if you use the -Dcrunch.platform=2 option?
> > > >
> > > > On Tue, Oct 14, 2014 at 6:23 AM, Danushka Menikkumbura <
> > > > danushka.menikkumbura@gmail.com> wrote:
> > > >
> > > > > Yes. I get it every time I try to build HBase module.
> > > > >
> > > > > FYI : org.apache.crunch.io.hbase.HFileTargetIT is the test in
> error.
> > > > >
> > > > > Thanks,
> > > > > Danushka
> > > > >
> > > > > On Tue, Oct 14, 2014 at 9:15 AM, Josh Wills <jw...@cloudera.com>
> > > wrote:
> > > > >
> > > > > > That can happen intermittently if the local HBase cluster gets
> hung
> > > > up--
> > > > > do
> > > > > > you get it regularly (i.e., every time you run?)
> > > > > >
> > > > > > J
> > > > > >
> > > > > > On Tue, Oct 14, 2014 at 6:08 AM, Danushka Menikkumbura <
> > > > > > danushka.menikkumbura@gmail.com> wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I am getting the following test failure while building Crunch.
> > Have
> > > > you
> > > > > > got
> > > > > > > an idea as to what may be the issue here?.
> > > > > > >
> > > > > > > 34787 [Thread-2057] INFO
> > > > > > >
> > > > >
> org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob
> > > -
> > > > > > > Job status available at: http://localhost:8080/
> > > > > > > 44548 [M:0;danushka:38318.oldLogCleaner] ERROR
> > > > > > > org.apache.hadoop.hbase.client.HConnectionManager - Connection
> > not
> > > > > found
> > > > > > > in the list, can't delete it (connection
> > > > > > >
> key=HConnectionKey{properties={hbase.zookeeper.quorum=localhost,
> > > > > > > hbase.rpc.timeout=60000,
> > hbase.zookeeper.property.clientPort=57963,
> > > > > > > zookeeper.znode.parent=/hbase, hbase.client.retries.number=350,
> > > > > > > hbase.client.pause=100}, username='danushka'}). May be the key
> > was
> > > > > > > modified?
> > > > > > > java.lang.Exception
> > > > > > > at
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:493)
> > > > > > > at
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:429)
> > > > > > > at
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.stop(ReplicationLogCleaner.java:141)
> > > > > > > at
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.cleanup(CleanerChore.java:276)
> > > > > > > at org.apache.hadoop.hbase.Chore.run(Chore.java:94)
> > > > > > > at java.lang.Thread.run(Thread.java:745)
> > > > > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > > > 21.625
> > > > > > sec
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Danushka
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Director of Data Science
> > > > > > Cloudera <http://www.cloudera.com>
> > > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Director of Data Science
> > > > Cloudera <http://www.cloudera.com>
> > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> > > >
> > >
> >
> >
> >
> > --
> > Director of Data Science
> > Cloudera <http://www.cloudera.com>
> > Twitter: @josh_wills <http://twitter.com/josh_wills>
> >
>
--
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>
Re: Test failure on the master branch
Posted by Danushka Menikkumbura <da...@gmail.com>.
I am on Ubuntu 14.04.1 LTS and Java 1.8.0_20.
BTW maybe it is a result of the commit
3f98411364cec32a0a8c6681dfaabd43caa4dd60?.
Thanks,
Danushka
On Tue, Oct 14, 2014 at 9:44 AM, Josh Wills <jw...@cloudera.com> wrote:
> No, master is compatible w/Hadoop 1; the error you're seeing is caused by
> the HBase testing code being flaky. I've experienced the flaky
> HFileTargetIT test on my machine before, but never on a regular basis. Can
> you give me the basics of your setup-- OS, java version, etc.?
>
> On Tue, Oct 14, 2014 at 6:38 AM, Danushka Menikkumbura <
> danushka.menikkumbura@gmail.com> wrote:
>
> > No I don't.
> >
> > That means the master branch is not compatible with Hadoop 1?
> >
> > Thanks,
> > Danushka
> >
> > On Tue, Oct 14, 2014 at 9:26 AM, Josh Wills <jw...@cloudera.com> wrote:
> >
> > > Hrm, okay. Do you get it if you use the -Dcrunch.platform=2 option?
> > >
> > > On Tue, Oct 14, 2014 at 6:23 AM, Danushka Menikkumbura <
> > > danushka.menikkumbura@gmail.com> wrote:
> > >
> > > > Yes. I get it every time I try to build HBase module.
> > > >
> > > > FYI : org.apache.crunch.io.hbase.HFileTargetIT is the test in error.
> > > >
> > > > Thanks,
> > > > Danushka
> > > >
> > > > On Tue, Oct 14, 2014 at 9:15 AM, Josh Wills <jw...@cloudera.com>
> > wrote:
> > > >
> > > > > That can happen intermittently if the local HBase cluster gets hung
> > > up--
> > > > do
> > > > > you get it regularly (i.e., every time you run?)
> > > > >
> > > > > J
> > > > >
> > > > > On Tue, Oct 14, 2014 at 6:08 AM, Danushka Menikkumbura <
> > > > > danushka.menikkumbura@gmail.com> wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I am getting the following test failure while building Crunch.
> Have
> > > you
> > > > > got
> > > > > > an idea as to what may be the issue here?.
> > > > > >
> > > > > > 34787 [Thread-2057] INFO
> > > > > >
> > > > org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob
> > -
> > > > > > Job status available at: http://localhost:8080/
> > > > > > 44548 [M:0;danushka:38318.oldLogCleaner] ERROR
> > > > > > org.apache.hadoop.hbase.client.HConnectionManager - Connection
> not
> > > > found
> > > > > > in the list, can't delete it (connection
> > > > > > key=HConnectionKey{properties={hbase.zookeeper.quorum=localhost,
> > > > > > hbase.rpc.timeout=60000,
> hbase.zookeeper.property.clientPort=57963,
> > > > > > zookeeper.znode.parent=/hbase, hbase.client.retries.number=350,
> > > > > > hbase.client.pause=100}, username='danushka'}). May be the key
> was
> > > > > > modified?
> > > > > > java.lang.Exception
> > > > > > at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:493)
> > > > > > at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:429)
> > > > > > at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.stop(ReplicationLogCleaner.java:141)
> > > > > > at
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.cleanup(CleanerChore.java:276)
> > > > > > at org.apache.hadoop.hbase.Chore.run(Chore.java:94)
> > > > > > at java.lang.Thread.run(Thread.java:745)
> > > > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > > 21.625
> > > > > sec
> > > > > >
> > > > > > Thanks,
> > > > > > Danushka
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Director of Data Science
> > > > > Cloudera <http://www.cloudera.com>
> > > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Director of Data Science
> > > Cloudera <http://www.cloudera.com>
> > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> > >
> >
>
>
>
> --
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>
>
Re: Test failure on the master branch
Posted by Josh Wills <jw...@cloudera.com>.
No, master is compatible w/Hadoop 1; the error you're seeing is caused by
the HBase testing code being flaky. I've experienced the flaky
HFileTargetIT test on my machine before, but never on a regular basis. Can
you give me the basics of your setup-- OS, java version, etc.?
On Tue, Oct 14, 2014 at 6:38 AM, Danushka Menikkumbura <
danushka.menikkumbura@gmail.com> wrote:
> No I don't.
>
> That means the master branch is not compatible with Hadoop 1?
>
> Thanks,
> Danushka
>
> On Tue, Oct 14, 2014 at 9:26 AM, Josh Wills <jw...@cloudera.com> wrote:
>
> > Hrm, okay. Do you get it if you use the -Dcrunch.platform=2 option?
> >
> > On Tue, Oct 14, 2014 at 6:23 AM, Danushka Menikkumbura <
> > danushka.menikkumbura@gmail.com> wrote:
> >
> > > Yes. I get it every time I try to build HBase module.
> > >
> > > FYI : org.apache.crunch.io.hbase.HFileTargetIT is the test in error.
> > >
> > > Thanks,
> > > Danushka
> > >
> > > On Tue, Oct 14, 2014 at 9:15 AM, Josh Wills <jw...@cloudera.com>
> wrote:
> > >
> > > > That can happen intermittently if the local HBase cluster gets hung
> > up--
> > > do
> > > > you get it regularly (i.e., every time you run?)
> > > >
> > > > J
> > > >
> > > > On Tue, Oct 14, 2014 at 6:08 AM, Danushka Menikkumbura <
> > > > danushka.menikkumbura@gmail.com> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I am getting the following test failure while building Crunch. Have
> > you
> > > > got
> > > > > an idea as to what may be the issue here?.
> > > > >
> > > > > 34787 [Thread-2057] INFO
> > > > >
> > > org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob
> -
> > > > > Job status available at: http://localhost:8080/
> > > > > 44548 [M:0;danushka:38318.oldLogCleaner] ERROR
> > > > > org.apache.hadoop.hbase.client.HConnectionManager - Connection not
> > > found
> > > > > in the list, can't delete it (connection
> > > > > key=HConnectionKey{properties={hbase.zookeeper.quorum=localhost,
> > > > > hbase.rpc.timeout=60000, hbase.zookeeper.property.clientPort=57963,
> > > > > zookeeper.znode.parent=/hbase, hbase.client.retries.number=350,
> > > > > hbase.client.pause=100}, username='danushka'}). May be the key was
> > > > > modified?
> > > > > java.lang.Exception
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:493)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:429)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.stop(ReplicationLogCleaner.java:141)
> > > > > at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.cleanup(CleanerChore.java:276)
> > > > > at org.apache.hadoop.hbase.Chore.run(Chore.java:94)
> > > > > at java.lang.Thread.run(Thread.java:745)
> > > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 21.625
> > > > sec
> > > > >
> > > > > Thanks,
> > > > > Danushka
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Director of Data Science
> > > > Cloudera <http://www.cloudera.com>
> > > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> > > >
> > >
> >
> >
> >
> > --
> > Director of Data Science
> > Cloudera <http://www.cloudera.com>
> > Twitter: @josh_wills <http://twitter.com/josh_wills>
> >
>
--
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>
Re: Test failure on the master branch
Posted by Danushka Menikkumbura <da...@gmail.com>.
No I don't.
That means the master branch is not compatible with Hadoop 1?
Thanks,
Danushka
On Tue, Oct 14, 2014 at 9:26 AM, Josh Wills <jw...@cloudera.com> wrote:
> Hrm, okay. Do you get it if you use the -Dcrunch.platform=2 option?
>
> On Tue, Oct 14, 2014 at 6:23 AM, Danushka Menikkumbura <
> danushka.menikkumbura@gmail.com> wrote:
>
> > Yes. I get it every time I try to build HBase module.
> >
> > FYI : org.apache.crunch.io.hbase.HFileTargetIT is the test in error.
> >
> > Thanks,
> > Danushka
> >
> > On Tue, Oct 14, 2014 at 9:15 AM, Josh Wills <jw...@cloudera.com> wrote:
> >
> > > That can happen intermittently if the local HBase cluster gets hung
> up--
> > do
> > > you get it regularly (i.e., every time you run?)
> > >
> > > J
> > >
> > > On Tue, Oct 14, 2014 at 6:08 AM, Danushka Menikkumbura <
> > > danushka.menikkumbura@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > I am getting the following test failure while building Crunch. Have
> you
> > > got
> > > > an idea as to what may be the issue here?.
> > > >
> > > > 34787 [Thread-2057] INFO
> > > >
> > org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob -
> > > > Job status available at: http://localhost:8080/
> > > > 44548 [M:0;danushka:38318.oldLogCleaner] ERROR
> > > > org.apache.hadoop.hbase.client.HConnectionManager - Connection not
> > found
> > > > in the list, can't delete it (connection
> > > > key=HConnectionKey{properties={hbase.zookeeper.quorum=localhost,
> > > > hbase.rpc.timeout=60000, hbase.zookeeper.property.clientPort=57963,
> > > > zookeeper.znode.parent=/hbase, hbase.client.retries.number=350,
> > > > hbase.client.pause=100}, username='danushka'}). May be the key was
> > > > modified?
> > > > java.lang.Exception
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:493)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:429)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.stop(ReplicationLogCleaner.java:141)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.cleanup(CleanerChore.java:276)
> > > > at org.apache.hadoop.hbase.Chore.run(Chore.java:94)
> > > > at java.lang.Thread.run(Thread.java:745)
> > > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 21.625
> > > sec
> > > >
> > > > Thanks,
> > > > Danushka
> > > >
> > >
> > >
> > >
> > > --
> > > Director of Data Science
> > > Cloudera <http://www.cloudera.com>
> > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> > >
> >
>
>
>
> --
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>
>
Re: Test failure on the master branch
Posted by Josh Wills <jw...@cloudera.com>.
Hrm, okay. Do you get it if you use the -Dcrunch.platform=2 option?
On Tue, Oct 14, 2014 at 6:23 AM, Danushka Menikkumbura <
danushka.menikkumbura@gmail.com> wrote:
> Yes. I get it every time I try to build HBase module.
>
> FYI : org.apache.crunch.io.hbase.HFileTargetIT is the test in error.
>
> Thanks,
> Danushka
>
> On Tue, Oct 14, 2014 at 9:15 AM, Josh Wills <jw...@cloudera.com> wrote:
>
> > That can happen intermittently if the local HBase cluster gets hung up--
> do
> > you get it regularly (i.e., every time you run?)
> >
> > J
> >
> > On Tue, Oct 14, 2014 at 6:08 AM, Danushka Menikkumbura <
> > danushka.menikkumbura@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I am getting the following test failure while building Crunch. Have you
> > got
> > > an idea as to what may be the issue here?.
> > >
> > > 34787 [Thread-2057] INFO
> > >
> org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob -
> > > Job status available at: http://localhost:8080/
> > > 44548 [M:0;danushka:38318.oldLogCleaner] ERROR
> > > org.apache.hadoop.hbase.client.HConnectionManager - Connection not
> found
> > > in the list, can't delete it (connection
> > > key=HConnectionKey{properties={hbase.zookeeper.quorum=localhost,
> > > hbase.rpc.timeout=60000, hbase.zookeeper.property.clientPort=57963,
> > > zookeeper.znode.parent=/hbase, hbase.client.retries.number=350,
> > > hbase.client.pause=100}, username='danushka'}). May be the key was
> > > modified?
> > > java.lang.Exception
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:493)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:429)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.stop(ReplicationLogCleaner.java:141)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.cleanup(CleanerChore.java:276)
> > > at org.apache.hadoop.hbase.Chore.run(Chore.java:94)
> > > at java.lang.Thread.run(Thread.java:745)
> > > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.625
> > sec
> > >
> > > Thanks,
> > > Danushka
> > >
> >
> >
> >
> > --
> > Director of Data Science
> > Cloudera <http://www.cloudera.com>
> > Twitter: @josh_wills <http://twitter.com/josh_wills>
> >
>
--
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>
Re: Test failure on the master branch
Posted by Danushka Menikkumbura <da...@gmail.com>.
Yes. I get it every time I try to build HBase module.
FYI : org.apache.crunch.io.hbase.HFileTargetIT is the test in error.
Thanks,
Danushka
On Tue, Oct 14, 2014 at 9:15 AM, Josh Wills <jw...@cloudera.com> wrote:
> That can happen intermittently if the local HBase cluster gets hung up-- do
> you get it regularly (i.e., every time you run?)
>
> J
>
> On Tue, Oct 14, 2014 at 6:08 AM, Danushka Menikkumbura <
> danushka.menikkumbura@gmail.com> wrote:
>
> > Hi,
> >
> > I am getting the following test failure while building Crunch. Have you
> got
> > an idea as to what may be the issue here?.
> >
> > 34787 [Thread-2057] INFO
> > org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob -
> > Job status available at: http://localhost:8080/
> > 44548 [M:0;danushka:38318.oldLogCleaner] ERROR
> > org.apache.hadoop.hbase.client.HConnectionManager - Connection not found
> > in the list, can't delete it (connection
> > key=HConnectionKey{properties={hbase.zookeeper.quorum=localhost,
> > hbase.rpc.timeout=60000, hbase.zookeeper.property.clientPort=57963,
> > zookeeper.znode.parent=/hbase, hbase.client.retries.number=350,
> > hbase.client.pause=100}, username='danushka'}). May be the key was
> > modified?
> > java.lang.Exception
> > at
> >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:493)
> > at
> >
> >
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:429)
> > at
> >
> >
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.stop(ReplicationLogCleaner.java:141)
> > at
> >
> >
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.cleanup(CleanerChore.java:276)
> > at org.apache.hadoop.hbase.Chore.run(Chore.java:94)
> > at java.lang.Thread.run(Thread.java:745)
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.625
> sec
> >
> > Thanks,
> > Danushka
> >
>
>
>
> --
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>
>
Re: Test failure on the master branch
Posted by Josh Wills <jw...@cloudera.com>.
That can happen intermittently if the local HBase cluster gets hung up-- do
you get it regularly (i.e., every time you run?)
J
On Tue, Oct 14, 2014 at 6:08 AM, Danushka Menikkumbura <
danushka.menikkumbura@gmail.com> wrote:
> Hi,
>
> I am getting the following test failure while building Crunch. Have you got
> an idea as to what may be the issue here?.
>
> 34787 [Thread-2057] INFO
> org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob -
> Job status available at: http://localhost:8080/
> 44548 [M:0;danushka:38318.oldLogCleaner] ERROR
> org.apache.hadoop.hbase.client.HConnectionManager - Connection not found
> in the list, can't delete it (connection
> key=HConnectionKey{properties={hbase.zookeeper.quorum=localhost,
> hbase.rpc.timeout=60000, hbase.zookeeper.property.clientPort=57963,
> zookeeper.znode.parent=/hbase, hbase.client.retries.number=350,
> hbase.client.pause=100}, username='danushka'}). May be the key was
> modified?
> java.lang.Exception
> at
>
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:493)
> at
>
> org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:429)
> at
>
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.stop(ReplicationLogCleaner.java:141)
> at
>
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.cleanup(CleanerChore.java:276)
> at org.apache.hadoop.hbase.Chore.run(Chore.java:94)
> at java.lang.Thread.run(Thread.java:745)
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.625 sec
>
> Thanks,
> Danushka
>
--
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>